Explore OpenStack (9): In-depth block storage Service Cinder (Part 1)

After studying the neutron, continue Nova's peripheral study tour. This site is a research block storage service cinder.

0. Verifying the Environment

The environment includes:

1, a controller node, running NOVA-API, Nova-scheduler, Cinder-api, Cinder-scheduler, MySQL, RABBITMQ

2, a Nova compute node, running a virtual machine

3, three cinder volume nodes, each node uses Lvmiscsidriver to use local storage

4. Create a volume type, set volume_backend_name = Lvmbackend

cinder.conf on Block1 enabled_backends = Lvmdriver-b1[lvmdriver-B1]volume_group= cinder-Volumesvolume_driver=Cinder.volume.drivers.lvm.LVMISCSIDrivervolume_backend_name=lvmbackendcinder.conf on Block2 enabled_backends = Lvmdriver-b21,lvmdriver-b22storage_availability_zone=Az1[lvmdriver-b21]iscsi_ip_address=10.0.1.29Volume_group= cinder-Volumes1volume_driver=Cinder.volume.drivers.lvm.LVMISCSIDrivervolume_backend_name=Lvmbackend[lvmdriver-B22]volume_group= cinder-Volumes2volume_driver=Cinder.volume.drivers.lvm.LVMISCSIDrivervolume_backend_name=lvmbackendcinder.conf on the Block3.
Enabled_backends = Lvmdrier-network[lvmdriver-Network]volume_group=Systemvolume_driver=Cinder.volume.drivers.lvm.LVMISCSIDrivervolume_backend_name= Lvmbackend

Cinder's service is as follows:

[Email protected]:/home/s1# Cinder service-List+------------------+---------------------------+------+---------+-------+----------------------------+---------      --------+|            Binary | Host |  Zone | Status |         State | Updated_at | Disabled Reason |+------------------+---------------------------+------+---------+-------+----------------------  ------+-----------------+|         Cinder-backup | Controller | Nova |   Enabled | Up | -- on-11t16: $:00.000000| None | |         Cinder-scheduler | Controller | Nova |   Enabled | Up | -- on-11t16: $:01.000000|  None | |    Cinder-volume | [Email protected] | Nova |   Enabled | Up | -- on-11t16: $:08.000000|  None | |    Cinder-volume | [Email protected] | AZ1 |   Enabled | Up | -- on-11t16: $:06.000000|  None | |    Cinder-volume | [Email protected] | AZ1 |   Enabled | Up | -- on-11t16: $:05.000000|  None | | Cinder-volume | [Email protected] | Nova |   Enabled | Up | -- on-11t16: $:02.000000| None |+------------------+---------------------------+------+---------+-------+----------------------------+---- -------------+

Description

• Cinder run a Cinder-volume service for each backend
• You can specify a zone for the Cinder-volume host by setting the STORAGE_AVAILABILITY_ZONE=AZ1 in cinder.conf. You can select AZ when creating volume, and with Cinder-scheduler availabilityzonefilter you can create the volume into the specified AZ. The zone is nova by default.
• By setting the backend configuration section in cinder.conf iscsi_ip_address = 10.0.1.29 You can specify the network card used by the iSCSI session, thus separating the data network from the management network.
• With the above multi-node environment, it is important to note that each node uses NTP for time synchronization, otherwise there may be cinder-volume without any errors, but the condition is down.

1. About the storage in OpenStack block storage 1.1 OpenStack

1.2 Requirements for block storage of virtual machines

1.3 Cinder Overview

1.4 Cinder's internal architecture

• Three main components
  • The –CINDER-API component is responsible for providing an out-of-cinder REST API
  • The –cinder-scheduler component is responsible for allocating storage resources
  • The –cinder-volume component is responsible for encapsulating the driver, with different driver responsible for controlling different back-end storage
• RPC by Message Queuing (queue) implementation between components
• Cinder's development efforts are focused on scheduler and driver, in order to provide more scheduling algorithms, more functionality, and to point out more back-end storage
• Volume metadata and status stored in database

Basic functions of 1.5 cinder

1	Volume operations	Create a volume
2		Create a volume from an existing volume (clone)
3		Extended volume
4		Delete a volume
5	Volume-Virtual machine operation	Mount a volume to a virtual machine
6		Detaching a virtual machine volume
7	Volume-Snapshot operations	To create a snapshot of a volume
8		To create a volume from an existing volume snapshot
9		Deleting a snapshot
10	Volume-Mirroring operations	Create a volume from a mirror
11		To create a mirror from a volume

1.6 Cinder Plug-in

1.6.1 Lvmiscsidriver

each hyperviosor as an iSCSI initiator. Likeinitiator:iqn.1993-08.org.debian:01:8d794081cd6a alias:compute1 [Email protected]:/home/s1# cat/etc/iscsi/  INITIATORNAME.ISCSIINITIATORNAME=IQN.1993-08.ORG.DEBIAN:01:8D794081CD6A each LUN as an iSCSI target: for example, IQN- iqn.2010-10.org.openstack:volume-3f204086-609e-449f-90a1-3a0d2c92c525 there is a TCP session between each initiator and target. View iSCSI session on compute node: [email protected]:/home/s1# iscsiadm-m Session
TCP: [Ten] 192.168.1.24:3260,1 iqn.2010-10.org.openstack:volume-5cfc715d-a7b3-47b4-bded-44c0a228360c
TCP: [One] 192.168.1.19:3260,1 iqn.2010-10.org.openstack:volume-4039eb07-90eb-4a92-8fd3-e3514cb4969b
TCP: [192.168.1.29:3260,1] iqn.2010-10.org.openstack:volume-3f204086-609e-449f-90a1-3a0d2c92c525
TCP: [+] 10.0.1.29:3260,1 iqn.2010-10.org.openstack:volume-1b7f6669-06db-474e-bf78-4feea529be5b
TCP: [6] 192.168.1.24:3260,1 iqn.2010-10.org.openstack:volume-39363c5f-cf3c-4461-af83-00314839f05atcp: [9] 192.168.1.24:3260,1 IQN.2010-10.ORG.OPENSTACK:VOLUME-A0A7CCB3-8864-4FD0-AEE2-0E20D43BA8DD details for each target: tgtadm-- LLD iSCSI--op Show--mode Target
Target 1:iqn.2010-10.org.openstack:volume-136354c3-5920-46b9-a930-52c055c53295
System Information:
Driver:iscsi
State:ready
i_t Nexus Information:
i_t Nexus:2
INITIATOR:IQN.1993-08.ORG.DEBIAN:01:8D794081CD6A alias:compute1
connection:0
IP address:192.168.1.15
LUN Information:
lun:0
Type:controller
SCSI Id:iet 00010000
SCSI SN:BEAF10
size:0 MB, Block size:1
Online:yes
Removable Media:no
Prevent Removal:no
Readonly:no
Swp:no
Thin-provisioning:no
Backing store Type:null
Backing store Path:none
Backing Store Flags:
Lun:1
Type:disk
SCSI Id:iet 00010001
SCSI sn:beaf11
size:1074 MB, Block size:512
Online:yes
Removable Media:no
Prevent Removal:no
Readonly:no
Swp:no
Thin-provisioning:no
Backing store Type:rdwr
Backing store Path:/dev/cinder-volumes/volume-136354c3-5920-46b9-a930-52c055c53295
Backing Store Flags:
Account information:
S6kdhjsuru2meeyxptdz
ACL Information:
After the all volume is detached from the virtual machine, the corresponding TCP session is deleted. 1.6.2 IBM SVC/DS8K/XIV Plug-in

Comparison of 1.6.3 LVM plug-ins and vendor plugins

2. Cinder operation

Here are a few more interesting things to say.

2.1 Tranfer Volume: A user who transfers the ownership of volume from one tenant to another in tenant.

Two-Step walk:

1. Users using commands in the volume tenantCinder transfer-create generates tranfer transfer ID and authkey:[email protected]:/home/s1# cinder transfer-create D146a947-9c1e-489f-b7a3-6b9604d9fb49
+------------+--------------------------------------+
|                Property | Value |
+------------+--------------------------------------+
|           Auth_key | a94e45d06dd54500 |
|      Created_at | 2015-01-07t07:36:33.916921 |
| ID | b14d1d26-8249-4dd2-8213-258ccfe31542 |
|                 name | None |
| volume_id | d146a947-9c1e-489f-b7a3-6b9604d9fb49 |
+------------+--------------------------------------+ Current Tenant id:os-vol-tenant-attr:tenant_id | 96aacc75dc3a488cb073faa06a34b235 2. When a user in another tenant uses the command cinder transfer-accept accept transfer, you need to enter the transfer ID and auth_key[email protected]:~$ cinder Transfer-accept b14d1d26-8249-4dd2-8213-258ccfe31542 a94e45d06dd54500
+-----------+--------------------------------------+
|                Property | Value |
+-----------+--------------------------------------+
| ID | b14d1d26-8249-4dd2-8213-258ccfe31542 |
|                 name | None |
| volume_id | d146a947-9c1e-489f-b7a3-6b9604d9fb49 |
+-----------+--------------------------------------+ new Tenant id:os-vol-tenant-attr:tenant_id | 2F07AD0F1BEB4B629E42E1113196C04B in fact, for volume, is to modify the tenant ID (attribute: os-vol-tenant-attr:tenant_id) just. 2.2 Volume Migrate: Migrating volume from one backend to another backend

A variety of possible scenarios:

1. If volume does not have attach to virtual machine,    1.1 If it is a migration between different backend on the same storage, the driver that needs to be stored will directly support the migrate on the storage.     1.2 If it is a volume migration between backend on different stores, or the storage cinder driver does not support migration between backend on the same storage, Then the cinder default migration action is used: Cinder first creates a new volume, then copies the data from the source volume to the new volume, and then deletes the old volume. 2. If volume has been attach to the VM, Cinder creates a new volume, calls Nova to copy the data from the source volume to the new volume, and then deletes the old volume. Currently only supports compute Libvirt driver.  Note in the case of multiple backend, host must use the host full name. For example:  cinder migrate vol-b21-1 [email protected] 2.3 volume Backupopenstack Juno Version support volume Backup to   CEPH,SWIFT,IBM Tivoli Storage Manager (TSM). The  2.4 QoS support Cinder provides a QoS support framework that relies on plugin implemented by each vendor.   with IBM Svc as an example, you can use QoS as follows: (1) Create a QoS spec:cinder qos-create Qos-spec qos:iothrottling=12345 (2) associated QoS Spec to a volume  typecinder qos-associate 0e710a13-3c40-4d50-8522-72bddabd93cc (3) Create this volume type of volumecinder Create 1--volume-type svc-driver25--display-name volwit (4) View the volume, which is set with the throttling property, which limits the maximum I/O on the volume. SVC volume: throttling 12345 3 Cinder components

about using RABBITMQ-based RPC communication between components within the Rpc:cinder. Cinder-scheduler and Cinder-volume Create an RPC connection, start the consumer thread, and then wait for the queue message. When a polling query arrives at a message, the thread is created to process the related message.

3.1 Cinder-api

The primary service interface is responsible for accepting and processing external API requests and placing requests into the RABBITMQ queue for execution by the backend.

3.2 Cinder-scheduler

The purpose of Cinder-scheduler is to determine the location of volume in a multi-backend environment:

0. First, the state of the host is determined, and only the host with the service status up will be considered.

1. When creating the volume, the optimal host is chosen according to the filter and weight algorithm to create the volume.

2. When migrating volume, the filter and weight algorithms are used to determine whether the destination host meets the requirements.

If a host is selected, the RPC call Cinder-volume is used to perform the volume operation.

In order to maintain the state of the host, Cinder-scheduler accepts the Cinder-volume status escalation on the scheduled host:

2015-01-12 02:02:56.688 828 DEBUG Cinder.scheduler.host_manager [req-403ef666-5551-4f31-a130-7bcad8e9d1ec-----] rece Ived Volume service update from [email protected]: {u ' pools ': [{u ' pool_name ': U ' lvmbackend ', U ' qos_support ': False, U ' Allo CATED_CAPACITY_GB ': 1, U ' FREE_CAPACITY_GB ': 3.34, U ' location_info ': U ' lvmvolumedriver:block2:cinder-volumes1: Default:0 ', U ' total_capacity_gb ': 5.34, U ' reserved_percentage ': 0}], u ' driver_version ': U ' 2.0.0 ', U ' vendor_name ': U ' Open Source ', U ' volume_backend_name ': U ' lvmbackend ', U ' storage_protocol ': U ' iSCSI '} update_service_capabilities/usr/ lib/python2.7/dist-packages/cinder/scheduler/host_manager.py:434

3.2.1 Host Filtering algorithm

The default filter includes Availabilityzonefilter,capacityfilter,capabilitiesfilter. which

• Availabilityzonefilter will determine if cinder host's availability zone is the same as the destination AZ. The difference is filtered out.
• Capacityfilter will determine the size of the remaining space free_capacity_gb on the host, ensuring that the FREE_CAPACITY_GB is larger than the volume size. Not enough is filtered out.
• Capabilitiesfilter checks whether the properties of the host are exactly the same as the extra specs in the volume type. The inconsistency was lost by the country.

After filtering the filter above, Cinder-scheduler will get the host list that matches the condition, then enter the weighting link and select the optimal host according to the weighting algorithm. An empty list is reported to no valid host was found error.

Cinder.conf, if Scheduler_default_filters is not set, Cinder-scheduler will use the three filter by default.

3.2.2 Host Weighting algorithm

• Allocatedcapacityweigher: Host with the smallest used space wins. You can set the Allocated_capacity_weight_multiplier to be reversed with a positive value, which defaults to-1.

• Capacityweigher: Host with the largest available space wins. The capacity_weight_multiplier can be set to a negative value to invert the algorithm, which defaults to 1

• Chanceweigher: Randomly select a host from the filtered host

After this step, Cinder-scheduler will get a list of weighted_hosts, which will select the first host as the destination host for volume, add it to the Retry_hosts list, The Cinder-volume is then called by RPC to create the volume.

Cinder.conf, Scheduler_default_weighers is not set, Cinder-scheduler uses Capacityweigher by default.

3.3 Cinder-volume The service runs on storage nodes, manages storage spaces, handles read and write requests for maintenance status of cinder databases, interacts with other processes through Message Queuing and directly on block storage devices or software. Each storage node has a volume Service, and several such storage nodes can be combined to form a pool of storage resources. Cinder-volume will implement some common operations, such asCopy_volume_data, the attach source and target volume are implemented in the driver.py, and then the copy data is executed. Other operations requirecall the driver interface to implement the volume operation. 3.3.1 Volume create failed retry mechanism

Users can use scheduler_max_attempts in cinder.conf to configure the number of retries when the volume creation fails, with a default number of 3 and a value of 1 indicating that the retry mechanism is not used.

# Maximum number of attempts to schedule an volume (integer value)
#scheduler_max_attempts =3

The current number of retries is passed between Cinder-sheduler and Cinder-volume. If volume creation fails, Cinder-volume will re-call Cinder-scheduler through RPC to create Volume,cinder-scheduler and check that the current retry count is greater than the maximum number of retries. If not, it will choose the next host that can be used to recreate the volume. If the volume is still not created within the specified number of retries, then no valid host was found error is reported.

For example, the following retry process:

Cinder-volume:

1. Insufficient free space for volume creation on host [email protected] #lvmbackend (requested/avail): 5/0.0
2. Insufficient free space for volume creation on host [email protected] #lvmbackend (requested/avail): 5/4.0
3. Insufficient free space for volume creation on host [email protected] #lvmbackend (requested/avail): 5/1.0

Cinder-scheduler:no valid host was found

The next blog post will specifically analyze the volume process.

Explore OpenStack (9): In-depth block storage Service Cinder (Part 1)