Openstack storage Summary: explains how to use NFS as the cinder's back-end Storage

Brief description of NFS service

NFS is short for network file system, that is, Network File System. An agreement for the use of distributed file systems, developed by Sun, was published on April 9, 1984. The function is to allow different machines and different operating systems to share individual data with each other through the network, so that applications can access data on server disks through the network on the client, it is a way to share disk files between Unix-like systems.

 

The basic principle of NFS is "allow different clients and servers to share the same file system through a group of RPC", which is independent of the operating system, allows different hardware and operating system systems to share files.

 

NFS depends on the RPC protocol during file transfer or information transfer. Remote Procedure Call (RPC) is a mechanism that enables the client to execute programs in other systems. NFS itself does not provide information transmission protocols and functions, but NFS allows us to share data over the network, because NFS uses some other transmission protocols. These transmission protocols use this RPC function. NFS itself is a program that uses RPC. Or NFS is also an RPC server. Therefore, the RPC service must be started wherever NFS is used, whether it is an NFS server or an NFS client. In this way, the server and client can implement the corresponding program port through rpc. We can understand the relationship between RPC and NFS in this way: NFS is a file system, while RPC is responsible for information transmission.

 

NFS server

 

NFS Server Installation

yum -y install rpcbind nfs-utils

NFS server configuration

The configuration on the NFS server is relatively simple. It only involves modifications to the/etc/exports file. The configuration content is as follows:

/nfs/shared192.168.40.0/255.255.255.0(rw,sync)

The above configuration indicates "192.168.40.0/255.255.255.0". Users of this network segment can mount the/nfs/shared directory on the NFS server. After mounting, they have read and write permissions. Because they do not specify the method for compressing user permissions, therefore, even if you log on as the root user, it will be downgraded to nobody.

Start NFS service

service rpcbind startservice nfs startservice nfslock startchkconfig rpcbind onchkconfig nfs onchkconfig nfslock on

NFS service verification

The server uses the showmount command to query the NFS sharing status.

# Showmount-E // by default, you can view your shared services, provided that the DNS can resolve yourself, otherwise it is easy to report an error # showmount-A // display the directory information that has been connected to the client

The client uses the showmount command to query the NFS sharing status.

# Showmount-e NFS server IP Address

NFS system daemon

• NFSD: it is a basic NFS Daemon. Its main function is to manage whether the client can log on to the server;
• MOUNTD: it is the RPC installation daemon. Its main function is to manage NFS file systems. After the client successfully logs on to the NFS server through NFSD, it must pass the File Permission verification before using the files provided by the NFS service. It reads the NFS configuration file/etc/exports to compare the client permissions.

• Portmap: Mainly used for port ing. When the client tries to connect to and use the services provided by the RPC server (such as the NFS service), Portmap will provide the managed port corresponding to the service to the client, this allows the customer to request services from the server through this port.

Common NFS directories

Main configuration file of/etc/exports NFS service

/Usr/sbin/exportfs NFS service management command

Command for viewing/usr/sbin/showmount Client

/Var/lib/nfs/etab records the complete permission settings for the directories shared by NFS.

/Var/lib/nfs/xtab records client information that has been logged on

 

Remount the NFS Directory

If we modify/etc/exports after starting NFS, do we have to restart NFS? In this case, we can use the exportfs command to make the change take effect immediately.

Exportfs-ARV

 

Cinder node NFS client configuration and installation software

Yum install rpcbindnfs-utils

Start Related Services

servicerpcbind startservicenfslock startchkconfigrpcbind onchkconfignfs onchkconfignfslock on

Check NFS server sharing information

The IP address of the NFS server is 192.168.40.107.

[[email protected](keystone_admin)]# showmount -e 192.168.40.107Exportlist for 192.168.40.107:/nfs/shared192.168.40.0/255.255.255.0

Mount to local directory

cd /rootmkdir nfssharemount -tnfs 192.168.40.107:/nfs/shared /root/nfsshare/

View mounting result

[[email protected]~(keystone_admin)]# df -hFilesystem                           Size  Used Avail Use% Mounted on/dev/sda1                             97G  4.8G  87G   6% /tmpfs                                3.9G  4.0K 3.9G   1% /dev/shm/srv/loopback-device/swift_loopback  1.9G  67M  1.8G   4% /srv/node/swift_loopback192.168.40.107:/nfs/shared           444G 1.4G  420G   1% /root/nfsshare

 

Note that if the NFS server fails at this time or the client cannot connect to the server, the command will be slow because it will not return results until the file system searches for timeout, this principle is used for all commands for the file system, such as DF, ls, CP, etc.

 

Cinder node NFS backend storage Configuration

 

Create the/etc/cinder/nfsshares file and edit the file as follows:

192.168.40.107:/home/nfsshare

Set Configuration File Permissions

[[email protected]~]# chown root:cinder /etc/cinder/nfsshares[[email protected]~]# chmod 0640 /etc/cinder/nfsshares

The volume service for configuring cinder uses NFS

Modify/etc/cinder. conf to/etc/cinder/nfsshares. Run the following command:

openstack-config --set /etc/cinder/cinder.conf DEFAULT nfs_shares_config /etc/cinder/nfsshares

To configure the driver used by cinder volume, run the following command:

openstack-config --set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.nfs.NfsDriver

Restart the service

service openstack-cinder-volume restart<span style="font-size:14px;"> </span>

View the disk status on the client

Last Record added

Create a virtual machine, network hard disk, and mount the network hard disk to the host

Console:

Virtual Machine:

 

Attach a hard disk to a VM

Use the following command to format

mkfs.ext4 /dev/vdb

Then execute the following command:

 

Problem

The following exception occurs in Nova/compute. log during mounting.

 

2014-10-2312:23:28.193 1747 INFO urllib3.connectionpool [-] Starting new HTTP connection(1): 192.168.40.2482014-10-2312:23:28.395 1747 WARNING nova.virt.libvirt.utils[req-5bf92b88-6d15-4c41-8ed7-3325fdea0dcf 5832a2295dc14de79522ee8b42e7daac9207105ae2ac4ef3bdf5dfe40d99fd8d] systool is not installed2014-10-2312:23:28.449 1747 WARNING nova.virt.libvirt.utils[req-5bf92b88-6d15-4c41-8ed7-3325fdea0dcf 5832a2295dc14de79522ee8b42e7daac9207105ae2ac4ef3bdf5dfe40d99fd8d] systool is not installed2014-10-2312:23:28.451 1747 INFO urllib3.connectionpool [-] Starting new HTTP connection(1): 192.168.40.2482014-10-2312:23:28.960 1747 ERROR nova.virt.block_device[req-5bf92b88-6d15-4c41-8ed7-3325fdea0dcf 5832a2295dc14de79522ee8b42e7daac9207105ae2ac4ef3bdf5dfe40d99fd8d] [instance:eb1742c6-1e73-4656-b646-ca8442519e7a] Driver failed to attach volume a1862c54-0671-4cc5-9fce-5e5f8485c21fat /dev/vdb2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a] Traceback (most recent call last):2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File"/usr/lib/python2.6/site-packages/nova/virt/block_device.py", line239, in attach2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    device_type=self['device_type'], encryption=encryption)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line1267, in attach_volume2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    disk_dev)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File"/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py",line 68, in __exit__2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    six.reraise(self.type_, self.value, self.tb)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line1254, in attach_volume2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    virt_dom.attachDeviceFlags(conf.to_xml(), flags)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py",line 183, in doit2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    result = proxy_call(self._autowrap, f, *args, **kwargs)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py",line 141, in proxy_call2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    rv = execute(f, *args, **kwargs)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py",line 122, in execute2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    six.reraise(c, e, tb)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File "/usr/lib/python2.6/site-packages/eventlet/tpool.py",line 80, in tworker2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    rv = meth(*args, **kwargs)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]  File "/usr/lib64/python2.6/site-packages/libvirt.py", line419, in attachDeviceFlags2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a]    if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed',dom=self)2014-10-2312:23:28.960 1747 TRACE nova.virt.block_device [instance:eb1742c6-1e73-4656-b646-ca8442519e7a] libvirtError: internal error unable toexecute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk1'could not be initialized

Solution

This error is from libvirt. Make the following settings to check whether the kernel _use_nfs is off or on.

/usr/sbin/getseboolvirt_use_nfs

If it is off, do the following settings

/usr/sbin/setsebool -P virt_use_nfs on

Openstack storage Summary: explains how to use NFS as the cinder's back-end Storage