OpenStack Swift Installation Notes

After the smooth installation of the deployment of Keystone, the original thought that Swift installation and debugging will be like Keystone general, but the process is full of bumpy, in the combination of some functions in the source code, finally successfully completed the installation of Swift, and experimental upload download files and other functions, The following set up the swift installation process, and the problems encountered in the summary, I hope to meet the same problem of people have some inspiration or help. The installation process still refers to the official OpenStack installation manual, but adds some details. Before installing, let's briefly describe what Swift is and what the main features are. OpenStack Object Storage (Swift) is a multitenant, highly extensible object Storage system that manages large amounts of unstructured data through RESTful HTTP APIs, and must have at least identity services installed (KeyStone) before deploying object storage. The components that Swift contains include:

1. Proxy Server (Swift-proxy-server): accepts object store APIs and HTTP requests to upload files, modify metadata, and create containers, as well as provide a list of files or containers to the browser. To improve performance, the proxy server can use an optional cache, typically choosing memcache to deploy with a proxy server.
2. Account Services (Account-server): Manage accounts in the object store.
3. Container Service (Container-server): Manages the mapping of containers or folders in the object store.
4. Object Services (Object-server): Manages the actual objects on the storage node, such as files.
5. Wsgi middle tier: Process authentication, usually identity service (KeyStone).
6. Recurring processes: Perform various maintenance tasks in the cluster, such as replication (replicator) services to ensure consistency and availability of data in the cluster, among others: Auditor, Updater, and Reaper.

The proxy service relies on authentication and authorization mechanisms, and if authentication and authorization are done using identity services, the first step is to create the SWIFT authentication information and endpoint in Keystone before configuring Swift. Object storage is not available on the control node for SQL database. Since the Keystone has been successfully deployed, it is not here to repeat the installation process of Keystone and go directly to swift installation. This installation is performed on a single-node virtual machine, where the control node and storage node are deployed on the same host and the operating system is CentOS7.1. Start by creating Swift users, services, and endpoint in Keystone, with the following specific commands and results:

[[email protected] ~]$ Keystone user-create--name Swift--pass 123456+----------+------------------------------ ----+|              Property |  Value |+----------+----------------------------------+|                                  email | ||               Enabled |    True | | ID |   ED4F45AC3E8A4AC683D54B642E61AC04 | |              name | Swift | |              Username | Swift |+----------+----------------------------------+keystone user-role-add--user Swift--role admin--ten Ant service[[email protected] ~]$ Keystone service-create--name Swift--type object-store+-------------+--------   --------------------------+|              Property | Value |+-------------+----------------------------------+|                                  Description |   ||               Enabled |      True | | ID |     094cc0da43e348f8b792f77ef99f8e7e | |              name |  Swift | |   Type | Object-store |+-------------+----------------------------------+[[email protected] ~]$ Keystone ENDPOINT-CR Eate--region regionone--service Swift--publicurl ' http://localhost:8080/v1/AUTH_% (tenant_id) s '--internalurl ' http ://localhost:8080/v1/auth_% (tenant_id) s '--adminurl ' http://localhost:8080/v1/AUTH_% (tenant_id) s ' +-------------+   -----------------------------------------------------------------+|                              Property |   Value |+-------------+-----------------------------------------------------------------+|                    Adminurl |      http://locoalhost:8080/v1/| |                 ID | C0DA7D196E204E038766740E405DBBCE | | InternalUrl |  http://locoalhost:8080/v1/AUTH_2835009c452b4d408f95ff5a920fc877 | | Publicurl |    http://locoalhost:8080/v1/AUTH_2835009c452b4d408f95ff5a920fc877 | |                            Region |                      Regionone      ||                 service_id | 094cc0da43e348f8b792f77ef99f8e7e |+-------------+---------------------------------------------------------- -------+

Then install the Proxy-server,swift client, memcached, etc.: Yum install Openstack-swift-proxy python-swiftclient Python-keystone-auth-token python-keystonemiddleware memcached. The Python-keystone-auth-token in the above command cannot be installed and will be reported as exception information when using Yum: There is no available package Python-keystone-auth-token.  After confirming on OpenStack's official website that the problem is an error in the documentation, ignore the issue directly or perform a Yum install Openstack-swift-proxy python-swiftclient Python-keystonemiddleware memcached. After the installation has finished executing, download the proxy-server.conf file by executing the following command:

Curl-o/etc/swift/proxy-server.conf https://raw.githubusercontent.com/openstack/swift/stable/juno/etc/ Proxyserver.conf-sample

After the download is complete, the configuration parameters in the file should be modified, and the specific contents are as follows:

• Specify the port number, user name, and profile directory in [DEFAULT]:

[Default]...bind_port = 8080user = Swiftswift_dir =/etc/swift

• In [Pipeline:main], enable the appropriate module:

[Pipeline:main]pipeline = Authtoken cache healthcheck Keystoneauth proxy-logging proxy-server

• In [App:proxy-server], enable the user management feature:
  

[app:proxy-server]...allow_account_management = Trueaccount_autocreate = True

• Configure the appropriate operation role in [Filter:keystoneauth];

[Filter:keystoneauth]use = Egg:swift#keystoneauth...operator_roles = Admin,_member_

• To configure information about the authentication service in [Filter:authtoken]:

[filter:authtoken]paste.filter_factory = Keystonemiddleware.auth_token:filter_factory...auth_uri =/http Localhost:5000/v2.0identity_uri = Http://localhost:35357admin_tenant_name = Serviceadmin_user = Swiftadmin_password = Swift_pass #指定为使用keystone创建swift用户时设置的密码, such as 123456delay_auth_decision = True

• In [Filter:cache], configure the location of the memcached:
  

[Filter:cache]...memcache_servers = 127.0.0.1:11211

This completes the installation and deployment of Proxy-server, which can be started by using SYSTEMCTL, but does not start the service until after the Configuration Storage service is deployed. Before installing the storage service, specify the storage directory for the storage service, which requires a disk partition to be mounted to the storage directory. According to the official installation manual, the mounted disk must have a partitioned table, which means that the disk has been partitioned. After testing, the disk has several partitions that are not important, but partitions must be performed, and you can use the FDISK command to perform disk partitioning operations. Although Swift supports a variety of file systems, depending on the best performance and stability of the test XFS file system, after partitioning the disk, such as/DEV/SDB1, you need to format the/DEV/SDB1 using the command mkfs.xfs: MKFS.XFS/DEV/SDB1. After formatting is complete, create the Mount directory: Mkdir-p/srv/node/, and then edit/etc/fstab, adding the following statement:

/DEV/SDB1/SRV/NODE/XFS noatime,nodiratime,nobarrier,logbufs=8 0 2

Last mount device storage directory: Mount/srv/node, and modify the directory owner to run Swift users: Chown-r swift:swift/srv/. Mount Directory is a potential problem for swift, it may be a permissions issue, it may be related to the configuration file described later, the specific problem is described in detail later. After mounting the device, you also need to modify the/etc/rsyncd.conf file, as follows:

UID = Swiftgid = Swiftlog File =/var/log/rsyncd.logpid File =/var/run/rsyncd.pidaddress = Management_interface_ip_addres S[account]max connections = 2path =/srv/read only = Falselock File =/var/lock/account.lock[container]max connections = 2 Path =/srv/read = Falselock File =/var/lock/container.lock[object]max connections = 2path =/srv/read only = Falsel Ock file =/var/lock/object.lock

After you have finished preparing, execute the following command to install the storage service, including Account-server, Container-server, and Object-server:yum installed Openstack-swift-account Openstack-swift-container Openstack-swift-object. After the installation is complete, execute the following command to download the account-server.conf, container-server.conf, and object-server.conf files:

Curl-o/etc/swift/account-server.conf https://raw.githubusercontent.com/openstack/swift/stable/juno/etc/ Account-server.conf-samplecurl-o/etc/swift/container-server.conf https://raw.githubusercontent.com/openstack/ Swift/stable/juno/etc/container-server.conf-samplecurl-o/etc/swift/object-server.conf https:// Raw.githubusercontent.com/openstack/swift/stable/juno/etc/object-server.conf-sample

Three configuration files are similar to what is being modified, so only the account-server.conf is described, and the other two files are marked in different places.

• In [DEFAULT], configure the IP address, port number, user, directory and mount directory of the configuration file, you need to be aware of the Mount directory, which is the mount point of the parent directory, such as/DEV/SDB1 mount to the directory/srv/node directory, in the configuration file in the Mount directory should be /srv, not/srv/node.

[DEFAULT]...BIND_IP = 127.0.0.1bind_port = 6002 #不同的服务使用不同的端口user = Swiftswift_dir =/etc/swiftdevices =/srv/#设备挂载目录的上级目 Recorded

• In [Pipeline:main], set the appropriate module:

[Pipeline:main]pipeline = Healthcheck Recon Account-server#account-server In other configuration files should be changed to Container-server, Object-server

• In [Filter:recon], configure the recon cache directory:

[Filter:recon]...recon_cache_path =/var/cache/swift

Container-server.conf and object-server.conf only need to modify the above annotated place, the other place is exactly the same. After modifying all the profiles, change the owner of the/var/cache/swift to a user running swift, such as the Swift User: Chown-r swift:swift/var/cache/swift. After performing all of the above steps, the next step is to generate the ring file, the specific meaning of the command will not be explained in depth, you can enter the command directly swift-ring-builder to see the detailed description. First go to the/etc/swift directory, which is set in the above configuration files, and then generate the ring file for account, container and object respectively, because all the storage services are deployed on a single node, so the number of replicas is set to 1, the specific command is:

Swift-ring-builder Account.builder Create 8 1 1swift-ring-builder container.builder Create 8 1 1swift-ring-builder Object . Builder Create 8 1 1

The first parameter specifies the number of partition to be generated, such as 2^8=256, the second parameter executes the number of copies, here is 1, and the third parameter means the number of hours that the partition is moved at least once. After creating the ring file, you need to add the storage node or directory to the ring:

Swift-ring-builder account.builder Add r1z1-127.0.0.1:6002/node 100swift-ring-builder container.builder add R1z1-127.0.0.1:6001/node 100swift-ring-builder object.builder Add r1z1-127.0.0.1:6000/node 100

The port number must match the port number set in the corresponding configuration file, and the slash/trailing content matches the last level of the mount point, such as mounting the/DEV/SDB1 to/srv/node, then/after the content is node, and the value of devices in the configuration file is/srv/ , the following error occurs if you set the mismatch or do not match the Mount directory:

Object-replicator:node is not mountedproxy-server:error insufficient Storage 127.0.0.1:6002/node

You can also enter the following statement under the Python command line to verify that the settings are mounted properly, that the configuration is correct, and that if everything is correct, return true, otherwise false:

From swift.common.constraints import check_mountcheck_mount ('/srv ', ' node ') #第一个参数为配置文件中devices的值, the second parameter is when adding a device to the ring/ The values that follow

Then run the following command to rebalance the ring:

Swift-ring-builder Account.builder Rebalanceswift-ring-builder Container.builder rebalanceswift-ring-builder Object.builder rebalance

Note that the above generated ring file, the steps to increase the storage node to the ring are performed in the control Section store. Download the swift.conf file and modify it:

Curl-o/etc/swift/swift.conf Https://raw.githubusercontent.com/openstack/swift/stable/juno/etc/swift.conf-sample

• Configure the prefix and suffix of the hash path in [Swift-hash]:

[Swift-hash]...swift_hash_path_suffix = Hash_path_prefixswift_hash_path_prefix = Hash_path_suffix

• Configure the default storage policy in [storage-policy:0]:

[Storage-policy:0]...name = Policy-0default = yes

If you are deploying swift with multiple nodes, you will also need to copy the ring file, the swift.conf file generated by the above steps to all nodes, and modify the owner of the/etc/swift to be the Swift User: Chown-r swift:swift/etc/swift. Once the deployment configuration is complete, all services need to be started. In addition to Proxy-server, Account-server, Container-server, and Object-server, Swift also includes many other cyclical processes, and if you start, stop, and restart these services one at a time, will be tedious and boring, so it is recommended that you put all the commands that start, stop, or restart in the appropriate script so that you can simply execute the script, such as:

[email protected] ~]$ cat swift-stop.sh #! /bin/bashsystemctl Stop Openstack-swift-account.service Openstack-swift-account-auditor.service Openstack-swift-account-reaper.service Openstack-swift-account-replicator.servicesystemctl Stop Openstack-swift-container.service Openstack-swift-container-auditor.service Openstack-swift-container-replicator.service Openstack-swift-container-updater.servicesystemctl Stop Openstack-swift-object.service Openstack-swift-object-auditor.service Openstack-swift-object-replicator.service Openstack-swift-object-updater.service

Once all services have been started, it is possible to verify the ability to upload the downloaded files to Swift:

[Email protected] ~]$ source demo-openrc.sh [[email protected] ~]$ Swift stat        account:auth_ 1928446a6c364ace8a40043b5318bd9f     containers:0        objects:0          bytes:0x-put-timestamp:1434681204.28622     connection:keep-alive    x-timestamp:1434681204.28622     x-trans-id:tx60e7648bf53f4365a2c84-0055837f72   content-type:text/plain; Charset=utf-8

[[email protected] ~]$ Swift upload demo swift-stop.sh--object-name stopstop[[email protected] ~]$ Swift Listdemo[[email Protected] ~]$ Swift list Demostop

This completes the single-node deployment of Swift and, judging from the test results, everything is working. The successful deployment of Swift does not imply that all the functions and processes of swift are familiar, but there are many things to learn, such as the detailed use of Swift client commands, the specific process of uploading the downloaded file, the specific meaning of the ring file, how to adjust the number of partition, etc. Further study is needed.

OpenStack Swift Installation Notes