OpenStack Object Storage--swift Open source cloud computing

OpenStack Object Storage (Swift) is one of the subprojects of the OpenStack Open source Cloud project, known as Object storage, which provides powerful extensibility, redundancy, and durability. This article will describe swift in terms of architecture, principles, and practices. Swift is not a file system or a real-time data storage system, which is called object storage and is used for long-term storage of permanent types of static data that can be retrieved, adjusted, and updated as necessary. Examples of data types that are best suited for storage are virtual machine mirroring, picture storage, mail storage, and archived backups. With no central unit or master node, Swift provides greater scalability, redundancy, and durability. Swift, formerly the Rackspace Cloud files project, contributed to OpenStack as part of the open source project as Rackspace joined the OpenStack community in July 2010. The latest version of Swift is OpenStack Essex 1.5.1.

The Sina SAE team has nearly one year of research and operational experience with Swift. After delving into Swift's architecture and principles, fully mastering the Swift source code, and after a period of testing and operation, we decided to launch the SAE storage service based on Swift. Currently, the development has been completed and started one months ago on the line, and performed very well. So let's share some of our research and work on Swift.

Swift Features

In the OpenStack website, there are more than 20 features of Swift, the most interesting of which are the following.

Extremely high data persistence

Some friends often confuse data persistence (durability) with system availability (availability), which is also understood as the reliability of data, which is the likelihood that data will be lost to one day after it is stored in the system. For example, Amazon S3 data persistence is 11 9, which means that if you store 10,000 (4 0) files into S3, 10 million (7 0) years later, you may lose 1 of them. So how many 9 SLAs can Swift offer? The answers are given below. For Swift's deployment in the Sina test environment, we have theoretically calculated that Swift has a data replication of 3 in 5 zone, 5x10 storage nodes, and a data persistence SLA that can reach 10 9.

Fully Symmetric system architecture

"Symmetry" means that all nodes in Swift can be completely equal, which can greatly reduce the cost of system maintenance.

Unlimited scalability

There are two aspects of extensibility here, one is the unlimited scalability of data storage capacity, and the second is that Swift performance (such as QPS, throughput, etc.) can be linearly upgraded. Because Swift is a completely symmetrical architecture, the capacity to expand simply by adding machines, the system will automatically complete the data migration and so on, so that each storage node to regain balance.

No single point of failure

The single point of storage has always been a problem in the context of large-scale applications of internet services. For example, a database, the general Ha method can only take the decision from, and "master" is generally only one; there are other open source storage system implementations, the storage of metadata information has always been a headache, generally only a single point of storage, and this single point is easy to become a bottleneck, and once the point is different, tend to affect the entire cluster, typically such as HDFs. Swift's metadata store is completely distributed evenly, and, like object file storage, metadata is stored in multiple copies. There is no single point in the entire swift cluster, and the architecture and design guarantees that no single point of business is effective.

Simple, reliable

Simple to embody in the structure of beautiful, neat code, easy to understand, no use of some advanced distributed storage theory, but very simple principle. Reliance means that swift, after testing and analysis, can confidently take swift to its core storage business without worrying about Swift's failure, as it can be quickly resolved by logging, reading code, regardless of any problems.

Application Scenarios

Swift offers the same services as Amazon S3, and applies to many scenarios. The most typical application is as a storage engine for disk-like products, such as the Amazon S3 behind Dropbox. OpenStack can also be combined with mirror service glance to store mirrored files for them. In addition, Swift's unlimited scalability is ideal for storing log files and data backup warehouses.

Swift Architecture Overview

Swift has three main components: Proxy server, Storage server, and consistency server. The architecture is shown in Figure 1, where the storage and consistency services are allowed on storage node. The AUTH certification service has now been stripped out of swift, using OpenStack certification services Keystone to achieve a unified OpenStack certification management between projects.

Figure 1 Swift Deployment architecture

Main components

Proxy Server

Proxy server is a server process that provides swift APIs and is responsible for the communication between the rest of the swift components. For each client request, it queries the location of account, container, or object in the ring and forwards the request accordingly. Proxy provides the Rest-full API and conforms to the standard HTTP protocol specification, which allows developers to quickly build custom client and swift interactions.

Storage Server

Storage Server provides storage services on disk devices. There are three types of storage servers in Swift: Account, container, and object. Where the container server handles the list of objects, the container server does not know where the object is stored, only what object is stored in the specified container. These object information is stored in the form of a SQLite database file. The container server also does some tracking statistics, such as the total number of object and the use of container.

Consistency Servers

storing data on disk and providing an external rest-ful API is not a difficult problem, and the main problem is fault handling. The purpose of Swift's consistency servers is to find and resolve errors caused by data corruption and hardware failure. There are mainly three server:auditor, updater and replicator. Auditor runs a continuous scan of the disk in the background of each swift server to detect the integrity of objects, container, and accounts. If a data corruption is found, auditor moves the file to the quarantine area, where the replicator is responsible for replacing the data with a healthy copy. Figure 2 shows the process flow diagram for the quarantined object. In the event of high system load or failure, the data in the container or account will not be updated immediately. If the update fails, the update is queued on the local file system, and Updaters continues to handle these failed updates, where account updater and container Updater are responsible for the update of both the both. The function of replicator is to handle the correct location of the data and to maintain a reasonable copy of the data, designed to keep the system consistent while the SWIFT server is facing temporary failures such as network outages or drive failures.

Ring

The ring is Swift's most important component for recording the mapping between storage objects and physical locations. When it comes to querying account, Container, object information, you need to query the ring information for the cluster. The ring uses zone, Device, partition, and replica to maintain these mapping information. Each partition in the ring has 3 replica in the cluster (default). The location of each partition is maintained by the ring and stored in the map. The ring file is created when the system is initialized, and then each time the storage node is added or subtracted, the project in the ring file needs to be rebalanced to ensure that the number of files that are migrated is the least.

Figure 2 Processing flow graph for isolated objects

Principle

Swift uses algorithms and storage theories that are not complex, with several concepts.

Consistent hashing algorithm

Swift uses the consistent hashing algorithm to build a redundant, scalable, distributed object storage cluster. The main purpose of Swift's use of a consistent hash is to change the node number of the cluster as little as possible to change the mapping relationship of existing key and node. The idea of the algorithm is divided into the following three steps. The hash value of each node is computed first and assigned to a 0~232 ring interval. Second, the same method is used to compute the hash value of the stored object and assign it to the ring. It then begins a clockwise lookup from the location where the data is mapped, saving the data to the first node found. If more than 232 still cannot find the node, it is saved to the first node. Assuming that there are 4 node in this ring hash space, and if a Node5 is added, and the NODE5 is mapped between Node3 and NODE4 according to the algorithm, the affected will only be the objects (which map to NODE5) that traverse the Node3 counterclockwise to the NODE4. The distribution is shown in Figure 3.

Figure 3 Consistent Hashi structure

Replica

If the data in the cluster has only one copy on the local node, a failure can result in a permanent loss of data. Therefore, redundant replicas are required to ensure data security. Swift introduces the concept of replica with a default value of 3, which is based mainly on the NWR strategy (also known as the Quorum protocol). NWR is a strategy for controlling conformance levels in a distributed storage system. In Amazon's Dynamo cloud storage system, NWR is used to control consistency. where n represents the number of copies of the replica of the same data, W is the number of copies needed to ensure a successful update of a data object; R represents the number of replica to read a data. The formula is w+r>n to ensure that a data is not read and written by two different transactions at the same time; the formula W>N/2 guarantee that two transactions cannot write a single data concurrently. In distributed systems, the single point of data is not allowed to exist. The fact that the number of replica that are normally present on the line is 1 is very dangerous, because once the replica again fails, a permanent error in the data can occur. If we set N to 2, then as long as there is a storage node corruption, there will be a single point of existence, so n must be greater than 2. The higher the N, the higher the maintenance cost and the overall cost of the system. Industry usually sets N to 3. For example, for MySQL master-slave structure, the NWR values are n= 2, W = 1, R = 1, and the NWR policy is not met. And Swift's n=3, w=2, r=2, fully compliant with the NWR strategy, so the SWIFT system is reliable and does not have a single point of failure.

Zone

If all of the node is in a rack or a computer room, then once the outage, network failure, etc., will cause users can not access. A mechanism is therefore needed to isolate the physical location of the machine to satisfy the partitioning tolerance (p in Cap theory). Therefore, the ring introduced the concept of zone, the cluster node allocated to each zone. The replica of the same partition cannot be placed simultaneously on the same node or in the same zone. Note that the size of the zone can be customized according to business requirements and hardware conditions, can be a disk, a storage server, or a rack or even an IDC.

Weight

The ring introduced weight to address more partition when it comes to adding storage-capable node in the future. For example, 2TB capacity node has a partition number of twice times 1TB, so you can set 2TB weight to 200 and 1TB 100.

Figure 41 Swift Deployment clusters

Example analysis

Figure 4 is the swift cluster that the Sina SAE deployed in the test environment, cluster is divided into 5 zone, each zone is a storage server, each server consists of 12 2TB SATA disks, only the operating system installation disk needs RAID, the other disk as storage node, do not need raid. As mentioned earlier, Swift employs a fully symmetric system architecture that is well represented in this deployment case. The role of each storage server in Figure 4 is fully equivalent and the system configuration is identical, with all the SWIFT service packages installed, such as proxy server, Container server, and account server. The above load balancing (load balancer) does not belong to Swift's software package and, for security and performance reasons, typically blocks a layer of load-balancing equipment before the business. Of course, you can remove this layer of proxies and let proxy server directly receive the user's request, but this may not be appropriate for use in a production environment. Figure 4 shows the data stream for the upload file put and download file get requests, with two requests operating on the same object. When uploading a file, the put request selects a proxy Server randomly through load balancing, forwarding the request to the latter, by querying the local ring file, selecting the backend of 3 different zone to store the file, and sending the file to the three storage nodes at the same time. This process needs to meet the NWR policy (Quorum Kyoto), that is, 3 storage, write a successful number of copies must be greater than 3/2, that is, must ensure that at least 2 data write successfully, and then return the file to write successful message. When downloading a file, get requests also randomly select a proxy Server through load Balancing, the ring file on the latter can query which three nodes the file is stored in, and then go to the back-end query, at least 2 storage nodes "means" to provide the file, and then proxy Server to select a node to download files from.

Summary

Swift's simple, redundant, extensible architecture design ensures that it can be used for the underlying services of IaaS. The two-year operation of the Rackspace Cloud files service has made swift code more sophisticated and currently deployed in public and private cloud services around the world. With the continuous improvement and development of OpenStack, Swift will be more widely used.