OpenStack's three "generals"

Swift, Glance, Cinder - The three storage-related components under OpenStack are just as familiar as the components themselves:

Swift - conceptually similar to the Amazon S3 service, provides object storage, but swift is very extensible, redundant and persistent, and it is also compatible with the S3 API

Glance - includes a lot of features similar to the Amazon AMI catalog, providing virtual machine image storage and management. (Glance's background data was initially stored in Swift).

Cinder - similar to Amazon's EBS block storage service, provides block storage (Block Storage) is currently only used for virtual machine mounts.

(Amazon has been the illusionist and challenger of the early days of OpenStack design, so basically all of the key functional modules have counterparts.) In addition to the three components mentioned above, Nova is the OpenStack for important EC2 services in AWS Correspondence, and maintain compatibility with the EC2 API, there are different ways to achieve)

Among the three components, Glance is primarily a management of virtual machine images, so it's relatively simple; Swift is already well-established as an object store and even CloudStack supports it. Cinder is a relatively new block storage, good design ideas, and the combination of commercial storage opportunities, so manufacturers are more positive.

Swift

There is also a lot of articles on the Swift architecture and deployment discussion, in addition to the official website, which is not repeated here (also refer to my previous PPT presentation at Shanghai OpenStack China). From the development point of view, the recent structural adjustment is not too much, so I would like to talk mainly about the more applicable areas of application is good.

From the actual case I know, there are four areas in which Swift appears, (there should be more, hopefully everyone can see the actual use cases)

Network disk

Swift symmetric distributed architecture and multi-proxy multi-node design led to it from the gene suitable for multi-user concurrent application model, the most typical application is similar to the Dropbox network disk application, Dropbox has exceeded 100 million users by the end of last year Number, for this size of access, good architecture design is able to support the root cause.

Swift's symmetric architecture allows data nodes to be logically at the same level, with data and associated metadata on each node. And the core data structure of the metadata uses a hash loop. Consistent hash algorithm only needs to relocate a small part of data in the ring space for the increase or decrease of nodes, which has better fault tolerance and scalability. In addition the data is stateless, and each piece of data is completely stored on disk. These points combine to ensure the good scalability of the storage itself.

In addition, the application and the combination of Swift is that the HTTP protocol language, which makes the application and store the interaction easier, without having to consider the details of the underlying infrastructure, the application software without any modifications can make the system as a whole Expand to a very large extent.

2.IaaS public cloud

Swift linear expansion in the design, high concurrency and multi-tenant support and other features, making it very suitable as an IaaS choice, the public cloud is larger, more encountered a large number of virtual machines concurrent start this situation, so for Virtual machine image background storage Specifically, the actual challenge is the concurrent read performance of big data (more than G), Swift in the beginning of OpenStack as a mirror library backstage storage, after RACKSpace deployment of thousands of machines under the scale Years of practice, Swift has proven to be a mature choice.

In addition, if based on IaaS to provide top-level SaaS services, multi-tenancy is an inevitable issue, Swift architecture design itself is to support multi-tenancy, so more convenient docking.

Backup the archive

RackSpace's main business is the data backup and archiving, so Swift is also a long-standing test in this area, but they also extend a new business - "hot archive." Because of the long-tailed effect, data may be recalled for longer and longer periods of time, and hot archiving is a big step forward for minutes of application archival data reacquisition, as compared to hours in traditional tape drive archiving scenarios .

4. Mobile Internet and CDN

Mobile Internet and mobile games generate a lot of user data, the amount of data is not large but a large number of users, which is Swift to deal with areas.

With CDN, if you use Swift, cloud storage can respond directly to mobile devices, do not need a dedicated server to respond to the HTTP request, nor in the data transfer through the file system on the mobile device directly HTTP Upload cloud agreement. If you cache data that is frequently accessed by the platform and use some optimization mechanism to distribute the data from different locations to your users, this will increase the speed of access and I recently saw Swift's developer community discussing video The combination of web applications and Swift, theft is the direction of concern.

Glance

Glance is relatively simple, is a virtual machine image storage. Provide mirror services to front-end nova (or other virtual management platforms with Glance-client installed), including storage, querying and retrieval. The module itself does not store large amounts of data, need to mount background storage (Swift, S3 ...) to store the actual image data.

Glance mainly includes the following sections:

l API service: glance-api is mainly used to accept a variety of api Nova request, the request into the RBMQ background processing.

l Glacne-registry Used to interact with the MySQL database to store or retrieve mirrored metadata. Note that as we mentioned earlier in Swift, Swift does not save metadata in its own Storage Server. Metadata here means save In the MySQL database some information about the mirror, this metadata belongs to Glance.

l Image store: The background storage interface, through which to obtain the image, the background mount default storage is Swift, but also supports other images such as Amazon S3.

Glance looks a bit like a virtual storage in some ways, but also provides an API for more complete image management. So in theory other cloud platforms can also use it.

Glance relatively simple, limited to the cloud, so nothing to discuss more, it is better to look at the new out of the block storage components Cinder, I think the basic view of Cinder overall design is good, there are many details and functions need to be improved The place, from a mature product still a little distance.

Cinder

OpenStack to F version of a relatively large change, one of which is part of Nova previously in the persistent block storage function (Nova-Volume) separated, independent of the new component Cinder. It integrates a variety of back-end storage, using the API interface to provide block storage services to the outside world, the main core is the volume management, allows the volume, volume type, volume snapshot processing.

Cinder contains the following three main components

API service: Cinder-api is the main service interface, which is responsible for accepting and processing external API requests and placing the request into the RabbitMQ queue for execution by the backend. Cinder currently provides Volume API V2

Scheduler service: Process task queue tasks, and according to the predetermined strategy to select the appropriate Volume Service node to perform the task. The current version of cinder only provides a Simple Scheduler, which selects an active node with the fewest volumes to create the volume.

Volume service: The service runs on the storage node, manages storage space, reads and writes requests for maintenance state of the cinder database from the tower, and interacts with other processes through the message queue and directly on the block storage device or software. Each storage node has a Volume Service, a number of these storage nodes can be combined to form a storage resource pool.

Cinder supports different types and models of storage by adding drivers from different vendors. At present, there are several EMC and IBM commercial storage devices that can be supported, as well as support for NAS storage through LVM support for local storage and NFS protocol. Therefore, NetApp's NAS should be no problem as if Huawei is working hard. Some time ago I was still in the blueprints Cinder IBM GPFS distributed file system, the future version should be added

So far, Cinder has mostly interacted with Openstack's Nova internally to provide the volume Attach required for a virtual machine instance, but in theory it is also possible to provide block storage separately to the outside world.

Deployment, the three services can be deployed in a server, you can also independently deploy to different physical nodes

Now Cinder is not mature enough, there are several obvious problems have not been solved, one is not enough commercial storage support, and does not support the FC SAN, another single hidden trouble did not solve the internal schedule scheduling algorithm is too simple. In addition, because it integrates a variety of storage and added a layer of management there is a solution, but certainly have an impact on efficiency, performance certainly loss, but this is no good thing.

Openstack through more than two years of development, becoming more and more huge. Currently there are three kinds of optical storage: object storage, mirror storage and block storage. This is also to meet more diverse needs, reflecting the flexibility and speed of open source projects. In general, choosing a storage system that takes into account multiple applications in the future should be considered a long-term decision. Openstack as an open system, the most important is to solve the problem of hardware and software vendors lock, you can always choose a new hardware vendor, the new hardware and existing hardware composition of the cluster, unified management, of course, you can also replace the software Technical service providers, do not have to move applications. This is the advantage of open source itself!

【Editor's Choice】

Experts talk about how to avoid the five pitfalls of cloud storage IT giants have bet OpenStack is expected to become the de facto standard cloud computing warfare server OEM bet OpenStack [Editor: Xiao Yun TEL: (010) 68476606]