The architecture design of depth analysis Cloudfoundry

VMware suddenly released its first open source Paas--cloudfoundry this April. In the months since its release, the author has been concerned about its evolution and benefited from its architectural design, and felt the need to write to share it with you.

This article will be divided into two parts: the first part mainly introduces the architecture design of Cloudfoundry, from the module introduced by it, to the various parts of the message flow, how each module coordinated cooperation; The second part will be based on the first part, Use the architecture knowledge introduced in the first section to target how to deploy a private PAAs in your datacenter with Cloudfoundry.

The first part of the story will cite Pat's speech on the Cloudfoundry architecture in Vmwarecloud Forum on October 12. Pat was in charge of Cloudfoundry core, and his speech was well worth listening to. If you are present and understand what he is saying, this part can choose to skip directly. I would not be able to speak better than that, except that I would be specific about what I said.

I. Architecture and Modules

Overall, the Cloudfoundry architecture is as follows:

This architecture diagram, as well as the module architecture diagrams used below, are from the Pat's PPT. From the above, we can see that cloudfoundry mainly consists of the following major components:

1, Router: As the name suggests, Router component in Cloudfoundry is to all incoming request route. There are two main types of request to enter the router: first, from Vmcclient or STS, by the Cloudfoundry user, the management directives.

For example: List all your apps Vmcapps, submit a apps, and so on. This type of request is routed to the Applife management component, called the Cloudcontroller component, and the second is the request for an outside visit to your deployed apps. This part of the requests is routed to appexecution, or components called Deas. All the requests entering the Cloudfoundry system will pass through the router component, seeing that there may be friends who worry that router becomes a single point, thus becoming the bottleneck of the entire cloud.

But Cloudfoundry as a cloud system, the core of its design is to go to a single point of dependency, component parallel expansion, and can be replaced to ensure extensibility, this is cloudfoundry, and even all cloud computing system design principles, the later article will discuss how cloudfoundry do this, At present, as long as you know, the system can deploy a number of routers together to deal with the requests, but router upper loadbalance is not cloudfoundry implementation scope, Cloudfoundry only ensure that all request is stateless , so that the upper balanced load selection surface is very very large, for example, can be done through DNS, can also deploy hardware loadbalancer, or simple point, to get a ngnix as a loading balancer, are feasible.

Router component, the current version is a simple package for nginx. A friend familiar with Ngnix should know that it can be a socket file (. sock file) as input output. All router component servers that install Cloudfoundry install a nginx with the following configuration for ngnix.conf files:

Overall, the router component is structured as follows:

2, the DEA (Droplet Execution Agency): First of all to resolve what is called Droplet. Droplet in Cloudfoundry's concept refers to the source code that you submit, as well as the cloudfoundry of the running environment, plus some management scripts, such as start/stop these small scripts are all compressed together in the TAR package. There is also a concept called Stagingapp, which is the process of making the package described above and storing it well. Cloudfoundry will automatically save this droplet until you start an app, a server that deploys the DEA module will come and run it with a droplet copy. So if you extend your app to 10 instances, the droplet will be copied 10 copies and 10 DEA servers run.

The following figure is an architectural diagram of the DEA module:

The Cloud controller module (described below) sends basic apps management requests, such as Start/stop, to DEA,DEA.RB to receive these requests, and then find the right droplet from NFS. Said droplet is actually a running script, with the running environment of the TAR package, the DEA only need to take it to unpack, and the start script inside, you can let this app run. So, app is accessible and started, in other words, one of the ports on this server is on standby, and the app can receive and return the correct information as long as a request comes in from the port.

Then dea.rb to do some clean-up work: 1, the information to tell router module. As we said earlier, all the requests into the cloudfoundry are processed and forwarded by the router module, including user access to the app request, after an app is up, need to tell router, let it according to LoadBalance principles, Turn the appropriate request in, so that the app's instance can do the job; 2, some statistical work, for example, to the user and new deployment of a app told Cloudcontroller for quota control, etc. 3, The operation information is told to the Healthmanager module and the instance operation of the app is reported in real time. In addition, the DEA is responsible for some of the droplet query work, for example, if the user through the Cloudcontroller to query the log information of the app, the DEA needs to take the log back from the droplet and so on.

3, Cloudcontroller:cloudcontroller is the Cloudfoundry management module. The main tasks include:

(a) to read the additions and deletions of apps;

(b) Start and stop the application;

c) Staging apps (pack apps into a droplet);

D Modify the operating environment of the application, including instance, MEM, etc.

E Manage service, including the binding of service and app;

(f) cloud environmental management;

(g) Modification of cloud user information;

h) View Cloud Foundry, and log information for each app.

This may seem a bit complicated, but simply, it can be simple: the server side that interacts with VMC and Sts. VMC and STS Use the RESTful interface for Cloudfoundry Communications, on the other hand cloudcontroller is a typical Rubyon rails project, from VMC or STS to the JSON-formatted protocol, Then write Cloudcontroller Database, concurrent messages to each module to control the entire cloud. As with other ROR projects, all APIs for Cloudcontroller can be seen from the conf/routes.rb. The benefits of an open RESTful interface are THIRD-PARTY application development and integration, where organizations can automate the management of the entire cloud environment by deploying private clouds with Cloudfoundry. This part will be discussed in the second part.

The following figure is the Cloud controller architecture diagram:

The Tiyatien Manager and DEA are external modules, Ccdatabase is the Cloudcontroller Database, this is where the entire cloudfoundry can not do hp. Cloudcontroller database does not have a lot of concurrency, application-level access is handled by the service module below, the database is cloud configuration information. The reading operation is mainly from the DEA startup, as the basis for initializing the DEA, and the Healthmanager module will read the expected state information from here, and this part of the data will be compared with the actual state information obtained from the DEA.

NFS is a shared storage of multiple Cloudcontroller, Cloudcontroller one of the important tasks is Stagingapps. Droplets storage is unique in a clustered environment. While Cloudcontroller is a cluster run, in other words, every control request may be handled by different cloudcontroller, assuming a simple user scenario: we need to deploy an app to the Cloudfoundry. After we hit the simple push command, VMC began to work, after a round of user authentication, to see whether the number of apps deployed exceeded the predetermined amount, asked a bunch of related app questions, you need to send 4 instructions:

1. Send a post to "apps" to create an app;

2. Send a put to "apps/:name/application", upload app;

3. Send a get to "apps/:name/", get app status and see if it's started;

4. If it is not started, send a put to "apps/:name/" to make it start.

If steps 2nd and 4th are handled by different cloud controller, and there is no guarantee that they will find the same droplet, the 4th step will fail because the corresponding droplet cannot be found. How do you ensure that this sequence of instructions comes to the same droplet? With NFS, making Cloudcontroller shared storage is the easiest way. But this approach is not perfect in terms of security. On the October 12 Vmwarecloud forum, Pat told us that there would be big tweaks to the next version of Cloudfoundry, but before that part of the code was open, it was inconvenient for me to comment too much.

4, Healthmanager: Do things are not complex, simply said that from the various DEA to get the running information, and then statistical analysis, reports and so on. The statistics are compared with the cloudcontroller set of indicators and provide alert. Healthmanager module is not very perfect, but the cloudmanage stack, automation Tiyatien management, analysis is a very important area, and this can be expanded in many places, combined with orchestrationengine can make the cloud from management, Since the early warning, but with the BI technology, can be statistical operations, reasonable allocation of resources. Cloudfoundry is still developing in this regard.

5, Services:cloud Foundry Service module from the source code control to see that is an independent, plugin module, to facilitate the third party to integrate their services into the cloudfoundry ecosystem. The service was seen on the GitHub as an independent repository with the Cloudfoundry core project Vcap, Vcap-service. Service Module The design principle is to facilitate third-party service providers. In this regard Cloudfoundry did a very successful, from the GitHub view, the following services have been provided: a) MongoDB; b) MySQL; c) neo4j; d) PostgreSql; e) rabbitmq; f) Redis; g) Vblob. Base classes are placed in the base folder.

If a third party needs to develop its own cloudfoundry services, it needs to inherit the two underlying classes within it: node and gateway, while some operations, such as: provision, You can add your own logic to base provisioner.rb, as well as Service_error and Service_message. about how to write their own SERVICE,ELC blog will be introduced in detail, not in the scope of this article, from the framework to understand, as long as the relationship between services, know that a service and base through the inheritance relationship between the horizontal expansion, The Cloudfoundry and Apps invocation service is done with base to complete this simple architectural approach.

6, NATS (Message bus): From the cloudfoundry of the general architecture diagram, located in the center of each module is a component called NATS. Nats is a lightweight, cloudfoundry messaging system that supports the release and subscription mechanism developed by Derek, the architect of the architecture division. GitHub Open Source address is: Https://github.com/derekcollison/nats. Its core based on Eventmachine development, the code is not much, you can download and slowly study.

Cloudfoundry is a multi module distributed system, which supports the discovery of modules, error self-test, and low coupling between modules. The core principle is to publish the subscription mechanism based on the message. Each module on each server publishes multiple message topics to the Messagebus based on its own message category, while also subscribing to the message subject of the desired information content to the module in which it needs to interact. For example: A DEA is added to the Cloudfoundry cluster, it needs to yell at everyone to show that it's ready to serve, and it publishes a message with the theme "Dea.start":

@ Hello_message_json includes DEA Uuid,ip, port, version information, and so on.

For example, Cloudcontroller needs to start a droplet instance:

A the first time a DEA starts, it will first subscribe to the message subject of its UUID.

Other modules need to send a message through the ' Dea.#{uuid}.start ' topic to enable it to start, and once the DEA receives the message, it triggers the Process_dea_start (msg) method to handle the work required to start.

b Cloud controller or other modules to send a message, so that the UUID for XXX to start the DEA.

C when the DEA module receives the message, it triggers the Process_dea_start (msg) method. MSG is the message content sent by other modules, including: Droplet_id,instance_index, service, runtime and so on, Process_dea_start will get the necessary information to start the DEA, and then do a series of operations, For example, get droplet from NFS, unzip, modify the necessary environment configuration, run startup scripts, and so on. When everything is ready, and then need to send a message to router, tell it that the droplet is ready to serve the country, and later have the corresponding request remember to let it deal with.

d the Router module has subscribed to the "Router.register" message topic at startup.

After receiving the information from the previous DEA, the Register_droplet method is triggered to bind the droplet. To start a droplet instance work done.

We can see the whole cloudfoundry core is a set of message system, if want to understand the ins and outs of Cloudfoundry, to track its complex message mechanism is a very good way. On the other hand, Cloudfoundry is a distributed system based on message, and the message-oriented architecture is the basis of its node scaling, component self discovery and other cloud characteristics.

Cloud Foundry The framework of a brief introduction to this point, in fact, as the first open source Paas,cloudfoundry architecture has a lot to learn from the place, a lot of details of the processing is very subtle, these content if it is possible to continue in the follow-up article discussion, Although this topic is deep cloudfoundry, but also just to taste the end, the overall framework to introduce, the goal is to enable us to have enough background to use Cloudfoundry to build the private PAAs within the enterprise. To sum up, I learned from the structure of Cloudfoundry:

1, the message-based multi-component architecture is a simple and effective way to implement the cluster. The message can decouple the cluster nodes and enable them to be registered, and discover these functions which are very important in the large-scale data center.

2, the appropriate abstraction layer, the use of template mode, to facilitate the third party can facilitate the development of Cloudfoundry expansion function. Cloudfoundry in the DEA and service layer have done the abstraction layer processing, relatively should make the developer can easily develop runtime and service for Cloudfoundry. For example, when Cloudfoundry just launched, only support Node.js,java, Ruby, but the Third-party provider, the open source community to follow up quickly, for Cloudfoundry added Php,python support. Thanks to Cloudfoundry's ingenious DEA architecture, how to develop new runtime support will be discussed in subsequent posts.

Second, the source guide

The author has always felt that the best way to understand a technology is to read its source code, and Cloudfoundry is a completely open source of the PAAs platform, and because the newly developed, the code is not much, the main authors of the code is very good, read it very comfortable, very suitable for reading. and have to praise it again. It is entirely based on the message mechanism of the architecture design, the component extensibility, Third-party access and so do a good job, the reader can learn a lot of ideological things. I highly recommend that you read its source code. You can find all the Cloudfoundry code on GitHub: Https://github.com/cloudfoundry, you will see several different repositories, respectively:

1, Vcap:cloud Foundry Core, or called kernel;

2, Vcap-service:cloud Foundry service components. Cloud Foundry Service is provided as a plug-in, which is designed for its convenience to third party development service;

3. Vmc:vmware Cloud CLI. is a ruby application that interacts with the CLI of the Cloud Foundry. Mainly by analyzing the user input CLI, send restful request to cloudfoundry;

4. Vcap-java: If your app is developed in Java and needs to interact with cloud foundry, such as getting the current Serviceserver IP address, you may need this jar, which supports our common Java development Framework, It is also the bottom of the cloudfoundry of the RESTful request packaging;

5, Vcap-java-client:cloud foundry RESTful API Java encapsulation, unlike the project above, it is just a simple read cloudfoundry information, and put such as JavaBean;

6, Vcap-test:cloud foundry test cases;

7, Vcap-test-assets:cloud foundry some apps examples.