This blog post has been quietly in the draft box for more than half a year. If it is not asked for some reason or the loss caused by ignoring it, I do not know when to complete it. I am grateful for the setbacks I 've experienced during this period. I hope you can give me more determination!
This article attempts to describe in detail the complete process of creating a virtual machine in openstack, from the user's request to the successful running of the virtual machine, this includes sending of client requests, Keystone authentication, receiving of Nova-API requests, scheduling of Nova-scheduler, creation of Nova-computer, and distribution network of Nova-network. The functions and operations performed by each module during VM creation are described and discussed in detail.
For ease of description, this document assumes that all service IP addresses are localhost, and the Port uses the default port of the service. Openstack is version F. A rough flowchart showing how to create a virtual machine. Next, we will introduce each module in detail. If you have any errors, please click it!
Client
The first step to create a VM is to call the NOVA-API on the client to send a VM creation request. Currently, openstack provides two types of clients:
1. By specifying the command line parameters, you can call the NOVA-API to create a VM. The simplest command for creating a VM is as follows:
nova boot vm_name --flavor flavor_id --image image_uuid
2. webpage interaction page, horizon. This is a simple and easy-to-use web operation page that calls the NOVA-API to create virtual machines. After selecting relevant parameters, click Create.
These two clients have the same functions except the UI. To create virtual machines, they must be completed:
We can see that no matter whether it is Nova boot or horizon, the request is sent to the Nova-API through novaclient. Novaclient encapsulates the NOVA-API layer, such as obtaining the verification token-ID, constructing the HTTP request headers and body in a specific format, and parsing the request results. In fact, you do not need to use the Nova boot command line and horizon, or even the novaclient. We can set the headers and body of the HTTP request and directly request the NOVA-API. However, these three tools have done the tedious work for us, and we only need to fill in the parameters. Note that the Nova Command is actually an entry point of novaclient: novaclient. Shell. Main (), while Nova boot actually calls novaclient. v1_1.shell.do_boot ().
Keystone
It can be seen from the above that when the client initiates a VM creation request, Keystone needs to verify the user name and password of the client. Like the NOVA-API, keystone-all does not adopt any framework for API release. Instead, it uses the router and paste class libraries. The implementation style is slightly different from that of Nova-API. The keystone-all service listens to two ports: localhost: 5000, namely publicurl. Generally, users can access the port using a password. localhost: 35357, that is, adminurl, can only be accessed using the admin account and password. In the process of creating a virtual machine, there are two keystone APIs called (however, each client request basically calls these two APIs ):
1 http: // localhost: 5000/V2.0/tokens. The client requests this API to obtain the token_id and servicecatalog. Keystone will send the API request to Keystone. tokencontroller. Authenticate () for processing. This function is mainly completed:
- Verify the name, password, and tenant_id passed by the client. The user-related information, including the password-encrypted hash value, is saved in the keystone Database User table. Password encryption uses the sha512 algorithm, which is provided by the passlib library. Password verification: passlib. Hash. sha512_crypt.verify (password, hashed ).
- Generate a token_id for each request of the client according to the conf. Signing. token_format configuration item. Currently, two UUID and PKI are supported. The default value is UUID. Therefore, by default, a token_id is a random UUID. Token_id has a valid time starting from the time when token_id is generated. The validity period can be set through the conf. Token. Expiration configuration item, and the default value is 86400 S.
- To build servicecatalog, there are a bunch of endpoints, including Nova-API, glance-API, and cinder-API. According to the configuration item Conf. Catalog. Driver, you can obtain it from the database endpoint table or from the template file.
2 http: // localhost: 35357/V2.0/tokens/token_id, verify the token_id of the request. After receiving a request, the Nova-API first uses the token_id carried by the request to access the API to verify whether the request is valid, of course, the Nova-API needs to add admin account and password information to the body, which needs to be configured in the api-paste.ini. After receiving client requests, glance-API and cinder-API call this API to verify the token_id of the client. The request for this API is sent to Keystone. tokencontroller. validate_token () for processing. In fact, the request's token_id is used for a database query.
Nova-API
A Nova-API is a collection of services. As described in the Process Analysis of the openstack Nova-API service, by default, it contains EC2 (API compatible with EC2) and osapi_compute (API of openstack compute ), osapi_volume (openstack volume Service API), metadata (metadata Service API) and other API services. Each service provides different APIs. However, although EC2 and osapi_compute have different APIs, their functions are the same. Here, the API for creating a VM request is:
Http: // localhost: 8774/v2/project_id/servers, which is published by the osapi_compute service. This API is restful and requests this API using the POST method. After several layers of middleware processing, it is finally handed over to Nova. API. openstack. compute. servers. Controller. Create () for processing. They provide the following functions:
- Verify the token_id of the client. It is verified by Keystone. Middleware. auth_token.authprotocol. _ call _ (), which is a middleware that is configured through a api-paste.ini. As keystone mentioned above, it is implemented by requesting http: // localhost: 35357/V2.0/tokens/token_id.
- Check whether the VM creation parameters are valid and valid, which is implemented by Nova. API. openstack. compute. servers. Controller. Create. For example, check whether the Virtual Machine name complies with the naming rules, whether flavor_id exists in the database, and whether image_uuid is in the correct UUID format.
- Check whether the number of instances, vcpu, and Ram has exceeded the limit, and reserve required resources, which are managed by Nova. Quota. quotas. Resources owned by each project are limited, such as the number of virtual machines created, the number of vcpus, the memory size, and the number of volume. By default, all projects have the same number of resources, which are specified by quota_instances, quota_cores, quota_ram, quota_volumes, and other configuration items. Use the admin account to request http: // localhost: 8774/v2/admin_project/OS-quota-sets/project_id using the put method, and set a quota for a specific project.
- Check whether the length of metadata exceeds the limit and whether the number of inject files exceeds the limit. It is managed and detected by Nova. Quota. quotas. In general, all these parameters are null and can pass.
- Check whether the specified network and fixed IP are valid, for example, whether the network exists, whether the fixed IP belongs to the network, and whether the fixed IP is assigned. In general, this parameter is also null and is automatically allocated by the network.
- Check whether image, disk, and ramdisk exist and are available. This is a request to glance-API (http: // localhost: 9292/V1/images/image_id, obtain the returned data for judgment. Determine whether flavor meets the minimum image configuration requirements, such as memory and Virtual Disks.
- When all resources are sufficient and all passing parameters are valid, update the database, create an instance record, set vm_states to building, and set task_state to scheduling.
- Use RPC call to send all parameters to the Nova. scheduler. Manager. schedulermanager. run_instance () of the Nova-scheduler to run VM scheduling.
When the token_id verification is passed, the main task of the Nova-API is to check the resource quota and parameters, and create a database. If you use the dashboard, you will see that the VM is in the scheduling status on the page. However, this status lasts for a short time and is almost invisible unless there is a problem with the Nova-scheduler.
Nova-Scheduler
Unlike the external services provided by NOVA-API, each component of Nova calls each other using rabbitmq as the RPC call of the message queue. The implementation details of RPC are ignored here. You only need to know the service of the node on which the RCP call is called. The run_instance () of Nova-Scheduler only uses request_spec and filter_properties from the parameters received by the Nova-API. Other parameters are passed directly to the Nova-compute to create a VM. Request_spec contains the type, number, uuids, and other information of the virtual machine, which must be used as a reference during scheduling. Filter_properties contains the specified scheduling rule information, for example, force_hosts specifies to be scheduled to a specific node, and ignore_hosts does not schedule to certain nodes. After the NOVA-schedces receives a request from the Nova-API, Nova. scheduler. filter_scheduler.filterscheduler.scheduler_run_instances () performs the following tasks:
- Call Nova. scheduler. host_manager.hostmanager.get_all_host_states () to obtain all existing computing nodes and their information. The information includes the information in the compute_nodes table and the capabilities uploaded by each Nova-compute through the scheduled task _ publish_service_capabilitites, including the CPU, vcpu, memory, disk, and other sizes and usage.
- Use the specified filter to filter the nodes returned above and call Nova. schedters. Filters. hostfilterhander. get_filtered_hosts. These filters can be specified by the configuration item scheduler_default_filters, which can contain any implemented filters in Nova. schedters. filters. For example, ramfilter is used to filter hosts with insufficient memory to create virtual machines. computefilter is used to filter hosts with invalid services. You can also add a custom filter here and add it to the scheduler_default_filters configuration item. Note that the hosts specified by force_hosts and forces_nodes will be directly returned without filter.
- Use weighers to sort filtered hosts and call Nova. scheduler. weights. hostweighthander. get_weighed_objects () is implemented. In version F, only ramweigher is implemented and hosts is sorted by memory size. Later versions may think that weigher and weigherhander are used to sort the memory size. This is a little tricky. In June 2.2.4, we use a function Nova. scheduler. least_cost.weighted_sum () to sort the memory size. But in 2013.2, the original structure is returned.
- From the top n hosts, randomly select one for creating a VM. N can be set through the configuration item scheduler_host_subset_size. The default value is 1, that is, the optimal node is selected for creation. Update the resource information of the host stored by scheduler, such as memory, disk, and vcpu. The resource information used to schedule the next virtual machine is updated in a timely manner.
- Update the database to set the instance host = NULL, scheduled_at = now (), and then use RPC to call the NOVA-compute command of the host selected above to create a virtual machine.
Comparatively speaking, the logic is simple and the code is not difficult. However, during node-scheduler execution, the vm_state and tast_state of the virtual machine are not changed. So there will be no change in horizon.
Nova-compute
After Nova-schedute selects the host, it calls the host through RPC and calls the Nova. compute. Manager. computemanager. run_instance () of the Nova-compute service of the host (). It is used to create a virtual machine. However, before the introduction, we need to briefly mention the two scheduled tasks of Nova-compute:
- Update_available_resource. The task first obtains the number of CPUs, total memory size, and total disk size of the current computing node, and CALLS Nova. virt. libvirt... driver. lib1_driver. get_available_resource (). Then, find all the virtual machine information running on the node from the database, and calculate the number of vcpu, memory size, and disk size used by these virtual machines. Then, the total number of resources on the computing node is subtracted from the number of virtual machines used to obtain the free memory size and the free disk size. Then, update the database compute_node information. Note that the number of resources used by the Virtual Machine obtained from the database is not necessarily the number of resources consumed by the computing node. For example, 1) when the disk image of the virtual machine is in the qcow2 format, it must be smaller than or equal to the actual allocated size; 2) it is also possible that the data is not synchronized, making the statistical data inconsistent with the actual resource usage.
- _ Report_driver_status. This task is used to regularly update the capabilities of the current computing node and obtain resource data through lib1_driver. However, in terms of computing resources used, is the number of CPUs, memory, and disk usage returned by calling Multiprocessing, libvirt, and OS. And append the host_ip information. The scheduled task _ publish_service_capabilities is forwarded to Nova. scheduler. host_manager.hostmanager.service_states through rpc call.
These two scheduled tasks provide real-time host information for the NOVA-scheduler, so as to implement accurate scheduling. The following describes the main functions of run_instance:
- If the image size of the Instance exceeds the root size, an error is returned.
- Update the database, and set vm_state = building and task_state = NULL for the instance. The response on Horizon is very short.
- Allocate resources to virtual machines, including CPU, memory, and disks. Update the database and set the Host field of the instance.
- Assign the network. First, set the tast_state = networking of the instance, and then call the Nova. Network. Manager. NetworkManager. allocate_for_instance () of the Nova-network through rpc to return the network information. The detailed processing of the Nova-network is discussed in the following Nova-network module.
- Create a block device and set task_state = block_device_mapping. Because no block device is allocated, this step will not be performed and will be discussed later.
- Set task_state = spawning. If this type of instance is created on this computing node for the first time, the State will last for several minutes because the image needs to be downloaded.
- Generate XML based on the VM information configured above, write libvirt and XML files, and generate the console. log file.
- Download the image, such as kernel, ramdisk, and image, through the glance API. They are first placed in the _ base directory, and then copied to the instance directory. In this way, if the same virtual machine is created on the computing node, first check whether the _ base file has been downloaded. If the file has already been downloaded, copy it directly. For images, the qcow2 format is generally used as the backing_file of qemu-IMG to create an image, which saves space.
- Finally, use the XML generated above to call libvirt to create a VM.
,
Nova-networkNova-network is called by NOVA-compute before creating a virtual machine. network. manager. flatdhcpmanger. allocate_for_instance (), assign fixed IP and floating IP to the Virtual Machine (if auto_assign_floating_ip is true), and return network information. Here, the network adopts flatdhcp mode, and multihost = true. The Code shows that flatdhcpmanager inherits three classes: rpcallocatefixedip, floatingip, and NetworkManager. According to the python attribute access process (refer to the python object attribute access process), flatdhcpmanger is called. allocate_for_instance () First executes floatingip. allocate_for_instance (), and then it calls networkmanger. allocate_for_instance (). The main tasks are as follows:
- Obtain network information. You can specify network_uuid. Otherwise, all networks in the networks table will be obtained directly.
- Create a Virtual Interface for each network in the database and set the instance UUID and network ID to the corresponding value. Note that the Virtual Interface table is not the same as other tables. When you delete a virtual interface, the record is deleted instead of 1 ed = 1.
- Find a fixed IP whose network ID is equal to the network or null from the database and set instance UUID, host, allocated = true, virtual_interface_id. Update the conf file of dnsmasq (under the source Nova directory by default), send an HUP signal to dnsmasq, and re-read the configuration file. When the virtual machine dynamically obtains the IP address, dnsmasq reads the received MAC address, query the configuration file and return the IP address assigned to the virtual machine. Note that the dnsmasq startup parameter-DHCP-script =/usr/bin/NOVA-dhcpbridge is called when the VM requests and releases the IP address, the leased field used to set the fixed IP address.
- If auto_assign_floating_ip is true, a floating IP address is assigned to the VM. The allocated floating IP address does not belong to any project. The project_id and auto_assigned fields are set only after the allocation. Set the fixed_ip_id field to the above and assign it to the fixed IP address.
- Bind a floating IP address to the NIC of the outbound network, and set the NOVA-Network-prereouting, Nova-Network-output, and Nova-Network-float-SNAT tables of iptables Nat to perform SNAT and DNAT conversions. In this way, you can access the virtual machine from the outside through the floating IP address.
- Query the database by instance UUID and obtain information such as VIF, network, floating IP, and fixed IP. The database is constructed in the structure of Nova. Network. model. networkinfo and returned to Nova-compute.
