Nova Network-vif-plugged Event Analysis 1

In the process of creating a virtual machine, Nova-compute calls the Wait_for_instance_event function (nova/compute/ manage.py) for Network-vif-plugged event waits, this article attempts to describe the occurrence of this event, as well as the impact after the timeout. In the first section, two simple experiments were carried out to illustrate the role of the Network-vif-plugged event and the process of initiating a piece to event processing.

There are two parameters associated with this event in the Nova-compute configuration file-Vif_plugging_timeout, Vif_plugging_if_fatal, which is the maximum time to wait for an event, which is the way to handle a time-out exception. If in the specified time, the Nova-compute received the incident response, then the virtual machine can be created normally, then when the time-out phenomenon occurs, Nova-compute will be based on the configuration of Vif_plugging_is_fatal to take two processing methods.

If the timeout occurs and Vif_plugging_is_fatal is true, Nova executes Guest.poweroff first, stops the QEMU process, then executes the cleanup function, clears the network resource, uses the function _unplug_ VIFs, removing the OvS Port,ovs agent created by the PLUG_VIFS function detects an event that deletes the port and then notifies Neutron-server to delete the port information in neutron DB, Finally throw virtualinterfacecreateexception.

if Vif_plugging_is_fatal is false, the creation process continues even if a eventlet.timeout.Timeout exception occurs , which means that network-vif-plugged does not affect the creation of the virtual machine. So what is the role of this event? In the Nova-compute code, the Network-vif-plugged event is not the neutron-server of the neutron-client notification, so how does Neutron-server receive the event request and handle the event?

In Neutron-server's NeutronDbPluginV2 class (neutron/db/db_base_plugin_v2.py) initialization function, three types of events and their triggering actions are defined for the port:

Event.listen (MODELS_V2. Port, ' After_insert ', self.nova_notifier.send_port_status)

Event.listen (MODELS_V2. Port, ' after_update ', self.nova_notifier.send_port_status)

Event.listen (MODELS_V2. Port.status, ' Set ', self.nova_notifier.record_port_status_changed)

The record_port_status_changed function is used to define the port's _notify_event property, which records the completion state of the event. The "set" operation is monitored, indicating that when the state of the port is changed, the Network-vif-plugged event is triggered and the corresponding function is called to notify Nova-compute. So nova-compute to the Network-vif-plugged event is how to pass to Neutron-server, we experiment on the line to verify.

Experiment 1 set Vif_plugging_is_fatal to False, turn off the compute node's neutron-openvswitch-agent service, start the virtual machine, the virtual machine can start normally, but the virtual machine corresponding port status is down.

The virtual also cannot get an IP address.

The port status immediately becomes active when the ovs-agent is started, and the virtual machine can get the IP.

Experiments have shown that when Ovs-agent is turned off, the operation of Nova adding OvS Port cannot be captured, neutron does not update the port state of the VM, and therefore causes the Network-vif-plugged event to time out. That is to say, network-vif-plugged event to trigger, must be notified by Ovs-agent Neutron-server.

Under normal circumstances, how does this process happen, we observe through the following experiment.

Experiment 2 Under normal circumstances, the occurrence and processing of network-vif-plugged when the virtual machine is created.

(1) The Nova-compute of the compute node waits for commands such as Ovs-vsctl, and then waits for feedback from the Network-vif-plugged event.

2016-08-16 12:26:41.650 1231 Debugoslo_concurrency.processutils [ req-8f7c1a9d-7647-4207-824b-ef281a47d14850a809139b434c2287b4f82474242533 630b741632ba47b596e21eaf50f2b568---] Running cmd (subprocess): sudo nova-rootwrap/etc/nova/rootwrap.conf ovs-vsctl--timeout=120----if-exists del-port Qvo059c0374-b8--Add-port br-intqvo059c0374-b8--Set Interface qvo059c0374-b8external-ids:iface-id= 059c0374-b801-4ab3-b652-4b33e405eb15external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:3f : 68:6EEXTERNAL-IDS:VM-UUID=BC7D4A87-4DF4-413B-9364-39E7780F64A1 Execute/usr/lib/python2.7/site-packages/oslo_ concurrency/processutils.py:344

(2) The ovs-agent of the compute node detects that a new port is added.

12:26:41.776 23624 DEBUGneutron.agent.linux.async_process [-] Output received from [Ovsdb-clientmonitor Interface name, Ofport,external_ids--format=json]:{"Data": [["CF5AD2E5-5ADC-463A-845D-DCB156CE1BDC", "Insert", "Qvo059c0374-b8", [ "Set", []],["Map", [["Attached-mac", "fa:16:3e:3f:68:6e"],["Iface-id", "059c0374-b801-4ab3-b652-4b33e405eb15"],[" Iface-status "," active "],[" Vm-uuid "," BC7D4A87-4DF4-413B-9364-39E7780F64A1 "]]]]," headings ": [" Row "," Action "," Name "," Ofport "," External_ids "]}_read_stdout/usr/lib/python2.7/site-packages/neutron/agent/linux/async_ process.py:236

Then notify Neutron-server.

2016-08-16 12:26:42.161 23624 debugoslo_messaging._drivers.amqpdriver [req-e02bb72d-a5f4-4970-8e3e-961b189a72db--- --] Call msg_id:4f9c9facb1fa4969889783a986e2c561 Exchange ' neutron ' topic ' q-plugin ' _send/usr/lib/python2.7/ site-packages/oslo_messaging/_drivers/amqpdriver.py:454

(3) Neutron-server received the information.

2016-08-16 12:26:42.164 25184 Debugoslo_messaging._drivers.amqpdriver [-] Received message msg_id : 4f9c9facb1fa4969889783a986e2c561 reply toreply_42e9b47bc89d494bac7cd08e1590b22b __call__/usr/lib/python2.7/ site-packages/oslo_messaging/_drivers/amqpdriver.py:201

2016-08-16 12:26:42.165 25184 DEBUG neutron.plugins.ml2.rpc[req-e02bb72d-a5f4-4970-8e3e-961b189a72db-----] DEVICE059C0374-B801-4AB3-B652-4B33E405EB15 details requested by Agentovs-agent-gansu.saltmaster with host Gansu.saltmaster get_device_details/usr/lib/python2.7/site-packages/neutron/plugins/ml2/rpc.py:70

2016-08-16 12:26:42.226 25184 DEBUG neutron.plugins.ml2.db[req-e02bb72d-a5f4-4970-8e3e-961b189a72db-----] for port05 9C0374-B801-4AB3-B652-4B33E405EB15, host Gansu.saltmaster, got binding levels[< Neutron.plugins.ml2.models.portbindinglevel[object at 5ebd1d0]{port_id=u ' 059c0374-b801-4ab3-b652-4b33e405eb15 ', Host=u ' Gansu.saltmaster ', level=0, Driver=u ' Openvswitch ', segment_id=u ' 7c1db2be-4425-4f92-9913-d0169e4afc7b '}> ] get_binding_levels/usr/lib/python2.7/site-packages/neutron/plugins/ml2/db.py:184

2016-08-16 12:26:42.249 25184 DEBUG neutron.notifiers.nova[req-e02bb72d-a5f4-4970-8e3e-961b189a72db-----] Ignoring s Tate Changeprevious_port_status:down Current_port_status:build port_id059c0374-b801-4ab3-b652-4b33e405eb15 Record_ port_status_changed/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:194

(4) Neutron-server the status of the port from down to Actice and responds to the Nova event.

2016-08-16 12:26:44.634 25188 DEBUG neutron.notifiers.nova[-] Sending events: [{' Status ': ' Completed ', ' tag ': U ' 059c0374-b801-4ab3-b652-4b33e405eb15 ', ' name ': ' network-vif-plugged ', ' Server_uuid ': U ' BC7D4A87-4DF4-413B-9364-39E7780F64A1 '}] send_events/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py : 209

(5) in the Nova-api log, you can find

2016-08-16 12:26:44.822 31556 DEBUG nova.api.openstack.wsgi[req-efb87df1-0901-435f-8ba4-d7461794ef56 1333f6ff00fa428c984b3ba0a99f08440de50e7e0c7c4f148300936580762205---] Action: ' Create ', calling Method:<bound Method Serverexternaleventscontroller.create of<nova.api.openstack.compute.server_external_events. Serverexternaleventscontrollerobject at 0x7afd0d0>>, body: {"Events": [{"Status": "Completed", "tag": " 059c0374-b801-4ab3-b652-4b33e405eb15 "," name ":" Network-vif-plugged "," Server_uuid ":" BC7D4A87-4DF4-413B-9364-39E7780F64A1 "}]} _process_stack/usr/lib/python2.7/site-packages/nova/api/openstack/ wsgi.py:696

Note: Neutron-server called the Create feature of Nova-client server_external_events at notify, sending a create request to NOVA-API. (roughly so, the details forget J)

2016-08-16 12:26:44.879 31556 INFOnova.api.openstack.compute.server_external_events [ req-efb87df1-0901-435f-8ba4-d7461794ef561333f6ff00fa428c984b3ba0a99f0844 0de50e7e0c7c4f148300936580762205---] Creating Event Network-vif-plugged:059c0374-b801-4ab3-b652-4b33e405eb15 Forinstance Bc7d4a87-4df4-413b-9364-39e7780f64a1

(6) Nova-compute received the reply of the event

2016-08-16 12:26:45.050 1231 DEBUG nova.compute.manager[req-efb87df1-0901-435f-8ba4-d7461794ef56 1333f6ff00fa428c984b3ba0a99f08440de50e7e0c7c4f148300936580762205---] [instance: BC7D4A87-4DF4-413B-9364-39E7780F64A1] Received eventnetwork-vif-plugged-059c0374-b801-4ab3-b652-4b33e405eb15external_instance_event/usr/lib/python2.7/ site-packages/nova/compute/manager.py:6707

(7) Neutron-server get Nova-api's reply

12:26:45.056 25188 INFO Neutron.notifiers.nova [-] Nova Eventresponse: {u ' status ': U ' completed ', U ' tag ': U ' 059c0374-b801-4ab3-b652-4b33e405eb15 ', U ' name ': U ' network-vif-plugged ', U ' server_uuid ': U ' BC7D4A87-4DF4-413B-9364-39E7780F64A1 ', U ' Code ': 200}

Summarize:

In this paper, the significance of network-vif-plugged event and the normal processing flow are illustrated by experiments, and it is hoped to provide reference for investigating the Network-vif-plugged event timeout and then the virtual machine creation.

Next work:

Nova-api received after the incident is how to inform Nova-compute, has not been studied clearly, personal guess is through RABBITMQ, still need to be verified.

Nova Network-vif-plugged Event Analysis 1