Nova-conductor and amqp (2)

Source code version: Version H

References:

Http://bingotree.cn /? P = 242

Http://blog.csdn.net/gaoxingnengjisuan/article/details/12231633

1. First, check the service startup script.

/Usr/bin/NOVA-Conductor

import sysfrom nova.cmd.conductor import mainif __name__ == "__main__":    sys.exit(main())

Nova/CMD/conductor. py

def main():    ...    server = service.Service.create(binary=‘nova-conductor‘,                                    topic=CONF.conductor.topic,                                    manager=CONF.conductor.manager)    service.serve(server, workers=CONF.conductor.workers)    service.wait()

 

Ii. Analysis of RPC service creation

Nova/service. py

Service Class: @ classmethoddef create (CLS, host = none, binary = none, topic = none, Manager = none, report_interval = none, periodic_enable = none, periodic_fuzzy_delay = none, periodic_interval_max = none, db_allowed = true ):... service_obj = Cls (host, binary, topic, Manager, report_interval = report_interval, periodic_enable = periodic_enable, priority = enabled, priority = enabled, db_allowed = db_allowed) return service_obj

 

Iii. Analysis Service Operation

Nova/service. py

Def serve (server, workers = none): Global _ launcher if _ Launcher: Raise runtimeerror (_ ('serve () can only be called once ')) "service is Nova. openstack. common. service "" _ launcher = service. launch (server, workers = workers)

Nova/openstack/common/service. py

def launch(service, workers=None):    if workers:        launcher = ProcessLauncher()        launcher.launch_service(service, workers=workers)    else:        launcher = ServiceLauncher()        launcher.launch_service(service)    return launcher

Refer to the Service Startup Process (http://www.cnblogs.com/littlebugfish/p/4022907.html) of the Nova-API, that is, starting the coroutine (using eventlet) to run the service, primarily the start function. Note: If you specify multiple workers in the configuration file, multiple processes will listen to the message queue, and the process that obtains the message is responsible for processing. Next, let's look at the code of the Start function, as shown below:

Nova/service. py

Service Class: def start (Self ):... self. manager. pre_start_hook () If self. backdoor_port is not none: Self. manager. backdoor_port = self. backdoor_port "creates amqp connections to facilitate the creation of consumer" "self. conn = RPC. create_connection (New = true) log. debug (_ ("creating consumer connection for service % s") % self. topic) rpc_dispatcher = self. manager. create_rpc_dispatcher (self. backdoor_port) self. conn. create_consumer (self. topic, rpc_dispatcher, fanout = false) node_topic = '% S. % s' % (self. topic, self. host) self. conn. create_consumer (node_topic, rpc_dispatcher, fanout = false) self. conn. create_consumer (self. topic, rpc_dispatcher, fanout = true) self. conn. consume_in_thread () self. manager. post_start_hook () log. debug (_ ("Join ServiceGroup membership for this service % s") % self. topic) self. servicegroup_api.join (self. host, self. topic, self) If self. periodic_enable: If self. periodic_fuzzy_delay: initial_delay = random. randint (0, self. periodic_fuzzy_delay) else: initial_delay = none self. TG. add_dynamic_timer (self. periodic_tasks, initial_delay = initial_delay, periodic_interval_max = self. periodic_interval_max)

Self. Conn. create_consumer (node_topic, rpc_dispatcher, fanout = false ). Self. conn mainly refers to Nova. openstack. common. RPC. impl_qpid.connection (here, the backend Implementation of amqp is qpid, which can be found in Nova. the create_consumer code called is as follows:

def create_consumer(self, topic, proxy, fanout=False):    proxy_cb = rpc_amqp.ProxyCallback(        self.conf, proxy,        rpc_amqp.get_connection_pool(self.conf, Connection))    self.proxy_callbacks.append(proxy_cb)    if fanout:        consumer = FanoutConsumer(self.conf, self.session, topic, proxy_cb)    else:        consumer = TopicConsumer(self.conf, self.session, topic, proxy_cb)    self._register_consumer(consumer)    return consumer

Based on the specific backend amqp implementation, this function will send the queue and exchange creation (the first time it will be created and will be reused later) to the amqp server and bind the request, in this way, the consumer and the specific queue can be bound and monitored. The self. Conn. consume_in_thread function is used to start the consumer thread. It uses evelent. Spawn to create a coroutine to keep running and waiting for the message. A new coroutine will be created to run the remotely called function when the message arrives. When a message arrives in the queue, call proxy_cb for processing, that is, the _ call _ function of the proxycallback object. The Code is as follows:

Nova/openstack/common/rpc/amqp. py

Proxycallback class: def _ call _ (self, message_data): If hasattr (local. store, 'context'): del local. store. context rpc_common. _ safe_log (log. debug, _ ('Received % s'), message_data) self. msg_id_cache.check_duplicate_message (message_data) ctxt = unpack_context (self. conf, message_data) "parse message" "method = message_data.get ('method') ARGs = message_data.get ('args', {}) version = message_data.get ('version ') namespace = message_data.get ('namespace ') if not method: log. warn (_ ('no Method for message: % s') % message_data) ctxt. reply (_ ('no Method for message: % s') % message_data, connection_pool = self. connection_pool) Return "process messages" self. pool. spawn_n (self. _ process_data, ctxt, version, method, namespace, argS) def _ process_data (self, ctxt, version, method, namespace, argS): ctxt. update_store () Try: rval = self. proxy. dispatch (ctxt, version, method, namespace, ** ARGs )...

We can see that the message processing is mainly through the new coroutine (using eventlet) to execute self. _ process_data function, and self. the _ process_data function mainly calls self. proxy. what is the content of the dispatch function?

3.1 first find self. Proxy

According to self. layer-by-layer tracing of the proxy, which is passed in when the above proxycallback is constructed, that is, rpc_dispatcher = self in the Start function. manager. create_rpc_dispatcher (self. backdoor_port), self. when creating the service, the manager has already been set as the conductormanager. The create_rpc_dispatcher Function Code of conductor manager is as follows:

def create_rpc_dispatcher(self, *args, **kwargs):    #self.compute_task_mgr = ComputeTaskManager()    kwargs[‘additional_apis‘] = [self.compute_task_mgr]    return super(ConductorManager, self).create_rpc_dispatcher(*args,            **kwargs)

Based on the class inheritance relationship:

Nova/manager. py

Manager class: def create_rpc_dispatcher (self, backdoor_port = none, additional_apis = none): APIs = [] If additional_apis: APIs. extend (additional_apis) base_rpc = baserpc. baserpcapi (self. SERVICE_NAME, backdoor_port) APIs. extend ([self, base_rpc]) serializer = objects_base.novaobjectserializer () return rpc_dispatcher.rpcdispatcher (APIS, serializer)

Nova/openstack/common/rpc/dispatcher. py

Rpcdispatcher class def _ init _ (self, callbacks, serializer = none): # Callbacks is a list that contains computetaskmanager, conductormanager, and baserpcapi self. callbacks = callbacks if serializer is none: serializer = rpc_serializer.noopserializer () self. serializer = serializer super (rpcdispatcher, self ). _ init __()

Therefore, the final self. Proxy is the rpcdispatcher object. Note that the callbacks attribute of the rpcdispatcher object contains a series of manager classes, which are used to process the method specified in the message. Refer:

 

3.2 analyze the self. Proxy. Dispatch Function

That is, the dispatch function of the rpcdispatcher class. The Code is as follows:

def dispatch(self, ctxt, version, method, namespace, **kwargs):    if not version:        version = ‘1.0‘    had_compatible = False    for proxyobj in self.callbacks:        try:            cb_namespace = proxyobj.RPC_API_NAMESPACE        except AttributeError:            cb_namespace = None        if namespace != cb_namespace:            continue           ...                 if not hasattr(proxyobj, method):            continue        if is_compatible:            kwargs = self._deserialize_args(ctxt, kwargs)            result = getattr(proxyobj, method)(ctxt, **kwargs)            return self.serializer.serialize_entity(ctxt, result)    ...

This function matches the manager according to the namespace, and then analyzes whether the functions in the Manager class match the functions in the message. If they match, the corresponding functions in the Manager class are called for processing. After observation, the rpcapi of components that provide RPC services externally. py and manager. the classes in The py file are matched, and they are matched through the namespace. The names have the same prefix, so if you want to track rpcapi. for subsequent implementation of functions in Py, you only need to view the manager of this component. the corresponding function of the manager class in The py file.

Nova-conductor and amqp (2)