< ext >openstack Ceilometer host monitoring module extension

The Openstack ceilometer monitoring Extension (http://eccp.csdb.cn/blog/?p=352) focuses on the expansion of virtual machine monitoring items, which is relatively straightforward. How to increase the monitoring of host, service, etc. on the basis of Ceilometer framework? This article takes host monitoring as an example and details the extension method.

First, the overall introduction of the Ceilometer capture monitoring data to the persistent storage process, on the compute node through the pollster polling mechanism to obtain the host's monitoring data, through publisher call RPC to send the monitoring data to the message queue, The collector end receives the data sent by the corresponding pipeline according to the agreed topic, and calls the storage interface for persistent storage, as shown in the process.

Secondly, in the framework of how to expand the new monitoring module, let its natural idea is to modify the above module, the focus is the configuration file settings, need to define the sending and receiving end of the topic, secrete and method to send the data to the collector side for storage. The following is a detailed description of how each module is modified.

first, compute data acquisition pollster Module

The module is mainly responsible for data collection, collecting host CPU, memory, network and other information, packaged into a custom data format and sent to the message queue via publisher, which is similar to the Ceilometer monitoring item extension.
1. Define the base class for the host to get data in ceilometer/ceilometer/compute/plugin.py.

class Serverpollster (plugin. Pollsterbase):    @abc. Abstractmethod    defget_samples(self, Manager, Cache, instances):

2, in the Ceilometer/ceilometer/compute under the new Server_pollsters package, under the package new server_cpu.py, server_mem.py and so on. Inheriting the Serverpollster class implements the Get_samples method, which encapsulates the obtained data into a defined data format, and can refer to the virtual Machine acquisition data schema.
3, with the expansion of monitoring items, in the ceilometer/setup.cfg to increase the configuration of monitoring items pollster, finally, after all the modifications are completed, reinstall Ceilometer.

ii. sending data publisher module

1, modify the Ceilometer/pipeline.yaml configuration file, configure the way to send and receive data.

-  name:server_pipeline  counters:          "Server.cpu"    "server.mem"  transformers :  Publishers:  -RPC://?target=record_server_data&meter_type=   Server

Where name is the name of pipeline, interval is the time interval for polling, counters is the pollster name of the pipeline; Transformers if the data sent is to be processed further before publisher, Transformers can be set to convert, publishers specifies the way to send data to collector, target specifies the receiver of the collector side, and Meter_type, topic, and method of RPC are obtained according to the.
At the same time, you need to set the ceilometer/ceilometer.conf file, set the server's topic, secret, method information, specifically configured as follows:

[PUBLISHER_RPC] server_topic=Server  server_secret=True  server_method=record_server_data 

2, due to the addition of a new topic send channel, so need to make corresponding changes to ceilometer/ceilometer/publisher/rpc.py. First, you need to modify the initialization function to read the pipeline and Ceilometer configuration file information. Second, modify the Publish_counters function, since the pipeline file will poll all counters every 10 minutes, ceilometer defaults to topic as metering, with the same data encapsulation format. However, due to the different formats of data encapsulation for different monitoring modules, separate processing is required, and only data with counter type sample is publish for 10 minute polling. Specific as follows:

if  Self. Meter_type = =' Metering ':  meters = [      Meter_message_from_counter_beta (        Counter         Self. topic_secret[ Self. Meter_type],        Source         forCounterinchCountersifType (counter) = = Sample.Sample      ]Else:  meters = [      Meter_message_from_counter_beta (        Counter         Self. topic_secret[ Self. Meter_type],        Source         forCounterinchCounters      ]topic = Self. meter_type

third, receive Data collector module

1. The ceilometer/ceilometer/collector/service.py file Initialize_service_hook defines the data that the worker uses to receive the RPC sent over, with the following configuration:

 for  in CFG. CONF. Publisher_rpc.iteritems ():  if k.endswith (' topic '): Self. conn.create_worker (               V,  rpc_dispatcher.      Rpcdispatcher([Self]),      ' Ceilometer.collector. ' + V,  )  

2. Define the corresponding target to receive the collected data as follows:

def Record_server_data (self, Context, data): For in     self.dispatchers:        ' server ')

The last parameter specifies the type of topic that is sent this time, and when the Dispatch.record_data method is called, it is used to invoke the corresponding method of storing the data, which in turn makes the data persisted to different database tables.
3, Dispatch also on the original basis to do the corresponding adjustment, the adjustment according to the current data received topic, call different methods, database storage, as follows:

 def record_data(self, context, data, Meter_type):  if  notIsinstance (data, list):    data = [Data]   forMeterinchData    ifSelf.secret_method[meter_type] and      Publisher_rpc.verify_signature (meter,        self.secret_method[meter_type][0]):      Try:        ifMeter.get (' timestamp '):          meter[' timestamp '] =            Self.time_to_date (meter[' timestamp '])          method = GetAttr (self.storage_conn.__class__,            self.secret_method[meter_type][1])          Method (Self.storage_conn, meter)      exceptException asErr:        Log.error (' Failed to record metering data:%s ', err)        Log.exception (ERR)      Else:        Log.warning (          ' message signature invalid, discarding message:%r ',          Meter

Iv. Persistence of storage modules

1, due to the new host data module monitoring, need to add a database table, so the storage module also need to make corresponding changes. First, you need to modify the ceilometer/ceilometer/storage/base.py file to add an interface:

@abc. Abstractmethod     defrecord_server_data(self, data):

2, in the Ceilometer/ceilometer/storage/impl_mongodb.py connection class implemented in the interface.
3, design the corresponding database table structure, set the model corresponding server class in ceilometer/ceilometer/storage/sqlalchemy/models.py.
4. Define the corresponding table in ceilometer/ceilometer/storage/sqlalchemy/migrate_repo/versions/001_add_meter_table.py.

When the above configuration is complete, the source isolation environment will reinstall the Ceilometer and restart the service to see the new server tables in the MongoDB database, and the table stores the data for the corresponding counter.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

< ext >openstack Ceilometer host monitoring module extension