How to Use the ansible callback plug-in to parse the execution results, ansiblecallback

How to Use the ansible callback plug-in to parse the execution results, ansiblecallback

Recently I was writing a Batch Inspection tool to push the script to each machine using ansible for execution, and then return the execution result in json format.

 

As follows:

# Ansible node2-m script-a/root/python/health_check.py

node2 | SUCCESS => {    "changed": true,     "rc": 0,     "stderr": "Shared connection to 192.168.244.20 closed.\r\n",     "stdout": "{'cpu_iowait': '0.00', 'swap_out': 0, 'cpu_usr': '0.00', 'cpu_idle': '100.00', 'swap_total': '1999', 'swap_used': '78', 'load_average_5': '0.11', 'mem_util': '92.0', 'uptime': '5', 'load_average_1': '0.03', 'cpu_sys': '0.00', 'mem_total': '475', 'swap_in': 0, 'load_average_15': '0.06', 'disk': ['Filesystem      Size  Used Avail Use% Mounted on\\n', '/dev/sda3        18G  8.6G  8.1G  52% /\\n', 'tmpfs           238M     0  238M   0% /dev/shm\\n', '/dev/sda1       190M   27M  154M  15% /boot\\n'], 'numa': '1'}\r\n",     
    "stdout_lines": [        "{'cpu_iowait': '0.00', 'swap_out': 0, 'cpu_usr': '0.00', 'cpu_idle': '100.00', 'swap_total': '1999', 'swap_used': '78', 'load_average_5': '0.11', 'mem_util': '92.0', 'uptime': '5', 'load_average_1': '0.03', 'cpu_sys': '0.00', 'mem_total': '475', 'swap_in': 0, 'load_average_15': '0.06', 'disk': ['Filesystem      Size  Used Avail Use% Mounted on\\n', '/dev/sda3        18G  8.6G  8.1G  52% /\\n', 'tmpfs           238M     0  238M   0% /dev/shm\\n', '/dev/sda1       190M   27M  154M  15% /boot\\n'], 'numa': '1'}"    ]}

 

Then, redirect the result to a text file and use another script to parse and summarize the text file. The final result is as follows:

ip              uptime          cpu_usr         cpu_sys         cpu_iowait      cpu_idle        load_average_1  load_average_5  ...           192.168.244.30  24              0               0               6               94              0.02            0.08            ...              192.168.244.20  24              0               0               0               100             0               0.01            ...           

 

But I always feel that this method is a bit low. It seems to be a common requirement to parse the returned results?

 

It doesn't make sense. The official team will turn a blind eye to this kind of requirement. In fact, the official team provides a callback plug-in to implement the callback function, which defines a number of scenarios, such as inaccessible hosts and failed task execution, the execution of the task is successful, which corresponds to different methods, so that different operations can be triggered in different scenarios. For example, if the execution of the playbook fails, an email will be sent, after successful execution, the returned results are saved to the database.

 

The official gave a sample, specific visible: https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/log_plays.py

 

Based on the above example, we have customized the development. I originally wanted to implement all functions in the callback plug-in. However, debugging of the callback plug-in is quite troublesome. I cannot use the print function. If something goes wrong, for example, the list subscript is out of bounds, the error message is only provided when ansible is executed, and the specific number of error lines is not specified.

 

Finally, I gave up my idea of all in one. I only Parsed the returned results and saved them to the sqlite3 database. Then I summarized them based on the data IN the database.

 

The Code is as follows:

from __future__ import (absolute_import, division, print_function)__metaclass__ = typeimport osimport timeimport jsonimport sqlite3from ansible.module_utils._text import to_bytesfrom ansible.plugins.callback import CallbackBaseclass CallbackModule(CallbackBase):    """    logs playbook results, per host, in /var/log/ansible/hosts    """    CALLBACK_VERSION = 2.0    CALLBACK_TYPE = 'notification'    CALLBACK_NAME = 'performance_check'    CALLBACK_NEEDS_WHITELIST = False    def __init__(self):        super(CallbackModule, self).__init__()    def runner_on_failed(self, host, res, ignore_errors=False):        pass    def runner_on_ok(self, host, res):        performance_data=PerformanceData()        create_table_sql = 'CREATE TABLE performance_data(ip varchar(20) primary key, uptime varchar(20),cpu_usr DECIMAL,cpu_sys DECIMAL, cpu_iowait DECIMAL,cpu_idle DECIMAL,load_average_1 DECIMAL,load_average_5 DECIMAL,load_average_15 DECIMAL, mem_total INTEGER,mem_util DECIMAL,swap_total INTEGER,swap_used INTEGER,swap_in INTEGER,swap_out INTEGER,numa TINYINT)'        
        insert_sql = 'insert into performance_data values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'        insert_value = str_to_json(host,res)        performance_data.create_table(create_table_sql)        performance_data.insert_command(insert_sql,insert_value)        performance_data.quit()    def runner_on_skipped(self, host, item=None):        #self.log(host, 'SKIPPED', '...')        pass    def runner_on_unreachable(self, host, res):        #self.log(host, 'UNREACHABLE', res)        pass    def runner_on_async_failed(self, host, res, jid):        #self.log(host, 'ASYNC_FAILED', res)        pass    def playbook_on_import_for_host(self, host, imported_file):        pass    def playbook_on_not_import_for_host(self, host, missing_file):        passclass PerformanceData():    def __init__(self):        self.conn = sqlite3.connect("/tmp/data.db")        self.cursor = self.conn.cursor()    def create_table(self,create_table_sql):        self.cursor.execute(create_table_sql)    def insert_command(self,insert_sql,insert_value):        self.cursor.execute(insert_sql,insert_value)    def query(self,query_sql):        self.cursor.execute(query_sql)        results=self.cursor.fetchall()        return results    def quit(self):        self.conn.commit()        self.conn.close()def str_to_json(host,res):            result= res["stdout"].strip(" ").replace("'",'"').strip('\n').strip('"')            results= '{"'+host+'":'+result+'}'            result_with_host = json.loads(results)            value=result_with_host[host]            return (host,value['uptime'],float(value['cpu_usr']),float(value['cpu_sys']),float(value['cpu_iowait']),               float(value['cpu_idle']), float(value['load_average_1']), float(value['load_average_5']), float(value['load_average_15']),               int(value['mem_total']), float(value['mem_util']),int(value['swap_total']),int(value['swap_used']),int(value['swap_in']),               int(value['swap_out']), int(value['numa'])               )

 

The above text parsing script is attached here. it seems better to implement my all in one idea. Haha ~

#coding: utf8import re,json,sqlite3def get_ip_success():    with open(r'C:\Users\Administrator\Desktop\2.txt') as f:        ip_unreachable = []        ip_failed = []        ip_success=[]        line_num=0        for line in f.readlines():            if re.search('UNREACHABLE', line):                ip=line.split()[0]                ip_unreachable.append(ip)                flag=0            elif re.search('FAILED',line):                ip = line.split()[0]                ip_failed.append(ip)                flag=0            elif re.search('SUCCESS',line):                ip = line.split()[0]                flag=1                line_num=1            elif flag == 1 and line_num == 7:                line= line.strip(" ").replace("'",'"').strip('\n').strip('"')                stdout_lines= '{"'+ip+'":'+line+'}'                stdout_lines_with_ip = json.loads(stdout_lines)                ip_success.append(stdout_lines_with_ip)            line_num =line_num + 1    return ip_successdef os_status_generator(ip_success):    for os_status in ip_success:        for key,value in os_status.iteritems():            yield (key,value['uptime'],float(value['cpu_usr']),float(value['cpu_sys']),float(value['cpu_iowait']),               float(value['cpu_idle']), float(value['load_average_1']), float(value['load_average_5']), float(value['load_average_15']),               int(value['mem_total']), float(value['mem_util']),int(value['swap_total']),int(value['swap_used']),int(value['swap_in']),               int(value['swap_out']), int(value['numa'])               )class OsStatus():    def __init__(self,ip_success):        try:            self.conn = sqlite3.connect(":memory:")            self.cursor = self.conn.cursor()            self.cursor.execute('''CREATE TABLE os_status                     (ip varchar(20) primary key, uptime varchar(20),cpu_usr DECIMAL,cpu_sys DECIMAL,cpu_iowait DECIMAL,cpu_idle DECIMAL,                      load_average_1 DECIMAL,load_average_5 DECIMAL,load_average_15 DECIMAL,mem_total INTEGER,mem_util DECIMAL,                      swap_total INTEGER,swap_used INTEGER,swap_in INTEGER,swap_out INTEGER,numa TINYINT)''')            self.cursor.executemany("insert into os_status values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",os_status_generator(ip_success) )        except Exception as e:            print e;    def query(self,sql):        self.cursor.execute(sql)        results=self.cursor.fetchall()        column_size=len(results[0])        column_name= [column[0] for column in self.cursor.description]        for i in range(column_size):            print column_name[i].ljust(15),        print        for each_result in results:            for i in range(column_size):                print str(each_result[i]).ljust(15),            print    def quit(self):        try:            self.cursor.close()            self.conn.close()        except Exception as e:            print e;ip_success=get_ip_success()os_status=OsStatus(ip_success)sql = "select * from os_status"os_status.query(sql)

 

Finally, let's talk about how to enable the callback plug-in function in ansible, Which is disabled by default.

Enable two options:

callback_plugins   = /usr/share/ansible/plugins/callbackbin_ansible_callbacks = True

These two are required, and the other option is

callback_whitelist = performance_check

Performance_check corresponds to the "CALLBACK_NAME" defined in the callback plug-in above ",

Another related parameter is "CALLBACK_NEEDS_WHITELIST". If it is set to False, the callback_whitelist option is not required. Otherwise, you must specify "CALLBACK_NAME" in the callback_whitelist option ".