Salt + custom returner + fluent + mysql for result collection

Background: There are a lot of optional returner items in salt, but they all need to be configured in minion. I think this is quite tricky, and we use fluent to collect data on our platform, so a reutren is customized, and then collected and processed using fluent. The import process is as follows: 1: mysql table structure: CREATETABLE 'fluent '; createtable' salt _ returns

Background: There are a lot of optional returner items in salt, but they all need to be configured in minion. I think this is quite tricky, and we use fluent to collect data on our platform, therefore, a reutren is customized, and then collected and processed using fluent. The import process is as follows: 1: mysql TABLE structure: create table 'fluent '; create table 'Salt _ returns

Background: There are a lot of optional returner items in salt, but they all need to be configured in minion. I think this is quite tricky, and we use fluent to collect data on our platform, therefore, a reutren is customized, and fluent is used to collect, process, and store data.

The process is as follows:

1: mysql table structure:

CREATE TABLE `fluent`;CREATE TABLE `salt_returns` (  `id` mediumint(9) NOT NULL AUTO_INCREMENT,  `jid` char(20) DEFAULT NULL,  `host_id` varchar(48) DEFAULT NULL,  `time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,  `fun` varchar(30) DEFAULT NULL,  `return` text,  `success` char(5) DEFAULT NULL,  PRIMARY KEY (`id`),  KEY `idx_host_id` (`host_id`))

2: Custom returners

Create the Default User-Defined return directory. Although this directory is default, it is not created by default :(

      mkdir /srv/salt/_returners

Custom reuters:

This is mainly the definition of the returner (ret) function.

cat /srv/salt/_returners/lcoal_return.py#coding=utf8import jsondef __virtual__():    return 'local_return'def returner(ret):    '''    Return data to the local file    '''    result_file = '/var/local/salt/returner'    result = file(result_file,'a+')    result.write(str(json.dumps(ret.values()))[1:-1]+'\n')    result.close()

Synchronize to all nodes:

salt '*' saltutil.sync_returners

Execute Command

salt '*' cmd.run 'ls /var' --return local_return

View results:

cat /var/log/salt/returner"cmd.run", "20130524052158870765", "cache\ncvs\ndb\nempty\ngames\nlib\nlocal\nlock\nlog\nmail\nnis\nopt\npreserve\nrun\nspool\ntmp\nwww\nyp", "minion1", true

3fluent Collection Client Configuration:

  type tail  path /var/log/salt/returner  pos_file /tmp/return_pos.log  tag os.salt  format /\"(?.*)\", \"(?\d+)\", (?.*), \"(?.*)\", (?.*)/    type forward    flush_interval 1s        host         port 

Server Configuration:

  type forward  port 24224  bind 0.0.0.0  type mysql  host localhost  database fluent  username fluent  password fluent  key_names jid,id,fun,return,success  sql INSERT INTO salt_returns (jid,host_id,fun,`return`,success) VALUES (?,?,?,?,?)  flush_interval 5s

Result query:

select * from salt_returns where success is not NULL and fun='cmd.run' limit 1;+------+----------------------+-----------------------------------------+---------------------+---------+---------+---------+| id | jid | host_id | time | fun | return | success |+------+----------------------+-----------------------------------------+---------------------+---------+---------+---------+| 2571 | 20130531184127393793 | test | 2013-05-31 10:38:29 | cmd.run | "/root" | true |+------+----------------------+-----------------------------------------+---------------------+---------+---------+---------+

Original article address: salt + custom returner + fluent + mysql for result collection. Thanks to the original author for sharing.