Using the HTTP API interface to read pillar data--ext_pillar

Most of the time, in the use of pillar we are directly using SLS files to store data, but in fact pillar can support a variety of data storage methods, such as: MySQL, MONGO, JSON and so on, these can be in the official website or code to see Ext_piilar code;

pillar Supported data storage Module List address: Http://docs.saltstack.com/en/latest/ref/pillar/all/index.html#all-salt-pillars

Demand:

First of all to say why there is this demand, sometimes we need to pillar data stored in the CMDB, or to pull the CMDB data to provide pillar use, this time to edit pillar under the SLS file is a little less elegant; Ext_pillar is to solve this problem, It serves as a hub for pillar data mapping and data storage (CMDB).

Recently write code to publish, need to use the pillar data (the platform to provide a version number, code URL, in the package after the code push to repo, the version number to update to pillar data, for Saltstack), think of ext_pillar this thing, OMS operations platform used to MySQL database, just want to use this module directly, in contact with a little touch of sadness, the document is very small ~ ~ to find a translation of the article is MongoDB, think again to set up a MongoDB is a bit over; Fat fertilizer to my advice is not to use MySQL, HTTP API interface method is recommended;

Pillar is a powerful tool that can not only store security data, but also serve as a business data store, and use the Ext_pillar docking CMDB system, state to describe the business process logic, and the real data from the CMDB; I never thought I could play like this, this green manure, Jacky two people first achieved, very experienced

Speaking so much, say so to implement this HTTP API's Ext_pillar (no CMDB)

1. Implement backend data- based on business scenarios, design data Structures (DICT) to meet business, control over your hands, what you want to be, what you can achieve, the key points that match your business

2. Implement Ext_pillar, access to HTTP access to back-end data

3. Configure the Salt master configuration file to restart the master

4. Pillar Test

Realize:

1. Backend data implementation.

The use of HTTP is the JSON data, not only to generate JSON data, but also to change the JSON data; First look at the Pillar Data map SLS file format

Hdworkers:

ver:2014102202

The above data format conversion under dict,{' hdworkers ': {' ver ': ' 2014102202 '}, I just need to implement a simple version number mapping on the line, then the complex data, we can design, the following code ( code sucks, MO spray ~)

 # -*- coding: utf-8 -*-import jsonimport osclass buildjson (object):      '     build json data (base and minion_id  etc..)      "    def base_data (Self,args):          '         build base data          '         info =  {}        ret = dict (Info,**args)          self.write_data (' base ', ret)     def build_data (Self,id,args):         if not os.path.exists ('/home/api/pillar/%s '  %   (ID)):             with open ('/home/api/ Pillar/base ')  aS f:                obj  = f.readlines () [0]            ret  = eval (obj)             self.write_data (id,ret )         with open ('/home/api/pillar/%s '  %  (ID))   As f:            data = f.readlines () [0]        cov_data = eval (data)          if not cov_data.has_key (Args.keys () [0]):             ret = dict (Cov_data,**args)              self.write_data (Id,ret)          Else:  &nBsp;         cov_data.update (args)              self.write_data (Id,cov_data)     def write_ Data (Self,file,ret):         f = open ('/home/api/pillar/%s ')  %  (file), ' w+ ')                     f.write (str (ret))         f.close ()

#data = {' hdworkers ': {' ver ': ' 2014103105 '}}

#bapi = Buildjson ()

#bapi. Base_data (data)

#bapi. Build_data (' test-01 ', data)

Generate base data, then call Build_data (inherit base data, update the data at the same time), and some data will be in the ID, but the base is not the ~, so the above is I write, crossing can play on their own, there is good feedback to me, I am in the revision ~

2. Implement Ext_pillar, can be accessed by HTTP mode

Because it is combined with the OMS platform, the above generated files, I made the localtion settings in Nginx, so that the data can be accessed through HTTP; otherwise ext_pillar can't play.

Nginx configuration process I just ignore, directly paste the results

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/4D/91/wKiom1RTRYnBk6SRAABWZ4qyD_U841.jpg "title=" Clipboard.png "alt=" Wkiom1rtrynbk6sraabwz4qyd_u841.jpg "/>

ID data can be accessed via HTTP, OK, go down, with Ext_pillar

Ext_pillar implementation

more/usr/lib/python2.6/site-packages/salt/pillar/oms.py

# -*- coding: utf-8 -*-' Author: pengyaoa module to pull data  from OMS system via its API into the Pillar  dictionaryconfiguring the wolf system ext_pillar==================================.  code-block:: yaml  ext_pillar:  - oms:       api: http://oms.example.com/api/pillar/module documentation==================== ' # Import  python libsimport loggingimport urllib2import json# set up logginglog  = logging.getlogger (__name__) def ext_pillar (MINION_ID, PILLAR, API):      '     read pillar data from oms system via  its API.     '     pillar_url = api +  ' /" + minion_id    Log.info ("querying oms system pillar for %r"  % (minion_id))      try:        request = urllib2.urlopen (pillar_url). Read ()         ret = eval (Request)          result = json.loads (Json.dumps (ret))     except exception,  e:        log.exception (              ' query oms system failed! error: %s '  % (e)         )         return { }    return result

The code is simple, URLLIB2 request minion_id data, json.loads convert the data to dict back to Saltstack. Note that the data requested by Urllib is STR, and I convert STR first to dict in the pass to JSON to avoid becoming Unicode ... (Bright JSON is not a bit more ~)

3. Configure the Salt master configuration file, pillar test

Configure the master configuration file according to the Ext_pillar code so that master will load the pillar data

Cat/etc/salt/master

Ext_pillar:

-OMS: #代码名字, oms.py
API:/ http/x.x.x.x/pillar/# API prefix,/HTTP//access domain/IP/pillar/minion_id to access data

Restart Master

/etc/init.d/salt-master restart

4. Pillar Test

Salt ' minion_id ' Pillar.item hdworkers

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/4D/91/wKioL1RTRv3DLHntAACU_G68Z-I985.jpg "title=" Clipboard.png "alt=" Wkiol1rtrv3dlhntaacu_g68z-i985.jpg "/>

I do not demonstrate the data generation process, as long as the back-end data updates, execute the corresponding Minion ID pillar will be able to obtain the latest data, because pillar is dynamic ~

Hope this article can help everyone, play happy ~ ~ ~ no longer and I do not find the same article.

This article is from the "Binbin" blog, make sure to keep this source http://binbin158.blog.51cto.com/2659767/1570433

Using the HTTP API interface to read pillar data--ext_pillar