Using salt skillfully to realize automatic discovery of CMDB configuration

With the development of the Internet + new situation, more and more enterprises are stepping into the dual state (stability-sensitive dual state) It era, the information environment is more and more complex, both the IoE three layer architecture, and VCE, OpenStack and other cloud virtualization architecture and Internet-based distributed big Data architecture. Therefore, enterprises need to establish a suitable configuration management library (CMDB), like the human "brain" unified storage from the infrastructure to the business application at all levels of configuration information, in order to coordinate the "body" (Operation and maintenance system) parts to complete complex operations.

CMDB is so important, but it is the most difficult to build in the operation of the part, even a lot of IT operations and the big guys are lamenting the CMDB is the most painful point of operation, they summed up the CMDB construction has 3 difficulties, and ensure that the accuracy of the configuration data is difficult. The current industry mainly relies on process control, manual maintenance and automatic discovery to solve the problem of fresh and accurate data. Process Control and human maintenance are not involved, since we are human beings, not "God", it means "lazy", will be wrong. In this information explosion of the big data era, and then rely on human operations, we have been hurt. Therefore, the key to accurate data is to rely on automatic discovery.

Today, I would like to talk about a configuration information automatic discovery Scheme, hope to liberate the hands and brains of the OPS, so we can do something more meaningful, such as drinking coffee, accompany the family.

I. Selection of collection tools

In order to realize the automatic discovery of the configuration, it is necessary to have a handy basic collection tool (work if good, its prerequisite). Some big guys might say, "Easy, write a chant yourself." But now is the Internet Sharing era, the predecessors have stepped on a lot of pits, and contribute their efforts, we do not need to repeat the wheel, big can "take doctrine", find a most suitable for their own open source tools to use.

When it comes to open source automated configuration management tools, you have to say puppet, Chef, Ansible, and Saltstack, the four carriages. The internet on the four tools have a lot of detailed introduction and contrast, here no longer discuss, mainly about our choice of reasons and conclusions.

First, the four tools are implemented in two main factions: Ruby (Puppet, Chef) and Python (Ansible, Saltstack). Many seniors have said that to really apply an open source technology, you have to really take control of it. So the choice of technology type is very important, Python is known as the operation of the human language, its simple and unified, efficient operation of the characteristics more suitable for the use of OPS. And the most important thing is that Python has a lot of perfect and powerful OS, network programming aspects of the third-party library, these are not comparable to Ruby. So, from a technical point of view, Ansible and saltstack These two tools are more suitable for our Ops to play.

Second, Ansible and saltstack The biggest difference is the use of proxy technology, of course saltstack can also be achieved by SALT-SSH command-like ansible agentless execution capability, Just Saltstack's design philosophy is to implement an efficient and scalable configuration management system by executing agents (minion).

Summary down, there are agents and non-agent comparative advantages mainly include:

1) No need to save sensitive information such as account password, to avoid information leakage, but also easy for administrators to change the password regularly;

2) can collect richer configuration information and related relations, such as the configuration file analysis to obtain deep-level configuration information;

3) can be quickly aware of configuration changes, no agent mode can only be timed polling, increase the consumption of network traffic;

4) can achieve fast file transfer, such as easy to implement the collection of configuration files.

By comparing the above analysis, we choose Saltstack as the basic tool to configure automatic acquisition.

Second, the construction of the collection framework

Saltstack is a powerful, centralized configuration of automated management tools, it can be grains, pillar to achieve the configuration of thousands of servers automatically collected and managed (about Saltstack I do not popularize, we ask Google or degrees Niang). But now it only provides a better command-line tool, of course, the official also provides a simple Web UI called Halite, but this web interface is really a bit down, the function is limited, sorry saltstack this configuration Automation management "artifact" title Ah!

Therefore, our goal is to use the Saltstack framework and expansion capabilities to achieve the configuration of automated acquisition, data acquisition can be delivered to the configuration management library or centralized management. The entire collection frame logic is designed as shown in:

Saltstack contains both master and minion from the deployment. Only one master implementation centralized discovery scheduling and management is required within a network domain, and minion is installed on each managed host to receive and execute scripts and to feed the results back to master. Because this article is not the focus of introduction SALTSATCK, so its installation deployment is not introduced, we can still ask the omniscient Google and degrees Niang.

Third, the realization of automatic discovery

The framework is determined, so our next major task is to write an Autodiscover dispatch script ("Autodiscover discovery" in the framework) that collects data from master and processes it centrally.

What is the script written with? Of course, the OPS special language Python, which can be integrated with Saltstack natural, the main logic of the script is to get the configuration information from master through the Grains module and store or push it to other required systems. At the command line, we can use:

Salt ' * ' Grains.items

Or

Salt ' * ' Grains.item {key1} {Key2} ...

Get all configuration information or specify configuration information for the keyword name, such as Get tomcat_8080 configuration information:

You may find it strange that the regular Grains.item {key} command gets only 1 of the data, but here you get a multi-attribute complex object. This is because multiple services are installed on a host in the actual environment, that is, there are multiple configuration items, so we use the Python dictionary type (dict) to store the property information for each configuration item so that all the information about the configuration item can be obtained with a single key.

Note: How to do the discovery collection of configuration information, how to let grains the collection results of storing configuration items in dictionary format will be discussed in section Fifth, "Expansion of information acquisition".

We can also get the key by: Salt ' * ' grains.ls:

Now that we've used the powerful python, there's no need to call Saltstack's command line through the shell in the actual development, The return result of a dictionary type (DICT) can be obtained directly from the Salt.client module, and the Python dictionary type can be directly exported to a JSON format string for parsing the JSON string.

The code looks like this:

Iv. Expansion of configuration acquisition

With the configuration of the auto-capture framework, what we need is a constant building blocks, which is to constantly expand the rich saltstack grains capture script according to the management requirements and configuration model, and place the script file under the Salt Master's grains directory (the default is/srv/salt/ _grains), and then sync the script file to all Minion hosts via the "Salt ' * ' saltutil.sync_grains" command.

We take the configuration discovery of the Tomcat service as an example to collect information such as installation path, Web service port, Tomcat version information, maximum number of threads, and the script is as follows:

With a custom capture script, the configuration information for a Tomcat instance is found as follows:

Does it feel like a bit of a machine so easy! Access to more configuration information leave a kiss to your feet. Of course, the application of saltstack is not just to realize the configuration discovery so simple, we can dig deeper, play its greatest potential, realize the dual-state it automation operation and maintenance!

Author profile: Pang Hui Fu

? Senior Architect

? More than 10 years experience in the development of IT operations management software

? Committed to the research and promotion of automated operation and maintenance solutions

Using salt skillfully to realize automatic discovery of CMDB configuration