SaltStack's salt-master salt-minion starts the Master and Minion certified salt '*' test. ping execution process

Salt-master startup process

Environment introduction

We use the following environment to analyze the SaltStack source code:

System: CentOS-7
Python version: 2.7.5
Saltstack version: 2015.5.2 (Lithium)

We use the following method to install salt-master:

Rpm-Uvh http://mirrors.opencas.cn/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
Yum install salt-master

 
Scenario description

This section analyzes the salt-master startup process. We use the following command to start salt-master:

/Usr/bin/salt-master-d

Overall analysis

The startup process of the salt-master can be divided into two parts: one is to parse the configuration file and the command line parameters, and the other is to start the required process.
For parsing command line parameters, we mainly use the python standard module optparse, and parse the master configuration file using the yaml module.
The main process is:
1. Import salt. syspaths as the default value of the command line parameter, and import salt. config as the default value of the master configuration file;
2. Use the add_option of optparse to register command line options in ConfigDirMixIn, LogLevelMixIn, RunUserMixin, and other classes;
3. Call the parse_args function, parse the command line parameters, and parse the master configuration file;
4. Call self. master. start () and register sigusr1 and sigusr2 signals to output stack and profile information respectively;
5. Start the required processes, such as the maintenance process, publisher process, and master event process.

 
Detailed analysis

 
Function call flowchart

 
Core functions

 Class relationship diagram

MasterOptionParser class

This class is mainly used to parse command line parameters and yaml configuration files. Inherits the python Standard optparse. optionParser class. Previously, these six classes were ConfigDirMixIn, LogLevelMixIn, RunUserMixin, DaemonMixIn, PidfileMixin, and SaltfileMixIn respectively used to register the configuration file path, log level, Daemon, and pid file path, saltfile path and other command line options. The parse_args method of optparse. OptionPaser is rewritten to add parsing for the master yaml configuration file.

Maintenance process

The Maintenance process is used for general Maintenance of the master, such as regular Maintenance tasks.

Publisher Server

Bound to the default port 4505 zmq pub and publish_pull.ipc pull type ipc, as long as the data is pushed to publish_pull.ipc, all clients of subcribe 4505 publisher will receive the data. The specific role of this server has not been viewed yet.

EventPublisher Server

If you bind master_event_pub.ipc pub and master_event_pull.ipc pull to push data to master_event_pull.ipc, the client subscribing to master_event_pub.ipc will receive the message.

ReqServer Server

Multiple Mworker processes are started, and each process connects to workers. ipc REP, zmq_device is enabled, and tcp port 4506 ROUTER and workers are bound. ipc DEALER, so long as the REQ data is sent to 4506, the load will be balanced to the Mworker process. After the Mworker process receives the request, it will execute the _ handle_payload function, that is, it will execute the corresponding module.

Salt-minion startup process

Features

Salt-minion is responsible for receiving messages from salt-master and executing corresponding commands.

 
Environment introduction
 

System: CentOS-7
Python version: 2.7.5
Saltstack version: 2015.5.2 (Lithium)

 
Scenario description


This section analyzes the salt-minion startup process. We use the following command to start salt-minion:

/Usr/bin/salt-minion-d

 
Overall analysis

The overall analysis of the salt-minion startup process is as follows:
1. Use the parsers. MinionOptionParser class to Parse command line parameters and configuration files. This class inherits the MasterOptionParser class, which is similar to the salt-master parsing process;
2. Load necessary modules, such as grains and pillar;
3. Start to connect to the Req Server of the salt-master and evaluate the verification status with the master. If the master does not add the minion key to the trust list, the minion will be cyclically verified until the verification passes;
4. The message is sent through the master, and minion has been started;
5. Subscribe to the salt-master pub server.
 
Detailed analysis
 
Function call flowchart

 

Salt-minion startup process

 
Core functions
 
Class relationship diagram

MinionOptionParser class

The MinionOptionParser class inherits the MasterOptionParser and overwrites the setup_config method of MasterOptionParser. It is changed from the master configuration file to the minion configuration file. For details about MasterOptionParser, refer to SaltStack source code parsing-salt-master startup process.

Minion class

This should be an important class, including all the methods for communicating with the master. For example, _ do_socket_recv is responsible for receiving the master pub information; _ do_event_poll is responsible for processing the event; authenticate is responsible for processing the verification with the master.

Master and Minion authentication process

Preface

When the Minion process starts, it first connects to the Master's Req Server for authentication and continues until the authentication is successful. Next we will analyze the authentication process.

 
Overview

1. Before the Minion process is started, Minon sends a payload message with cmd _ auth to the Master Req Server;
2. After the Master Req Server receives the message from Minion, it uses the _ auth method for authentication;
3. Start to make the following judgment on the key file. If the key is in the minions_rejected directory, the connection is rejected. If the key is in the mini-ons directory, verify that the pub key is equal. If not, add it to the mini-ons_denied directory; if the key is not in the minions_pre directory, the key is written to the minions_pre Directory. If the minions_pre does not exist, if the pub key verification fails, the key is added to the minions_denied directory.

 
Activity diagram of the minion authentication request process

Function call diagram of the authentication request sent by minion

 
Master node processes minion verification request function call diagram

Salt '*' test. ping execution process

Preface

In this article, we will learn the whole process of implementing the salt '*' test. ping command. There are many components involved, which will help us to further understand the operating mechanism of SaltStack.

 
Overview

Salt '*' test. ping involves many components, including Master Req Server, Master Publisher, Minion, and Master EventPubliser. The following describes the entire implementation process:

1. Execute the salt '*' test. ping command on the salt-master machine;
2. The salt sends a message with the publish command to the Master Req Server;
3. After receiving the message, the Master Req Server pushes the message to publish_pull.ipc;
4. Master Publisher sends messages to Minion Publish;
5. After receiving the message, Minion starts a new process to execute the function specified by fun in the message;
6. After the function is executed, the result is returned to the Master Req Server;
7. The Master Req Server pushes the result to the Master EventPublisher;
8. Because the salt client subscribes to Master EventPublisher, it will receive the returned result and then directly output it to the terminal. The entire process is complete.

 
Activity diagram

To better understand the entire process, the following is an activity diagram:

 
Salt '*' test. ping function call diagram

 
Diagram of calling the Master Req Server to process the salt test. ping Message function

 
Call map of the message function processed by Minion in Master Pub

 
Figure of the function call returned by the Master Req Server for processing Minion data