Zeromq in saltstack

Basic Introduction

Pre-reading

Environment Description

Zeromq patterns in Salt

Salt master

Salt minion

Salt

Summary

Basic Introduction

The underlying network architecture of salt is implemented using zeromq (2014.1 and earlier versions, starting from 2014.7, with raet added to salt). The official manual provides a brief description. so today we will look at which zeromq pattern is used inside salt? How do components collaborate?

Pre-reading

0mq-the Guide: sockets and patterns

Environment Description

Centos6.4

Salt 2014.1.10, default configuration

Zeromq patterns in Salt

Salt master

The salt master is the control node of the salt center. It provides command issuing, file collection, and other services for the salt environment.

When the master node is started, the name "reqserver" is started first. When the reqserver is initialized, the following zeromq patterns is created immediately:

Clients:

Zeromq pattern: zmq. Router

Listen address: TCP: // 0.0.0.0: 4506

Listen mode: bind

Role: salt master RET interface, supporting authentication (auth), file service, result collection, and other functions

Workers:

Zeromq pattern: zmq. Dealer

Listen address: IPC: // var/run/salt/master/workers. IPC

Listen mode: bind

Role: salt master task processing process interface

At the same time, clients and workers create a zeromq. device:

Zmq. Device (zmq. queue, self. Clients, self. Workers)

Through zmq. device, after the client receives the request, it forwards the request to the workers process interface for processing.

Next, the master starts publisher and immediately creates the following zeromq patterns:

Pub:

Zeromq pattern: zmq. Pub

Listen address: TCP: // 0.0.0.0: 4505

Listen mode: bind

Role: salt master pub interface, which provides the remote command sending Function

Pull:

Zeromq pattern: zmq. Pull

Listen address: IPC: // var/run/salt/master/publish_pull.ipc

Listen mode: bind

Role: salt master remote command execution pull Interface

After receiving data, the pull interface sends the data from the pub interface:

Package = pull_sock.recv ()

Pub_sock.send (Package)

Next, the master starts eventpublisher to implement event bus and creates the following zeromq patterns:

EPUB:

Zeromq pattern: zmq. Pub

Listen address: IPC: // var/run/salt/master/master_event_pub.ipc

Listen mode: bind

Role: salt master event pub interface, to facilitate other or third-party applications to subscribe to events on Event Bus

Epull:

Zeromq pattern: zmq. Pull

Listen address: IPC: // var/run/salt/master/master_event_pull.ipc

Listen mode: bind

Role: salt master event pull Interface

When the epull interface receives the packet, it sends the data on the pub interface:

Package = self. epull_sock.recv ()

Self. epub_sock.send (Package)

After eventpublisher is started, the salt master will continue to start the halite and reactor systems, which are not described at the moment. subsequently, salt will start multiple work processes (the default value is 5. In a large environment, we recommend that you increase the number of worker_threads in the configuration file to increase the number of such processes) for task processing, each worker process creates the following zeromq patterns:

Socket

Zeromq pattern: zmq. Rep

Listen address: IPC: // var/run/salt/master/workers. IPC

Listen mode: connect

Role: salt master task processing process, process and verify Minion, Get Master configuration, mine, pillar, fileserver file acquisition, Minion event fire to master event interface, collect minions return results and other tasks

Salt minion

Salt Minion is the operating node of the salt environment. After a remote command is sent from the master, it is executed on the host and the result is returned to the master.

Salt minion obtains the Master Address from the configuration file at startup. If it is a domain name, it will be resolved. after the resolution, the system will connect to the RET interface of the master for key authentication. if the authentication succeeds, the master's publish_port is obtained. This is why you only need to specify the minion ret_port (corresponding to master_port In the minion configuration file) in the minion configuration file.

After obtaining the publish_port (4505 by default) of the master, a local event interface of Minion is created:

EPUB:

Zeromq pattern: zmq. Pub

Listen address: IPC: // var/run/salt/minion/minion_event _ {id_hash} _ pub. IPC

Listen mode: bind

Role: salt minion event pub interface, so that other or third-party applications can obtain event information through this event bus.

Epull:

Zeromq pattern: zmq. Pull

Listen address: IPC: // var/run/salt/minion/minion_event _ {id_hash} _ Pull. IPC

Listen mode: bind

Role: salt minion event pull Interface

After the epull interface receives the data, it checks whether it needs to be processed. If it needs to be processed, it executes the operation and then transmits the data packet to the Epub interface:

# Check the Event System

If Socks. Get (self. epull_sock) = zmq. Pollin:

Package = self. epull_sock.recv (zmq. Noblock)

Log. debug ("handling event % R", package)

Try:

If package. startswith ('module _ refresh '):

Self. module_refresh ()

Elif package. startswith ('pillar _ refresh '):

Self. pillar_refresh ()

Elif package. startswith ('grains _ refresh '):

If self. grains_cache! = Self. opts ['grains']:

Self. pillar_refresh ()

Self. grains_cache = self. opts ['grains']

Elif package. startswith ('fire _ master '):

Tag, Data = salt. utils. event. minionevent. Unpack (Package)

Log. debug ("forwarding master event tag = {tag}". Format (TAG = data ['tag'])

Self. _ fire_master (data ['data'], data ['tag'], data ['events'], data ['pretag'])

Self. epub_sock.send (Package)

Failed t exception:

Log. debug ("exception while handling events", exc_info = true)

After the event interface is created, the following zeromq pattern will be created:

Socket:

Zeromq pattern: zmq. sub

Listen address: "TCP:/": TCP: // {master_ip}: 4505

Listen mode: connect

Purpose: Subscribe to tasks from the master pub interface.

The remote command execution is established through zeromq pub/sub pattern. That is, when the master node sends the Operation Command, all minion can receive the command, then, Minion checks whether the local machine is target match. If match is used, execute the command. after the execution is complete, the RET interface sent to the master through sreq will be created, and the following zeromq pattern will be created:

Socket:

Zeromq pattern: zmq. req

Listen address: "TCP:/": TCP: // {master_ip}: 4506

Listen mode: connect

Purpose: Send the execution result to the master.

For more information about how minion executes tasks, visit: http://devopstarter.info/yuan-ma-jie-du-saltstackyun-xing-ji-zhi-zhi-job-runtime/

Salt

The salt master and salt minion have established the corresponding zeromq pattern. How does one process the data flow when a remote execution command is issued? Take test. Ping as an example:

1. Execute:

Salt '*' test. Ping

The corresponding Python execution is:

Client = salt. cli. saltcmd ()

Client. Run ()

Internally, it is called again:

Local = salt. Client. localclient ()

Pai_fun = Local. pai_cli ()

For full_ret in resource_func (kwargs ):

RET, out = self. _ format_ret (full_ret)

Self. _ output_ret (Ret, out)

2. During localclient object initialization, the serial object used to serialize the sent data will be created, and the masterevent object. masterevent object will create the following zeromq pattern:

Sub:

Zeromq pattern: zmq. sub

Listen address: IPC: // var/run/salt/master/master_event_pub.ipc

Listen mode: connect

Purpose: Subscribe to data from the master event pub interface.

3. When running cmd_cli, the Operation Command is first encapsulated into the following content through run_job: </P>

{'Tgt _ type': 'glob', 'jid': '', 'key': 'samples/lla32ydvumvklpaunsmtbwdboqpis = ', 'tgt ':'*', 'arg ': [], 'fun': 'test. ping ', 'wargs': {'show _ timeout': false}, 'cmd': 'Publish', 'ret ': '', 'user': 'root '}

The RET interface that will be sent to the local master. The following zeromq pattern will be created during the process:

Socket:

Zeromq pattern: zmq. req

Listen address: TCP: // 127.0.0.1: 4506

Listen mode: connect

Purpose: Send the encapsulated command to the master RET interface.

4. after the master RET interface receives the data sent in 3, it will pass the chminions. check_minions: obtain the required minions and generate a jid. Then, perform the fire_event operation on the master event interface, and then use the master private key (master. PEM), and then create the following zeromq pattern:

Pub_socket:

Zeromq pattern: zmq. Push

Listen address: IPC: // var/run/salt/master/publish_pull.ipc

Listen mode: connect

Purpose: send commands to the master pull interface.

After the master pull interface receives data, it will quickly send the previously received data on the master pub interface.

At the same time, the jid And minions encapsulated results are returned to 3. 3, when pai_cli obtains data, it calls get_cli_event_returns to listen to the event bus on the master end and filter out the event corresponding to the jid of this task, used to obtain execution results

5. at this time, Minion can receive a message from the master pub interface through pub/Sub. after receiving the message, Minion decrypts the message through the local master pub_key (minion_master.pub) and ensures that the message comes from the master. after the decryption is complete, the local target matches. If the target matches, it indicates that it needs to be executed and a new process is derived for execution. otherwise, ignore it directly.

6. After Minion is executed, the encapsulated result will be sent to the RET interface of the master through AES Encryption Through _ return_pub.

7. after the master RET receives the data sent in step 6, it decrypts the data through AES and then uses _ return to first perform fire_event on the local event interface, the results are stored locally on the master.

8. because fire_event is carried out in 7, the get_cli_event_returns in 4 can be captured. Because the iterator is used, each received result can be immediately displayed, once the result of the captured minions is greater than or equal to the number of previously obtained minions, it means that all minions have returned results and quit.

Summary

Salt uses the flexible and efficient patterns of zeromq to make the salt network topology very flexible and efficient. pub/sub is used to implement an efficient remote execution command issuing mechanism; router/req is used to implement authentication and asynchronous remote execution result returning; dealer/rep is used, implement a multi-process task processing mechanism. Use pull/pub to implement event bus so that other or third-party applications can quickly use pub/sub to receive messages on Event bus.

I love salt, I love zeromq!

Powered by redmine? 2006-2014 Jean-Philippe Lang

This article is from the "davideylee" blog and will not be reposted!

Zeromq in saltstack