Syslog, UNIX domain sockets from a simple requirement to a Linux environment

Demand:

Back to Top

Working in a Linux (DEBIAN8) environment to run a server program, implemented in the Python language, the code has different priority logs need to be recorded, the development time is the use of Python logging module output to the file, the sample code is as follows:

　　

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

 1 import logging, os 2  3 logger = none 4 def  get_logger (): 5     global logger 6      if not logger: 7         logger =  Logging.getlogger (' Serverlog ')  8         logger.setlevel ( Logging.info)  9         filehandler = logging. Filehandler (os.environ[' HOME '] +  '/server.log ',  encoding= ' UTF8 ') 10          filehandler.setformatter (logging. Formatter ("% (asctime) s - % (levelname) s - % (message) S")) 11          logger.addhandler (Filehandler) 12     return logger13  14 def some_func (): 15     get_logger (). info ("Call some_func") 16 17 if __name__ ==  ' __main__ ':18      some_func ()

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

Running the above code will result in a Server.log file under the home directory.

Later the data analysis department said they wanted to get some logs in real time, and they had a server dedicated to logging, so how do you send them the logs? I did not have the relevant experience before, the data analysis department colleagues said that this demand they are looking for operations and maintenance personnel to help. OPS colleagues gave a simple solution: the product writes the logs to the syslog, and then they are responsible for forwarding the logs with certain keywords to the data Analysis Department, which, under the guidance of the OPS colleague, changes the code to this:

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

 1 import logging 2 import logging.handlers 3  4 logger  = none 5 def get_logger ():  6     global logger  7     if not logger: 8          logger = logging.getlogger (' Serverlog ')  9          logger.setlevel (Logging.info) 10 11         sys_ Handler = logging.handlers.sysloghandler ('/dev/log ',  facility= logging.handlers.SysLogHandler.LOG_LOCAL0) 12         syslog_tag  =  ' Serverlog ' 13         sys_handler.setformatter ( Logging. Formatter (syslog_tag +  ":% (asctime) s - % (name) s - % (levelname) s - % ( Message) (S ")) 14 15     &Nbsp;   logger.addhandler (Sys_handler) 16     return logger17  18 def some_func (): 19     get_logger (). info ("Call some_func") 20  21 if __name__ ==  ' __main__ ': 22     some_func ()

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

The above code modifies the output form of the log, the intuitive feeling is from the file Server.log to the/dev/log, but/dev/log corresponds to Sysloghandler, not filehandler, so certainly not an ordinary file. At this point, I have two questions: first, here I did not output the log to the home directory of the Server.log file, but the program is run to generate such a file, second, how to tell the log sent to the data analysis Department of the server.

If you don't understand, ask:

Q: How do I generate a Server.log file under the new code, and how does the log content be forwarded to the data Analysis department's server?

A: This is/etc/init.d/rsyslog this daemon based on/etc/rsyslog.conf This configuration file will output the log to different files, including network files, that is, other servers. See/etc/rsyslog.conf This configuration is clear.

Q:ok, what does the Python code do with the output of the file to/dev/log and Rsyslog?

A:python Sysloghandler will send logs to Rsyslog, they use UNIX domain socket communication, specifically see logging module source code will know

UNIX domain sockets:

Back to Top

According to the above dialogue, the Python program first send the log to rsyslog this program, and then Rsyslog to process the received log data, so first look at the logging code:

Sysloghandler This class in logging.handlers.py, the core code is as follows:

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

 1     def __init__ (self, address= (' localhost ',  syslog_udp_port),  2                   facility=log_user, socktype=socket. SOCK_DGRAM): 3          "" " 4          Initialize a handler. 5  6          If address is specified as a string, a  unix socket is used. to log to a 7          local syslogd,  "Sysloghandler (address="/dev/log ")"  can be used.  8         if facility is not specified,  LOG_USER is used. 9          "" "10         logging. Handler.__init__ (self) 11 12         self.address =  address13         self.facility = facility14          self.socktype = socktype15 16          if isinstance (address, basestring):17              self.unixsocket = 118              self._connect_unixsocket (address) 19          else:20              self.unixsocket = 021              self.socket = socket.socket (socket.af_Inet, socktype) 22             if  Socktype == socket. sock_stream:23                  self.socket.connect (address) 24         self.formatter  = none25 26     def _connect_unixsocket (self, address):27          self.socket = socket.socket (socket.AF_UNIX,  Socket. SOCK_DGRAM) 28         # syslog may require  Either dgram or stream sockets29         try : 30             self.socket.connect (Address) 31         except socket.error:32             self.socket.close () 33              self.socket = socket.socket (socket.AF_ Unix, socket. SOCK_STREAM) 34              Self.socket.connect (Address)

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

It is clear in __init__.doc that if address is a string (the default value is a tuple), a UNIX socket (Unix domain socket) will be established. If address is "/dev/log" (as in our previous Python code), then output to the native SYSLOGD program. Also, on line 27th self.socket = Socket.socket (Socket.af_unix, socket. SOCK_DGRAM) The value of the first parameter family for Socket.socket is Af_unix, not the Af_inet (IPV4) or Af_inet6 (IPV6) that we often use. So what is a UNIX domain socket?

　　

UNIX domain sockets are a way of interprocess communication (ipc:inter-process communication), as well as pipelines, named pipes, message queues, shared memory, sockets, and so on. What is the difference between a UNIX domain socket and a commonly used socket (a narrow-sense Internet socket) that UNIX domain sockets can only communicate between processes on the same host, and a common socket localhost ' to communicate on the same host, what are the advantages of UNIX domain sockets?

First: There is no need to go through the network protocol stack

Second: No package unpacking, calculation of checksums, maintenance of serial numbers and answers, etc.

So, the advantage is good performance, a word, fast.

Here is a simple server client example to look at the UNIX domain socket using the method and process:

Server: uds_server.py

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

 1 ADDR =  '/tmp/uds_tmp '  2  3 import socket, os 4   5 def main (): 6     try: 7          sock = socket.socket (Socket.af_unix, socket. SOCK_STREAM)  8         if os.path.exists (ADDR):  9              os.unlink (ADDR) 10          sock.bind (ADDR) 11          sock.listen (5) 12         while true:13                  connection,  Address = sock.accept () 14                  print  "Data : %s " % connection.recv (1024x768);15                  connection.send ("Hello uds client") 16                  connection.close () 17      finally:18         sock.close () 19 20  if __name__ ==  ' __main__ ': 21         main ()

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

Client: uds_client.py

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

1 ADDR = '/tmp/uds_tmp ' 2 3 Import Socket 4 5 def main (): 6 sock = Socket.socket (Socket.af_unix, socket. SOCK_STREAM) 7 Sock.connect (ADDR) 8 sock.send (' Hello UNIX domain Socket server ') 9 print ' Client R Ecieve ', Sock.recv (1024x768) sock.close () if __name__ = = ' __main__ ': Main ()

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

First: Run the server Python uds_server.py, this time in the/tmp directory generated files, with LS to view the details as follows:

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170323140416611-1663002555.png "style=" margin:0px;padding:0px;border:0px; "/>

As you can see, the file type (the first field) is S, which represents the socket file. (PS: If the process uses a command pipeline to communicate with the intermediate file, LS displays the file type P)

Running the client Python uds_client.py, there are corresponding outputs on both the client and server side, there is no big difference between the usage and the normal socket.

Log forwarding Process:

Back to Top

After understanding the UNIX domain socket concept, the following is relatively simple, the first is/dev/log this file, we use LS to view the information of this file

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170321200554424-555977189.png "style=" margin:0px;padding:0px;border:0px; "/>

Can see this file is a symbolic link file, the real file is/run/systemd/journal/dev-log, then to view this file

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170321200742736-1156458900.png "style=" margin:0px;padding:0px;border:0px; "/>

OK, is a socket file, composite expected, according to the previous UNIX domain socket example, Rsyslog should also listen to this file, we take a look at

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170324101520533-1212316745.png "style=" margin:0px;padding:0px;border:0px; "/>

Lsof FD can list all the processes that use this file (the concept of Linux files is relatively broad), in fact we see only Systemd and systemd-j two unknown so process. So, look directly at the UNIX domain socket that Rsyslog uses.

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170322091124143-462804263.png "style=" margin:0px;padding:0px;border:0px; "/>

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170324101545783-1464834217.png "style=" margin:0px;padding:0px;border:0px; "/>

Well, you can see RSYSLOGD used socket domain socket is/run/systemd/journal/syslog, not/run/systemd/journal/dev-log, these two files in the same directory, Then let's see what other processes are using/run/systemd/journal/syslog.

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170322194059533-186007552.png "style=" margin:0px;padding:0px;border:0px; "/>

SO,SYSTEMD and Rsyslogd both use this file and feel like the application process (e.g. Python program above) passes the log through/run/systemd/journal/dev-log (/dev/ Log behind the real file) sent to the SYSTEMD, and then systemd the log through/run/systemd/journal/syslog sent to RSYSLOGD, is not so, Google a bit, The discovery of this article Understand-logging-in-linux, is really such a process:

　　

SYSTEMD have a single monolithic log management program, Systemd-journald. This runs as a service managed by SYSTEMD.

• It reads/dev/kmsg for kernel log data.

• It reads/dev/log (a symbolic link to/run/systemd/journal/dev-log) for application log data from the GNU C library ' s SYSL OG () function.

• It listens on the af_local stream sockets at/run/systemd/journal/stdout for log data coming from systemd-managed services.

• It listens on the af_local datagram sockets at/run/systemd/journal/socket for log data coming from programs that speak the Systemd-specific Journal protocol (i.e. SD_JOURNAL_SENDV () et al.).

• It mixes these all together.

• It writes to a set of System-wide and Per-user journal files, in/run/log/journal/or/var/log/journal/.

• If It can connect (as a client) to an af_local datagram Socket At/run/systemd/journal/syslogit writes journal data There, If forwarding to Syslog is configured.

　　

OK, until now, we know how the application log is forwarded to Rsyslog, then rsyslog How to handle the received log, the secret is in/etc/rsyslog.conf, before opening this configuration file, we first look at Rsyslog official website Simple description:

RSYSLOG is the rocket-fast system for log processing.

The original R is the meaning of Rocket-fast! Rockets generally fast! The official website claims to be able to process millions logs per second. RSYSLOGD in the partial Linux environment is the default SYSLOGD program (at least on the author's machine), D is daemon meaning, background process. When the system starts, it starts the process to process the logs (including the log of the operating system itself and the user process). Open the modified/etc/rsyslog.conf, then the time to witness the miracle.

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170323142740033-1321897632.png "style=" margin:0px;padding:0px;border:0px; "/>

The original movement is monitored. This file is provided by the system and it is clearly not advisable to make changes directly to this file. such as the red part, you can add your own profile under the Rysyslog.d folder, custom log filter rules. So look at the new tmp.conf under the Rsyslog.d folder.

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

1 $FileOwner USERNAME 2 $FileGroup USERNAME 3 $FileCreateMode 0644 4 $DirCreateMode 0755 5 $Umask 0022 6 $template Serverl OG, "/home/username/server.log" 7 $template logformat, "%msg%\n" 8 if $syslogfacility-text = = ' Local0 ' and $syslogtag conta Ins ' Serverlog ' then-?serverlog; Logformat 9 #if $syslogfacility-text = = ' Local0 ' and $syslogtag contains ' Serverlog ' then @someip:p ort11 & Stop

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

Let's review the corresponding application code:

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

 1 import logging 2 import logging.handlers 3  4 logger  = none 5 def get_logger ():  6     global logger  7     if not logger: 8          logger = logging.getlogger (' Serverlog ')  9          logger.setlevel (Logging.info) 10 11         sys_ Handler = logging.handlers.sysloghandler ('/dev/log ',  facility= logging.handlers.SysLogHandler.LOG_LOCAL0) 12         syslog_tag  =  ' Serverlog ' 13         sys_handler.setformatter ( Logging. Formatter (syslog_tag +  ":% (asctime) s - % (name) s - % (levelname) s - % ( Message) (S ")) 14 15     &Nbsp;   logger.addhandler (Sys_handler) 16     return logger17  18 def some_func (): 19     get_logger (). info ("Call some_func") 20  21 if __name__ ==  ' __main__ ': 22     some_func ()

650) this.width=650; "src="/img/fz.gif "alt=" Copy Code "style=" Margin:0px;padding:0px;border:none; "/>

Note: The configuration file needs to be combined with the application code, such as the 11th line in the code facility=logging.handlers.sysloghandler.log_local0 and configuration $syslogfacility-text = = ' Local0 ' The code line 12th Syslog_tag = ' Serverlog ' corresponds to the configuration file $syslogtag contains ' Serverlog '. For the Syslogtag settings in Python code, refer to this question and answer on StackOverflow.

When we modify the configuration, we need to restart the RSYSLOGD by command/etc/init.d/rsyslog restart, reboot and then run the previous Python file.

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170323165530158-761295228.png "style=" margin:0px;padding:0px;border:0px; "/>

Send to remote server:

The tmp.conf file above commented out the 10th line, which is the function of sending the log that satisfies the condition to the other machine specified, and ip:port is used to specify the remote RSYSLOGD program that accepts the log. By default, RSYSLOGD is listening on port 514. Assuming I need to send a syslog to the LAN 10.240.10.10, the 10th line changes to this:

If $syslogfacility-text = = ' Local0 ' and $syslogtag contains ' Serverlog ' then @10.240.10.10

So 10.240.10.10 primarily turns on RSYSLOGD remote monitoring and specifies the output rule for the remote log, for example:

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201704/ 1089769-20170410132458782-973391177.png "style=" margin:0px;padding:0px;border:0px; "/>

This configuration allows RSYSLOGD to listen on port 514 using both UDP and TCP protocols and output non-native logs to files corresponding to the remote host name. Note that the above changes need to be restarted rsyslogd to take effect.

Summarize:

Back to Top

The process of logging from the application to the final log file (or remote server) is as follows:

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1089769/201703/ 1089769-20170323163540080-1733313578.png "width=" 865 "height=" "style=" margin:0px;padding:0px;border:0px; "/ >

Syslog, UNIX domain sockets from a simple requirement to a Linux environment