Python daemon and script Singleton running details, python daemon

Python daemon and script Singleton running details, python daemon

This article mainly introduces the Python daemon process and script Singleton running. I think it is quite good. I will share it with you and give you a reference. Let's take a look at it with xiaobian.

I. Introduction

The most important feature of a daemon is that it runs in the background. It must be isolated from the environment before running, these environments include unclosed file descriptors, control terminals, sessions and process groups, working directories, and file creation masks. They can be started from the startup script/etc/rc at system startup. d, which can be started by the inetd daemon, or by the job planning process crond, or by a user terminal (usually shell.

Python sometimes needs to ensure that only one script instance is run to avoid data conflicts.

Ii. Python daemon

1. function implementation

#! /Usr/bin/env python # coding: UTF-8 import sys, OS ''' fork the current process as a daemon. Note: If your daemon is started by inetd, do not do this! Inetd completes all the things that need to be done, including redirecting standard file descriptors. Only chdir () and umask () are required () '''def daemonize (stdin = '/dev/null', stdout ='/dev/null', stderr = '/dev/null '): # redirect standard file descriptor (directed to/dev/null by default) try: pid = OS. fork () # The parent process (the session Group header process) exits, which means that a non-session Group header process can never obtain the control terminal again. If pid> 0: sys. exit (0) # The parent process exits except T OSError, e: sys. stderr. write ("fork #1 failed: (% d) % s \ n" % (e. errno, e. strerror) sys. exit (1) # Remove the OS from the parent environment. chdir ("/") # chdir check that the process does not maintain any directory for use; otherwise, you cannot umount a file system. It can also be changed to the OS directory where the files that are important to the daemon program run are located. umask (0) # Call umask (0) to have full control over everything written, because sometimes you don't know what umask is inherited. OS. setsid () # After the setsid call is successful, the process becomes the new session leader and the new process leader, and is detached from the original logon session and process group. # Execute the second fork try: pid = OS. fork () if pid> 0: sys. exit (0) # The second parent process exits except T OSError, e: sys. stderr. write ("fork #2 failed: (% d) % s \ n" % (e. errno, e. strerror) sys. exit (1) # The process is already a daemon. Redirect the standard file descriptor for f in sys. stdout, sys. stderr: f. flush () si = open (stdin, 'R') so = open (stdout, 'a + ') se = open (stderr, 'a +', 0) OS. dup2 (si. fileno (), sys. stdin. fileno () # The dup2 function atomically closes and copies the file descriptor OS. dup2 (so. fileno (), sys. stdout. fileno () OS. dup2 (se. fileno (), sys. stderr. fileno () # example function: print a number and timestamp def main (): import time sys. stdout. write ('daemon started with pid % d \ n' % OS. getpid () sys. stdout. write ('daemon stdout output \ n') sys. stderr. write ('daemon stderr output \ n') c = 0 while True: sys. stdout. write ('% d: % s \ n' % (c, time. ctime () sys. stdout. flush () c = c + 1 time. sleep (1) if _ name _ = "_ main _": daemonize ('/dev/null','/tmp/daemon_stdout.log ', '/tmp/daemon_error.log') main ()

You can run the ps-ef | grep daemon. py command to view the inheritance of running in the background. The error running log is recorded in/tmp/daemon_error.log, and the standard output log is recorded in/tmp/daemon_stdout.log.

2. Class implementation

#! /Usr/bin/env python # coding: UTF-8 # python simulates the linux Daemon import sys, OS, time, atexit, string from signal import SIGTERM class Daemon: def _ init _ (self, pidfile, stdin = '/dev/null', stdout ='/dev/null', stderr = '/dev/null '): # You need to obtain the debugging information and change it to stdin = '/dev/stdin', stdout = '/dev/stdout', and stderr = '/dev/stderr' to run as root. Self. stdin = stdin self. stdout = stdout self. stderr = stderr self. pidfile = pidfile def _ daemonize (self): try: pid = OS. fork () # For the first fork, the child process is generated, and if pid> 0: sys. exit (0) # exit the main process worker t OSError, e: sys. stderr. write ('fork #1 failed: % d (% s) \ n' % (e. errno, e. strerror) sys. exit (1) OS. chdir ("/") # modify the operating directory OS. setsid () # set a new session to connect to the OS. umask (0) # reset the file creation permission try: pid = OS. fork () # The second fork, forbidding the process to open the terminal if pid> 0: s Ys. exit (0) failed t OSError, e: sys. stderr. write ('fork #2 failed: % d (% s) \ n' % (e. errno, e. strerror) sys. exit (1) # redirect file descriptor sys. stdout. flush () sys. stderr. flush () si = file (self. stdin, 'R') so = file (self. stdout, 'a + ') se = file (self. stderr, 'a + ', 0) OS. dup2 (si. fileno (), sys. stdin. fileno () OS. dup2 (so. fileno (), sys. stdout. fileno () OS. dup2 (se. fileno (), sys. stderr. fileno () # register the exit function and determine whether the process ate exists based on the file pid. Xit. register (self. delpid) pid = str (OS. getpid () file (self. pidfile, 'W + '). write ('% s \ n' % pid) def delpid (self): OS. remove (self. pidfile) def start (self): # Check whether the pid file exists to detect whether a process exists. try: pf = file (self. pidfile, 'R') pid = int (pf. read (). strip () pf. close () handle T IOError: pid = None if pid: message = 'pidfile % s already exist. daemon already running! \ N' sys. stderr. write (message % self. pidfile) sys. exit (1) # Start monitoring self. _ daemonize () self. _ run () def stop (self): # obtain the pid from the pid file. try: pf = file (self. pidfile, 'R') pid = int (pf. read (). strip () pf. close () handle T IOError: pid = None if not pid: # No error message = 'pidfile % s does not exist. daemon not running! \ N' sys. stderr. write (message % self. pidfile) return # Kill the process try: while 1: OS. kill (pid, SIGTERM) time. sleep (0.1) # OS. system ('hadoop-daemon. sh stop datanode ') # OS. system ('hadoop-daemon. sh stop tasktracker ') # OS. remove (self. pidfile) Before t OSError, err: err = str (err) if err. find ('no such Process')> 0: if OS. path. exists (self. pidfile): OS. remove (self. pidfile) else: print str (err) sys. exit (1) def restart (self): self. stop () self. start () def _ run (self): "run your fun" "while True: # fp = open ('/tmp/result', 'a + ') # fp. write ('Hello World \ n') sys. stdout. write ('% s: hello world \ n' % (time. ctime (),) sys. stdout. flush () time. sleep (2) if _ name _ = '_ main _': daemon = Daemon ('/tmp/watch_process.pid', stdout = '/tmp/watch_stdout.log ') if len (sys. argv) = 2: if 'start' = sys. argv [1]: daemon. start () elif 'stop' = sys. argv [1]: daemon. stop () elif 'restart' = sys. argv [1]: daemon. restart () else: print 'unknown command' sys. exit (2) sys. exit (0) else: print 'usage: % s start | stop | restart' % sys. argv [0] sys. exit (2)

Running result:

It is designed as a template when Daemon is designed. In other files, from daemon import Daemon is defined as a subclass, And the run () method is rewritten to implement its own functions.

class MyDaemon(Daemon):   def run(self):     while True:       fp=open('/tmp/run.log','a+')       fp.write('Hello World\n')       time.sleep(1) 

Insufficient: signal processing signal. signal (signal. SIGTERM, cleanup_handler) is not currently installed, and the callback function delpid () when the registration program exits is not called.

Then, write a shell command and add it to the boot service to check whether the daemon process is started every 2 seconds. If the daemon process is not started, it starts and automatically monitors the recovery program.

#/bin/sh while true do  count=`ps -ef | grep "daemonclass.py" | grep -v "grep"`  if [ "$?" != "0" ]; then    daemonclass.py start  fi  sleep 2 done 

3. python ensures that only one script instance can be run

1. Lock the file itself

#! /Usr/bin/env python # coding: UTF-8 import fcntl, sys, time, OS pidfile = 0 def ApplicationInstance (): global pidfile = open (OS. path. realpath (_ file _), "r") try: fcntl. flock (pidfile, fcntl. LOCK_EX | fcntl. LOCK_NB) # create an exclusive lock, and other processes that are locked will not block the wait t: print "another instance is running... "sys. exit (1) if _ name _ = "_ main _": ApplicationInstance () while True: print 'running... 'Time. sleep (1)

Note: The open () parameter cannot use w; otherwise, the file itself will be overwritten; The pidfile must be declared as a global variable; otherwise, the life cycle of the local variable ends, the file descriptor is recycled by the system because the reference count is 0 (if the entire function is written in the main function, it does not need to be defined as global ).

2. Open the custom file and lock it

#! /Usr/bin/env python # coding: UTF-8 import fcntl, sys, time pidfile = 0 def ApplicationInstance (): global pidfile = open ("instance. pid "," w ") try: fcntl. lockf (pidfile, fcntl. LOCK_EX | fcntl. LOCK_NB) # create an exclusive lock, and other processes that are locked will not block wait t IOError: print "another instance is running... "sys. exit (0) if _ name _ = "_ main _": ApplicationInstance () while True: print 'running... 'Time. sleep (1)

3. Detect the PID in the file

#!/usr/bin/env python #coding: utf-8 import time, os, sys import signal  pidfile = '/tmp/process.pid'  def sig_handler(sig, frame):   if os.path.exists(pidfile):     os.remove(pidfile)   sys.exit(0)  def ApplicationInstance():   signal.signal(signal.SIGTERM, sig_handler)   signal.signal(signal.SIGINT, sig_handler)   signal.signal(signal.SIGQUIT, sig_handler)    try:    pf = file(pidfile, 'r')    pid = int(pf.read().strip())    pf.close()   except IOError:    pid = None     if pid:    sys.stdout.write('instance is running...\n')    sys.exit(0)    file(pidfile, 'w+').write('%s\n' % os.getpid())  if __name__ == "__main__":   ApplicationInstance()   while True:     print 'running...'     time.sleep(1) 

4. Check for specific folders or files

#!/usr/bin/env python #coding: utf-8 import time, commands, signal, sys  def sig_handler(sig, frame):   if os.path.exists("/tmp/test"):     os.rmdir("/tmp/test")   sys.exit(0)  def ApplicationInstance():   signal.signal(signal.SIGTERM, sig_handler)   signal.signal(signal.SIGINT, sig_handler)   signal.signal(signal.SIGQUIT, sig_handler)   if commands.getstatusoutput("mkdir /tmp/test")[0]:     print "instance is running..."     sys.exit(0)  if __name__ == "__main__":   ApplicationInstance()   while True:     print 'running...'     time.sleep(1) 

You can also detect a specific file to determine whether the file exists:

import os import os.path import time     #class used to handle one application instance mechanism class ApplicationInstance:     #specify the file used to save the application instance pid   def __init__( self, pid_file ):     self.pid_file = pid_file     self.check()     self.startApplication()     #check if the current application is already running   def check( self ):     #check if the pidfile exists     if not os.path.isfile( self.pid_file ):       return     #read the pid from the file     pid = 0     try:       file = open( self.pid_file, 'rt' )       data = file.read()       file.close()       pid = int( data )     except:       pass     #check if the process with specified by pid exists     if 0 == pid:       return       try:       os.kill( pid, 0 )  #this will raise an exception if the pid is not valid     except:       return       #exit the application     print "The application is already running..."     exit(0) #exit raise an exception so don't put it in a try/except block     #called when the single instance starts to save it's pid   def startApplication( self ):     file = open( self.pid_file, 'wt' )     file.write( str( os.getpid() ) )     file.close()     #called when the single instance exit ( remove pid file )   def exitApplication( self ):     try:       os.remove( self.pid_file )     except:       pass     if __name__ == '__main__':   #create application instance   appInstance = ApplicationInstance( '/tmp/myapp.pid' )     #do something here   print "Start MyApp"   time.sleep(5)  #sleep 5 seconds   print "End MyApp"     #remove pid file   appInstance.exitApplication() 

The above OS. kill (pid, 0) is used to detect whether a pid process is still alive. If the pid process has stopped, an exception is thrown. If it is running, no kill signal is sent.

5. socket listens to a specific port

#!/usr/bin/env python #coding: utf-8 import socket, time, sys   def ApplicationInstance():   try:       global s     s = socket.socket()     host = socket.gethostname()     s.bind((host, 60123))   except:     print "instance is running..."     sys.exit(0)  if __name__ == "__main__":   ApplicationInstance()   while True:     print 'running...'     time.sleep(1) 

You can use the decorator to implement this function for reuse (the effect is the same as above ):

#! /Usr/bin/env python # coding: UTF-8 import socket, time, sys import functools # implement def ApplicationInstance (func) using the decorator: @ functools. wraps (func) def fun (* args, ** kwargs): import socket try: global s = socket. socket () host = socket. gethostname () s. bind (host, 60123) failed T: print ('already has an instance... ') return None return func (* args, ** kwargs) return fun @ ApplicationInstance def main (): while True: print 'running... 'Time. sleep (1) if _ name _ = "_ main _": main ()

Iv. Summary

(1) daemon and single script are important in actual application, and there are many methods. You can choose the appropriate one to modify them and make them into a separate class or template, then implement customization by subclass.

(2) The Automatic Recovery of daemon monitoring processes avoids nohup and & usage, and the use of shell scripts can save a lot of trouble for starting and disabling servers from time to time.

The above is all the content of this article. I hope it will be helpful for your learning and support for helping customers.