[Python journey] Article 6 (7): development of simple host batch management tools

[Python journey] Article 6 (7): development of simple host batch management tools
With the learning and use of the Paramiko module, as well as the multi-thread and multi-process knowledge in Python, you can develop a simple host batch management tool. Obviously, when you manage hosts in batches, it is best for programs to execute concurrently, because it can maximize the CPU performance. Therefore, you need to use Python multithreading or multi-process, based on the Learning Needs, here we mainly use multi-process for development. Of course, this will have some problems, which will be explained later. The main content is as follows:

1. host batch management tool function 2. design framework 3. implementation: database information and program source code 4. practical demonstration 5. program insufficiency 6. lessons 7. future improvement ideas

 

1. host batch management tool function the host here mainly refers to the Linux server. The required functions are as follows: (1) Batch command execution can execute commands entered by the Administrator in batches on hosts in the management list through this program. (2) batch file distribution when multiple server hosts need the same file, you can use this program to remotely batch distribute specified files. (3) Support for custom port implementation (1) (2) All functions depend on the Paramiko module, while the Paramiko module is based on SSH, although the SSH port number of most Linux servers uses 22 by default, for security reasons, you can also modify the default port number, such as modifying the SSH remote port number to 52113. (4) The user defined here refers to the user of the program. The program is understood as a batch management system, to use the system, you must have the account and User Name of the system. Each account and user name should have a list of hosts that can be managed according to the permission requirements, for example, ordinary O & M personnel can only manage some server hosts, while the O & M director should be able to manage more hosts, and their management permissions should be different. Therefore, the permissions of the corresponding management system accounts are different. (5) After logging on to the system, the O & M personnel of the logging function should record the operations, time, success or failure of the remote server host. 2. the design framework is based on the needs of the above functions. implementation: database information and program source code according to requirements and design framework, do the following: (1) database information 1) the management system login information database stores the username and password that can be logged on to the system. Only the username that exists here can be logged on, as shown below: The manager_system database is created:

mysql> show databases;+--------------------+| Database           |+--------------------+| information_schema || manager_system     || mysql              |+--------------------+

 

3 rows in set (0.03 sec) creates two different types of tables in the manager_system database:

mysql> use manager_systemmysql> show tables;+--------------------------+| Tables_in_manager_system |+--------------------------+| manager1_server          || manager2_server          || users                    |+--------------------------+

 

3 rows in set (0.00 sec) the users table is used to store user information. The manager‑server table is used to store the list of manageable hosts for users. The users table is used to store system user information:

mysql> describe users;+-----------+------------------+------+-----+---------+----------------+| Field     | Type             | Null | Key | Default | Extra          |+-----------+------------------+------+-----+---------+----------------+| id        | int(10) unsigned | NO   | PRI | NULL    | auto_increment || username  | char(20)         | NO   |     | NULL    |                || password  | char(20)         | NO   |     | NULL    |                || real_name | char(20)         | NO   |     | NULL    |                |+-----------+------------------+------+-----+---------+----------------+

 

4 rows in set (0.01 sec) Table users stores two user information:

mysql> select * from users;+----+----------+----------+-----------+| id | username | password | real_name |+----+----------+----------+-----------+|  1 | manager1 | 123456   | zhangsan  ||  2 | manager2 | 123456   | lisi      |+----+----------+----------+-----------+

 

2 rows in set (0.00 sec) That is to say, only users manager1 and manager2 can log on to the system. Other users cannot log on to the system unless they apply to the Administrator for registration. 2) The management system user host list database still uses the manager_system database, but creates different user-based tables in the database, as follows:

mysql> use manager_systemmysql> show tables;+--------------------------+| Tables_in_manager_system |+--------------------------+| manager1_server          || manager2_server          || users                    |+--------------------------+3 rows in set (0.00 sec)

 

Table [name] _ server is used to store the user's host list:

mysql> describe manager1_server;+-------------+------------------+------+-----+---------+----------------+| Field       | Type             | Null | Key | Default | Extra          |+-------------+------------------+------+-----+---------+----------------+| id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment || ip          | char(20)         | NO   |     | NULL    |                || username    | char(20)         | NO   |     | NULL    |                || password    | char(20)         | NO   |     | NULL    |                || port        | int(11)          | NO   |     | NULL    |                || server_type | char(20)         | NO   |     | NULL    |                |+-------------+------------------+------+-----+---------+----------------+

 

6 The rows in set (0.00 sec) Table stores host information that can be managed by users:

mysql> select * from manager1_server;+----+---------------+-----------+----------+-------+-------------+| id | ip            | username  | password | port  | server_type |+----+---------------+-----------+----------+-------+-------------+|  1 | 192.168.1.124 | oldboy    | 123456   |    22 | DNS Server  ||  2 | 192.168.1.134 | yonghaoye | 123456   | 52113 | DHCP Server |+----+---------------+-----------+----------+-------+-------------+

 

2 rows in set (0.00 sec) where the ip address is the ip address of the remote host, server_type is the server type, and username and password are the user passwords for ssh login to the remote host, as long as the management system user enters the management system and manages the remote host, the user name and password of the remote host do not need to be entered. In addition to convenience, this also has certain security. Note the port number here. We can see that the port number of one host is 22, and the other is 52113, which is the custom port number mentioned above. Therefore, this must be manually specified by the Administrator when adding hosts. (2) After preparing the source code of the program with the above basic data, let's take a look at the source code of the program. Some comments will be given, but based on the previous introduction, the code should also be easier to understand:

Import MySQLdb, OS, paramiko, sys, timefrom multiprocessing import Process, Pool # database connection class Connect_mysql: conn = MySQLdb. connect (host = 'localhost', user = 'root', passwd = '000000', db = 'manager _ system', port = 123456) cur = conn. cursor () def _ init _ (self, username, password = 'null'): self. username = username self. password = password # contect to the login table def login_check (self): # connect to the management system account information database and verify the user name Password information try: self.cur.exe cute ("select * from users where username = '% s' and password =' % S'" % (self. username, self. password) qur_result = self. cur. fetchall () # return the tuple if qur_result = (): # database do not have this user return 0 else: return 1 # database has this user self. cur. close () self. conn. close () distinct T MySQLdb. error, e: print '\ 033 [31; 1 mMysql Error Msg: % s \ 033 [0m' % e # contect to the s Erver table def return_server (self): # connect to the user host list database and return the table information self.cur.exe cute ("select * from % s_server" % self. username) qur_result = self. cur. fetchall () return qur_resultdef ssh_run (host_info, cmd, sysname): # ip address, username, password, port = host_info [1], host_info [2], host_info [3], host_info [4] date = time. strftime ('% Y _ % m _ % D') date_detial = time. strftime ('% Y _ % m _ % d % H: % M: % s') f = file ('. /log/% s _ % s_rec Ord. log '% (sysname, date), 'a +') # operation log record, record the/log directory in all directories of the program try: s. connect (ip, int (port), username, password, timeout = 5) stdin, stdout, stderr = s.exe c_command (cmd) cmd_result = stdout. read (), stderr. read () print '\ 033 [32; 1 m ------------- % s -------------- \ 033 [0m' % ip for line in rows _result: print line, print '\ 033 [32; 1 m ----------------------------- \ 033 [0m' handle T: log = "Time: % s | Type: % s | Detial: % s | Serve R: % s | Result: % s \ n "% (date_detial, 'cmd batch', cmd, ip, 'failed') f. write (log) f. close () print '\ 033 [31; 1 mSomething is wrong of % s \ 033 [0m' % ip else: log = "Time: % s | Type: % s | Detial: % s | Server: % s | Result: % s \ n "% (date_detial, 'cmd batch', cmd, ip, 'success ') f. write (log) f. close () return 1def distribute_file (host_info, file_name, sysname): # ip address of the batch file distribution function, username, password, port = host_info [1], host_info [2 ], Host_info [3], int (host_info [4]) date = time. strftime ('% Y _ % m _ % D') date_detial = time. strftime ('% Y _ % m _ % d % H: % M: % s') f = file ('. /log/% s _ % s_record.log' % (sysname, date), 'a + ') # log record try: t = paramiko. transport (ip, port) t. connect (username = username, password = password) sftp = paramiko. SFTPClient. from_transport (t) sftp. put (file_name, '/tmp/% s' % file_name) t. close () failed T: log = "Time: % s | Type: % s | Deti Al: % s | Server: % s | Result: % s \ n "% (date_detial, 'stribute file', file_name, ip, 'failed') f. write (log) f. close () print '\ 033 [31; 1 mSomething is wrong of % s \ 033 [0m' % ip else: log = "Time: % s | Type: % s | Detial: % s | Server: % s | Result: % s \ n "% (date_detial, 'stribute file', file_name, ip, 'success ') f. write (log) f. close () print "\ 033 [32; 1 mDistribute '% s' to % s Successfully! \ 033 [0 m "% (file_name, ip) OS. system ('clear') print '\ 033 [32; 1 mWelcome to the Manager System! \ 033 [0m' while True: # main program username = raw_input ('username :'). strip () password = raw_input ('password :'). strip () if len (username) <= 3 or len (password) <6: print '\ 033 [31; 1 mInvalid username or password! \ 033 [0m'continue # Begin to login p = Connect_mysql (username, password) mark = p. login_check () if mark = 0: # login failed print '\ 033 [31; 1 mUsername or password wrong! Please try again! \ 033 [0m' elif mark = 1: # login success print '\ 033 [32; 1 mLogin Success! \ 033 [0m 'print 'The server list are as follow:' # seek for the server list managed by the system user p = Connect_mysql (username) server_list = p. return_server () for server in server_list: print '% s: % s' % (server [5], server [1]) while True: print ''' What do you want to do? # Main program menu 1. execute the command batch.2.Distribute file (s) batch.3.Exit. '''choice = raw_input ('\ 033 [32; 1 myur choice: \ 033 [0m '). strip () if '1' <= choice <= '4': pass else: continue # Execute the command batch. if choice = '1': # Execute Command Block s = paramiko in batches. SSHClient () # Call Paramiko module s. load_system_host_keys () s. set_missing_host_key_policy (paramiko. autoAddPolicy () p = Pool (processes = 3) # set the process Pool data result_list = [] While True: cmd = raw_input ('\ 033 [32; 0 mEnter the command (or quit to quit): \ 033 [0m') if cmd = 'quit ': break for h in server_list: result_list.append (p. apply_async (ssh_run, [h, cmd, username]) # the usename is system name # Call related function and execute multi-process concurrency for res in result_list: res. get () s. close () # Distribute file (s) batch. elif choice = '2': # batch distribution file Block s = paramiko. SSHClient () # Call Paramiko module s. load_system_host_key S () s. set_missing_host_key_policy (paramiko. autoAddPolicy () p = Pool (processes = 3) result_list = [] # save the suanfa that come from the apply_async while True: file_name = raw_input ('the file you want to distribute (or quit to quit ):'). strip () start = time. time () if file_name = 'quit': break file_chcek = OS. path. isfile (file_name) log_list = [] if file_chcek = False: print '\ 033 [31; 1 mThe file does n Ot exist or it is a directory! \ 033 [0m'continue for h in server_list: result_list.append (p. apply_async (distribute_file, [h, file_name, username]) # the list save the suanfa for res in result_list: res. get () # run the suanfa end = time. time () print '\ 033 [31; 1 mCost time: % ss \ 033 [0m' % str (end-start) s. close () # Exit the system elif choice = '3': # Exit the system sys. exit ('\ 033 [32; 1 mWelcome to use our system! \ 033 [0m ')

 

The program has a small amount of code and its main functions are also annotated. As long as paramiko simulates SSH command execution and SFTP file distribution, Python multi-process and process pool usage, it is easy to understand.