Python scripts for cluster detection and management

Python scripts for cluster detection and management
The python script implements cluster detection and management in the following scenario: A production machine room has many testing machines and production machines (about 30 machines ), because of the chaotic management, it is unclear which machine is used and which machine is not used, which leads to an idea: Use one machine to manage all machines, recording Device owner, device usage status, and other information .... so why python is simple enough and supports a wide range of third-party libraries. My initial thought was that I had never been in touch with these things since I was just a member of my work. I had to make things when I moved to a certain Department. (The project is okay, so I have to make things, you can't try it out )... I wanted to create a simple automated testing tool. However, this general testing method and test case cannot be used for the moment (neither input nor output can be determined... so what do you do? I often find my colleagues asking 208 who uses it? Who uses 201? That IP is mine !!! Have you unplugged my network cable? 242 which machine is it? Suddenly, spring is coming. Can I use a system to detect IP addresses and device users, or even run a script or command on a certain device as needed? After talking about this short and poor idea with leader, make sure you can do it. Let's get started !!! Design Concept the general idea of the system: 1. to obtain various information about all servers, you need to deploy an agent on any server as the node for obtaining information, and regularly send server information data to the management server node. 2. As a comprehensive management node, the server receives and stores information submitted by the agent. 3. For ease of use, Web pages are used for presentation. Select 1 as the development tool. development language: python is easy to use, and a wide range of third-party libraries are not required. database: mysql is simple and easy to use. webpy: The web framework is simple and easy to deploy. bootstrap: the front-end framework does not care too much about front-end issues. 5. paramiko: python library, which complies with the SSH2 protocol and supports remote server connections in the form of encryption and authentication. Connect to the agent server through SSH: remotely run commands and transmit files. 6. scapy: python library, which can be used to send, sniff, parse, and forge network data packets. This is used to scan IP address 7. mySQLdb: connect to mysql 8. shell and python script interfaces: Provide other users with shell script interfaces. 1. the front-end is a new thing for me. I have never done it before. The transition between page animation effects and script runtime needs to be considered. I started to consider using countdown, but this time is uncontrollable, later, ajax was used to solve this problem.. The agent must be automatically deployed on each machine, and the refresh time can be controlled through the server. 3. creating a scalable table is very important, and some important information needs to be written to the disk. When the database fails, you can obtain data from the disk. the connection to the database will time out if no operation is performed for a long time ...... project Structure -- webpy 1. website. py is the main program of webpy, and url ing is set. model. py is the url ing Class of webpy, processing requests and returning 3. static storage of static resources 4. scripts is used to store the processing scripts. the names mentioned here are some problems connecting to the database using MyQSLdb to connect to mysql. Here I didn't use the database interface provided by webpy, instead, it encapsulates a set of ssh remote connection server paramiko for ssh connection, data transmission, command execution, and script def executecmd (cmd, host, port = 22, user = 'root ', Passwd = 'root'): try: s = paramiko. SSHClient () s. set_missing_host_key_policy (paramiko. autoAddPolicy () s. connect (host, port, user, passwd, timeout = 10) except t Exception as e: s. close () print e print 'connet error... 'Return try: stdin,stdout,stderrcmds.exe c_command (cmd) # print 'host: % s ...... '% host res = stdout. readlines () failed t Exception as e: print 'exec _ commmand error... 's. close () return res d Ef executefile (file, host, port = 22, user = 'root', passwd = 'root'): try: s = paramiko. SSHClient () s. set_missing_host_key_policy (paramiko. autoAddPolicy () s. connect (host, port, user, passwd, timeout = 5) t = paramiko. transport (host, port) t. connect (username = user, password = passwd) sftp = paramiko. SFTPClient. from_transport (t) failed t Exception as e: s. close () print e print 'connet error... 'Return ''' try: Filename = OS. path. basename (file) if filename. find ('. sh')> = 0: sftp. put (path + '/' + file, '/tmp/tmp_test.sh') stdin,stdout,stderrcmds.exe c_command ('sh/tmp/tmp_test.sh 2>/dev/null', timeout = 5) else: sftp. put (path + '/' + file, '/tmp/tmp_test.py') stdin,stdout,stderrcmds.exe c_command ('python/tmp/tmp_test.py ', timeout = 5) initialize stdin,stdout,stderrcmds.exe c_command ('rm-rf/tmp/tmp_test * 2>/dev/null') res = stdout. re Adlines () s.exe c_command ('rm-rf/tmp/tmp_test * 2>/dev/null') except t Exception as e: s.exe c_command ('rm-rf/tmp/tmp_test * 2>/dev/null') print 'timeout error... 'print e return ''return res copy code IP scan use scapy for IP scan copy code def pro (ip, cc, handle): global dict dst = ip + str (cc) packet = IP (dst = dst, ttl = 20)/ICMP () reply = sr1 (packet, timeout = TIMEOUT) if reply: print reply. src, 'is online' tmp = [1, repl Y. src] handle. write (reply. src + '\ n') # handle. write (reply. src + "is online" + "\ n") def main (): threads = [] ip = '2017. 168.1.1's = 2 e = 254 f = open ('IP. log', 'w') for I in range (s, e): t = threading. thread (target = pro, args = (ip, I, f) threads. append (t) print "main Thread begins at", ctime () for t in threads: t. start () for t in threads: t. join () print "main Thread ends at", ctime () Copy code batch add ssh-key copy code home _ Dir = '/home/xx' id _ rsa_pub =' % s/. ssh/id_rsa.pub '% home_dir if not id_rsa_pub: print 'id _ rsa. pub Does not exist! 'Sys. exit (0) file_object = open ('% s /. ssh/config '% home_dir, 'w') file_object.write ('stricthostkeychecking no \ n') file_object.write ('userknownhostsfile/dev/null') file_object.close () def up_key (host, port, port, user, passwd): try: s = paramiko. SSHClient () s. set_missing_host_key_policy (paramiko. autoAddPolicy () s. connect (host, port, user, passwd) t = paramiko. transport (host, port) t. connect (username = user, p Assword = passwd, timeout = 3) sftp = paramiko. SFTPClient. from_transport (t) print 'create Host: % s. ssh dir ...... '% host stdin,stdout,stderrcmds.exe c_command ('mkdir ~ /. Ssh/') print 'upload id_rsa.pub to Host: % s ...... '% host sftp. put (id_rsa_pub, "/tmp/temp_key") stdin,stdout,stderrcmds.exe c_command ('cat/tmp/temp_key> ~ /. Ssh/authorized_keys & rm-rf/tmp/temp_key ') print 'host: % s @ % s auth success! \ N' % (user, host) s. close () t. close () failed t Exception, e: # import traceback # traceback. print_exc () print 'connect error... 'print 'delete' + host + 'from database... 'delip (host) # delete from mysql *** try: s. close () t. close () failed T: pass