Rapid cluster deployment using python and shell

Recently, ganglia has been deployed one by one in the cluster. cacti is very troublesome and has written a small program for automatic batch deployment. The principle is that python pexpect uses ssh to copy files and execute system commands. I use it to deploy ganglia and so on, but other things can also be deployed in batch through this script, you only need to write the deployment script by yourself. It mainly proposes a solution to check whether it is helpful to everyone.

First, let's define a concept. The server that stores python and scripts is called the master node or server, and the node to be installed is called the controlled node or client. Server and client are used as the proxies below.

The first is the configuration file. I need to first define a configuration file to specify some paths of the server and the specific information of the client.

#-*-Coding: UTF-8-*-log_dir = '. /logs/'# defines the Log Path, but I haven't written it yet, we plan to use log4py for client_tmp_dir = '/tmp/' # define the script storage path ssh_port = '22' # SSH port script_dir = 'on the client '. /shells/'# server-side script storage path node_list = [{'IP': '2017. 168.1.1 ', 'user': 'root', 'passwd': '000000', 'cmd': 'sh/tmp/dpkg_client_ubuntu_x.x86_64.sh '}, # cmd refers to the Command executed on the client. Other statements are not interpreted as {'IP': '100. 168.1.2 ', 'user': 'root', 'passwd': '000000', 'cmd': 'sh/tmp/dpkg_client_ubuntu_x.x86_64.sh'},]

Next is the main program

#! /Usr/bin/env python #-*-coding: UTF-8-*-import osimport sysimport platform # Use the import method to introduce conf. pyimport confimport subprocess ''' determines the server (local) operating System type and version class ''' class System: def GetBranch (self): Branch = platform. dist () [0] return Branch def GetRelease (self): Release = platform. dist () [1] return Release def GetInstaller (self): if self. getBranch () in ['ubuntu ', 'debian']: installer = 'apt-get' elif self. getBranc H () in ['redhat', 'fedora ', 'cento']: installer = 'yum' elif self. getBranch () in ['suse']: installer = 'zypper 'else: installer = 'unknown 'Return install''' get the corresponding pexpect package based on the operating system version and try to introduce it. Because pexpect is not installed on the default Operating System, this part supports RH, Ubuntu, debian, SuSE ''' try: import pexpectexcept ImportError: installer = System () inst = install. getInstaller () if (inst = 'apt-get') or (inst = 'zypper '): cmd =' % s install python-pex Pect '% (inst) elif inst = 'yum': cmd = '$ s install p;ct' % (inst) else: cmd = 'echo "Not support yet :)"'; try: fd = subprocess. popen (cmd, shell = True, stdin = subprocess. PIPE, stdout = subprocess. PIPE, stderr = subprocess. PIPE) out = fd. stdout. readlines () err = fd. stderr. readlines () all = out + err all = "". join (all) failed t OSError, e: all = "Cannot run command, Exception:" + e + OS. linesep import pexpec T # print all '''pexpect execution class, divided into two methods, ssh and scp, automatically determine whether to connect for the first time, and automatically complete yes or enter a password for the response. '''Class login CT: # define the ssh method. The entry variables include ip, port, username, password, and run command def ssh (self, ip, port, user, passwd, cmd ): # create a connection sub-process object ssh = pexpect. spawn ('ssh-p % s @ % s "% s" '% (port, user, ip, cmd) r = ''try: # determine whether to connect for the first time. If it is the first time, answer yes and enter the password. Otherwise, enter the password I = ssh. continue CT (['password: ', 'continue connecting (yes/no )? '], Timeout = 5) if I = 0: ssh. sendline (passwd) elif I = 1: ssh. sendline ('yes') ssh. verify CT ('password: ') ssh. sendline (passwd) Does T pexpect. EOF: ssh. close () else: r = ssh. read () ssh. secondary CT (pexpect. EOF) ssh. close () return r # define the scp method. The entry variables include ip, port, username, and password. The file name to be copied to the client must be copied to the client path def scp (self, ip, port, user, passwd, srcfile = "index.html", distpath): # create a connection sub-process object ssh = pexpect. spawn ('SC P-P % s @ % s: % s' % (port, file, user, ip, distpath) r = ''try: # determine whether to connect for the first time. If it is the first time, answer yes and enter the password. Otherwise, enter the password I = ssh. continue CT (['password: ', 'continue connecting (yes/no )? '], Timeout = 5) if I = 0: ssh. sendline (passwd) elif I = 1: ssh. senline ('yes') ssh. verify CT ('password: ') ssh. sendline (passwd) Does T pexpect. EOF: ssh. close () else: r = ssh. read () ssh. secondary CT (pexpect. EOF) ssh. close () return r # create objects in the conf file to facilitate writing. Use packages = conf directly without creation. package_dirlogs = conf. log_dirc_tmp = conf. client_tmp_dirport = conf. ssh_portscripts = conf. script_dirnodes = conf. node_listexpect = Expect () # execute the server script on the local machine. This script will install Ganglia gmetad, gmond, cacti, nagios, gangliaweb, mysql, apache, and other operating systems. system ("sh" + scripts + "dpkg_server_ubuntu_x.x86_64.sh") # list the hosts, user names, and passwords configured in the conf list cyclically. Run the command for I in range (len (nodes )): ip = nodes [I] ['IP'] user = nodes [I] ['user'] passwd = nodes [I] ['passwd'] cmd = nodes [I] ['cmd'] # copy the script executed by the local client to the conf at the client end. the path defined in py is client_tmp_dir r = wrong CT. scp (ip, port, user, passwd, scripts + 'dpkg _ client_ubuntu_x.x86_64.sh ', c_tmp) print r # Run the copied script on the client, which contains gmond, nagios-client, snmpd, and so on r = CT. ssh (ip, port, user, passwd, cmd) print r

I have not written the logic for automatically determining whether the client operating system is installed with different scripts. If you are interested, you can modify the script yourself. In fact, the script itself is not difficult. The core is pexpect. I mainly don't want to use puppet, the main reason I don't want to use puppet is that I don't want to learn ruby or his complicated configuration files. Not only is monitoring deployment, but you can write shell scripts on your own and deploy anything at will. Nginx and php can all be done by script.

This article was posted on the "practice test truth" blog and declined to be reproduced!