Server Status Monitoring based on embedded webserver

In fact, this embedded webserver is used for server status monitoring during easyhadoop's second refactoring. You can extract and write something separately.

The main idea is to use a python script to obtain various status information of the linux server, and then use webserver to send json data to http. The master node reads the json returned by webserver and generates a system monitoring report. The code is simple and convenient for development and deployment.

The main thing to use is the third-party embedded web module of python called cherrypy. The main reason for choosing cherrypy is that development is fast and learning is fast, it took me more than a day to learn how to write. Of course, you can also use simpleHTTPserver, which comes with python. But that is really too simple. Cherrypy has the advantages of multi-thread and multi-concurrency. It is not as heavyweight as Tornado and Django. Because we return json, we cannot use any html template or database functions. You can also choose web. py, but it is better than cherrypy. I think web. py also draws on many ideas of cherrypy.

In fact, Ganglia, Cacti, and Nagios are installed in all my clusters. But I want to see the real-time chart generation and write such a program myself. A friend who is doing automated O & M can serve as a reference and add his/her own methods, so can he monitor others.

The main process is as follows:

How can I read various data in Linux? --- parse data into json and send it to http server ---> monitor server scan data to generate charts

#! /Usr/bin/python #-*-coding: utf8-*-import sysimport cherrypyimport platformimport osimport time # python 2.4 is simplejson, python 2.6 or above is jsontry: import jsonexcept ImportError: import simplejson as json # Pretend to make an index out class Index (object): # The following sentence indicates the index method, which is exposed to the http server @ cherrypy. expose def index (self): return "hello cherrypy" class Node (object): ''' url/node/dist/''' # obtain the Distribution Branch and version number of the target machine, architecture type, Master The host name and so on. json @ cherrypy is returned. expose def dist (self): dist_json = ''sysinstaller ='' installer = ''ostype = platform. dist () if (ostype [0] in ['ubuntu ', 'debian', 'ubuntu ', 'debian']): sysinstaller = 'apt-get' installer = 'dpkg 'elif (ostype [0] in ['suse']): sysinstaller = 'zypper 'installer = 'rpm 'elif (ostype [0] in ['cento', 'cento', 'redhat ', 'redhat']): sysinstaller = 'yum 'installer = 'rpm' machine = p Latform. machine () hostname = platform. node () dist_json = {'OS. system ': ostype [0],' OS. version': ostype [1], 'OS. release ': ostype [2],' OS. sysinstall ': sysinstaller,' OS. installer ': installer,' OS. arch ': machine,' OS. hostname': hostname} return json. dumps (dist_json, sort_keys = False, indent = 4, separators = (',',':')) '''url/node/GetCpuInfo/''' # obtain the CPU model, and return json @ cherrypy. expose def GetCpuInfo (self): cpu = [] Cpuinfo = {} f = open ("/proc/cpuinfo") lines = f. readlines () f. close () for line in lines: if line = 'N': cpu. append (cpuinfo) cpuinfo ={} if len (line) <2: continue name = line. split (':') [0]. strip (). replace ('', '_') var = line. split (':') [1]. strip () cpuinfo [name] = var return json. dumps (cpuinfo, sort_keys = False, indent = 4, separators = (',',':')) '''url/node/GetMemInfo/''' # obtain the memory usage details @ cherr Ypy. expose def GetMemInfo (self): mem = {} f = open ("/proc/meminfo") lines = f. readlines () f. close () for line in lines: if len (line) <2: continue name = line. split (':') [0] var = line. split (':') [1]. split () [0] mem [name] = long (var) * 1024.0 mem ['memused'] = mem ['memtotal']-mem ['memfree']-mem ['buffers']-mem ['cached'] return json. dumps (mem, sort_keys = False, indent = 4, separators = (',',':'))''' Url/node/GetLoadAvg // ''' # obtain the system load details @ cherrypy. expose def GetLoadAvg (self): loadavg = {} f = open ("/proc/loadavg") con = f. read (). split () f. close () loadavg ['lavg _ 1'] = con [0] loadavg ['lavg _ 5'] = con [1] loadavg ['lavg _ 15'] = con [2] loadavg ['nr '] = con [3] loadavg ['last _ pid'] = con [4] return json. dumps (loadavg, sort_keys = False, indent = 4, separators = (',', ':') ''' url/node/GetIfInfo/eth (x) ''' gets the traffic information of the specified Nic, This is a bit complicated @ cherrypy. expose def GetIfInfo (self, interface): dist_json = self. dist () f = open ("/proc/net/dev") lines = f. readlines () f. close () intf ={} for line in lines [2:]: con = line. split () # if is used for centos. When the traffic of centos is high, the strings in the NIC information will be connected, so you need to split them separately, else is formatted in ubuntu or other systems. if con [0] [-1] is used properly. isdigit () = True: offset = con [0]. split (':') intf ['interface'] = str (offset [0]) intf ['receivebytes '] = str (Fset [1]) intf ['incluepackets '] = str (con [1]) intf ['receiveerrs'] = str (con [2]) intf ['incluedrop'] = str (con [3]) intf ['receivefifo '] = str (con [4]) intf ['receiveframes '] = str (con [5]) intf ['receivecompressed'] = str (con [6]) intf ['external'] = str (con [7]) intf ['transmitbytes '] = str (con [8]) intf ['transmitpackets '] = str (con [9]) intf ['transmiterrs'] = str (con [10]) intf ['transmitdrop'] = str (Con [11]) intf ['transmitfifo '] = str (con [12]) intf ['transmitframes'] = str (con [13]) intf ['transmitcompressed '] = str (con [14]) intf ['transmitmulticast'] = str (con [15]) else: intf ['interface'] = str (con [0]) intf ['receivebytes '] = str (con [1]) intf ['incluepackets '] = str (con [2]) intf ['receiveerrs'] = str (con [3]) intf ['incluedrou'] = str (con [4]) intf ['receivefifo '] = str (con [5]) intf ['receiveframes'] = Str (con [6]) intf ['receivecompressed '] = str (con [7]) intf ['receivemulticast'] = str (con [8]) intf ['transmitbytes '] = str (con [9]) intf ['transmitpackets'] = str (con [10]) intf ['transmiterrs'] = str (con [11]) intf ['transmitdrop'] = str (con [12]) intf ['transmitfifo '] = str (con [13]) intf ['transmitframes'] = str (con [14]) intf ['transmitcompressed '] = str (con [15]) intf ['transmitmulticast'] = str (con [16]) ret Urn json. dumps (intf, sort_keys = False) # obtain the interface and traffic information of all NICs @ cherrypy. expose def GetIfTraffic (self): ifs = [] nettraffic = {} f = open ("/proc/net/dev") lines = f. readlines () f. close () for line in lines [2:]: con = line. split () ifname = con [0]. split (':') if (ifname [0]. strip ()! = 'Lo'): ifs. append (ifname [0]. strip () else: continue for interface in ifs: nettraffic [interface] = self. getIfInfo (interface) return json. dumps (nettraffic) # obtain the partition information and usage of the hard disk @ cherrypy. expose def GetHddInfo (self ): hdds = [] mount = {} file_system = [] type = [] size = [] used = [] avail = [] used_percent = [] mounted_on = [] hdds = OS. popen ('df-lhT | grep-v tmpfs | grep-v boot | grep-v usr | grep -V tmp | sed \ '1d ;//! N; s/\ n //; s/[] * []/\ t/g ;\''). readlines () for line in hdds: file_system.append (line. replace ('\ n ',''). replace ('\ t ',''). split () [0]) type. append (line. replace ('\ n ',''). replace ('\ t ',''). split () [1]) size. append (line. replace ('\ n ',''). replace ('\ t ',''). split () [2]) used. append (line. replace ('\ n ',''). replace ('\ t ',''). split () [3]) avail. append (line. replace ('\ n ',''). replace ('\ t ',''). split () [4]) used_percent.append (line. replace ('\ n ',''). replace ('\ t ',''). split () [5]) mounted_on.append (line. replace ('\ n ',''). replace ('\ t ',''). split () [6]) mount ['file _ system'] = file_system mount ['type'] = type mount ['SIZE'] = size mount ['used'] = used mount ['avail '] = avail mount ['used _ percent '] = used_percent mount ['mounted _ on'] = mounted_on dist_json = json. dumps (mount) return dist_json # obtain CPU usage information. You need to install sysstat to support @ cherrypy. expose def GetCpuDetail (self): dist_json = self. dist () dist = json. loads (dist_json) if (dist ['OS. system '] in ['centos', 'centos ', 'redhat', 'redhat']): if (int (dist [' OS. version']. split ('. ') [0]) <6): # For CentOS only cmd = 'mpstat 1 1 | sed \ '1d; 2d; 3d; 4d \ '| awk \' {print "{\\" user \\\ ":\\" \ "$3 \"\\\", \ "nice \": \ "\" $4 \ "\", \ "sys \\\": \\\" \ "$5 \" \\\ ", \\\" iowait \\\ ": \\\" \ "$6 \"\\\", \ "irq \": \ "\" $7 \ "\", \ "soft \\\": \ "\" $8 \ "\", \ "steal \": \ "\" $9 \"\\\", \ "idle \": \ "\" $10 \ "\"} "} \ ''else: cmd = 'mpstat 1 1 | sed \ '1d; 2d; 3d; 4d \ '| awk \' {print "{\\" user \\\": \ "\" $3 \ "\", \ "nice \": \ "\" $4 \"\\\", \ "sys \": \ "\" $5 \ "\", \ "iowait \\\": \ "\" $6 \ "\", \ "irq \": \ "\" $7 \"\\\", \ "soft \": \ "\" $8 \ "\", \ "steal \\\": \\\" \ "$9 \" \\\ ", \\\" idle \\\": \ "\" $11 \ "\"} "} \ ''else: cmd = 'mpstat 1 1 | sed \ '1d; 2d; 3d; 4d \ '| awk \' {print "{\\" user \\\ ":\\" \ "$3 \"\\\", \ "nice \": \ "\" $4 \ "\", \ "sys \\\": \\\" \ "$5 \" \\\ ", \\\" iowait \\\ ": \\\" \ "$6 \"\\\", \ "irq \": \ "\" $7 \ "\", \ "soft \\\": \ "\" $8 \ "\", \ "steal \": \ "\" $9 \"\\\", \ "idle \": \ "\" $11 \ "\"} "} \'' cpu = OS. popen (cmd ). readline (). strip () return cpuif "_ main _" = _ name __:# server configuration settings = {'global': {# bind port 'server. socket_port ': 60090, # Set the ip address. If it is safe enough, use 0.0.0.0. Otherwise, write the ip address of the server 'server. socket_host ': '0. 0.0.0 ', 'server. socket_file ': '', 'server. socket_queue_size ': 100, 'server. protocol_version ': 'http/1.1', 'server. log_to_screen ': True, 'server. log_file ': '', 'server. reverse_dns ': False, 'server. thread_pool ': 200, 'server. environment ': 'production', 'Engine. timeout_monitor.on ': False }}# configure and map routes and start webserver cherrypy. config. update (settings) cherrypy. tree. mount (Index (), '/') cherrypy. tree. mount (Node (), '/node') cherrypy. engine. start ()

It doesn't matter if the chart generation end writes data in any language. The data is in json format.

Of course, I also use him to monitor hadoop and hbase. Add some code related to hadoop and hbase.

This article was posted on the "practice test truth" blog and declined to be reproduced!