First, Introduction
Graphite is a python-written web application, using the Django Framework, Graphite used to collect all the server's timely state, user request information, memcached hit rate, rabbitmq the status of the message server, The load state of the UNIX operating system, the graphite server needs to be updated approximately 4,800 times per minute, and the graphite is easy to use on any operating system using a simple text protocol and drawing capabilities.
There are three components of the graphite:
Graphite-web:web interface
Carbon: equivalent to network interface
Whisper: Equivalent to RRDtool
Graphite Official documents:
Http://graphite.wikidot.com/documentation
http://graphite.readthedocs.org/en/latest/
Second, installation graphite
1. Install Epel source
RPM-IVH http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpmsed-i ' [email protected]^#@@ '/ Etc/yum.repos.d/epel.reposed-i ' [email protected]@#[email protected] '/etc/yum.repos.d/epel.repo
2, install the appropriate version of the Django package, the version is too high will appear bug
Yum Install Python-simplejsonwget https://kojipkgs.fedoraproject.org//packages/Django14/1.4.14/1.el6/noarch/ DJANGO14-1.4.14-1.EL6.NOARCH.RPMRPM-IVH django14-1.4.14-1.el6.noarch.rpm
3, installation Graphite
Yum Install Graphite-web Python-carbon python-whisper
4. Install MySQL Database
Yum install MySQL mysql-server mysql-pythonservice mysqld startchkconfig mysqld onmysqladmin-uroot Password 123456mysql- UROOT-P123456-E ' CREATE database graphite; '
5. Modify the Graphite configuration file
# cat >>/etc/graphite-web/local_settings.py << eofsecret_key = ' 123qwe ' allowed_hosts = [' * ']time_zone = ' as Ia/shanghai ' DEBUG = truedatabases = {' default ': {' NAME ': ' Graphite ', ' ENGINE ': ' DJANGO.DB.BACKENDS.MYSQ L ', ' USER ': ' Root ', ' PASSWORD ': ' 123456 ', ' HOST ': ' 127.0.0.1 ', ' PORT ': ' 3306 '}}from graphi Te.app_settings Import *eof
6. Synchronizing the database
Mkdir-p/opt/graphite/storagecd/etc/graphite-web/django-admin syncdb--settings=local_settings--pythonpath=. Yesroot[email protected]123456123456
7. Modify the Graphite Data Catalog
Chown-r Apache.apache/opt/graphite/storage
8. Start the service
/etc/init.d/carbon-cache startchkconfig carbon-cache on/etc/init.d/httpd startchkconfig httpd on
Third, visit the exhibition graphite
1, Chrome browser access to GHIPTE address:
650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/5C/11/wKiom1UaGbvgMgkrAAIEjT1B2Zk181.jpg "title=" 1.png " Width= "650" height= "322" border= "0" hspace= "0" vspace= "0" style= "width:650px;height:322px;" alt= " Wkiom1uagbvgmgkraaiejt1b2zk181.jpg "/>
2. Provide a script to monitor network card traffic
[[email protected] ~]# cat network_traffic.py #!/usr/bin/env pythonfrom Subprocess import popen,pipeimport socketimport shleximport timeimport sysimport osdef get_traffic (f): p = popen (Shlex.split (f), stdout=PIPE,stderr= PIPE) result = p.stdout.read () d = [i For i in result.split (' \ n ') [3:] if i] dic_traffic = {} for i in d: Devname = i.split (': ') [0].strip () receive = i.split (': ') [1].split () [0].strip () Transmit = I.split (': ') [1].split () [8].strip () dic_traffic[devname] = {' in ': Receive, ' out ':transmit} return dic_traffic if __name__ == ' __main__ ': try: HOST = ' 127.0.0.1 ' port = 2003 s = socket.socket () s.connect ((host,port)) except:print couldn ' t connect to % (server) s on port % (port) d, Is carbon-agent.py running? " % {' server ': HOST, ' Port ':P ort}sys.exit (1) while True: cur_traffic = get_traffic (' Cat /proc/net/dev ') time.sleep (5) five_s_traffic = get_traffic (' Cat /proc/net/dev ') diff _dic = {} for k in cur_traffic: traffic_in = int (five_s_traffic[k][' in ']) - int (Cur_traFfic[k][' in ']) traffic_out = int (five_s_traffic[k][' out ') - int ( cur_traffic[k][' out ']) diff_dic[k] = {"in": traffic_in, ' out ': Traffic_out}now = int (Time.time ()) For k,v in diff_dic.items (): net_in = ' network.%s_in %s %s\n ' % (k,v[' in '],now) net_out = ' network.%s_out %s %s\n ' % (k,v[' out '],now) s.sendall (net_in) s.sendall (Net_out) time.sleep (5)
3, backstage mode running monitoring network card traffic script
[email protected] ~]# python network_traffic.py &
Iv. Installation of Diamond
Diamond: Collector, for data collection
Diamond's GitHub official site: Https://github.com/python-diamond/Diamond/wiki
1. Install Diamond
Yum install gcc gcc-c++ python-configobj python-pip pip install diamond==3.4.421 (sometimes not successful) if the download installation is unsuccessful you can use the following method wget https://pypi.python.org/packages/source/d/diamond/diamond-3.4.421.tar.gz#md5= 080ab9f52a154d81f16a4fd27d11093atar XF diamond-3.4.421.tar.gzcd diamond-3.4.421python setup.py Install
2. Configuration
CD/ETC/DIAMOND/CP diamond.conf.example diamond.conf Major modification of three profiles: [[email protected] diamond]# vim/etc/diamond/ Diamond.conf[[graphitehandler]]//59 Line host = localhost[[default]]//173 row interval = 10//time to collect [[email protected] Diamond ]# vim/etc/diamond/handlers/archivehandler.conf#log_file =./storage/archive.log//note this line [[email protected] diamond]# vim/etc/diamond/handlers/graphitehandler.conf host = localhost
3. Start the Diamond service
chmod +x/etc/init.d/diamond/etc/init.d/diamond Startchkconfig Diamond on
Five, continue to visit the display Diamond Automatic collection of information
1, Chrome browser access to GHIPTE address:
You will find that there is a servers directory in the graphite, this directory is the information that Diamond automatically collects
650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/5C/0B/wKioL1UaG8fRUruvAAPm1blb_Gk327.jpg "title=" Diamond.png "width=" 650 "height=" "border=" 0 "hspace=" 0 "vspace=" 0 "style=" width:650px;height:320px; "alt=" Wkiol1uag8fruruvaapm1blb_gk327.jpg "/>2, where two Python scripts are available to collect httpcode of the Web site, is based on the diamond Way
[[email protected] httpcode]# ll total dosage 8-rwxr-xr-x 1 root root 1356 March 11:12 filerev.py-rwxr-xr-x 1 root root 3737 March 31 11:12 httpcode.py
3, run the collection Httpcode script
First delete the servers directory generated by the original diamond [[Email protected] httpcode]# rm -rf /var/lib/carbon/whisper /servers/then manually run the diamond's httpcode script [[Email protected] httpcode]# diamond -f -l -r ./httpcode.py -c /etc/diamond/diamond.conferror: pidfile exists. server already running? #需要手动停止diamond服务 [[Email protected] httpcode]# /etc/init.d/diamond stopStopping diamond: [Determine][[email protected] httpcode]# diamond -f -l -r ./httpcode.py -c /etc/diamond/diamond.conf[ 2015-03-31 11:13:56,198] [mainthread] changed uid: 0 () GID: 0 () . [2015-03-31 11:13:56,198] [mainthread] loaded handler: diamond.handler.graphite.graphitehandler[2015-03-31 11:13:56,201] [MainThread] GraphiteHandler: Established connection to graphite server localhost:2003. [2015-03-31 11:13:56,202] [mainthread] loaded handler: diamond.handler.archive.archivehandler[2015-03-31 11:13:56,206] [mainthread] loading collectors from: . [2015-03-31 11:13:56,209] [mainthread] loaded module: httpcode[2015-03-31 11:13:56,209] [mainthread] loaded collector: httpcode. httpcodecollector[2015-03-31 11:13:56,209] [mainthread] initialized collector: Httpcodecollector[2015-03-31 11:13:56,210] [mainthread] skipped loading disabled collector: httpcodecollector[2015-03-31 11:13:56,210] [mainthread] started task scheduler. [2015-03-31 11:13: 57,211] [mainthread] stopping task scheduler. [2015-03-31 11:14:01,217] [mainthread] stopped task scheduler. [2015-03-31 11:14:01,217] [mainthread] Exiting. If there is no error, the View browser will find a servers directory, but the directory is not generated, I am really puzzled. Originally this configuration was not started in the configuration file [[Email protected] httpcode]# vim httpcode.py......config = super (httpcodecollector, self). Get_default_config () Config.update ({ ' path ': ' Weblog ', ' enabled ': ' True ' #开启此选项 }) if you collect with diamond, you do not need this option, Because diamond has a configuration file for the class, opening in the configuration file will look more uniform than opening in the script
4. Browser view
Chrome refreshes the GHIPTE Web page to see
Ghipte, servers, ec2-54-201-82-69, weblog (custom), HTTP, the following monitoring graphs appear
We can generate 200 status codes using Ab-c 100-n http://localhost/
Use the Refresh Ghipte browser page to generate a 304 status code
650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/5C/0B/wKioL1UaHCijRVkoAAMpNTmsVaU913.jpg "title=" Manual httpcode.png "width=" 650 "height=" 322 "border=" 0 "hspace=" 0 "vspace=" 0 "style=" width:650px;height:322px; "alt=" Wkiol1uahcijrvkoaampntmsvau913.jpg "/>
At present, the mainstream open source monitoring has cacti, Nagios, Zabbix and so on, the community is active, powerful
Graphite Although functionally and in the community can not be compared with this, but in the flexibility is worth mentioning, the lightweight monitoring program, more importantly, graphite is written by Python, so in the troubleshooting, scripting, etc. will be very handy
Also very grateful for more Python open source contributions!!!
This article is from the "Zheng" blog, make sure to keep this source http://467754239.blog.51cto.com/4878013/1626736
Python Performance monitoring Graphite