Tsar Open Source: Taobao's extensive use of system monitoring tools

Tsar is a collection of information on the server system and application of the collection reporting tools, such as the collection of server System information (CPU,MEM, etc.), as well as application data (Nginx, Swift, etc.), the collected data stored on the server disk, can always query historical information, Data can also be sent to the Nagios alarm.

Tsar can be more convenient to add modules, only need to follow the requirements of Tsar to write data collection function and presentation function, you can add the custom module to the Tsar.

Tsar Installation

Tsar is currently hosted on GitHub, download the compile and install steps:

$git clone git://github.com/kongjian/tsar.git
$cd tsar
$make
$make Install

After installation:

Scheduled Task configuration:/etc/cron.d/tsar, responsible for every minute call Tsar perform the acquisition task; log file rotation configuration:/etc/logrotate.d/tsar, the local storage of Tsar will be rotated every month; Tsar Profile path:/etc/ Tsar/tsar.conf,tsar Collection module and output of the specific configuration; module path:/usr/local/tsar/modules, dynamic Library so file for each module;

Tsar Configuration

Tsar just installed, there is no historical data, want check is normal, execute tsar-l, see if there is real-time information output:

Tsar configuration is mainly in the/etc/tsar/tsar.conf, commonly used are:

Add a module that adds mod_<yourmodname> on to the config file to open or close a module, modify mod_<yourmodname> on/off Output_stdio_mod The ability to configure the output module when executing Tsar Output_file_path the file to which the data was collected by default (if modified, it needs to modify the rotation configuration/etc/logrotate.d/tsar) output_interface Specify the data output purpose of the Tsar, the default file is saved locally, nagios/db output to the Monitoring Center/database, these two functions also need to be combined with other configuration, see the following

Tsar use

View historical data, tsar-l/--list view available modules List-l/--live View real-time data, tsar-l--cpu-i/--interval specify interval, history, tsar-i 1--cpu--modname specified module, Tsar- -CPU-S/--SPEC Specifies the field, Tsar--cpu-s sys,util-d/--date The specified date, YYYYMMDD or n represents n days ago-c/--check View the last acquisition data-d/--detail be able to specify the view primary field or Are all fields of the module-h/--help help function

Advanced Features

Output to Nagios

Configuration: First configure Output_interface File,nagios to increase Nagios output

Then configure the Nagios server and port, as well as the time between sending

# # # #The IP address or the host running the NSCA daemon
server_addr nagios.server.com
# # # #The port on which the Dae Mon is Running-default is 5667
server_port 8086
# # #The cycle of Send alert to Nagios
cycle_time 300

Because it is Nagios's passive monitoring mode, you need to specify the location of the NSCA and the configuration file location

# # # #nsca client program
Send_nsca_cmd/usr/bin/send_nsca
send_nsca_conf/home/a/conf/amon/send_nsca.conf

Next specify which modules and fields need to be monitored, and a total of four thresholds correspond to different alarm levels in Nagios

# # # #tsar mod alert config file
# # # #threshold [hostname.] Servicename.key;w-min;w-max;c-min;cmax;
Threshold cpu.util;50;60;70;80;

Output to MySQL

Configuration: First configure Output_interface file,db, increase db output

Then configure which module data needs to be output

Output_db_mod Mod_cpu,mod_mem,mod_traffic,mod_load,mod_tcp,mod_udpmod_io

Then configure the destination address and port that the SQL statement sends

Output_db_addr console2:56677

The destination address listens to the TCP data on this port, and the data is put into storage, can refer to tsar2db.

Module development

A good feature of Tsar is the ability to increase their own collection, this time need to write module code, compiled so file.

First installs the Tsardevel, when installs, if executes makes Tsardevel, will install the module development basic file to the system and then executes the tsardevel, can generate a module directory in the current module:

[KONGJIAN@V132172.SQA.CM4 tsar]$ tsardevel test
build:make
install:make Install
uninstall:make Uninstall
[kongjian@v132172.sqa.cm4 tsar]$ ls test
Makefile  mod_test.c  mod_test.conf

Modify the Read_test_stats,set_test_record in mod_test.c as required make;make install to complete the new module configuration file and so settings, execute Tsar--test can see the effect.

Alternatively, you can pass parameters to the custom module through the configuration file by modifying the mod_test on Myparameter in the configuration file and then in the Read_test_stats function in Mod_test.c, You can get the contents of the configuration file just by using the parameter parameter.