Logs primarily include system logs, application logs, and security logs. System operations and developers can use the log to understand the server hardware and software information, check the configuration process errors and the cause of the error occurred. Frequently analyze logs to understand the load of the server, performance security, so as to take timely measures to correct errors.
Typically, the logs are stored on different devices that are scattered. If you manage hundreds of dozens of of servers, you are also using the traditional method of logging in to each machine in turn. This is not feeling very cumbersome and inefficient. It is imperative that we use centralized log management, for example: Open source syslog, to summarize log collection on all servers.
Centralized management of the log, log statistics and retrieval has become a more troublesome thing, generally we use grep, awk and WC and other Linux commands to achieve retrieval and statistics, but for higher requirements of query, sorting and statistics and the large number of machines still use such a method is a little too hard.
Open source real-time log analysis Elk platform can perfectly solve our problems above, elk by Elasticsearch, Logstash and Kiabana three open source tools. Official website: https://www.elastic.co
Elasticsearch is an open source distributed search engine, it features: distributed, 0 configuration, automatic discovery, Index auto-shard, index copy mechanism, RESTful style interface, multi-data source, automatic search load, etc.
Logstash is a fully open source tool that collects, analyzes, and stores your logs for later use (e.g., search)
Kibana is also an open source and free tool that Kibana can provide for Logstash and ElasticSearch log Analytics friendly Web interface that can help you summarize, analyze, and search for important data logs.
This article is from "Walker--->" blog, please be sure to keep this source http://liumissyou.blog.51cto.com/4828343/1850973
ELK (Elasticsearch+logstash+kibana) Log Analysis tool