The continuous progress of information technology, on the one hand, makes the banking information and data logical concentration continuously improve, on the other hand, it becomes a security hidden danger of the banking steady operation. As an intelligent IT management operation and maintenance platform, Splunk can help the banking industry to meet, respond and solve the emerging risks, perfect IT system, establish good risk management, improve risk control ability, and realize the new development of banking in Network economy era.
What is Splunk? Splunk is a powerful log management tool that can add logs, produce graphical reports, and, most of all, its search capabilities-known as "Google for IT"-in a variety of ways. Splunk has a free and paid version, the main difference is the Daily Index capacity (index is the basis of the search function), the free version of the maximum of 500M per day. When using the free version, if within 30 days, 7 days of index data amount of more than 500M, then you can not search (it's a pity!). ）。 Depending on your needs, you can choose to purchase a daily index capacity.
Splunk is in the difficult situation of this technology emerged, with flexible, agile data management capabilities for the enterprise to bring an innovation revolution in IT management:
support for data formats across platforms and applications
Supports the processing of all it data in the form of text, such as standard syslog and unstructured event logs, SNMP events, XML literals, system and application configurations, and Splunk can choose the most appropriate input method for data from different sources.
As shown in the image on the right, splunk file upload, share directory and other ways to index data analysis and real-time monitoring; Network devices and application server-generated logs can be collected through the TCP/UDP port listening method Even the data generated by the script run can be indexed and analyzed as splunk data sources.
fast data definition and transformation
The prerequisite for effective management of IT data is to extract valuable information from the data for definition, search, and in-depth analysis.
Compared with the traditional IT management platform, Splunk's definition of data is flexible and fast, it can quickly define the data of interest according to the change of the actual situation, and the whole process simply counts down the mouse and spends a few minutes on the management interface. The following is an example of defining a log source host: Splunk provides the ability to extract and define data intelligently, simply by filling in sample data with a sample of the data that needs to be defined, Splunk can use regular expressions to intelligently analyze the structure of the data, relative position, etc. Help managers quickly identify and extract useful information for definition and analysis.
Of course, if you are a familiar regular expression manager, you will have a deeper understanding of Splunk's flexible and powerful capabilities. Because of the different standards of various network equipment manufacturers, the physical characteristics of the interface, network device interface There are many representations, how to strictly define the interface of all the devices without creating omissions? This is a headache for many IT managers. In Splunk, simply enter a short, short regular expression in the search bar, and all network device logs contain network interfaces that can be clean sweep.
All data definitions can be added, deleted, and modified in real time as the situation changes and management needs, and the only thing managers need to do is reuse the Smart data definition feature or make minor changes to regular expressions.
Simple and flexible search language
Splunk an interesting nickname-"Google for IT," which implies that IT managers in Splunk can be as easy to manage data search as it is with Google search engines.
In the search bar, simply enter fail*, select the time range, and all events related to the failure of the operation are returned as results in a matter of seconds. Based on the results of the search, we can do further in-depth analysis, which simply selects the value of a specific field on the left (defined data): This data drill-down search in the problem location of the efficiency of the traditional IT management platform has a huge advantage, the layer of problem-based troubleshooting will become a legacy, Gone.
Fast report generation and content-rich dashboard
Search results in Splunk can be quickly converted into reports saved, and saved reports can be used as technical and managerial knowledge or real-time monitoring. Search statements and reports saved in Splunk are organized to form a rich collection of knowledge based on specific types and requirements, and are presented as dashboard.
How do I install Splunk? Splunk supports a variety of operating systems, but if you want to collect Windows logs through WMI, Splunk must be installed on the Windows operating system. I use the Windows 2003 Standard Server here, the specific installation steps are very simple, according to the wizard step-by-step.
How do I configure Splunk? As mentioned earlier, Splunk can collect logs in a number of ways, mainly including listening for syslog messages, accessing WMI, monitoring log files, and FIFO queues. I'll take a few typical configurations here: 1) configuring logs for Cisco network devices through a syslog configuration commands on Cisco network devices are generally: logging <syslog server IP address>logging Trap <severity>splunk uses UDP 514 ports to listen for syslog messages by default. such as: Logging 172.29.1.1logging trap warning2) through the syslog to collect the Linux host log on the Linux host configuration is generally: Modify the/etc/syslog.conf configuration, add the following two lines: # Send Syslog to Splunk Server
*.<severity> @<syslog server IP address> such as: # Send syslog to Splunk server
*.debug @172.29.1.13) collects logs for Windows hosts through WMI
- First make sure that the account that runs the Splunk service (shown as SPLUNKD in Service Manager) has permission to read WMI information from the remote Windows machine.
- Then just do a simple configuration on the Splunk server: the Splunk installation path defaults to C:\Program Files\splunk. Under C:\Program files\splunk\etc\system\local file, modify the inputs.conf file to add the following:
[script://$SPLUNK _home\bin\scripts\ SPLUNK-WMI.PY]
Interval = 10
Source = WMI
SourceType = WMI
Disabled = 0
- Then create a new text file in the same directory, named Wmi.conf, and add the following:
[wmi:< Name;]
Server = < Remote Windows Host IP Address>
Interval = 60
Event_log_file = < Event log Type>
Disabled = 0 For example, monitoring the event Log:[wmi:appandsys of application and system on a Windows host with IP address 172.29.1.30]
Server = 172.29.1.30
Interval = 60
Event_log_file = Application, System
Disabled = 0
In fact, you can also collect Windows logs via Syslog, which can be-ntsyslog with a free tool. （syslog Windows Event LogDetailed in the article)
How do I use Splunk? The splunk uses the B/S mode with the default port of 8000. To access the Splunk server in this article, simply enter [Url]http://172.29.1.1:8000[/url] in the browser. Free version is not required user authentication can log in, 30 days Trial Enterprise Edition is required authentication, login account is: User for admin, password for changeme. After logging in, you can clearly see the log errors that occurred in the past hour. Click any point in time and the Splunk will open the appropriate verbose log. This is a great help for us to monitor the entire enterprise IT system and analyze problems. So how does Google for it reflect? Splunk provides a set of keyword search rules that allow for very precise searches with this set of rules. For example, if I want to see all the relevant logs for users Jackie Chen and Michael Jordan in the last 24 hours, you can enter the following keywords in the search office. Splunk also allows users to keep their own search rules so that they don't have to enter the keyword every time they search for the same content. With this, I set up the corresponding search for all the network devices, Windows hosts and Linux hosts, and saved them in a new dashboard, so that I can clearly understand the log of all the devices as soon as I open this dashboard every day. The use of Splunk is very wide, and I mentioned just a little bit of application. You can go to [url]www.splunk.com[/url] to find more information. AnywaySplunk is a top-notch log analytics software, if you often use grep, awk, sed, sort, uniq, tail, head to analyze the log, then you need to Splunk. Can handle the regular log format, such as Apache, squid, System log, Mail.log these. Index all logs first, then cross-query to support complex query statements. And then show it in an intuitive way. Logs can be sent to the Splunk server via file, or it can be transmitted in real time via the network. or a distributed log collection. In short, a variety of log collection methods are supported.
Open source software: Http://www.splunk.com/en_us/download-5.html?ac=get_splunk_download
[References]
1. Super log server-splunk-facing the sea, spring flowers-51CTO Technical Blog http://jackiechen.blog.51cto.com/196075/150222/
2, Splunk Learning-the City of Ice-blog channel-Csdn.net http://blog.csdn.net/wangqi0079/article/details/8582400
3, Splunk in China | Splunk in China http://10data.com/splunk/
4. Splunk home, documents and downloads-log analysis software-open source China community Http://www.oschina.net/p/splunk
5, splunk_ Baidu Encyclopedia http://baike.baidu.com/view/2209060.htm
6, operational Intelligence, Log Management, Application Management, Enterprise Security and Compliance | Splunk http://www.splunk.com/
splunk-Cloud Computing & Big Data ERA Super log analysis and monitoring tool
