How to analyze Linux logs

How to analyze Linux logs

There is a lot of information in the log that you need to process, although sometimes it is not easy to extract. In this article, we will introduce some basic log analysis examples that you can do now (you only need to search ). We will also involve some more advanced analysis, but these require your initial efforts to make appropriate settings, which will save a lot of time in the future. Examples of advanced data analysis include generating a summary count, filtering valid values, and so on. We will first show you how to use multiple different tools in the command line, and then show you how a log management tool can automatically complete most of the heavy work, making log analysis easier. Search by Grep Text Search is the most basic way to find information. The most common tool for text search is grep. This command line tool is available in most Linux releases and allows you to search logs using regular expressions. Regular expressions are a pattern written in special languages that can recognize and match texts. The simplest mode is to enclose the string you want to search for with quotation marks. Regular expression This is an example of finding "user hoover" in the authentication log of the Ubuntu system: $ Grep "userhoover"/var/log/auth. log Accepted passwordfor hoover from 10.0.2.2 port 4792 ssh2 Pam_unix (sshd: session): session opened for user hoover by (uid = 0) Pam_unix (sshd: session): session closed for user hoover It may be difficult to construct precise regular expressions. For example, if we want to search for a number similar to Port 4792, it may also match the timestamp, URL, and other unnecessary data. In the example below in Ubuntu, it matches an Apache log that we don't want. $ Grep "4792"/var/log/auth. log Accepted passwordfor hoover from 10.0.2.2 port 4792 ssh2 74.91.21.46--[31/Mar/2015: 19: 44: 32 + 0000] "GET/scripts/samples/search? Q = 4972 HTTP/1.0 "404 545 "-""-" Surround search Another useful trick is that you can use grep for surround search. This will show you how to match the first or last lines. It helps you debug things that cause errors or problems. Option B displays the first few rows, and Option A displays the following rows. For example, we know that when a person fails to log on as an administrator and their IP address does not have reverse resolution, it means they may not have a valid domain name. This is very suspicious! $ Grep-B 3-A 2 'invalid user'/var/log/auth. log Apr 28 17: 06: 20ip-172-31-11-241 sshd [12545]: reverse mapping checking getaddrinfo for216-19-2-8.commspeed.net [216.19.2.8] failed-possible break-in attempt! Apr 28 17: 06: 20ip-172-31-11-241 sshd [12545]: Received disconnect from 216.19.2.8: 11: Bye [preauth] Apr 28 17: 06: 20ip-172-31-11-241 sshd [12547]: Invalid user admin from 216.19.2.8 Apr 28 17: 06: 20ip-172-31-11-241 sshd [12547]: input_userauth_request: invalid user admin [preauth] Apr 28 17: 06: 20ip-172-31-11-241 sshd [12547]: Received disconnect from 216.19.2.8: 11: Bye [preauth] Tail You can also use grep and tail together to obtain the last few lines of a file, or track logs and print them in real time. This is useful when you make interactive changes, such as starting a server or testing code changes. $ Tail-f/var/log/auth. log | grep 'invalid user' Apr 30 19: 49: 48ip-172-31-11-241 sshd [6512]: Invalid user ubnt from 219.140.64.136 Apr 30 19: 49: 49ip-172-31-11-241 sshd [6514]: Invalid user admin from 219.140.64.136 The detailed introduction of grep and regular expressions is not in the scope of this guide, but Ryan's Tutorials has a more in-depth introduction. The log management system provides higher performance and powerful search capabilities. They usually index data and perform parallel queries, so you can quickly search for GB or TB of logs in a few seconds. In contrast, grep takes several minutes, in extreme cases, or even hours. The log management system also uses a query language similar to Lucene, which provides simpler syntax to retrieve numbers, domains, and others. Use Cut, AWK, and Grok to parse Command line tool Linux provides multiple command line tools for text parsing and analysis. It is useful when you want to quickly parse a small amount of data, but it may take a long time to process a large amount of data. Cut The cut command allows you to parse fields from logs with delimiters. Separators are equal signs or commas that can separate fields or key-value pairs. Suppose we want to parse the user from the following log: Pam_unix (su: auth): authentication failure; logname = hoover uid = 1000 euid = 0 tty =/dev/pts/0 ruser = hoover rhost = user = root We can use the cut command below to obtain the text of the eighth field after division by equal signs. This is an example of Ubuntu: $ Grep "authentication failure"/var/log/auth. log | cut-d' = '-f 8 Root Hoover Root Nagios Nagios AWK You can also use awk to provide more powerful field resolution functions. It provides a scripting language that allows you to filter out almost anything irrelevant. For example, if we have the following line of log in Ubuntu, we want to extract the name of the user who failed to log on: Mar 24 08: 28: 18ip-172-31-11-241 sshd [32701]: input_userauth_request: invalid user guest [preauth] You can use the awk command as follows. First, use a regular expression/sshd. * invalid user/to match the sshd invalid user row. Then, use {print $9} to print the ninth field based on the default delimiter space. In this way, the user name is output. $ Awk '/sshd. * invalid user/{print $9}'/var/log/auth. log Guest Admin Info Test Ubnt You can read it in the Awk User Guide.