Java program Logging (3/3)-How to analyze log

1. Description

As a program ape, we often do one thing: Get a log file and find the information you want from it.

This article summarizes what tools I used to analyze the log file to get the information I want, my work environment in recent years is server on Linux, the work machine is windows, so I use the tool mainly is some command line tools on Linux, Of course they can also be used on Windows, see the following tools in detail.
First of all, I'm just plain Linux users, so if some Linux commands or tools are not efficient, please educate me.

Digression: My view is that as a Java program ape, you just need to understand the operating system you are using to the extent that you need to work. It's good to be an expert, but it's not necessary. For example: I use Windows every day, but I always do not know how to edit the Group Policy or the way the register to adjust the maximum number of network connections, almost all of the optimization tools have this function.

2. Tools

• Tools to connect remote Linuxserver: Mremoteng official site seems to be a wall. To be able to download in the Chinese army celestial pole.
• Tools for running the Linux command line on Windows: Cygwin or Git bash if neither of you is faking, my advice is to install a git bash. Basic enough, and easy to install.

• Text Editor: VI. Occasionally under Windows with notepad++ because the normal case of the log file will have a dozens of or even hundreds of m, with the VI directly to view the comparison directly. and open faster.
• Text analytics: Excel, just write something at home this time. So I'm going to try the WPS table in Jinshan WPS using

3. Analyze Log3.1 Search text

Analysis log first is to find the log you need, in general, we need the log is so positioned

1. Find the appropriate line based on the exception information or request information
2. Find the relevant log in the context based on information such as time, thread name, keyword, etc. of the line

Let's talk about what the above two steps should do, respectively.

3.1.1 Find the corresponding line, and some of the files

Normally, when we start analyzing log, we have this information to assist us in our search.

• The number of references when the error occurred
• Exception information when an error occurs
• Time
• Assuming that there is no information, just know the error, then we can try to search "error", "Warn", "Exception" and the Like

3.1.1.1 Assuming that the log is in a specific file, I usually open the file using VI read-only mode (if the file name is Access.log)

Vi-r Access.log

Go in and let it show the line number:

: Set Nu

Jump to the first line:

Gg

or a

1G

Skip to the last line:

0G

or a

G

Skip to Line 11th:

11G

Search for strings from the head of the file Hello World:

:/hello World

Search string forward from end of file Hello World:

:? Hello World

Skip to the next place that matches your search criteria:

N

Skip to the previous location that matches your search criteria:

N

VI inside with go abbreviation letter G Jump to jump, the letter n is the abbreviation of next Exit VI command is:

: q!

3.1.1.2 If I don't know which file it is, I usually use the grep command sample for example: Print the search results to the screen, and flip the page with the SPACEBAR

Grep-nr "Hello World"./*.log | More

Print search results to file HelloWorld.log

Grep-nr "Hello World"./*.log > HelloWorld.log

3.1.2 Find Context

Follow the above steps to find the clue. If there is not much interference information, you can browse up and down. So if there is a lot of interference information, I generally use the following methods to obtain the information I need without disturbing the log.

• Filter the required log according to the thread name and keyword of the request I usually use the grep command above to do
• All logs in the time period are obtained based on a parsed start and end time (such as the entry of a Web request and the write-back time of the resulting information). I usually open with VI first, then find the line number of the first line of the desired content and the line number of the last line, and then use the SED command to get a few of the lines. The result of this command is to save the contents of line No. 200 to 266 in Source.log to the Result.log file.

  Sed-n 200,266p source.log > Result.log

3.2 Log Analysis In addition to finding clues in log, sometimes it is necessary to do a batch analysis of the log, I have encountered an analogy to analyze the occurrence of some kind of error, the relevant ID of the error, and so on. Need to use some sort of statistic or something.

3.2.1 Simple case I usually extract each scene into a file, then use the WC (don't laugh, it is the abbreviation of Word count) command to calculate the number of rows, such as the following see

Wc-l Result.log

Complex I usually use Excel or Jinshan wps for file analysis.

3.2.2 using Excel or Kingsoft wps for file analysis

1. First, start a new worksheet
2. and copy the text you want to analyze.
3. Then repeat using the * column * (menu: Data-to-columns) to split the log file into a single table
4. And then use the filtering sort PivotChart to analyze

3.3 Some other useful commands

• Open a log file. Watch it update as the system executes

Tail-f Abc.log

• Monitor changes to multiple files at the same time

Tail-f./*.log

• Look at the 20 lines at the beginning of a file

Head-20 Abc.log

• See 50 lines at the end of a file

Tail-50 Abc.log

Java program Logging (3/3)-How to analyze log