Understand Linux system logs

1. Log Overview

Logs are very important for security. They record all kinds of events that occur in the system every day. You can use them to check the cause of the error or the traces left by the attacker when the system is attacked. The main functions of logs are audit and monitoring. It can also monitor the system status in real time, monitor and track intrusions, and so on.

In Linux, there are three major log subsystems:

Connection time log-the log is executed by multiple programs and written to/var/log/wtmp and/var/run/utmp, login and other programs to update the wtmp and utmp files, enables the system administrator to track who is logged on to the system at any time.

Process statistics-executed by the system kernel. When a process terminates, each process writes a record to the process Statistics file (pacct or acct. Process statistics are used to provide command usage statistics for basic services in the system.

Error Log -- executed by syslogd (8. Various system Daemon Processes, user programs, and kernels report noteworthy events to files/var/log/messages through syslog (3. In addition, many UNIX programs create logs. Servers that provide network services such as HTTP and FTP also maintain detailed logs.

Common log files are as follows:

Access-log records HTTP/web transmission

Acct/pacct record user commands

Aculog records MODEM activities

Btmp record failure record

Lastlog records the last successful logon events and the last unsuccessful logon events.

Messages records information from syslog (some links to the syslog file)

Sudolog records commands issued by using sudo

Sulog logs the use of the su command

Syslog records information from syslog (usually linked to the messages file)

Utmp records each user currently logged on

Wtmp a permanent record of the entry and exit times of a user upon each login

Xferlog records FTP sessions

Utmp, wtmp, and lastlog log files are the key for most UNIX log subsystems to be reused-keep records of user logon entry and exit. The information about the current logon user is recorded in the file utmp; the logon entry and exit records are recorded in the file wtmp; the last logon file can be viewed using the lastlog command. Data exchange, shutdown, and restart are also recorded in the wtmp file. All records contain timestamps. These files (lastlog is usually not large) grow rapidly in systems with a large number of users. For example, the wtmp file can grow infinitely unless it is intercepted regularly. Many systems configure wtmp to be used cyclically in units of one day or one week. It is usually modified by the script run by cron. These scripts are renamed and the wtmp file is recycled. Generally, wtmp is named wtmp.1 after the first day, wtmp.1 is changed to wtmp.2 after the second day, and so on until wtmp.7.

Each time a user logs on, the login program checks the user's UID in the lastlog file. If the logon time is found, the user's Last Logon Time, exit time, and host name are written to the standard output, and the login program records the new Logon Time in lastlog. After a new lastlog is written, the utmp file is opened and the user's utmp record is inserted. This record is always used when a user logs on and exits. The utmp file is used by various command files, including who, w, users, and finger.

Next, the login program opens the file wtmp and appends the user's utmp record. When a user logs on and exits, the same utmp record with the updated timestamp is appended to the file. The wtmp file is used by the program last and ac.

2. Specific commands

Wtmp and utmp files are binary files, and they cannot be cut or merged by tail commands (using cat commands ). You need to use the information contained in the two files by who, w, users, last, and ac.

Who: The who command queries the utmp file and reports to each user currently logged on. The default output of Who includes the user name, terminal type, logon date, and remote host. Example: who (Press ENTER) display

Chyang pts/0 Aug 18 :06
Ynguo pts/2 Aug 18 :32
Ynguo pts/3 Aug 18 :55
Lewis pts/4 Aug 18 :35
Ynguo pts/7 Aug 18
Ylou pts/8 Aug 18

If the wtmp file name is specified, the who command queries all previous records. The command who/var/log/wtmp will report every login since the wtmp file was created or deleted.

The w: w command queries the utmp file and displays information about each user in the current system and the processes it runs. Example: w (Press ENTER): 3: 36pm up 1 day, 6 users, load average: 0.23, 0.29, 0.27

User tty from login @ IDLE JCPU PCPU WHAT
Chyang pts/0 202.38.68.242 pm 0.08 s 0.04 s-bash
Ynguo pts/2 202.38.79.47 pm 0.00 s 0.14 s 0.05 s w
Lewis pts/3 202.38.64.233 pm 30: 39 0.27 s 0.22 s-bash
Lewis pts/4 202.38.64.233 pm 6.00 s 4.03 s sh/home/users/
Ynguo pts/7 simba. nic. ustc. e pm 0.00 s 0.47 s 0.24 s telnet mail
Ylou pts/8 202.38.64.235 pm 1: 09 m 0.10 s 0.04 s-bash

Users: users prints the current logon user with a single line. Each displayed user name corresponds to a logon session. If a user has more than one login session, the user name will display the same number of times. Example: users (Press ENTER): chyang lewis ylou ynguo

Last: The last command searches back for wtmp to display the users that have logged on since the first file creation. For example:

Chyang pts/9 202.38.68.242 Tue Aug 1)
Cfan pts/6 202.38.64.20.tue Aug 1)
Chyang pts/4 202.38.68.242 Tue Aug 1)
Lewis pts/3 202.38.64.233 Tue Aug 1)
Lewis pts/2 202.38.64.233 Tue Aug 1)

If the user is specified, only recent activities of the user are reported last time. For example, the last ynguo (Press ENTER) display:

Ynguo pts/4 simba. nic. ustc. e Fri Aug 4)
Ynguo pts/4 simba. nic. ustc. e Thu Aug 3)
Ynguo pts/11 simba. nic. ustc. e Thu Aug 3)
Ynguo pts/0 simba. nic. ustc. e Thu Aug 3)
Ynguo pts/0 simba. nic. ustc. e Wed Aug 2 0:04-0:16 1 + 02: 12)
Ynguo pts/0 simba. nic. ustc. e Wed Aug 2)
Ynguo pts/9 simba. nic. ustc. e Thu Aug 1)

Ac: the ac Command reports the user connection time (hours) based on the logon entry and exit times in the current/var/log/wtmp file. If no signs are used, the total time is reported. Example: ac (Press ENTER): total 5177.47

Ac-d (Press ENTER) to display the total connection time of each day

Aug 12 all 261.87
Aug 13 total 351.39
Aug 14 Overall 396.09
Aug 15 total 462.63
Aug 16 total 270.45
Aug 17 total 104.29
Today total 179.02

Ac-p (Press ENTER) to display the total connection time of each user

Ynguo 193.23
Yucao 3.35
Rong 1, 133.40
Hdai 1, 10.52
Zjzhu 52.87
Zqzhou 13.14
Liangliu 24.34
Total 5178.24

Lastlog: The lastlog file is queried every time a user logs on. You can use the lastlog command to check the last logon time of a specific user and format and output the last logon log/var/log/lastlog. It displays the logon name, port number (tty), and last logon time according to the UID sorting. If a user has Never logged on, lastlog displays "** Never logged **. Note that you need to run this command as root, for example:

Rong 5 202.38.64.187 Fri Aug 18 15:57:01 + 0800 2000
Dbb ** Never logged in **
Xinchen ** Never logged in **
Pb9511 ** Never logged in **
Xchen 0 202.38.64.190 Sun Aug 13 10:01:22 + 0800 2000

In addition, you can add one parameter. For example, last-u 102 will report users whose UID is 102; last-t 7 indicates limiting the report of the previous week.

3. Process statistics

UNIX can track every command run by every user. If you want to know what important files were messed up last night, the process Statistics subsystem can tell you. It is helpful for tracking an intrusion. Unlike the connection time log, the process Statistics subsystem is not activated by default and must be started. In Linux, The accton command is used to START process statistics and must be run as root. The form of the Accton command is accton file, which must exist first. Run the touch command to create the pacct file touch/var/log/pacct, and then run accton: accton/var/log/pacct. Once accton is activated, you can use the lastcomm command to monitor the commands executed in the system at any time. To disable statistics, you can use the accton command without any parameters.

The lastcomm Command reports the previously executed files. Without parameters, the lastcomm command displays information about all commands recorded in the lifecycle of the current statistics file. It includes the command name, user, tty, CPU time consumed by the command, and a timestamp. If the system has many users, the input may be very long. The following example:

Crond F root ?? 0.00 secs Sun Aug 20
Promisc_check.s S root ?? 0.04 secs Sun Aug 20
Promisc_check root ?? 0.01 secs Sun Aug 20
Grep root ?? 0.02 secs Sun Aug 20
Tail root ?? 0.01 secs Sun Aug 20
Sh root ?? 0.01 secs Sun Aug 20
Ping S root ?? 0.01 secs Sun Aug 20
Ping6.pl F root ?? 0.01 secs Sun Aug 20
Sh root ?? 0.01 secs Sun Aug 20
Ping S root ?? 0.02 secs Sun Aug 20
Ping6.pl F root ?? 0.02 secs Sun Aug 20
Sh root ?? 0.02 secs Sun Aug 20
Ping S root ?? 0.00 secs Sun Aug 20
Ping6.pl F root ?? 0.01 secs Sun Aug 20
Sh root ?? 0.01 secs Sun Aug 20
Ping S root ?? 0.01 secs Sun Aug 20
Sh root ?? 0.02 secs Sun Aug 20
Ping S root ?? 1.34 secs Sun Aug 20
Locate root ttyp0 1.34 secs Sun Aug 20
Accton S root ttyp0 0.00 secs Sun Aug 20

One problem with process statistics is that the pacct file may grow very rapidly. In this case, you need to run the sa command interactively or through the cron mechanism to keep the log data under system control. Sa command reports, cleans and maintains process statistical files. It can compress the information in/var/log/pacct to the abstract file/var/log/