Linux Log Management

Logs are very important for security. They record all kinds of events that occur in the system every day. You can use them to check the cause of the error or the traces left by the attacker when the system is attacked. The main functions of logs are audit and monitoring. It can also monitor the system status in real time, monitor and track intrusions, and so on.

In Linux, there are three major log subsystems:

Connection time log-the log is executed by multiple programs and written to/var/log/wtmp and/var/run/utmp, login and other programs to update the wtmp and utmp files, enables the system administrator to track who is logged on to the system at any time.

Process statistics-executed by the system kernel. When a process terminates, each process writes a record to the process Statistics file (pacct or ACCT. Process statistics are used to provide command usage statistics for basic services in the system.

Error Log -- executed by syslogd (8. Various system Daemon Processes, user programs, and kernels report noteworthy events to files/var/log/messages through Syslog (3. In addition, many UNIX programs create logs. Servers that provide network services such as HTTP and FTP also maintain detailed logs.

Common log files are as follows:

Access-log records HTTP/web transmission

Acct/pacct record user commands

Aculog records modem activities

Btmp record failure record

Lastlog records the last successful logon events and the last unsuccessful logon events.

Messages records information from syslog (some links to the syslog file)

Sudolog records commands issued by using sudo

Sulog logs the use of the su command

Syslog records information from syslog (usually linked to the messages file)

Utmp records each user currently logged on

Wtmp a permanent record of the entry and exit times of a user upon each login

Xferlog records FTP sessions

Utmp, wtmp, and lastlog log files are the key for most UNIX log subsystems to be reused-keep records of user logon entry and exit. The information about the current logon user is recorded in the file utmp; the logon entry and exit records are recorded in the file wtmp; the last logon file can be viewed using the lastlog command. Data exchange, shutdown, and restart are also recorded in the wtmp file. All records contain timestamps. These files (lastlog is usually not large) grow rapidly in systems with a large number of users. For example, the wtmp file can grow infinitely unless it is intercepted regularly. Many systems configure wtmp to be used cyclically in units of one day or one week. It is usually modified by the script run by cron. These scripts are renamed and the wtmp file is recycled. Generally, wtmp is named wtmp.1 after the first day, wtmp.1 is changed to wtmp.2 after the second day, and so on until wtmp.7.

Each time a user logs on, the login program checks the user's uid in the lastlog file. If the logon time is found, the user's Last Logon Time, exit time, and host name are written to the standard output, and the login program records the new Logon Time in lastlog. After a new lastlog is written, the utmp file is opened and the user's utmp record is inserted. This record is always used when a user logs on and exits. The utmp file is used by various command files, including WHO, W, users, and finger.

Next, the login program opens the file wtmp and appends the user's utmp record. When a user logs on and exits, the same utmp record with the updated timestamp is appended to the file. The wtmp file is used by the program last and AC.

Specific commands
Wtmp and utmp files are binary files, and they cannot be cut or merged by tail commands (using cat commands ). You need to use the information contained in the two files by WHO, W, users, last, and AC.

WHO: The who command queries the utmp file and reports to each user currently logged on. The default output of WHO includes the user name, terminal type, logon date, and remote host. Example: WHO (Press ENTER) display

Chyang pts/0 Aug 18 :06

Ynguo pts/2 Aug 18 :32

Ynguo pts/3 Aug 18 :55

Lewis pts/4 Aug 18 :35

Ynguo pts/7 Aug 18

Ylou pts/8 Aug 18

If the wtmp file name is specified, the WHO command queries all previous records. The command who/var/log/wtmp will report every login since the wtmp file was created or deleted.

The W: W command queries the utmp file and displays information about each user in the current system and the processes it runs. Example: w (Press ENTER): 3: 36pm up 1 day, 6 users, load average: 0.23, 0.29, 0.27.

User tty from login @ idle jcpu pcpu what

Chyang pts/0 202.38.68.242 PM 0.08 s 0.04 s-Bash

Ynguo pts/2 202.38.79.47 PM 0.00 s 0.14 s 0.05 W

Lewis pts/3 202.38.64.233 PM 30: 39 0.27 s 0.22 s-Bash

Lewis pts/4 202.38.64.233 PM 6.00 s 4.03 s sh/home/users/

Ynguo pts/7 Simba. Nic. USTC. e PM 0.00 s 0.47 s 0.24 s Telnet mail

Ylou pts/8 202.38.64.235 pm 1: 09 m 0.10 s 0.04 s-Bash

　　
Users: Users prints the current logon user with a single line. Each displayed user name corresponds to a logon session. If a user has more than one login session, the user name will display the same number of times. Example: Users (Press ENTER): chyang Lewis ylou ynguo

Last: The last command searches back for wtmp to display the users that have logged on since the first file creation. For example:

Chyang pts/9 202.38.68.242 Tue Aug 1)

Cfan pts/6 202.38.64.20.tue Aug 1)

Chyang pts/4 202.38.68.242 Tue Aug 1)

Lewis pts/3 202.38.64.233 Tue Aug 1)

Lewis pts/2 202.38.64.233 Tue Aug 1)

If the user is specified, only recent activities of the user are reported last time. For example, the last ynguo (Press ENTER) display:

Ynguo pts/4 Simba. Nic. USTC. e Fri Aug 4)

Ynguo pts/4 Simba. Nic. USTC. e Thu Aug 3)

Ynguo pts/11 Simba. Nic. USTC. e Thu Aug 3)

Ynguo pts/0 Simba. Nic. USTC. e Thu Aug 3)

Ynguo pts/0 Simba. Nic. USTC. e Wed Aug 2 0:04-0:16 1 + 02: 12)

Ynguo pts/0 Simba. Nic. USTC. e Wed Aug 2)

Ynguo pts/9 Simba. Nic. USTC. e Thu Aug 1)

AC: the AC Command reports the user connection time (hours) based on the logon entry and exit times in the current/var/log/wtmp file. If no signs are used, the total time is reported. Example: AC (Press ENTER): Total 5177.47

Ac-D (Press ENTER) to display the total connection time of each day

Aug 12 all 261.87

Aug 13 Total 351.39

Aug 14 Overall 396.09

Aug 15 Total 462.63

Aug 16 total 270.45

Aug 17 total 104.29

Today total 179.02

Ac-P (Press ENTER) to display the total connection time of each user

Ynguo 193.23

Yucao 3.35

Rong 1, 133.40

Hdai 1, 10.52

Zjzhu 52.87

Zqzhou 13.14

Liangliu 24.34

Total 5178.24

Lastlog: The lastlog file is queried every time a user logs on. You can use the lastlog command to check the last logon time of a specific user and format and output the last logon log/var/log/lastlog. It displays the logon name, port number (TTY), and last logon time according to the UID sorting. If a user has never logged on, lastlog displays "** never logged **. Note that you need to run this command as root, for example:

Rong 5 202.38.64.187 Fri Aug 18 15:57:01 + 0800 2000

DBB ** never logged in **

XinChen ** never logged in **

Pb9511 ** never logged in **

Xchen 0 202.38.64.190 sun Aug 13 10:01:22 + 0800 2000

In addition, you can add one parameter. For example, last-u 102 will report users whose uid is 102; last-T 7 indicates limiting the report of the previous week.

Process statistics
UNIX can track every command run by every user. If you want to know what important files were messed up last night, the process Statistics subsystem can tell you. It is helpful for tracking an intrusion. Unlike the connection time log, the process Statistics subsystem is not activated by default and must be started. In Linux, The accton command is used to START process statistics and must be run as root. The form of the accton command is accton file, which must exist first. Run the touch command to create the pacct file touch/var/log/pacct, and then run accton: accton/var/log/pacct. Once accton is activated, you can use the lastcomm command to monitor the commands executed in the system at any time. To disable statistics, you can use the accton command without any parameters.

The lastcomm Command reports the previously executed files. Without parameters, the lastcomm command displays information about all commands recorded in the lifecycle of the current statistics file. It includes the command name, user, tty, CPU time consumed by the command, and a timestamp. If the system has many users, the input may be very long. The following example:

Crond F root ?? 0.00 secs sun Aug 20

Promisc_check.s s root ?? 0.04 secs sun Aug 20

Promisc_check root ?? 0.01 secs sun Aug 20

Grep root ?? 0.02 secs sun Aug 20

Tail root ?? 0.01 secs sun Aug 20

Sh root ?? 0.01 secs sun Aug 20

Ping s root ?? 0.01 secs sun Aug 20

Ping6.pl F root ?? 0.01 secs sun Aug 20

Sh root ?? 0.01 secs sun Aug 20

Ping s root ?? 0.02 secs sun Aug 20

Ping6.pl F root ?? 0.02 secs sun Aug 20

Sh root ?? 0.02 secs sun Aug 20

Ping s root ?? 0.00 secs sun Aug 20

Ping6.pl F root ?? 0.01 secs sun Aug 20

Sh root ?? 0.01 secs sun Aug 20

Ping s root ?? 0.01 secs sun Aug 20

Sh root ?? 0.02 secs sun Aug 20

Ping s root ?? 1.34 secs sun Aug 20

Locate root ttyp0 1.34 secs sun Aug 20

Accton s root ttyp0 0.00 secs sun Aug 20

One problem with process statistics is that the pacct file may grow very rapidly. In this case, you need to run the SA command interactively or through the cron mechanism to keep the log data under system control. SA command reports, cleans and maintains process statistical files. It can compress the information in/var/log/pacct to the abstract file/var/log/savacct and/var/log/usracct. These summaries contain system statistics by command name and user name. By default, sa reads them first and then the pacct file so that the report can contain all available information. The output of SA has the following mark items:

Avio-average number of I/O operations performed each time

CP-sum of user and system time, measured in minutes

CPU -- same as CP

K -- average CPU time used by the kernel, in the unit of 1 K

K * sec-CPU storage integrity, in 1 k-core seconds

Re-real-time, measured in minutes

S-system time, measured in minutes

Total number of TiO -- I/O operations

U-user time, in minutes

For example:

842 173.26re 4.30cp 0 avio 358 K

2 10.98re 4.06cp 0 avio 299 K find

9 24.80re 0.05cp 0 avio 291 K *** other

105 30.44re 0.03cp 0 avio 302 K Ping

104 30.55re 0.03cp 0 avio 394 K Sh

162 0.11re 0.03cp 0 avio 413 K security. sh *

154 0.03re 0.02cp 0 avio 273 K ls

56 31.61re 0.02cp 0 Avi 823 K ping6.pl *

2 3.23re 0.02cp 0 Avi 822 K ping6.pl

35 0.02re 0.01cp 0 avio 257 k md5sum

97 0.02re 0.01cp 0 avio 263 K initlog

12 0.19re 0.01cp 0 Avi 399 K promisc_check.s

15. 0.09re. 0.00cp 0 avio 288 K grep

11 0.08re 0.00cp 0 avio 332 K awk

You can also provide a summary report based on the user rather than the command. For example, Sa-M is shown as follows:

885 173.28re 4.31cp 0avk

Root 879 173.23re 4.31cp 0avk

Alias 3 0.05re 0.00cp 0avk

Qmailp 3 0.01re 0.00cp 0avk

Syslog Device
Syslog has been adopted by many log functions and is used in many protection measures-any program can record events through Syslog. Syslog records system events, writes to a file or device, or sends a message to users. It can record local events or events on another host through the network.

The SYSLOG device depends on two important files:/etc/syslogd (Daemon) and/etc/syslog. conf configuration file. Traditionally, most syslog information is written to the/var/adm or the/var/log directory information file (messages. *). A typical syslog record includes the name of the generated program and a text message. It also includes a device and a priority range (but not in the day ).

　　
Each Syslog message is assigned to one of the following main devices:

Log_auth -- Authentication System: Login, Su, Getty, etc.

Log_authpriv -- same as log_auth, but only log on to the selected readable file by a single user

Log_cron -- Cron Daemon

Log_daemon -- Other System daemprocesses, such as routed

Log_ftp -- File Transfer Protocol: FTPD, tftpd

Log_kern -- message generated by the kernel

Log_lpr-system printer Buffer Pool: LPR, lpd

Log_mail -- email system

Log_news-network news system

Log_syslog-Internal messages generated by syslogd (8)

Log_user -- messages generated by random user processes

LOG_UUCP--UUCP Subsystem

Log_local0 ~ Log_local7 -- reserved for local use

Syslog assigns several different priorities to each event:

Log_emerg-Emergency

Log_alert-problems that should be corrected immediately, such as system database damages

Log_crit-important cases, such as hard disk errors

Log_err -- Error

Log_warning -- warning information

Log_notice -- not an error, but it may need to be processed

Log_info -- intelligence information

Log_debug -- contains information about intelligence, which is usually used to debug a program.

The syslog. conf file indicates the Log action recorded by the syslogd program. The program queries the configuration file at startup. This file consists of a single entry of different programs or message categories, each occupying a row. Provides a selection domain and an action domain for each type of message. These fields are separated by tabs: select a domain to specify the type and priority of the message. The action domain indicates the action that syslogd performs when receiving a message that matches the selection criteria. Each option is composed of a device and a priority. When a priority is specified, syslogd records a message with the same or higher priority. Therefore, if "crit" is specified, all messages marked as crit, alert, and emerg will be recorded. The action fields in each row indicate where to send a specified message to the selected domain. For example, if you want to record all the mail messages to a file, as follows:

# Log all the mail messages in one place

Mail. */var/log/maillog

Other devices also have their own logs. Uucp and news devices can produce many external messages. It stores these messages in its own logs (/var/log/Spooler) and limits the level to "Err" or higher. For example:

# Save mail and news errors of level err and higher in aspecial file.

Uucp, news. crit/var/log/Spooler

When an emergency message arrives, all users may want to get it. You may also want to receive and save your own logs.

# Everybody gets emergency messages, plus log them on anther Machine

*. Emerg *

*. Emerg @ linuxaid.com.cn

Alert messages should be written to the root and tiger personal accounts:

# Root and tiger get alert and higher messages

*. Alert root, Tiger

Sometimes syslogd will generate a large number of messages. For example, the kernel ("kern" device) may be lengthy. You may want to record kernel messages to/dev/console. The following example shows that the kernel log is commented out:

# Log all kernel messages to the console

# Logging much else clutters up the screen

# Kern. */dev/console

You can specify all devices in a row. In the following example, messages of info or higher level are sent to/var/log/messages, except for mail. Level "NONE" disables a device:

# Log anything (could t mail) of level info or higher

# Don't log private authentication messages!

*. Info: mail. None; authpriv. None/var/log/messages

In some cases, the log can be sent to the printer, so that it is useless for the network robbery server to modify the log. Generally, a wide range of logs are required. Syslog device is a notable target for attackers. A system that maintains logs for other hosts is particularly vulnerable to server attacks.

A small command logger provides a shell command interface for syslog (3) system log files so that you can create entries in the log files. Usage: logger For example: logger this is a test!

It will generate the following syslog record: Aug 19 22:22:34 Tiger: this is a test!

Be sure not to trust logs completely, because attackers can easily modify logs.

5. Program logs

Many programs maintain logs to reflect the security status of the system. The su command allows the user to obtain permissions of another user, so its security is very important. Its file is sulog. Sudolog is also available. In addition, Apache has two logs: access_log and error_log.

6. Other log tools

Chklastlog

Ftp://coast.cs.purdue.edu/pub/tools/unix/chklastlog/

Chkwtmp

Ftp://coast.cs.purdue.edu/pub/tools/unix/chkwtmp/

Dump_lastlog

Ftp://coast.cs.purdue.edu/pub/tools/unix/dump_lastlog.Z

Spar

Ftp://coast.cs.purdue.edu/pub/tools/unix/TAMU/

Swatch

Http://www.lomar.org/komar/alek/pres/swatch/cover.html

Zap

Ftp://caost.cs.purdue.edu/pub/tools/unix/zap.tar.gz

Log Classification Method

Http://csrc.nist.gov/nissc/1998/proceedings/paperD1.pdf