CentOS System Fault Analysis and troubleshooting

CentOS System Fault Analysis and troubleshooting

I. Log Analysis and Management

1. log file classification

Kernel and system logs

System Service syslog Management

User logs

Records Information about system user logon and exit.

Program log

Log files managed independently by various applications, with different recording formats

2. logs are saved under/var/log.

Kernel and public message log:/var/log/messages

Scheduled task log:/var/log/cron

System Boot log:/var/log/dmesg

Mail System log:/var/log/maillog

User Logon log:/var/log/lastlog/var/log/secure

/Var/log/wtmp/var/run/utmp

3. syslogd log management

Configuration File:/etc/syslog. conf

The format is as follows:

Service Category. Log-level Log message sending location

4. Log message level

0 EMERG (urgent): Causes host system unavailability

1 ALERT (warning): The problem must be resolved immediately

2 CRIT (severe): serious

3 ERR (error): running error

4 WARNING: events that may affect system functions

5 NOTICE: it does not affect the system, but it is worth noting

6 INFO: General information

7 DEBUG: program or system debugging information

5. System and kernel log formats

Time tag host name subsystem name message

6. User log analysis

/Var/log/lastlog: Recent user logon event

/Var/log/wtmp: User Logon, logout, and system startup/shutdown events

/Var/run/utmp: details of each user currently logged on

/Var/log/secure: security events related to user authentication

7. Program logs

Web Service:/var/log/httpd/

Access_log and error_log

Proxy service:/var/log/squid/

Access. log, cache. log, squid. out, And store. log

FTP service:/var/log/xferlog

Ii. system startup troubleshooting

1. Repair MBR sectors

1) Back up MBR sector data

Dd if =/dev/sda of =/backup/sda. mbr. bak bs = 512 count = 1

2) simulate MBR failure

Dd if =/dev/zero of =/dev/sda bs = 512 count = 1

3) boot RHEL5 to the First Aid Mode

Boot: linux rescue

4) restore the MBR sector from the backup file

Dd if =/tempdir/sda. mbr. bak of =/dev/sda bs = 512 count = 1

2. the/etc/inittab file is lost.

# Rpm-ivh -- replacepkgs initscripts *

Iii. troubleshooting of software packages

1. Rebuilding the RPM Database

Rpm -- rebuilddb or rpm -- initdb

2. The so file is missing.

Add the so file path after/etc/ld. so. conf

Ldconfig

Iv. File System Disk troubleshooting

1. fsck-yt ext3/dev/sdb1 fix ext3 in/dev/sdb1 Partition

2. Run df-I/dev/sdb1 to view the I node usage of/dev/sdb1.

3. Run fuser-mv/dev/cdrom to view the process information of the user using cdrom.

Fuser-k/dev/cdrom: Disable cdrom process a in use.

4. Check Disk Bad Sectors

Mkfs-t ext3-c/dev/sdb1 | grep bad

Badblocks-sv/dev/sdb1

V. View System Performance

1. uptime

/Proc/loadavg

2. vmstat

Free

3. Disk I/o

Iostat-dkt 5

4. mpstat