/*************************************** **********
** Reprinted from: http://iceskysl.1sters.com /? P = 363
** For copyright infringement, contact lxs_lover520 # QQ.com.
** I will handle it as soon as possible
**************************************** **********/.
Symptom description
There is a server in the project. Its Operating System is RHEL 3, which is mainly used in the lab environment and FTP servers of third-party systems. Recently, there have been frequent "false dead" situations, the specific performance is Ping, but ssh, FTP, telnet, and so on cannot be logged on. The server runs normally on the LAN console. Each time, a remote restart is used to temporarily solve the problem.
In-depth analysis
Today, another host Maintenance Engineer found that/var/log/messages. X contains the following information:
Jul 30 17:36:15 WWW login (pam_unix) [11819]: session opened for user siteview by (uid = 0)
Jul 30 17:36:15 www-siteview [11819]: Login on pts/0 by siteview from 192.168.168.42
Jul 30 17:36:15 WWW login (pam_unix) [11819]: session closed for user siteview
Jul 30 17:47:17 WWW syslogd:/var/log/Secure: no space left on Device
Jul 30 19:11:58 WWW Pam-securetty [1611]: couldn't open/etc/securetty
After searching, we found that the reason was that the audit subsystem laus (Linux Auditing System) enabled in the kernel had too many logs and occupied/var space. When I log on again, due to the inability to write logs, the log may be suspended, causing a false death.
Solution:
After locating the problem, you can find a solution. Currently, there are four feasible solutions:
1. manually delete log files under/var/log/audit. D on a regular basis
# Delete
Find./-mtime + 7-type F-exec RM {}/;
2. Configure Crob to automatically delete or back up data
Seven_days_old = '/usr/locale/Ebin/date' + % d'-date' 7 days ago''
If [-F/var/log/audit/bin. $ {seven_days_old}]
Then
Rm/var/log/audit/bin. $ {seven_days_old}
Fi
3. Modify the/etc/audit. conf configuration file.
You can modify audit. conf to configure it for automatic processing.
1) changed to automatically deleted:
Running y = "/usr/sbin/audbin-S/var/log/audit. d/Save. % u-c-t 20%-N 'rm-F % F '";
[The specified y line as above shocould (according to the docs) remove Old 'save' files when the filesystem comes within 20% of full (change the figure after-t to specify how close to full You Want To Start deleting old files).]
2) changed to automatic backup: (/backup is another partition that you want to save data)
Running y = "/usr/sbin/audbin-S/var/log/audit. d/Save. % u-c-t 20%-N 'mv % F/backup '";
4. Disable the audtid service.
If you do not need to review the information, disable the audtid Service ("chkconfig-del auditd") to improve the system performance. if you want to completely disable audit in the kernel, you can delete the/dev/audit file and restart it. You can enable it later by adding audit = 1 to the kernel line in grub. conf.
Reference:
- Laus related: http://blog.chinaunix.net/u/6303/showart_275806.html
- Console login hangs RHEL 3: http://www.linuxquestions.org/questions/linux-general-1/console-login-hangs-rhel-3-348072/
- Linux console hangs: http://www.usenet-forums.com/linux-networking/68241-linux-console-hangs-2.html
- /Directory space is insufficient, clear/var/log/audit. D: http://blog.zhangjianfeng.com/article/386