Use logs for troubleshooting in Linux

The main reason people create logs is the wrong line. Usually you will diagnose what problems occur in your Linux system or application. An error message or a series of events can give you clues to find the root cause, explain how the problem occurred, and point out how to fix it. Here are a few examples of using logs to solve.

Logon Failure Reason

If you want to check if your system is secure, you can check the authentication log for failed logins and login successes but suspicious users. Authentication failures occur when someone logs on with improper or invalid credentials, which typically occurs when using SSH for remote logins or SU to other local users for access. These are recorded by the plug-in verification module (PAM). You'll see strings like Failed Password and user unknown in your log. A successful authentication record would include strings such as Accepted Password and session opened.

Examples of failures:

Pam_unix (Sshd:auth): Authentication failure; Logname= uid=0 euid=0 tty=ssh ruser= rhost=10.0.2.2

Failed Password for invalid user hoover from 10.0.2.2 Port 4791 ssh2

Pam_unix (Sshd:auth): Check pass; User Unknown

PAM Service (sshd) ignoring Max retries; 6 > 3

Examples of success:

Accepted password for Hoover from 10.0.2.2 Port 4792 ssh2

Pam_unix (Sshd:session): Session opened for user Hoover by (uid=0)

Pam_unix (Sshd:session): Session closed for user Hoover

You can use grep to find out which users have the most number of failed logins. These are potential attackers who are trying and accessing failed accounts. This is an example of an Ubuntu system.

$ grep "Invalid user"/var/log/auth.log | Cut-d '-F 10 | Sort | uniq-c | Sort-nr

Oracle

Postgres

Nagios

Ten Zabbix

6 test

Because there is no standard format, you need to use different commands for each application's log. Log management system, you can automatically analyze the log, and effectively categorize them to help you extract keywords, such as user name.

The log management system can use the automatic parsing feature to extract user names from the Linux logs. This allows you to see the user's information and be able to filter by clicking.

The log management system also allows you to view the chart with time as an axis, making it easier to spot anomalies. If someone fails to log on once or two times within a few minutes, it may be a real user and forget the password. However, if you have hundreds of failed logins and are using a different user name, it is more likely that you are trying to attack the system.

Reason for restart

Sometimes, a server goes down because of a system crash or reboot. How do you know when it happened and who did it?

Shutdown command

If someone runs the shutdown command manually, you can see it in the validation log file. Here, you can see that someone has telnet from the IP 50.0.134.125 as an Ubuntu user and then shuts down the system.

Mar 18:36:41 ip-172-31-11-231 sshd[23437]: Accepted publickey for Ubuntu from 50.0.134.125 Port 52538 ssh

Mar 18:36:41 ip-172-31-11-231 23437]:sshd[Pam_unix (sshd:session): Session opened for the user Ubuntu by (uid=0)

Mar 18:37:09 ip-172-31-11-231 sudo:ubuntu:tty=pts/1; Pwd=/home/ubuntu; User=root; Command=/sbin/shutdown-r now

Kernel initialization

If you want to see all the causes of server restarts (including crashes), you can look for them from the kernel initialization log. You need to search for kernel class (kernel) and CPU initialization (Initializing) information.

Mar 18:39:30 ip-172-31-11-231 kernel: [0.000000] Initializing cgroup Subsys cpuset

Mar 18:39:30 ip-172-31-11-231 kernel: [0.000000] Initializing cgroup Subsys CPU

Mar 18:39:30 ip-172-31-11-231 kernel: [0.000000] Linux version 3.8.0-44-generic ([email protected]) (GCC version 4. 6.3 (Ubuntu/linaro 4.6.3-1ubuntu5)) #66 ~precise1-ubuntu SMP Tue Jul 04:01:04 UTC (Ubuntu 3.8.0-44.66~precise1-gen Eric 3.8.13.25)

Detecting Memory problems

There are a number of reasons for a server crash, but a common cause is memory exhaustion.

When your system is running out of memory, the process is killed and the process that uses the most resources is usually killed. An error occurs when the system uses all memory and the new or existing process tries to use more memory. Look for a string such as out of Memory in your log file or a kernel warning message like kill. This information indicates that the system intentionally kills the process or application, rather than allowing the process to crash.

For example:

[33238.178288] out of Memory:kill process 6230 (Firefox) score/Sacrifice child

[29923450.995084] Select 5230 (docker), adj 0, size 708, to kill

You can use tools like grep to find these logs. This example is in Ubuntu:

$ grep "Out of Memory"/var/log/syslog

[33238.178288] out of Memory:kill process 6230 (Firefox) score/Sacrifice child

Keep in mind that grep also uses memory, so just running grep can also lead to out-of-memory errors. This is another reason why you should centrally store logs!

Timed Task error Log

Cron A daemon is a scheduler that can run a process at a specified date and time. If the process fails or does not complete, a cron error appears in your log file. Depending on your release version, you can find this log in/var/log/cron,/var/log/messages, and/var/log/syslog several locations. There are many reasons for cron task failure. Typically, the problem occurs in the process rather than the cron daemon itself.

By default, the output of the cron task sends an e-mail message through Postfix. This is a log that shows that the message has been sent. Unfortunately, you can't see the contents of the message here.

Mar 16:35:01 PSQ110 postfix/pickup[15158]: c3edc5800b4:uid=1001 from=

Mar 16:35:01 PSQ110 postfix/cleanup[15727]: C3edc5800b4:message-id=<[email protected]>

Mar 16:35:01 PSQ110 postfix/qmgr[15159]: C3edc5800b4:from=<[email protected]>, size=607, nrcpt=1 (queue active)

Mar 16:35:05 PSQ110 postfix/smtp[15729]: C3edc5800b4:to=<[email protected]>, relay= Gmail-smtp-in.l.google.com[74.125.130.26]:25, delay=4.1, delays=0.26/0/2.2/1.7, dsn=2.0.0, status=sent (2.0.0 OK 1425985505 f16si501651pdj.5-gsmtp)

You might consider logging the standard output of cron to a log to help you locate the problem. This is an example of how you can use the Logger command to redirect the Cron standard output to the syslog. Using your script instead of the echo command, Hellocron can be set to the name of any application you want.

*/5 * * * * echo ' Hello world ' 2>&1 |/usr/bin/logger-t hellocron

It creates a log entry:

APR 22:20:01 ip-172-31-11-231 cron[15296]: (Ubuntu) CMD (Echo ' Hello world! ' 2>&1 | /usr/bin/logger-t Hellocron)

APR 22:20:01 ip-172-31-11-231 Hellocron:hello world!

Each cron task records different logs based on the specific type of task and how the data is output.

You may want to have a clue about the source of the problem in the log, or you can add additional log records as needed.

free pick up brother even it education original Cloud Computing Training video/Detailed Linux tutorials, details of the website customer service: http://www.lampbrother.net/linux/ or hooking up with q2430675018~.

Welcome to the Linux Communication Group 478068715

Use logs for troubleshooting in Linux