Security O & M: general handling process after the server is attacked

Security O & M: general handling process after the server is attacked
Security is always relative, and even secure servers may be attacked. As a security O & M personnel, the principle to be grasped is to do a good job in system security protection to fix all known dangerous behaviors. At the same time, the system can quickly and effectively handle attack behaviors after being attacked, minimize the impact of attacks on the system.

I. general ideas for handling server attacks


The attack on the system is not terrible. What is terrible is that the system is helpless in the face of the attack. The following describes in detail the general solution after the server is attacked.


1. Cut off the network
All attacks come from the network. Therefore, after learning that the system is being attacked by hackers, the first thing to do is to disconnect the network connection of the server. In this way, apart from the attack source, it can also protect other hosts on the server's network.
2. Search for attack sources
You can view suspicious information by analyzing system logs or log files. You can also view ports opened by the system, processes running, and suspicious programs. This process should be traced and analyzed based on experience and comprehensive judgment capabilities. The following sections describe how to deal with this process.
3. analyze the causes and approaches of intrusion
Since the system has been intruded, there are many causes, either system vulnerabilities or program vulnerabilities. You must check the cause, in addition, you need to identify the attack path and find the attack source. Only by knowing the cause and path of the attack can you delete the attack source and fix the vulnerability at the same time.
4. Back up user data
After the server is attacked, you must immediately back up user data on the server and check whether the attack source is hidden. If the attack source is in user data, you must completely delete it and back up the user data to a safe place.
5. reinstall the system
Never think that you can completely clear the attack source, because no one knows the attack program better than the hacker. After the server is attacked, the safest and easiest way is to reinstall the system, because most attack programs are attached to system files or kernels, You can reinstall the system to completely clear the attack source.
6. Fixed program or System Vulnerabilities
After detecting a system or application vulnerability, You must fix the system vulnerability or change the program bug, because only after fixing the vulnerability can the program be officially run on the server.
7. Restore data and connect to the network
Copy the backup data to the newly installed server, enable the service, and then enable the network connection of the server to provide external services.


2. Check and lock suspicious users

When a server is found to be under attack, you must first cut off the network connection. However, in some cases, if you cannot immediately cut off the network connection, you must log on to the system to check whether there are any suspicious users, if a suspicious user logs on to the system, immediately lock the user and then disconnect the user's remote connection.
1. log on to the system to view suspicious users
Log on as the root user and run the "w" command to list all users who have logged on to the system, as shown in figure 1-11.



With this output, you can check whether there are suspicious or unfamiliar user logins. You can also determine whether they are illegal users based on the user name, user logon source address, and the processes they are running.

2. Lock suspicious users
Once a suspicious user is found, it should be immediately locked. For example, after executing the "w" command above, the nobody user should be a suspicious user (because the nobody has no logon permission by default ), first, lock the user and perform the following operations:
[Root @ server ~] # Passwd-l nobody
After the user is locked, it is possible that the user is still logged on. Therefore, the user must be kicked offline. According to the output of the preceding "w" command, the user's logon pid value can be obtained, the procedure is as follows:
[Root @ server ~] # Ps-ef | grep @ pts/3
531 6051 6049 0? 00:00:00 sshd: nobody @ pts/3
[Root @ server ~] # Kill-9 6051
In this way, the suspicious user nobody is kicked out of the online environment. If this user attempts to log on again, it will no longer be able to log on.

3. Use the last command to view User Logon Events
The last command records the logs of all users logging on to the system and can be used to find logon events of unauthorized users. The output of the last command is from the/var/log/wtmp file, some experienced intruders will delete/var/log/wtmp to clear their whereabouts, but it will still reveal clues in this file.


3. view system logs

Viewing System logs is the best way to find attack sources. You can view/var/log/messages,/var/log/secure, and other system logs, these two log files can record the running status of the software and the logon status of remote users, and view. bash_history file, especially in the/root directory. bash_history file, which records all historical commands executed by the user.


4. Check and disable suspicious PROCESSES IN THE SYSTEM

There are many commands to check suspicious processes, such as ps and top, but sometimes the path is unknown only when the process name is known. You can run the following command:
First, you can use the pidof command to find the PID of the running process, for example, to find the PID of the sshd process, run the following command:
[Root @ server ~] # Pidof sshd
13276 12942 4284
Then go to the memory directory and view the information of the exe file under the PID directory:
[Root @ server ~] # Ls-al/proc/13276/exe
Lrwxrwxrwx 1 root 0 Oct 4/proc/13276/exe->/usr/sbin/sshd
In this way, the complete execution path corresponding to the process is found. If you want to view the file handle, you can view the following directory:
[Root @ server ~] # Ls-al/proc/13276/fd
In this way, you can find the complete execution information of any process. In addition, there are many similar commands that can help system O & M personnel find suspicious processes. For example, you can find the process PID through the specified port or tcp or udp protocol, and then find the relevant process:
[Root @ server ~] # Fuser-n tcp 111
111/tcp: 1579
[Root @ server ~] # Fuser-n tcp 25
25/TCP/IP: 2037
[Root @ server ~] # Ps-ef | grep 2037
Root 2037 1 0 Sep23? 00:00:05/usr/libexec/postfix/master
Postfix 2046 2037 0 Sep23? 00:00:01 qmgr-l-t fifo-u
Postfix 9612 2037 0? 00:00:00 pickup-l-t fifo-u
Root 14927 12944 0 00:00:00 pts/1 grep 2037
In some cases, the attacker's program is very hidden, such as the rootkits backdoor program. In such cases, commands such as ps, top, and netstat may have been replaced, if you use the system's own commands to check for suspicious processes, it becomes untrusted. In this case, you need to use third-party tools to check system suspicious programs, such as chkrootkit, RKHunter, and other tools described earlier, with these tools, you can easily find programs that have been replaced or tampered with by the system.


5. Check the integrity of the file system

Checking whether the file attributes have changed is the simplest and most direct way to verify the integrity of the file system, for example, you can check whether the size of the/bin/ls file on the compromised server is the same as that of the file on the normal system to verify whether the file is replaced, but this method is relatively low. In this case, you can use the rpm tool in Linux to complete the verification. The operation is as follows:
[Root @ server ~] # Rpm-Va
... L... c/etc/pam. d/system-auth
S.5...... c/etc/security/limits. conf
S.5..... T c/etc/sysctl. conf
S.5..... T/etc/sgml/docbook-simple.cat
S.5... T c/etc/login. defs
S.5...... c/etc/openldap/ldap. conf
S.5... T c/etc/sudoers
... 5... T c/usr/lib64/security/classpath. security
... L... c/etc/pam. d/system-auth
S.5...... c/etc/security/limits. conf
S.5...... c/etc/ldap. conf
S.5... T c/etc/ssh/sshd_config
The meanings of each tag in the output are described as follows:
? S indicates that the file length has changed.
? M indicates that the file access permission or file type has changed.
? 5 indicates that the MD5 checksum has changed.
? D indicates that the properties of the device node have changed.
? L indicates that the symbolic link of the file has changed.
? U indicates that the owner of the file/subdirectory/device node has changed.
? G indicates that the group of the file/subdirectory/device node has changed.
? T indicates that the last modification time of the file has changed.
If the "M" mark appears in the output result, the corresponding file may have been tampered with or replaced. In this case, you can uninstall the rpm package and reinstall it to clear the attacked file.
However, this command has a limitation, that is, it can only check all files installed through the rpm package method, there is no way to install files through the non-rpm package method. At the same time, if the rpm tool is replaced, you cannot use this method. At this time, you can copy an rpm tool from the normal system for detection.

You can also use chkrootkit and RKHunter to check the file system. The usage of chkrootkit and RKHunter will be introduced next time.