The experience of a Linux server being hacked and removed Trojans

I. BACKGROUND

At night to see a server traffic runs very high, obviously and usually not the same, the flow reached 800Mbps, the first feeling should be in the Trojan, was people as a broiler, in a large number of contracts.

Our server for the best performance, Firewall (iptables) or something is not open, but the server front of the physical firewall, and the machine is to do the port mapping, is not a common port, supposedly should be full of security, probably recently and Trojan destined bar, always let me meet, Also take this opportunity to record the discovery process.

II. discovery and tracking of processing

1. View Traffic Graph Discovery problem

When viewing the page is very card, sometimes not even respond.

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M02/7A/9A/wKiom1as-dyQlUCeAAFWXnyaceU258.png "title=" Qq20160131015343.png "alt=" Wkiom1as-dyqluceaafwxnyaceu258.png "/>

2. Top Dynamic Viewing process

I immediately telnet to the problem of the server, remote operation is very card, network card out of the traffic is very large, through the top found an abnormal process occupies a high resource, the name is not carefully see also really thought is a Web service process.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/7A/99/wKioL1as-nOSab6eAACPx1Gw2qc060.png "title=" Qq20160131014854.png "alt=" Wkiol1as-nosab6eaacpx1gw2qc060.png "/>

3, PS command to view the path of the process

Found this program file in/etc directory, is a binary program, I copied down, placed in the vicinity of this article for everyone in the virtual machine above the research, haha.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/7A/9A/wKiom1as-k-TydDMAADaZz0UV1k116.png "title=" Qq20160131014811.png "alt=" Wkiom1as-k-tyddmaadazz0uv1k116.png "/>

4. End the exception process and continue tracking

Killall-9 nginx1rm-f/etc/nginx1

After the process of killing, the traffic immediately down, remote also not Kaka, do not delete the program files, kill the abnormal process we think the processing is complete? Think also certainly not so simple, this is a Trojan Ah, certainly will also generate the program file (sure enough, I do not know, before I do not understand, the back is really generated) we have to continue to trace.

5. Check login log and log file secure

Check the account login record by command last, everything is OK. View system files Message I didn't find anything, but when I looked at the secure file, I found some exceptions, which are related to authentication, should I try to connect in the control contract?

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/7A/99/wKioL1as_jORDywFAAFogdyG4v8317.png "title=" Qq20160131021232.png "alt=" Wkiol1as_jordywfaafogdyg4v8317.png "/>

6, again PS viewing process

In fact, the first PS when there is this problem, then did not find, the second is self-study to view each process, self-study to find a less normal process, found a strange PS process.

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/7A/9A/wKiom1as_uDj_cZWAAA1rhjR00k722.png "title=" Qq20160131021947.png "alt=" Wkiom1as_udj_czwaaa1rhjr00k722.png "/>

I found a normal machine, check the size of the PS command, the normal is about 81KB, and then this machine above the PS is up to 1.2M, the command file must have been replaced.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7A/99/wKioL1as_52wGvNHAAASrsFyr24589.png "title=" Qq20160131022135.png "alt=" Wkiol1as_52wgvnhaaasrsfyr24589.png "/>

Then go to another PS directory, see the following several commands, and then I have to query the system of these commands, the discovery has become very large, all reached 1.2M, these system command files must have been replaced.

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M00/7A/9A/wKiom1as__TQk4K_AAAn0k9evMk278.png "title=" Qq20160131022328.png "alt=" Wkiom1as__tqk4k_aaan0k9evmk278.png "/>

7. Discovery of more unusual files

Viewing the timed task file crontab did not discover anything at once and then looked at the system boot file rc.local, and there was nothing unusual, and then went into the/ETC/INIT.D directory to see the more bizarre script file Dbsecurityspt, SELinux.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/7A/99/wKioL1atApfgw78ZAAB5n-v2zfM656.png "title=" Qq20160131023407.png "alt=" Wkiol1atapfgw78zaab5n-v2zfm656.png "/>

The first file can be seen that he is booting the exception file, the second should be related to the login, specifically I am not very clear, anyway, there must be a problem.

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M00/7A/99/wKioL1atBB_gx4vbAAAivNZ8pRs292.png "title=" Qq20160131024050.png "alt=" Wkiol1atbb_gx4vbaaaivnz8prs292.png "/>

Since and login, then find and SSH related, found the following file, is a hidden file, this is a Trojan file, we first recorded, so the program name and our service name is very similar, is to confuse us, their size is 1.2M, they may be a file.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/7A/9A/wKiom1atBQbz_CoYAAATGOPwJ50519.png "title=" Qq20160131024540.png "alt=" Wkiom1atbqbz_coyaaatgopwj50519.png "/>

I have looked at the Trojan like to appear in the directory/TMP, also found the exception file, from the name of the feeling as if it is a monitoring Trojan program.

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/7A/9A/wKiom1atBnyg7i6dAACRJQNJKns664.png "title=" Qq20160131025100.png "alt=" Wkiom1atbnyg7i6daacrjqnjkns664.png "/>

Think of here, replace the command should be a lot of, alone we go to find certainly can not solve, my suggestion is best to re-install the operating system, and do a good security policy, if not reload, I give my method below, specific line does not need to be verified.

Third, the Trojan manual removal

Now we summarize the approximate steps as follows:

1, simple to judge whether there is no Trojan horse

#有无下列文件cat/etc/rc.d/init.d/selinuxcat/etc/rc.d/init.d/dbsecuritysptls/usr/bin/bsd-portls/usr/bin/dpkgd# See if the size is normal ls-lh/bin/netstatls-lh/bin/psls-lh/usr/sbin/lsofls-lh/usr/sbin/ss

2, upload the following command to/root

PS Netstat SS lsof

3. Delete the following directories and files

RM-RF/USR/BIN/DPKGD (PS netstat lsof ss) Rm-rf/usr/bin/bsd-port #木马程序rm-F/usr/bin/.sshd #木马后门rm-F/tmp/g Ates.lodrm-f/tmp/moni.lodrm-f/ETC/RC.D/INIT.D/DBSECURITYSPT (start the above described Trojan variant program) Rm-f/etc/rc.d/rc1.d/ S97dbsecuritysptrm-f/etc/rc.d/rc2.d/s97dbsecuritysptrm-f/etc/rc.d/rc3.d/s97dbsecuritysptrm-f/etc/rc.d/rc4.d/ S97dbsecuritysptrm-f/etc/rc.d/rc5.d/s97dbsecuritysptrm-f/etc/rc.d/init.d/selinux (default is start/usr/bin/bsd-port/getty) Rm-f/etc/rc.d/rc1.d/s99selinuxrm-f/etc/rc.d/rc2.d/s99selinuxrm-f/etc/rc.d/rc3.d/s99selinuxrm-f/etc/rc.d/rc4.d/ S99selinuxrm-f/etc/rc.d/rc5.d/s99selinux

4. Find out the abnormal procedure and kill

5, remove the Trojan Horse command and reinstall (or upload the normal program to copy the past is OK)

#ps/root/chattr-i-a/bin/ps && rm/bin/ps-fyum Reinstall Procps-y or Cp/root/ps/bin#netstat/root/chattr-i-a /bin/netstat && Rm/bin/netstat-fyum Reinstall net-tools-y or Cp/root/netstat/bin#lsof/root/chattr-i-a/bin/ Lsof && Rm/usr/sbin/lsof-fyum Reinstall lsof-y or cp/root/lsof/usr/sbin#ss/root/chattr-i-a/usr/sbin/ss & ;& rm/usr/sbin/ss-fyum-y Reinstall Iproute or Cp/root/ss/usr/sbin

Iv. scanning of antivirus tools

1. Install anti-virus tool ClamAV

Yum-y Install ClamAV Clamav-milter

2. Start the service

Service CLAMD Restart

3. Update the virus database

Because ClamAV is not the latest version, there is a warning message. You can ignore or upgrade the latest version.

[[Email protected] ~]# freshclam clamav update process started at  sun jan 31 03:15:52 2016warning: can ' t query  Current.cvd.clamav.netwarning: invalid dns reply. falling back to http  mode. reading cvd header  (MAIN.CVD): warning: main.cvd not found on  Remote serverwarning: can ' T read main.cvd header from db.cn.clamav.net   (ip: 185.100.64.62) trying again in 5 secs ... Clamav update process started at sun jan 31 03:16:25 2016warning:  can ' t query current.cvd.clamav.netwarning: invalid dns reply. falling  back to http mode. reading cvd header  (MAIN.CVD): trying host db.cn.clamav.net  (200.236.31.1) ... okmain.cvd is up to date  (Version: 55, sigs: 2424225, f-level: 60, builder: neo) Reading  CVD header  (DAILY.CVD): ok  (IMS) daily.cvd is up to date  ( Version: 21325, sigs: 1824133, f-level: 63, builder: neo) Reading CVD  header  (BYTECODE.CVD): ok  (IMS) bytecode.cvd is up to date  ( Version: 271, sigs: 47, f-level: 63, builder: anvilleg)

4. Scanning method

You can use Clamscan-h to view the appropriate help information

Clamscan-r/etc--max-dir-recursion=5-l/root/etcclamav.logclamscan-r/bin--max-dir-recursion=5-l/root/ Binclamav.logclamscan-r/usr--max-dir-recursion=5-l/root/usrclamav.logclamscan-r--remove/usr/bin/ Bsd-portclamscan-r--remove/usr/bin/clamscan-r--remove/usr/local/zabbix/sbin

5. View Log Discovery

/bin/netstat:linux.trojan.agent found for viruses

grep found/root/usrclamav.log

/usr/bin/.sshd:linux.trojan.agent FOUND

/usr/sbin/ss:linux.trojan.agent FOUND

/usr/sbin/lsof:linux.trojan.agent FOUND

Appendix: Linux.backdoor.gates.5

After inquiry information, this trojan should be linux.backdoor.gates.5, find a document, the content is as follows:

Some users have a deep-rooted belief that there are currently no malicious software that can really threaten the Linux kernel operating system, but this concept is facing more and more challenges. Compared with April, the number of Linux malware detected by Doctor Web technicians in May 2014 has set a new record, and in June these malware lists added a series of newer Linux Trojans, This new Trojan family is named Linux.BackDoor.Gates.

Described here is a trojan in the malware family Linux.BackDoor.Gates: Linux.backdoor.gates.5, this malware combines the functionality of a traditional backdoor and a DDoS attack Trojan to infect a 32-bit Linux version, according to its characteristics can be determined, is with the Linu The X.dnsamp and Linux.ddos family Trojan is the same as the hand of a virus writer. The new Trojan is made up of two functional modules: The basic module is the backdoor that can execute the instructions issued by the Outlaws, and the second module is saved to the hard disk during the installation process for DDoS attacks. Linux.backdoor.gates.5 collects and forwards to criminals the following information about infected computers during the run:

• The number of CPU cores (read from/proc/cpuinfo).
  

• CPU speed (read from/proc/cpuinfo).

• CPU usage (read from/proc/stat).

• Gate ' A's IP (read from/proc/net/route).

• The MAC address of Gate ' A (read from/proc/net/arp).

• Network interface information (read from/proc/net/dev).

• The MAC address of the network device.

• Memory (using the Memtotal parameter in/proc/meminfo).

• The amount of data sent and received (read from/proc/net/dev).

• The operating system name and version (by calling the uname command).

When started, Linux.backdoor.gates.5 checks the path of its startup folder to implement four behavioral patterns based on the results of the check.

If the path of the backdoor executable file is inconsistent with the path of the Netstat, lsof, and PS tools, the Trojan will be disguised as a daemon to boot in the system and then initialized to decompress the configuration file during initialization. The configuration file contains all kinds of data necessary for the Trojan to run, such as the Management Server IP address and port, the backdoor installation parameters and so on.

Depending on the G_IGATSISFX parameter value in the profile, the Trojan or active connection manages the server, or waits for a connection: After a successful installation, the backdoor detects the IP address of the site to which it is connected and then takes the site as a command server.

The Trojan checks the file/tmp/moni.lock during installation, if the file is not empty, reads the data (PID process) and "kills" the ID process. The linux.backdoor.gates.5 then checks to see if the DDoS module and backdoor-owned processes are started in the system (they will also be "killed" if started). If the configuration file is set with a special flag g_iisservice, the Trojan is set to self-boot by writing to the command line in the file/etc/init.d/#!/bin/bash\n<path_to_backdoor> Then linux.backdoor.gates.5 create the following symbolic Links:

Ln-s/etc/init.d/dbsecurityspt/etc/rc1.d/s97dbsecuritysptln-s/etc/init.d/dbsecurityspt/etc/rc2.d/ S97dbsecuritysptln-s/etc/init.d/dbsecurityspt/etc/rc3.d/s97dbsecuritysptln-s/etc/init.d/dbsecurityspt/etc/ Rc4.d/s97dbsecurityspt

If a flag G_bdobackdoor is set in the configuration file, the Trojan will also attempt to open the/root/.profile file to check if its process has root privileges. The backdoor then copies itself to the/usr/bin/bsd-port/getty and starts. In the final phase of the installation, Linux.backdoor.gates.5 creates a copy again in the folder/usr/bin/, named the corresponding name set in the configuration file, and supersedes the following tools:

/bin/netstat/bin/lsof/bin/ps/usr/bin/netstat/usr/bin/lsof/usr/bin/ps/usr/sbin/netstat/usr/sbin/lsof/usr/sbin/ Ps

The Trojan completes the installation and begins to invoke the basic functionality.

When two other algorithms are executed, the Trojan also pretends to be a daemon that starts on the infected computer, checking that its components are started by reading the corresponding. lock file (starting the component if it is not started), but using a different name when saving the file and registering from startup.

After you set up the connection to the command server, Linux.backdoor.gates.5 receives the configuration data from the server and the commands that the zombie computer needs to complete. According to the instructions of the criminals, the Trojan can implement automatic Updates, initiate or stop a DDoS attack on the remote site that specifies the IP address and port, execute the commands contained in the configuration data, or make additional commands by establishing a connection to the remote site with the specified IP address.

The main DDoS attack target of the portal program is China's server, but the object of the rogue attack also includes other countries. The geographic distribution of DDoS attacks to take advantage of this Trojan:

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7A/9A/wKiom1atDxbAPpm7AAGEMXS_d8w613.png "title=" Qq20160131032907.png "alt=" Wkiom1atdxbappm7aagemxs_d8w613.png "/>

This article is from the "Little Water Drop" blog, please make sure to keep this source http://wangzan18.blog.51cto.com/8021085/1740113

The experience of a Linux server being hacked and removed from a Trojan horse program