Remote Host OS Fingerprint Recognition

Operating System (OS) identification is an important information to be collected for intrusion or security detection. It is the basis for analyzing vulnerabilities and various security risks. The security status of a remote host can be further evaluated only when the operating system type and version of the remote host are determined.
Since the TCP/IP protocol stack is described only in the RFC documentation, there is no unified implementation standard, as a result, companies have made different interpretations of the RFC documentation when writing TCP/IP protocol stacks for their own operating systems, as a result, different operating systems have different TCP/IP protocols. Therefore, the TCP/IP protocol stack in the operating system can be used as a special "fingerprint ", determine the operating system type by identifying the differences in the TCP/IP protocol stack of different operating systems.
1. Active protocol stack Fingerprint Recognition
1) FIN Detection
Send a FIN group (or packet without ACK or SYN flag) to an opened port on the target host and wait for a response. Many operating systems, such as Windows NT, cisco ios, HP/UX, and IRIX, return a Reset, and some do not respond.
2) BOGUS mark Detection
Send a SYN packet containing the undefined TCP mark and TCP header to the target host. Some operating systems such as Linux will include this undefined mark in the response, some other operating systems will close the connection when they receive such a package.
3) initialize the serial number sampling test
Find the rule between the value of the initialization serial number (ISN) and the specific operating system. For example, the initialization serial number of early UNIX systems increases by 64 K, while some new UNIX systems such as Solaris, IRIX, FreeBSD, Digital UNIX, and Cray randomly increase the value of the initialization serial number.
4) Don't Fragment bit Detection
Some operating systems will set the IP header "Don't Fragment (DF) bit" (not sharding bit) to improve performance. To monitor this bit, you can determine to distinguish remote OS.
5) TCP initial window size detection
This method checks the size of the window contained in the returned data packet. Some operating systems set this field to a unique value when implementing the TCP/IP protocol stack. For example, if AIX is 0x3F25, Windows NT and BSD are 0x402E.
6) TCP option Detection
The operating system is determined based on the returned packet content based on some TCP options set in the sent TCP packet.
7) ACK value detection
Find the differences and rules in setting ACK serial numbers for different operating systems. Some operating systems will set it to the serial number of the TCP packet confirmed, while others will add the serial number of the TCP packet confirmed to 1 as the ACK serial number to return.
8) ICMP error message Suppression
Some operating systems limit the rate at which ICMP error messages are returned. Send some UDP packets to a randomly selected high port and count the number of inaccessibility error messages received in a given period of time.
9) ICMP Error Message Reference
When an error message needs to be sent, the Information volume of the original network package referenced by different operating systems is different. The operating system type can be roughly determined by detecting the messages referenced in the returned ICMP error message.
10) ICMP error message echo integrity
Some operating systems modify the referenced IP address header when returning an ICMP error message to the TCP/IP protocol stack. You can roughly judge the operating system by checking its changes to the IP address header.
11) TOS service type
The TOS field used to detect messages inaccessible to ICMP ports. Most operating systems will be 0, while others will not.
12) segment Processing
Different TCP/IP protocol stacks have different processing methods for overlapping fragments. Some operating systems use new data to overwrite old data during restructuring, while others use the opposite.
2. Passive protocol stack Fingerprint Recognition
Active protocol stack fingerprint recognition needs to actively send data packets to the target, but because data packets do not appear in this order when the network is used normally, these data packets are quite noticeable in network traffic, it is easy to be captured by IDS. Passive protocol stack fingerprint recognition is required to improve concealment. The principle is similar to the fingerprint recognition of the active protocol stack. However, it never actively sends data packets, but passively captures the packets returned by the remote host to analyze the operating system type. Generally, let's look at the four aspects.
1) TTL value
The operating system sets the survival time for the outbound data packets.
2) Windows Size
The size of the TCP window set by the operating system. This size is included when the FIN information package is sent.
3) DF
You can check whether the operating system has set an inaccurate part bit.
4) TOS
Check whether the service type is set in the operating system.
Analyze these properties and compare the results with the property library to determine the remote operating system type. Of course, the detected system cannot be 100% correct, nor can it rely on the single signal features above to determine the operating system type. However, by viewing multiple signal features, we can combine enough differences, this greatly improves the accuracy of remote host system judgment.