Http Fingerprint Recognition Technology

The editor judges from the author. This is also an old article.

Author: the wings of desire Source: langke Alliance

Http fingerprint recognition has become a new topic in application security. Http server and Http application security have become an important part of network security. from the perspective of network management, maintaining monitoring and tracking of various web servers makes Http fingerprint recognition easy. Http fingerprint recognition can automate information systems and security policies, on a special platform or web server that has configured audit policies, the security testing tool can use Http fingerprint recognition to reduce the configuration required for testing.

This article will introduce the existing Http Fingerprint Recognition Technology and discuss some problems encountered and provide feasible solutions.

I. Fingerprint Recognition Theory

The fingerprint is defined as follows: 1 refers to the impression of any surface fingertip or the ink on the paper that is dipped in the ink on the fingertip; 2 refers to the thing that can be used for identification: such as characteristics, traces, or characteristics that reveal the origin, evidence of the characteristics of an object or material.

Fingerprint recognition can be divided into two steps: one is to collect and classify fingerprints, and the other is to compare the unknown fingerprints stored in the database to find the most suitable ones.

When collecting fingerprints, it is necessary to capture all the main features of an object. collecting more details can greatly help the second step. when comparing fingerprints, it is very likely that several fingerprints are not properly matched, Because tiny differences between fingerprints can easily lead to incorrect identification, this also requires high technology for fingerprint recognition.

Fingerprint recognition technology is now a known technology for network security. operating System fingerprint recognition is a common task in any network evaluation. Currently, many operating system fingerprint recognition technologies are available. why is operating system fingerprint recognition successful? That is because each operating system has a slight difference in TCP/IP protocol implementation. Currently, it is popular to use the TCP/IP stack for post-operating system identification. Related tools such as Nmap; some other tools, such as Xprobe, use ICMP to identify operating systems.

The principle of Http fingerprint recognition is also the same: records the tiny differences in Http protocol execution by different servers. http fingerprint recognition is much more complex than TCP/IP stack fingerprint recognition, because custom Http server configuration files, adding plug-ins or components make it easy to change Http response information, this makes it difficult to identify; however, the custom TCP/IP stack behavior needs to be modified on the core layer, so it is easy to identify.

2. Get Banner

Viewing the Http response header information is the simplest and most basic for Http fingerprint recognition. We can use a Tcp client such as netcat. Generally, we use telnet to port 80, then, send a command to get the response information. Here we use netcat as an example. The following is the response information of three different Http servers:

1: Apache 1.3.23 server:
Http/1.1 200 OK
Date: Mon, 08 Sep 2003 17:10:49 GMT
Server: Apache/1.3.23
Last-Modified: Mon, 08 Sep 2003 03:48:19 GMT
ETag: "32417-c4-3e5d8a83"
Accept-Ranges: bytes
Content-Length: 196
Connection: close
Content-Type: text/html

2: Microsoft IIS 5.0 server:
Http/1.1 200 OK
Server: Microsoft-Microsoft IIS/5.0
Expires: Mon, 08 Sep 2003 01:41:33 GMT
Date: Mon, 08 Sep 2003 16:41:33 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Mon, 08 Sep 2003 15:32:21 GMT
ETag: "b0aac0542e25c31: 89d"
Content-Length: 7369

3: Netscape Enterprise 4.1 server:
Http/1.1 200 OK
Server: Netscape-Enterprise/4.1
Date: Mon, 08 Sep 2003 16:19:04 GMT
Content-type: text/html
Last-modified: Mon, 08 Sep 2002 15:37:56 GMT
Content-length: 57
Accept-ranges: bytes
Connection: close

Iii. Fuzzy server Banner information

In many cases, obtaining Banner proves to be a good method for Http fingerprint recognition. however, the network administrator will choose to modify or blur the Banner information of the server by configuring or adding plug-ins. of course, this setting automatically blocks many Http server attacks.

It is very easy for the server to return different Banner information. For an open-source Http server like Apache, users can modify the Banner information in the source code, then the Http service will take effect again. For Http servers that do not have open source code, such as Microsoft's IIS or Netscape, they can be modified in the Dll file where Banner information is stored. Related articles have been discussed, I will not repeat it here. Of course, this modification has a good effect.

Another method to blur Banner information is to use plug-ins that provide custom Http response information. for example, the commercial software ServerMask can provide such a function, which is a plug-in of the IIS server. ServerMask not only blur the Banner information, in addition, it will re-combine the sequence of items in the Http response header to imitate servers such as Apache. It even has the ability to act as any Http server to process every request. this software can be found:

Http: // www.port80software.com/products/servermask

The following is an example of an Http server with custom Banner information. the Apache server is defined as an Unknown Server:

Http/1.1 403 Forbidden
Date: Mon, 08 Sep 2003 02:41:27 GMT
Server: Unknown-Webserver/1.0
Connection: close
Content-Type: text/html; charset = iso-8859-1

The following is an example of an IIS server using the ServerMask plug-in:

Http/1.1 200 OK
Server: Yes we are using ServerMask
Date: Mon, 08 Sep 2003 02:54:17 GMT
Connection: Keep-Alive
Content-Length: 18273
Content-Type: text/html
Set-Cookie: It works on cookies too = 82.3S3.O12.NT2R0RE, 4147ON3P,. 4OO.; path =/
Cache-control: private

From the above example, we can see that it is far from enough to judge the Http server type from the Banner information.

Iv. Protocol Behavior

When the Http protocol is executed, almost all Http servers have their unique methods. If the Http request is legal and rule-based, the response information returned by the Http server complies with the RFC description. however, if we send malformed Http requests, the response information of these servers will be different. The difference in Http protocol behavior between different servers is the basic basis and principle of Http fingerprint recognition technology.

Let's take a few different examples. We will analyze the response information returned by the three different Http services for different requests. These requests are as follows:

1: Send basic Http requests to HEAD/Http/1.0
2: DELETE/Http/1.0 sends unpermitted requests, such as Delete requests
3: GET/Http/3.0 sends an invalid Http Request
4: GET/JUNK/1.0 sends an incorrect Http Request

Exp1: Basic Http Request

We first send the request HEAD/Http/1.0, then analyze the information in the Http response header, and analyze the sorting of items in the header information. The sent Request command is as follows:

C: \> nc apache.example.com 80 // press enter, the same below
HEAD/Http/1.0 // enter and press enter, the same below

Response Information:

1: Apache 1.3.23
Http/1.1 200 OK
Date: Mon, 08 Sep 17:10:49 GMT
Server: Apache/1.3.23
Last-Modified: Thu, 27 Feb 2003 03:48:19 GMT
ETag: "32417-c4-3e5d8a83"
Accept-Ranges: bytes
Content-Length: 196
Connection: close
Content-Type: text/html

2: IIS 5.0
Http/1.1 200 OK
Server: Microsoft-Microsoft IIS/5.0
Content-Location: Http: // iis.example.com/Default.htm
Date: Mon, 08 Sep 20:13:52 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Mon, 08 Sep 2003 10:10:50 GMT
ETag: W/"e0d362a4c335be1: ae1"
Content-Length: 133

3: Netscape Enterprise 4.1
Http/1.1 200 OK
Server: Netscape-Enterprise/4.1
Date: Mon, 08 Sep 2003 06:01:40 GMT
Content-type: text/html
Last-modified: Mon, 08 Sep 2003 01:37:56 GMT
Content-length: 57
Accept-ranges: bytes
Connection: close

Comparison result: the Server and Date items in the Apache header are sorted differently.

Exp2: Http DELETE request

This time, we will send DELETE/Http/1.0 requests. We will analyze the different response information of different Http servers to illegal requests. The sent Request command:

C: \> nc apache.example.com 80
DELETE/Http/1.0

Response Information:

1: Apache 1.3.23
Http/1.1 405 Method Not Allowed
Date: Mon, 08 Sep 2003 17:11:37 GMT
Server: Apache/1.3.23
Allow: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, PATCH, PROPFIND, PROPPATCH,
MKCOL, COPY, MOVE, LOCK, UNLOCK, TRACE
Connection: close
Content-Type: text/html; charset = iso-8859-1

2: IIS 5.0
Http/1.1 403 Forbidden
Server: Microsoft-Microsoft IIS/5.0
Date: Mon, 08 Sep 2003 20:13:57 GMT
Content-Type: text/html
Content-Length: 3184

3: Netscape Enterprise 4.1
Http/1.1 401 Unauthorized
Server: Netscape-Enterprise/4.1
Date: Mon, 08 Sep 2003 06:03:18 GMT
WWW-authenticate: Basic realm = "WebServer Server"
Content-type: text/html
Connection: close

Comparison results: Apache responded to "405 Method Not Allowed", IIS responded to "403 Forbidden", and Netscape responded to "401 Unauthorized, the response information is completely different.

Exp3: Illegal Http Protocol Version Request

This time, we will send illegal Http Protocol version requests, such as GET/Http/3.0 requests. In fact, Http 3.0 does not exist. Send the Request command:

C: \> nc apache.example.com 80
GET, Http, 3.0

Response Information:

1: Apache 1.3.23
Http/1.1 400 Bad Request
Date: Mon, 08 Sep 2003 17:12:37 GMT
Server: Apache/1.3.23
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset = iso-8859-1

2: IIS 5.0
Http/1.1 200 OK
Server: Microsoft-Microsoft IIS/5.0
Content-Location: Http: // iis.example.com/Default.htm
Date: Mon, 08 Sep 2003 20:14:02 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Mon, 08 Sep 2003 20:14:02 GMT
ETag: W/"e0d362a4c335be1: ae1"
Content-Length: 133

3: Netscape Enterprise 4.1
Http/1.1 505 Http Version Not Supported
Server: Netscape-Enterprise/4.1
Date: Mon, 08 Sep 2003 06:04:04 GMT
Content-length: 140
Content-type: text/html
Connection: close

Comparison result: Apache responds to "400 Bad Request". IIS ignores this Request, returns the OK response information, and returns the HTML data in the root directory of the website, netscape responded to the "505 Http Version Not Supported ".

Exp4: Incorrect rule Protocol Request

This test mainly refers to the response to the GET/JUNK/1.0. Request and sends the Request command:

C: \> nc apache.example.com 80
GET/JUNK/1.0

Response Information:

1: Apache 1.3.23
Http/1.1 200 OK
Date: Sun, 15 Jun 2003 17:17:47 GMT
Server: Apache/1.3.23
Last-Modified: Thu, 27 Feb 2003 03:48:19 GMT
ETag: "32417-c4-3e5d8a83"
Accept-Ranges: bytes
Content-Length: 196
Connection: close
Content-Type: text/html

2: IIS 5.0
Http/1.1 400 Bad Request
Server: Microsoft-Microsoft IIS/5.0
Date: Fri, 01 Jan 1999 20:14:34 GMT
Content-Type: text/html
Content-Length: 87

3: Netscape Enterprise 4.1
<HTML> <HEAD> <TITLE> Bad request </TITLE> </HEAD>
<BODY> <H1> Bad request </H1>
Your browser sent a query this server cocould not understand.
</BODY> </HTML>

Comparison result: Apache ignores the irregular protocol "JUNK" and returns some information about 200 "OK" and the root directory. IIS responds to "400 Bad Request ", netscape returns almost no Http header information, but returns HTML information to indicate that this is an incorrect request.

Test Summary: A table is listed below. We can easily identify different Http servers.

Server header information item sorting Delete request illegal version irregular Protocol
Apache/1.3.23 Date, Server 405 400 200
MS-IIS/5.0 Server, Date 403 200 400
Netscape4.1 Server, Date 401 505 no header

5. fingerprint recognition tools

Here we will introduce an Http fingerprint recognition tool Httprint, which can effectively determine the type of Http server by combining fuzzy logic technology with statistical principles. it can be used to collect and analyze the signatures generated by different Http servers. What is an Http signature? The Http signature is converted into ASCII code in hexadecimal notation as follows:

Microsoft-Microsoft IIS/5.0
Bytes
0D7645B5811C9DC52A200B4C9D69031D6014C217811C9DC5811C9DC52655F350
FCCC535BE2CE6923E2CE6923F2454256E2CE69272576B769E2CE6926CD2698FD
6ED3C295E2CE692009DB9B3E6ED3C2956ED3C2956ED3C2956ED3C295E2CE6923
6ED3C295

Apache/1.3.x
9E431BC86ED3C295811C9DC5811C9DC5050C5D32505FCFE84276E4BB630A04DB
0D7645B5970EE6BB811C9DC5CD37187C11DDC7D78398721EB06FE5D78A91CF57
Bytes
6ED3C295E2CE69262A200B4C811C9DC5811C9DC5811C9DC5811C9DC5811C9DC5
811C9DC5

Httprint first saves some Http signature information in a document, and then analyzes the results generated by the Http server.

When we find that there is no signature information in the database, we can use the report generated by Httprint to expand the signature database. When Httprint runs for the next time, the newly added signature information can also be used.

Httprint can be run on the GUI and command line, and can be run on Windows, Linux, and Mac OS X platforms.

The following is a help file under the command line:

D: \> Httprint
Usage: Httprint {-h -H -I <input file> a file containing the test address. The source file is input.txt.
-S <signatures> a file containing the httpsignature, namely signatures.txt
Options:
-O <output file> the default report file is "Httprintoutput.html". You can customize it yourself.
-Tp <ping timeout> Ping timeout time. The default value is 1000 ms. The maximum value is 30000 ms.
-T <timeout> connection and read timeout. The default value is 10000 ms. The maximum value is 100000 ms.
-R <retry> time. The default value is 3. The maximum value is 30.
-P0 does not Ping the host.
-? Help information.

Example:

Httprint-h www.target.com-s signatures.txt
Httprint-h Https: // www.target.com-s signatures.txt
Httprint-h Http: // www.target.com: 8080/-s signatures.txt
Httprint-h 10.0.1.1-10.0.1.254-s signatures.txt-o 10_0_20.x.html
Httprint-I input.txt-s signatures.txt-o output.html

The usage is very simple. The following is an output of Httprint:

D: \> Httprint-h Http: // www.target.com-s signatures.txt
Host: www.target.com is alive...
Finger Printing on Http: // www.target.com: 80/
Derived Signature:
Apache/1.3.26 (Unix)
9E431BC86ED3C295811C9DC5811C9DC5050C5D32505FCFE84276E4BBC184CB92
0d7645b5811c9dc52a200b4c811c9dc1_ddc7d78398721e811c9dc5811c9dc5
E2CE6923E2CE6923E2CE6923811C9DC568D17AAE2576B769E2CE6926811C9DC5
E2CE6923E1CE67B1811C9DC5E2CE6920E2CE69206ED3C2956ED3C295E2CE6920
E2CE69206ED3C295811C9DC568D17AAEE2CE6923

Banner Reported: Apache/1.3.26 (Unix)
Banner Deduced: Apache/1.3.26
Scores:
Microsoft-IIS/4.0: 47
Microsoft-IIS/5.0: 57
Microsoft-IIS/5.0 ASP. NET: 57
Microsoft-IIS/5.1: 57
Microsoft-IIS/6.0: 75
.................... // Skip some
Apache/2.0.x: 70
Apache/1.3.27: 79
Apache/1.3.26: 80
Apache/1.3. [4-24]: 78
...................... // Skip some
Com21 Cable Modem: 49

In the preceding example, Httprint displays the signature information of the server under test, compares the information with the existing signature in the database, and then rates Each fingerprint, the highest score is the most suitable. in this example, "Apache/1.3.26" is the most suitable one.

Httprint generates an HTML report, which will play a significant role in future analysis. Httprint is powerful. Although you can use software such as ServerMask to blur fingerprints, it can still be defeated by software such as Httprint that uses statistical principles for identification. The following is an example:

D: \> Httprint unknown.example.com
Reported signature:
Protected by ServerMask
CD2698FD6ED3C295811C9DC5811C9DC5811C9DC594DF1BD04276E4BB811C9DC5
0D7645B5811C9DC5811C9DC59D69031D6014C217811C9DC5811C9DC580FF2CD2
FCCC535BE2CE6923E2CE6923811C9DC5E2CE69272576B769E2CE69262CEAB43E
Bytes
6ED3C295
Best Match: Microsoft-IIS/5.0, Microsoft-IIS/5.1
Scores:
Microsoft-IIS/4.0: 86
Microsoft-IIS/5.0: 101
Microsoft-IIS/5.1: 101
Microsoft-IIS/6.0: 56
.............................
Apache/1.3.27: 35
Apache/1.3.26: 36
Apache/1.3.x: 34
..............................
Min iserv/0.01: 15

Although the server uses ServerMask to confuse the response information, Httprint still accurately determines that it is Microsoft-IIS/5.0 or 5.1

:

WEB Server fingerprint tool httprint build 301 (beta)

Free Software recommendations
Software name WEB Server fingerprint tool httprint
Latest Version build 301 (beta)
Software size: 793 KB
Last Updated: 2005-12-22
Software language English
Application Platform Windows/Linux/Mac/FreeBSD
Contact Person httprint@net-square.com
Released by net-square solutions pvt. ltd.
Free Software (no plug-ins)
The license agreement is free for personal, educational, and non-commercial use.
For commercial authorization, please contact the email above
We recommend that you use it for free for non-commercial use.
Used to collect WEB application fingerprints during WEB Security Testing

Software introduction:

Httprint is a WEB server fingerprint tool. It relies on the characteristics of Web servers to accurately identify Web servers, even though they may confuse facts by changing the server bannerstrings or by using plug-ins like mod_security or servermask. Httprint can also be used to detect network function devices without server flag strings, such as wireless access points, routers, switches, and cable modems. Httprint uses a text signature string and easily adds the signature to the signature database. For more information, see Introduction to HTTP fingerprinting.

Features:

* Identification of web servers despite the banner string and anyother obfuscation. httprint can successfully identify the underlyingweb servers when their headers are mangled by either patching thebinary, by modules such as mod_security.c or by commercial products such as ServerMask. click here to see an example of how httprint detects disguised servers.
* Inventorying of web enabled devices such as printers, routers, switches, wireless access points, etc. Click on the sample HTML report.
* Customisable web server signature database. To add newsignatures, simply cut and paste the httprint output against unknownservers into the signatures text file.
* Confidence Ratings. httprint now picks the best matchesbased on confidence ratings, derived using a fuzzy logic technique, instead of going by the highest weight. more details on thesignificance of confidence ratings can be found in section 8.4 of the Introduction to HTTP fingerprinting paper.
* [New] Multi-threaded engine. httprint v301 is acomplete re-write, featuring a multi-threaded restart, to processmultiple hosts in parallel. this greatly saves scanning time. * multi-threading is not yet supported in the FreeBSD version.
* [New] SSL information gathering. httprint nowgathers SSL certificate information, which helps you identify expiredSSL certificates, ciphers used, certificate issuer, and other such SSLrelated details.
* [New] Automatic SSL detection. httprint can detectif a port is SSL enabled or not, and can automatically switch to SSLconnections when needed.
* Automatic traversal of HTTP 301 and 302 redirects. manyservers who have transferred their content to other servers send adefault redirect response towards all HTTP requests. httprint nowfollows the redirection and fingerprints the new server pointed. this feature is enabled by default and can be turned off, if needed.
* Ability to import web servers from nmap network scans. httprint can import nmap's xml output files.
* Reports in HTML, CSV and XML formats.
* Available on Linux, Mac OS X, FreeBSD (command line only) and Win32 (command line and GUI ).

:

Platform version address md5

Win32 graphics/command line version 301 download a66408308c3f540030bbb0d59716b032
Http://net-square.com/httprint/httprint_win32_301.zip

Download af53704de9c1851bd439cbe3fab3e0ad from Linux 301
Http://net-square.com/httprint/httprint_linux_301.zip

Download 6b188cd60df6eca5409694fa40859f0d from Mac OS X 301
Http://net-square.com/httprint/httprint_macosx_301.zip

Download d5efd9463f671ce92f50ce3222f1774e from FreeBSD 301
Http://net-square.com/httprint/httprint_freebsd_301.zip