How HTTP Evasions works: Deflate compression bypasses the Firewall

How HTTP Evasions works: Deflate compression bypasses the Firewall

This is the second article in the HTTP Evasions series. This article focuses on browsers or other devices that support compression, which may cause firewall bypass. In short, the firewall is bypassed by sending a compressed response packet.

Related Links: Article 1 Analysis of HTTP escape principles: using HTTP 0.9 for bypassing

 HTTP/1.1 200 ok Content-Encoding: deflate insert compressed malware here

What is compression in HTTP?

Because the compression/decompression method is easier to implement than the high-bandwidth data transmission method, the compression method is widely used in the delivery of corresponding packages. In the browser, Content compression is identified by the Content-Encoding field in the http header, as follows:

HTTP/1.1 200 okContent-Encoding: gzipContent-type: text/html...HTML content compressed with gzip

This compression method is the format in which the browser sends a request to the server through the request package. The format in the request package is as follows:

GET / HTTP/1.0Accept-Encoding: gzip, deflate

In fact, the current browser provides the compression methods "gzip" and "deflate", which are specified in the HTTP standard. Chrome also supports the "sdch" compression method, and Opera even provides the "lzma" method. In this article, we will only discuss "gzip" and "deflate.

Many readers may think that 'gzip' compression is a bit familiar, because they think it is similar to using the 'gzip' command to get the * .tar.gz file. Currently, Gizp is widely used to encode wen content, and almost all firewalls support this compression method. However, the encoding method "deflate" is different.

What is the Deflate encryption method?

Deflate is very similar to gzip. In short, gzip is a deflate with some headers and tails.

Of course, there are other packages in Deflate: zlib, which is often used to compress PNG images. The zlib format is commonly used by browsers. It is confusing that this encoding method is named 'deflate' in the browser '. Such disorder is often prone to misunderstanding, making most browsers accept zlib and deflate compression at the same time because their names are deflate.

In general, the following compression methods will be decompressed by the same algorithm (DEFLATE ):

N gzip-deflate with the header and tail, RFC1952: all browsers. n deflate-deflate algorithm with the zlib header and tail, RFC1950: all browsers except IE: ndeflate-original deflate without header and tail, RFC1951: all browsers

What happens if the firewall cannot identify the Deflate algorithm?

The most rational approach is, of course, to isolate these contents, because these may be dangerous. However, if this is done, a lot of normal data will be intercepted by the firewall, which is obviously not expected to be seen. When the firewall cannot identify the compression method, the packet is directly sent through the firewall. This seemingly user-friendly solution greatly facilitates attackers. Therefore, as long as the deflate compression method is used, malicious software can smoothly pass through some firewalls and security systems.

Worse, some malware may not need compression. As long as you simply change the Content-Encoding Content to a compression method that is not recognized by the firewall, the firewall will stop scanning this file, and even the firewall will not decompress the text:

 HTTP/1.0 200 ok Content-Encoding: foobar ...plain uncompressed malware ...

Why can't those firewalls detect malicious software in Gzip compression but Deflate?

An HTTP response package consists of the response header and response body. The compression algorithm is recorded in the response header, and the content in the response body is actually compressed. The most typical solution is that the firewall only sends the content of the response body to anti-virus software for analysis. Because gzip has a very easy-to-recognize header, anti-virus software can successfully detect this compression method and analyze the compressed content.

However, the Deflate compression method does not have such a unique identifier header. The compressed text looks more random. Therefore, anti-virus software cannot detect which compression method is used to compress the compressed part. At this time, you need to inform the anti-virus software of the compression method before sending the compressed text to the anti-virus software.

Why can't some firewalls process Deflate (RFC1951), but other Deflate (RFC1950 )?

We can guess that the firewall authors didn't realize that there were two Deflate methods or they thought they only needed to handle the methods supported by all browsers. Because no normal web Developer will use some compression methods that cannot be decompressed by all browsers. Unfortunately, attackers do not do this, and even prefer this method.

Can we pre-compress the server without using the Deflate algorithm?

This is also the practice of some products. They know from the Accept-Encoding field in the request header of the browser that the browser can decompress those files, and this field tells the server to compress the content using the compression method described in the Accept-Encoding field. However, this idea is also problematic because attackers can still perform those operations on their own controllable servers. As shown in this connection (Link)

Therefore, the firewall should not only pin its hopes on the server, but not trust the server too much. Instead, it should check whether the compression algorithm is used. However, all firewalls only seem to assume that attackers must abide by such rules without detecting whether compression is actually used. In reality, both ZScaler URL checker and Comodo Web Inspector make the same mistake. They only request whether there are any compression requests in the package, and blindly believe that the server will not deliver the compressed data, for more information, see.

Can I think that Deflate is a hacker?

Gzip is the most popular compression method among the 10000 servers counted by Alexa. However, many servers still use Deflate to compress data. Therefore, we cannot simply think that a server uses Deflate compression for attack behavior.

Last

Obviously, you cannot trust suppliers too much, so it is best to check whether the providers that say they are advanced threat protection systems are bragging. Download the HTTP Evader tool for testing if you want.