X-forwarded-for in the HTTP request header

Transferred from: https://imququ.com/post/x-forwarded-for-header-in-http.html

X-forwarded-for in the HTTP request header

I have always believed that the HTTP protocol and other common network knowledge are prerequisites for the Web front-end development students. On the one hand, many of the front-end work, such as Web performance optimization, most of the rules with HTTP, HTTPS, SPDY and TCP and other characteristics of the protocol directly correspond to, if not from the protocol itself but blindly follow the doctrine, it is likely to backfire. On the other hand, with the development of Node, more and more front-end students began to write service-side programs, even the framework (ThinkJS is developed by front-end engineers, and has many front-end engineers user Node framework), master the necessary network knowledge, for the server-side program security, deployment, Operations and other tasks are essential.

My blog has a "http related" topic, in the future will be updated more content into, welcome attention. Today is the X-forwarded-for (XFF) in the HTTP request header.

Background

By name you know that x-forwarded-for is an extension header. The http/1.1 (RFC 2616) protocol does not define it, it was first introduced by Squid, the cache proxy software, used to represent the HTTP request-side real IP, has now become the de facto standard, widely used by the large HTTP proxy, load balancer and other forwarding services, and is written to the RFC 7239 (forwarded HTTP Extension) standard.

The x-forwarded-for request header format is very simple:

X-Forwarded-For: client, proxy1, proxy2

As you can see, the content of XFF consists of several parts separated by "comma + Space", the first is the device IP farthest from the server, and then the IP of each level of proxy device.

If an HTTP request arrives at the server before, after three agents Proxy1, Proxy2, Proxy3,ip respectively for IP1, IP2, IP3, the user real IP is IP0, then according to the XFF standard, the server end will receive the following information:

X-Forwarded-For: IP0, IP1, IP2

Proxy3 Direct Connect server, it will give XFF append IP2, indicating it is to help Proxy2 forward the request. No IP3,IP3 in the list can be obtained from the Remote Address field on the server side. We know that the HTTP connection is based on a TCP connection, the HTTP protocol does not have the concept of IP, the Remote Address is from a TCP connection, the device IP that establishes the TCP connection with the server, in this case is IP3.

Remote Address cannot be forged because a TCP connection requires three handshakes, if the source IP is forged, the TCP connection cannot be established, and there is no subsequent HTTP request. Different languages get Remote Address the same way, for example, PHP is $_SERVER["REMOTE_ADDR"] , Node is req.connection.remoteAddress , but the principle is the same.

Problem

With the above background knowledge, start to say the question. I used Node to write a simplest Web Server for testing. The HTTP protocol has nothing to do with the language, where Node is just for illustrative purposes and the same conclusion can be obtained in any other language. In addition, this article with Nginx is the same truth, if interested, to Apache or other Web Server is the same.

The following code listens on the 9009 port and, after receiving the HTTP request, outputs some information:

js  var http = require ( ' http '); Http.createserver (function  ( Req, res) {Res.writehead (200, { ' Content-type ':  Text/plain '}); Res.write ( "remoteaddress: ' + req.connection.remoteAddress + " n '); Res.write ( "x-forwarded-for: ' + req.headers[ ' x-forwarded-for ' ] +  \ n '); Res.write ( "X-real-ip: ' + req.headers[ ' X-real-ip ') + 9009,  ' 0.0.0.0 ');       

This code, in addition to the previously described Remote Address and X-Forwarded-For , there is another X-Real-Ip , this is a custom header. X-Real-Iptypically used by an HTTP proxy to represent a device IP that generates a TCP connection to it, the device may be a different proxy, or it may be a true requester. It is important to note that there is no X-Real-Ip standard at this time, and the proxy and the WEB application can contract with any custom header to pass this information.

The Node service can now be accessed directly with the domain name + port number, with an Nginx reverse proxy:

Nginxlocation / {    proxy_set_header X-Real-IP $remote_addr;    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-NginX-Proxy true; proxy_pass http://127.0.0.1:9009/; proxy_redirect off;}

My nginx listening 80 port, so I can access the service without a port to the Nginx forwarded.

To test direct access to the Node service:

SHELLcurl http://t1.imququ.com:9009/remoteAddress: 114.248.238.236x-forwarded-for: undefinedx-real-ip: undefined

Since my computer is connected directly to the Node service, the Remote Address is my IP. At the same time I did not specify an extra custom header, so the latter two fields are undefined.

Again to access the Nginx forwarded service:

SHELL127.0.0.1x-forwarded-for: 114.248.238.236x-real-ip: 114.248.238.236

This time, my computer was accessing the Node service via Nginx, and the Remote Address was actually an nginx local IP. These two lines in the front Nginx configuration work, adding two additional custom headers to the request:

proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

In fact, the deployment of WEB applications in a production environment generally uses the second approach above, the benefits are many, specifically which is not the focus of this article is not written. This introduces a vulnerability: many WEB applications obtain IP from the HTTP request header in order to obtain the user's true IP.

The HTTP request header can be constructed at will, and we'll test it by constructing the parameters of Curl -H X-Forwarded-Fox X-Real-Ip .

Direct access to the Node service:

SHELLcurl http://t1.imququ.com:9009/ -H ‘X-Forwarded-For: 1.1.1.1‘ -H ‘X-Real-Ip: 2.2.2.2‘remoteAddress: 114.248.238.236x-forwarded-for: 1.1.1.1x-real-ip: 2.2.2.2

For WEB applications, X-Forwarded-Fox and X-Real-Ip that is two ordinary request headers, nature does not do any processing as is output. This means that for direct-attach deployment, the IP information carried in the request header is not trusted except for the Remote Address obtained from the TCP connection.

Access to Nginx forwarded services:

SHELL‘X-Forwarded-For: 1.1.1.1‘ -H ‘X-Real-Ip: 2.2.2.2‘remoteAddress: 127.0.0.1x-forwarded-for: 1.1.1.1, 114.248.238.236x-real-ip: 114.248.238.236

This time, Nginx will X-Forwarded-For append my IP and overwrite the request header with my IP X-Real-Ip . This shows that with Nginx processing, the X-Forwarded-For last section and X-Real-Ip the entire content cannot be constructed and can be used to obtain the user's IP.

User IPs are often used in Web-security-related scenarios, such as checking the user's login area, IP-based access control, and so on. In this scenario, it is more important to ensure that the IP cannot be constructed. With the previous testing and analysis, for Web applications that are directly user-oriented, the Remote Address obtained from the TCP connection must be used, and for Web applications that have a reverse proxy such as Nginx deployed, you can use Nginx when the Set Header behavior is configured correctly Pass through X-Real-Ip or the X-Forwarded-Ip last section (in fact they must be equivalent).

So, how does the WEB application itself tell if the request is coming directly or is it being forwarded by a controlled proxy? Adding extra headers When forwarding is a method, but not insurance, because the request header is too easy to construct. If you must use this, this custom head should be long enough to be rare, but also to keep out of the leak.

Determine whether Remote address is a local IP is also a way, but not perfect, because in the Nginx server access, whether directly connected or go Nginx proxy, Remote address are 127.0.0.1. This problem is usually negligible, and more troubling, the reverse proxy server and the actual WEB application may not necessarily be deployed on the same server. Therefore, it is more reasonable to collect all proxy IP lists, the Web application to the Remote Address after the comparison to determine how to access.

In general, in order to simplify logic, the production environment will block the form of direct access to the Web app via a port with the ports, allowing access only through Nginx. Does that mean it's no problem? And not necessarily.

First, if the user is really through the proxy access Nginx, the X-Forwarded-For last section and X-Real-Ip get the IP of the proxy, security-related scenarios can only use this, but some scenarios such as the location of the weather according to IP, you need to obtain the user's real IP, when X-Forwarded-For the first IP can be To be in handy. At this point you need to pay attention to a problem, or take the previous example to test:

SHELL‘X-Forwarded-For: unknown, <>"1.1.1.1‘remoteAddress: 127.0.0.1x-forwarded-for: unknown, <>"1.1.1.1, 114.248.238.236x-real-ip: 114.248.238.236

X-Forwarded-ForThe last section is appended to the Nginx, but the previous part is from the Nginx received the request header, this part of the user input content is completely untrusted. Use with extreme care, in accordance with the IP format in order to use, or can easily lead to SQL injection or XSS and other security vulnerabilities.

Conclusion

1. WEB applications that provide services directly to the outside, in the security-related operations, only through the Remote Address to obtain IP, can not trust any request header;
2. Web applications that use Web server such as Nginx to reverse proxy, with the correct configuration, to use the X-Forwarded-For last section or X-Real-Ip to obtain the IP (because the Remote Address is the Nginx server's intranet IP); Prohibit WEB applications from providing services directly to external providers;
3. In non-security-related scenarios, such as the location of the weather via IP, you can X-Forwarded-For obtain the IP from the front position, but you need to verify the legitimacy of the IP format;

PS: Some articles on the internet suggest that this configuration Nginx, in fact, not reasonable:

proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $remote_addr;

After this configuration, security does improve, but it also causes all proxy information before the request arrives at Nginx to be erased, which does not provide a better service to the user who actually uses the proxy. Still should understand this intermediate principle, concrete scene concrete analysis.

This article link: https://imququ.com/post/x-forwarded-for-header-in-http.html

X-forwarded-for in the HTTP request header