The Web Proxy server is the intermediary entity of the network. The agent sits between the client and the server and acts as a "middleman", sending HTTP messages back and forth between the endpoints. This chapter describes all of the HTTP proxy server-related content, special support for proxy features, and the use of a proxy server when you encounter
To some of the thorny issues. -referenced from the HTTP authoritative guide
The application of the agent must be familiar to everyone, no longer too much description, simple and expressive.
1:fq
2:anonymous
3: "Increase speed"
Ha, this does not say everybody also understand, formerly in the school dormitory's small partner plays the game often uses, various unicom crosses the telecommunication and so on (OK, does not send ad~)
Of course, this non-trivial HTTP proxy can do that.
4:xxxx
(Some time ago a word is called "can't describe" well, everybody knows how to scatter:-)
Of course, agents have many functions, such as: LAN Internet management, many years ago, "Green @ Dam Plan"
Agent principle
The point here is that the Web proxy differs from the gateway:
The agent connects two or more applications that use the same protocol, while the gateway connects two or more endpoints that use different protocols. The gateway acts as a "protocol converter", even if the client and server are using different protocols, and the client can also complete the transaction with the server through it. Connect the HTTP protocol with the SMTP protocol when sending mail.
There are two kinds of agents: General agent and tunnel agent
General Agent:
-Referenced from the HTTP authoritative guide
The agent here is both a server and a client
Tunnel Proxy
-Referenced from the HTTP authoritative guide
The gateway allows a client to authenticate with the tunnel before it can be used
(to reduce the misuse of tunnels, the gateway should be only for specific well-known ports, such as port 443 for HTTPS)
Comparison
- Ordinary proxies can be used to host HTTP traffic;
- Tunnel proxies can be used to host any TCP traffic, including HTTP and HTTPS.
Here respectively take the market's Landing page, 58 of the main station page to do a simple test.
Wireshark appearances
Capturing HTTP ( as an example of the main station)
Capturing TLS (formerly SSL) (as an example of a market login)
Follow TCP Steam
You can see that the market is using VeriSign 's certificate .
Wireshark is very powerful, it is available on every platform. There are many examples on the internet, one of the best helpers to grab a bag.
1. Frame: Physical Layer-data frame 2. Ethernet II: Data Link Layer Ethernet frame header information3. Internet Protocol Version IP packet header information 4. Transmission Control Protocol: Data segment header information for the transport layer, such as TCP5. Hypertext Transfer Protocol: Application layer information, such as the HTTP protocol in
Since we are here, let's review the three-time handshake .
(-。 - ~ ~)
OK, simple diagram review, Four waves here will not unfold.
Another comb.
Wireshark Filter Usage 1:IP. ADDR/SRC/DST to filter target IP and source IP 2:Port tcp.port eq or udp.port eq tcp.dstport = = tcp.srcport tcp.port >= 1 and tcp.port <= 3: common Protocols support non- TCP udp arp ICMP http SMTP ftp DNS msnms IP SSL OICQ BOOTP 4: filter mac
。
。
。
And so on, there are many, specific can look at the official, very strong said.
Gee, it seems that the deviation of the run, to go back to the topic of this article continue to talk about proxy IP HTTP Head related
Level comparison
|
Remote_addr |
Http_x_forwarded_for |
Hide Identities |
Identify difficulty |
Agent not used |
Real IP |
No |
No |
|
Transparent proxy |
Proxy IP |
Real IP |
No |
Easy |
Anonymous proxy |
Proxy IP |
Proxy IP |
Is |
Easy |
Fraud Agent |
Proxy IP |
Random IP |
Is |
Easy |
High Anonymous Proxy |
Proxy IP |
No |
Is |
Difficult |
Here is a simple summary of the table, it is clear that the high-stealth proxy IP is the most challenging, but also the most common.
Http Head
- Http Common head and identify IP related to the following several, of which only the remote_ADDR This field is reliable . The rest of the fields can be forged. Because a TCP connection requires three handshakes, a TCP connection cannot be established if the source IP is forged.
- Because only the REMOTE_ADDR field of the previous layer can be recognized, it is cumbersome for multi-tier proxies. Such as:
- Remote_addr
- X-forwarded-for
- X-real-ip
- Proxy-connection
A brief discussion of -forwarded-for
x-forwarded-for [+] |
A de facto standard for identifying the originating IP address of a client connecting to a Web server through an HTTP ProX Y or Load Balancer |
X-forwarded-for:client1, Proxy1, proxy2x-forwarded-for:129.78.138.66, 129.78.64.103 |
Https://en.wikipedia.org/wiki/X-Forwarded-For
https://tools.ietf.org/html/rfc7239
The x-forwarded-for (xff< Span lang= "en-us") http header field was a common method for Identifying the Originating ip address of a client connecting To A web server through an http proxy or load balancer rfc 7239 standardized a new Forwarded header with similar purpose but more features compared to XFF. [1] an example of a forwarded header syntax:
forwarded:for=192.0.2.60; Proto=http; by=203.0.113.43
A simple understanding is one that comes out a non-standard but is commonly used by the industry to identify a proxy HTTPhaed.
in other words, If this parameter is not present in the case of direct connection
When configuring a reverse proxy such as nginx( about Nginx,
There are many references on the Internet, and I have also excerpted some http://www.cnblogs.com/wenthink/p/nginxsetting.html .
12 |
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; |
- Test Address:
- Http://wenthink.com/headdemo
Simple forgery : Curl http://wenthink.com/headdemo - h ' x-forwarded-for:192.168.8.8 '- H ' X-real-ip: 192.168.8.8'
Proxy-connection
PROXY-CONNECTION[23] |
Implemented as a misunderstanding of the HTTP specifications. Common because of mistakes in implementations of early HTTP versions. Have exactly the same functionality as standard Connection field. |
Proxy-connection:keep-alive |
This field is also a common non-standard requirement field, in short, the possibility that this field can be judged as a proxy IP is extremely high.
Do a simple configuration proxy IP here do not show
Again please out Wireshark
Well mask A bit more, in fact, is to verify that more than one proxy-connection this request header.
Recognition
What about the recognition of proxy IPs?
There are a lot of common sayings on the Internet.
- Reverse detection Scan IP is not open the 80,8080 and other proxy server often open port, obviously an ordinary user IP is not likely to open the port as above.
- HTTP x_forwarded_for: http proxy IP can be used to identify whether the proxy IP, if with XFF information, the IP is proxy IP no doubt.
- Keep-alive Message: If the keep-alive message with Proxy-connection , the IP is undoubtedly the proxy IP.
- View IP Ports : If an IP has a port larger than 10000, then the IP is mostly problematic, the normal home IP to open such a large port is almost impossible.
Common ways to type and how:HTTP TCP UDP / ping,telnet,nmap, Socket
Here, simply draw a basic flowchart
Ok
Simple analysis first come here, later may continue to supplement, perhaps no longer in this area.
Wireshark capture Packet Analysis/tcp/http/https and proxy IP identification