Method of using Squid to build HTTP proxy _linux

Recently in the process of doing some infrastructure construction, but also encountered the company's HTTP agent problem. The main is that many of the tools on the host only support Http_proxy settings without authentication information, such as only support such as: Export http_proxy= ' http://10.10.1.1:8090 ', but not export http_proxy= ' http ://tonybai:passwd@10.10.1.1:8090 ' This form of configuration.

or its command-line options provide only the Proxy_host and proxy_port two options, but do not support the carrying of authentication information. And inside the company to access external information must also through the company with identity authentication proxy server, in short, I am very unhappy. An idea arises as to whether an internal HTTP intermediary can be built, and the internal host accesses the intermediary through an agent configuration without authentication information, and the intermediary agent forwards all HTTP request internal to the company agent, along with the configured authentication information.

To the HTTP proxy this matter, I am completely a small white ah, so Google to come (coincides with the recent Google has not given force, the reason you understand).

First tried a bit of tinyproxy, this tool is very small and simple, under Ubuntu through the Apt-get can be directly installed,/etc/tinyproxy/tinyproxy.conf configuration is also very simple and clear. However, configuration lines that involve forwarding to upstream proxy server in the configuration file only support "upstream Host:port" and do not support the "upstream Tonybai:passwd@host:port" form, And there is no other place to support the configuration of authentication information. There are a lot of people in the official bugzilla to reflect this situation, but the latest version does not seem to add this feature, very regrettable!

So I'm going to change to a heavyweight agent tool-nginx. The default Nginx installed under Ubuntu 9.04 is version 0.65. The Nginx feature is powerful, configuration is not so "complex", but the problem is that Nginx itself seems to be more focused on load balancing and reverse proxies, and there is little information to satisfy my scenario. There are so many nginx configuration commands and variables that it is really difficult to figure out the meaning of these variables for a short time. Suit has tried several configurations and failed to succeed. Leafing through the domestic only a book Nginx-"actual combat nginx", but helpless too thick, turned over three chapters, simply put down. Change the tools!

The most traditional free HTTP proxy tool for open source is squid. It is estimated that its market share is among the top. Ubuntu 9.04 under the default installed squid is the 2.7 version, not very old, Squid official station has also provided 2.7 version of detailed configuration documents. But squid default configuration file is super large, a total of nearly 5k lines, although most of the content is commented out. As a start, the rows are filtered out without comment, and these are the actual configurations that are in effect.

On how Squid forwards incoming HTTP request to a superior HTTP proxy server with authentication, there is less information on the web, but let me find one. Try to follow this configuration recommendation. The/etc/squid/squid.conf configuration is summarized as follows:

Access_log/var/log/squid/access.log squid
debug_options all,1
hosts_file/etc/hosts
coredump_dir/var/ Spool/squid

ACL All-src all
ACL manager proto cache_object
ACL localhost src 127.0.0.1/32
ACL to_ localhost DST 127.0.0.0/8 0.0.0.0/32

ACL localnet src 10.0.0.0/8 # RFC1918 possible internal network
ACL localnet SRC 172.16.0.0/12 # RFC1918 possible internal network
ACL localnet src 192.168.0.0/16 # RFC1918 possible internal net Work

http_port 10.10.13.17:3128

http_access allow localnet
http_access allow
localhost http_ Access deny all

cache_peer proxy.yourcompany.com parent port_of_company_httpproxy 0 no-query default login=user: passwd
never_direct Allow localnet

After configuration, restart squid (sudo/etc/init.d/squid restart). To change the proxy configuration of the Chrome browser to the agent, try to open "baidu.com" and get into a long wait. So open squid's access log/var/log/squid/access.log, see the following failure message:

1353476636.008 0 10.10.13.235 tcp_denied/400 1709 get error:invalid-request–none/-text/html
1353476657.337 1 10.10.13.235 tcp_denied/400 1709 get error:invalid-request–none/-text/html
1353476691.420 0 10.10.13.235 tcp_denied/400 1678 get error:invalid-request–none/-text/htm

It's a mistake! Change to IE browser, the phenomenon is the same error. In/var/log/squid/cache.log, you can also find the following error:

2012/11/21 13:43:56| CLIENTTRYPARSEREQUEST:FD (10.10.13.235:4247) Invalid Request

Constantly modify the squid.conf configuration, constantly modify the browser proxy configuration, continuous failure. Always modifying the browser's proxy configuration made me feel very laborious, so I swapped the Curl tool to test the agent. Curl is a http_proxy environment variable that can be identified. Change the HTTP_PROXY environment variable to export http_proxy=http://10.10.13.17:3128, typing curl http://baidu.com at the command line, and get the following result:

$ Curl http://baidu.com
 
 

Go back to Access.log observation, actually saw the following success log:

1353476863.916 0 10.10.13.235 tcp_hit/200 677 get http://baidu.com/–none/-text/html

And then try to download the external files with wget, with subversion access to external SVN repository, RVM install Ruby package is successful! This is not what I want to be the result of! I was accidentally hit by mistake! Although so far I still do not know why the HTTP request issued by the browser can not be recognized ^_^.

Squid This HTTP proxy function is very powerful, itself is a lot of enterprises as a company-level HTTP proxy tool. Its configuration reference can be written a thick book (the market already has this kind of book), fortunately my scene does not use those strange configuration, at present this kind of condition is sufficient!