How to Use DNS + geoip + nginx + varnish as a world-class CDN

How to Use DNS + geoip + nginx + varnish as a world-class CDN

How to use bind, geoip, nginx, and varnish to create your own efficient CDN network?
CDN stands for content distrubtion Network, which is a content delivery network. In short, it refers to Server Load balancer in all regions. The concept of a whole region can be national or global. The unified DNS server performs address forwarding and selects the region server closest to the user for load balancing. In essence, Server Load balancer is expanded from an equipment room to a global Server Load balancer. At the same time, the localized content can be implemented by the local server. Make the browser automatically select the region.
For example, in China, China is divided into two major regions: China Netcom in the north and China Telecom in the south. Mutual access between the two networks is slow. As a large website, one solution is to set up all servers at a dual-line or three-line ISP, and the ISP will provide the route selection. In this way, the cost of the line will be relatively high. Another method is to set up the server on both sides, one in the south and one in the north. Then, the server selects the server. If the IP address is in China Telecom, the server forwards the request to the server in the south, if it is Netcom, it will forward it to the server in the north.
By expanding the scope, requests from the United States can be handed over to the U.S. server for processing, which also shortens the waiting time on the route. This is the content delivery network.
All nodes on the network can be viewed as virtual servers. Server Load balancer can be performed between nodes on servers in different regions.
The preparation work is as follows: You need to download the following software to implement the above functions
Nginx, bind, geoip, varnish
Next, compile and install bind9 and geoip.

1. # Tar-xzvf bind-9.2.4.tar.gz
2. # Tar-xzvf GeoIP-1.4.6.tar.gz
3. # Cd GeoIP-1.4.6
4. #./Configure-Prefix =/usr/local/geoip
5. # Make
6. # Make install
7. # CD ..
8. # Patch-P0 <bind-9.2.4-geodns-patch/patch. Diff // patch bind9 to allow bind9 to directly support the geoip Library
9. # Cd bind-9.2.4
10. # Cflags = "-I/usr/local/geoip/include" ldflags = "-L/usr/local/geoip/lib-lgeoip ". /configure-Prefix =/usr/local/Bind
11. # Make
12. # Make install

After BIND is installed, create named. conf.

1. View "us "{
2. // Match the North American client US & Canada
3. Match-clients {country_us; country_ca ;};
4. // Provide recursive service to internal clients only.
5. Recursion no;
6. Zone "cdn.xianglei.com "{
7. Type master;
8. File "PRI/xianglei-us.db ";
9. };
10. Zone "." In {
11. Type hint;
12. File "named. ca ";
13. };
14. };
15. View "Latin "{
16. // Match to South America
17. Match-clients {country_ar; country_cl; country_br ;};
18. Recursion no;
19. Zone "cdn.xianglei.com "{
20. Type master;
21. File "PRI/xianglei-latin.db ";
22. };
23. Zone "." In {
24. Type hint;
25. File "named. ca ";
26. };
27. };

You can also match with Europe, Africa, and so on, and then start to make nginx and varnish
Note: The above content is what you want to do on the master node server. The master node server is only responsible for forwarding DNS requests.
As agreed, the BIND server is called a dynamic node server, and nginx + varnish is called a boundary server.
The following content is what the secondary node server needs to do, that is, the server actually placed in a region

1. #./Configure-Prefix =/usr/local/nginx-with-http_realip_module
2. # Make
3. # Make install

And configure nginx

1. HTTP {
2. Include mime. types;
3. Default_type application/octet-stream;
4. Sendfile on;
5. Keepalive_timeout 65;
6. Upstream dynamic_node {
7. Server 1.1.1.1: 80; #1.1.1.1 is the IP address of the primary DNS Node
8. }
9. Server {
10. Listen 8080;
11. SERVER_NAME cdn.xianglei.net;
12. Location ~ * \. (GIF | JPG | JPEG | PNG | WMV | Avi | MPG | MPEG | MP4 | HTM | HTML | JS | CSS | MP3 | SWF | ICO | FLV) $ {
13. Proxy_set_header X-real-IP $ remote_addr;
14. Proxy_pass http: // dynamic_node;
15. Proxy_store/var/www/cache $ URI;
16. Proxy_store_access User: RW group: RW all: R;
17. }

Above we use nginx to cache static files only, and cache static files in the/var/www/cache folder. If you do not have any, you need to create this folder. Nginx listens to port 8080. This is because varnish is used to listen to port 80 for dynamic file forwarding. In fact, nginx is used as a reverse proxy and cache server for static files, and varnish is the reverse proxy that truly allows users to see webpages and dynamic files, the separate storage of static and dynamic files can greatly improve the efficiency.
Finally, we will configure the varnish service.

1. # Tar-xzvf varnish-2.1.2.tar.gz
2. #./Configure-Prefix =/usr/local/Varnish
3. # Make
4. # Make install

Then the varnish Option

1. Backend default {
2. . Host = "127.0.0.1 ″;
3. . Port = 8080 ″;
4. }
5. Sub vcl_recv {
6. If (req. url ~ "\. (JS | CSS | JPG | JPEG | PNG | GIF | GZ | tgz | bz2 | TBZ | MP3 | Ogg | SWF) $ "){
7. Return (lookup );
8. }
9. }
10. Sub vcl_fetch {
11. If (req. url ~ "\. (JS | CSS | JPG | JPEG | PNG | GIF | GZ | tgz | bz2 | TBZ | MP3 | Ogg | SWF) $ "){
12. Unset obj. http. Set-cookie;
13. }
14. }

For other configuration content, see the varnish configuration document.
Summary:
The benefits of doing so are:
1. The root cause solves the uncertainty of DNS in round-robin and enables fast response on DNS. It also avoids high load when nginx + geoip is used in the past. After all, DNS computing is much smaller than nginx.
2. Reduce the server load pressure and operating costs of large websites. After all, the price and service fee of f5bigip and dual-line are too high.
3. Easy scalability. If the load pressure in a region is high, you only need to add the Web server of the border server group in the region. You do not need to consider the jump issue.
Let me think about other advantages.

How to Use DNS + geoip + nginx + varnish as a world-class CDN