Nginx supports caching of 302 redirected files to local

Think of a requirement like this:

When using Nginx reverse proxy, we need to cache the content pulled from the back-end server, under normal circumstances, if the required content is on the back-end server, then it is easy to use Nginx

The Proxy_cache module is able to implement caching. However, there is a situation where the content is not on the back-end server, but instead returns a redirect address from the back-end server, redirected to a third-party server,

So this time the Nginx cache is not what we need the source content, but a file containing only redirect links, then how can we deal with this situation?

X-accel allows redirection inside Nginx to an address returned from the back-end server, which can replace you in the backend server to handle some authentication, login or other things, and can also

Let Nginx handle the content from the redirect address to the terminal (browser), so that in the back-end server, you can handle more requests. This function is what we often mention.

X-sendfile.

This feature differs from the standard Nginx module because it does not rely on standard protocol directives but instead handles HTTP headers from upstream servers in a special way, and we use the method of

When you send a header message, carry the URI of the previous X-accel-redirect property, and when Nginx receives the message header, discovers that it has the X-accel-redirect attribute, then it will be treated as

A normal request (similar to a browser-access Nginx) uses the URI that is carried behind it and the location in the current Nginx configuration file to match, and then Nginx will point to the URI that matches the header of the message.

The following is a configuration instance

#user nobody;worker_processes 1;error_log logs/error.log Debug; #pid logs/nginx.pid;events {worker_connecti ONS 1024;}    HTTP {include mime.types;      Default_type Application/octet-stream;    Sendfile on;    #tcp_nopush on;    #keepalive_timeout 0;    Keepalive_timeout 65;    #gzip on;    Proxy_cache_path/home/mpeg/cache/nginx levels=1:2 keys_zone=ts:500m max_size=1g;        server {Listen 8889;        server_name localhost;        #charset Koi8-r;        #access_log Logs/host.access.log Main;        Add_header X-via $server _addr;        Add_header X-cache $upstream _cache_status;    Location/{Proxy_pass http://192.168.15.159:8999;    Proxy_set_header X-real-ip $remote _addr;          Proxy_cache ts;        Proxy_cache_valid 302 24h; } location ~* ^/internal_redirect/(. *?)    /(. *) {internal;    Proxy_set_header X-real-ip $remote _addr;          Proxy_cache ts;    Proxy_cache_valid 302 24h; # EXtract download URL from the request set $download _uri;     Set $download _host $;    # Compose Download URL set $download _url $download _host/$download _uri;        Proxy_pass $download _url;}        Error_page 502 503 504/50x.html;        Location =/50x.html {root html; }    }}

A demo in the real World

#user nobody;worker_processes 8; #error_log logs/error.log; #error_log logs/error.log Notice;error_log Logs/error.log Notice; #pid logs/nginx.pid;events {worker_connections 1024;}    HTTP {include mime.types;    Default_type Application/octet-stream; Log_format Main ' $remote _addr-$remote _user [$time _local] "$request" "$status $body _bytes_sent"    $http _referer "'" $http _user_agent "" $http _x_forwarded_for ";    Access_log Logs/access.log Main;    Sendfile on;    #tcp_nopush on;    #keepalive_timeout 0;    Keepalive_timeout 65;    #gzip on;    Proxy_cache_path/home/mpeg/cache/nginx levels=1:2 inactive=1h keys_zone=ts:500m max_size=2g;        server {Listen 8889;        server_name localhost;        #charset Koi8-r;        #access_log Logs/host.access.log Main; Add_header x-cache $upstream _cache_status; #cache for redirect ts:/internal_redirect/http://192.168.15.159:7090/home/mpeg/Record_ott/143252147.ts#cache for redirect vmhls:/internal_redirect/http://192.168.15.159:7090/vmhls?& filename=/home/mpeg/cl/052_800&startoffset=93717248&endoffset=98178864 location ~* ^/internal_redirect/(. *?)    /(. *) {internal;    Proxy_set_header X-real-ip $remote _addr;          Proxy_cache ts;    Proxy_cache_valid 302 24h;    # Extract Download URL from the request set $download _uri;     Set $download _host $;    # Compose Download URL set $download _url $download _host/$download _uri? $args; Proxy_pass $download _url;} Location ~* Live {#no cache for sub m3u8:/live/0000000000010078.m3u8?&nodename=cl&usercode=192_168_15_111_ 0000000000010078. #no cache for main m3u8:/live/1.m3u8location ~*. m3u8 {proxy_pass http://192.168.15.110:899    9;       Proxy_set_header X-real-ip $remote _addr;        } #cache for ts:/live/0000000000010078/1.ts?&nodename=cl&usercode=192_168_15_111_0000000000010078 ... Location ~*. ts {PRoxy_pass http://192.168.15.110:8999;    Proxy_set_header X-real-ip $remote _addr;          Proxy_cache ts;        Proxy_cache_valid 302 24h; }}location ~* VOD {#no cache for sub M3u8:/vod/000000000010078_800.m3u8#no The cache for main m3u8:/vod/000000000010078.m3u8l    Ocation ~*. m3u8 {proxy_pass http://192.168.15.110:8999;       Proxy_set_header X-real-ip $remote _addr;            } #cache for tss:/vod/000000000010052_800&startoffset=0&endoffset=4463308/1.ts location ~*. ts {    Proxy_pass http://192.168.15.110:8999;    Proxy_set_header X-real-ip $remote _addr;          Proxy_cache ts;        Proxy_cache_valid 302 24h;        }} #error_page 404/404.html;        # REDIRECT Server error pages to the static page/50x.html # Error_page 502 503 504/50x.html;        Location =/50x.html {root html; }    }}

Note: If the header message returned to Nginx has the following settings

X-accel-redirect:/internal_redirect/http://192.168.15.110:7090/home/mpeg/cl/1.ts

Then, after the above match, the following results can be obtained:

$ = http:/$2 =/192.168.15.110:7090/home/mpeg/cl/1.ts$download_url = Http://192.168.15.110:7090/home/mpeg/cl/1.ts

This is the result of a regular expression match.
Finally, take a look at the entire execution process:
Assume client (192.168.15.13), Nginx (192.168.15.14:8889), Back-end server (192.168.15.159:8999), third-party server (192.168.15.160)

The data flow should be like this

Client--->nginx---> Backend server---(redirect third-party servers, return third-party server addresses to Nginx)--->nginx---( Nginx identifies the X-accel-redirect and accesses the redirected connection it points to)
|
Return to client <------------------------------Nginx download and cache to this <-----------------------------------------|

Finally, to clean up the cache file, there is no need to use the Cache cleanup module, just use one command:

Find/-maxdepth 2-type f-mmin +100 | Xargs-n 1 rm-f

Of course, if the proxy cache module comes with the ability to clean up the cache file to meet your requirements, then you can also use the self-feature of the emotional path cache

Resources:

http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-remote/
Http://wiki.nginx.org/X-accel
http://forum.nginx.org/read.php?2,258887,258957#msg-258957

Nginx supports caching of 302 redirected files to local