Nginx uses the srcache_nginx module to build the cache, nginxsrcache_nginx

Nginx uses the srcache_nginx module to build the cache, nginxsrcache_nginx

In nginx, lua can be embedded, so that nginx can execute lua scripts to handle high concurrency and non-blocking requests. In openresty, you can use nginx to directly build srcache_nginx + redis caches, instead of using dynamic languages (QPS can be easily improved)

Let's take a look at the srcache-nginx-module workflow in openresty.

Okay, you don't have to talk about it.

1. Install pcre

cd /usr/local/srcwget -c ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gztar zxf pcre-8.38.tar.gz

Drizzle7

cd /usr/local/src/ wget http://openresty.org/download/drizzle7-2011.07.21.tar.gz tar xzvf drizzle-2011.07.21.tar.gz cd drizzle-2011.07.21/ ./configuremakemake installexport LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

JIT (Just-In-Time Compiler)

wget -c http://luajit.org/download/LuaJIT-2.0.2.tar.gztar xzvf LuaJIT-2.0.2.tar.gzcd LuaJIT-2.0.2make install PREFIX=/usr/local/luajitecho "/usr/local/luajit/lib" > /etc/ld.so.conf.d/usr_local_luajit_lib.confldconfigexport LUAJIT_LIB=/usr/local/luajit/libexport LUAJIT_INC=/usr/local/luajit/include/luajit-2.0

Nginx

cd /usr/local/srcwget -c http://nginx.org/download/nginx-1.9.9.tar.gzgit clone https://github.com/simpl/ngx_devel_kit.gitgit clone https://github.com/openresty/set-misc-nginx-module.gitgit clone https://github.com/openresty/memc-nginx-module.gitgit clone https://github.com/openresty/echo-nginx-module.gitgit clone https://github.com/openresty/lua-nginx-module.gitgit clone https://github.com/openresty/srcache-nginx-module.gitgit clone https://github.com/openresty/drizzle-nginx-module.gitgit clone https://github.com/openresty/rds-json-nginx-module.gitwget http://people.freebsd.org/~osa/ngx_http_redis-0.3.7.tar.gztar zxf nginx-1.9.9.tar.gzcd nginx-1.9.9./configure \--prefix=/usr/local/nginx-1.9.9 \--add-module=../memc-nginx-module \--add-module=../srcache-nginx-module \--add-module=../ngx_devel_kit \--add-module=../ngx_image_thumb \--add-module=../redis2-nginx-module \--add-module=../echo-nginx-module  \--add-module=../lua-nginx-module \--add-module=../set-misc-nginx-module \--add-module=../ngx_http_redis-0.3.7 \--with-pcre=../pcre-8.38 \--with-pcre-jitmake && make install

Redis

cd /usr/localwget http://download.redis.io/releases/redis-3.0.6.tar.gztar zxf redis-3.0.6.tar.gz cd redis-3.0.6make./src/redis-server &

Configuration File

daemonize yespidfile /var/run/redis-6379.pidport 6379bind 127.0.0.1timeout 0tcp-keepalive 0loglevel noticelogfile stdoutdatabases 16stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yesdbfilename dump.rdbslave-serve-stale-data yesslave-read-only yesrepl-disable-tcp-nodelay noslave-priority 100maxmemory 8096mb    maxmemory-policy volatile-ttlappendonly noappendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mblua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64activerehashing yesclient-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yes

Simple nginx Configuration

user  www www;worker_processes  auto;error_log  logs/error.log  info;pid        logs/nginx.pid;events {    use epoll;    worker_connections  65536;}http {    include       mime.types;    default_type  application/octet-stream;    charset  utf-8;    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '        '$status $body_bytes_sent "$http_referer" '        '"$http_user_agent" "$http_x_forwarded_for"';    log_format srcache_log '$remote_addr - $remote_user [$time_local] "$request" '        '"$status" $body_bytes_sent $request_time $bytes_sent $request_length '       '[$upstream_response_time] [$srcache_fetch_status] [$srcache_store_status] [$srcache_expire]';        server_tokens off;    keepalive_timeout           60 20;    client_header_timeout       3m;    client_body_timeout         3m;    send_timeout                3m;    client_header_buffer_size           16k;    large_client_header_buffers         4 32k;    server_names_hash_max_size          512;    server_names_hash_bucket_size       64;    sendfile        on;    tcp_nopush      on;    tcp_nodelay     on;    gzip  on;    gzip_min_length  1k;    gzip_buffers     4 16k;    gzip_http_version 1.0;    gzip_comp_level 2;    gzip_types       text/plain application/x-javascript text/css application/xml;    gzip_vary on;    upstream memcache {        server 192.168.1.30:12000;        keepalive 10;    }    upstream redis {        server 127.0.0.1:6379;        keepalive 20;    }    server    {        listen 90 default;        server_name _;        return 444;    }    include vhosts/*.conf;}

Ii. srcache + memcache

server {    listen       8099;    server_name  192.168.1.30;    root /data/www;    index  index.php index.html index.htm;    default_type text/plain;    access_log  logs/host.access.log  main;    location /hello{        echo "This is a test";    }    location = /lua-version {        content_by_lua '            if jit then                ngx.say(jit.version)            else                ngx.say(_VERSION)                    end           ';    }    location /memc {        internal;        memc_connect_timeout 100ms;        memc_send_timeout 100ms;        memc_read_timeout 100ms;        set $memc_key $query_string;        set $memc_exptime 300;        memc_pass memcache;    }    location ~ \.php$ {        charset        utf-8;        default_type   text/html;        set $key $uri$args;        srcache_fetch GET /memc $key;        srcache_store PUT /memc $key;        add_header X-Cached-From $srcache_fetch_status;         add_header X-Cached-Store $srcache_store_status;        fastcgi_pass   127.0.0.1:9000;        fastcgi_index  index.php;        include        fastcgi_params;        fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;    }    error_page   500 502 503 504  /50x.html;    location = /50x.html {        root   html;    }}

First visit

Connection:keep-aliveContent-Encoding:gzipContent-Type:text/html; charset=UTF-8Date:Wed, 20 Jan 2016 16:32:32 GMTKeep-Alive:timeout=20Server:nginxTransfer-Encoding:chunkedVary:Accept-EncodingX-Cached-From:MISSX-Cached-Store:STORE

Second access

Connection:keep-aliveContent-Encoding:gzipContent-Type:text/html; charset=UTF-8Date:Wed, 20 Jan 2016 16:33:17 GMTKeep-Alive:timeout=20Server:nginxTransfer-Encoding:chunkedVary:Accept-EncodingX-Cached-From:HITX-Cached-Store:BYPASS

Which access needs can be customized?

Iii. srcache + redis

Redis configuration test

server {    listen       9001;    server_name  192.168.1.30;    root /data/www;    index  index.php index.html index.htm;    default_type text/plain;    access_log  logs/host.access.log  main;    location /testx{        echo '1';    }    location ~ .*\.php {        srcache_store_private on;        srcache_methods GET;        srcache_response_cache_control off;        set $key $uri;        set_escape_uri $escaped_key $key;        srcache_default_expire 172800;        srcache_fetch GET /redis_get $key;        srcache_store PUT /redis_set key=$escaped_key&exptime=$srcache_expire;        add_header X-Cached-From $srcache_fetch_status;        set_md5 $md5key $key;        add_header X-md5-key $md5key;        add_header X-Cached-Store $srcache_store_status;        add_header X-Key $key;        add_header X-Query_String $query_string;        add_header X-expire $srcache_expire;        add_header X-uri $uri;        access_log logs/9001-access.log srcache_log;        include fastcgi_params;        fastcgi_pass  127.0.0.1:9000;        fastcgi_index index.php;        fastcgi_connect_timeout 60;        fastcgi_send_timeout 180;        fastcgi_read_timeout 180;        fastcgi_buffer_size 128k;        fastcgi_buffers 4 256k;        fastcgi_busy_buffers_size 256k;        fastcgi_temp_file_write_size 256k;        fastcgi_intercept_errors on;        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;        fastcgi_split_path_info ^(.+\.php)(.*)$;        fastcgi_param PATH_INFO $fastcgi_path_info;    }    location = /redis_get {        internal;        set_md5 $redis_key $args;        redis_pass redis;    }    location =/show{        echo $request_uri;         echo $args;    }    location = /redis_set {        internal;        set_unescape_uri $exptime $arg_exptime;        set_unescape_uri $key $arg_key;        set_md5 $key;        redis2_query set $key $echo_request_body;        redis2_query expire $key $exptime;        redis2_pass redis;    }    location = /one {        set $value 'first';        redis2_query set one $value;        redis2_pass redis;    }    location = /get {        set_unescape_uri $key $arg_key;  # this requires ngx_set_misc        redis2_query get $key;        redis2_pass redis;    }    error_page   500 502 503 504  /50x.html;    location = /50x.html {        root   html;    }}

Check whether the log cache is hit

Iv. lua

Lua processing stage and scope of use in Nginx

init_by_lua            httpset_by_lua             server, server if, location, location ifrewrite_by_lua         http, server, location, location ifaccess_by_lua          http, server, location, location ifcontent_by_lua         location, location ifheader_filter_by_lua   http, server, location, location ifbody_filter_by_lua     http, server, location, location iflog_by_lua             http, server, location, location iftimer

Lua code

ngx.req.read_body()  -- explicitly read the req bodylocal data = ngx.req.get_body_data()if data then    ngx.say("body data:")    ngx.print(data)    returnend-- body may get buffered in a temp file:local file = ngx.req.get_body_file()if file then    ngx.say("body is in file ", file)else    ngx.say("no body found")endlocal res = ngx.location.capture("/foo/index.php")if res then   ngx.say("status: ", res.status)   ngx.say("body:")   ngx.print(res.body)end

Nginx Configuration

location /lua_test {        content_by_lua_file conf/lua_test.lua;                                                                                      }  

 

 

References

Https://github.com/openresty/lua-nginx-module

Http://chenxiaoyu.org/2011/10/30/nginx-modules.html

Http://www.ttlsa.com/nginx/nginx-lua-redis/