Use the srcache_nginx module in Nginx to build the cache

Use the srcache_nginx module in Nginx to build the cache

In nginx, lua can be embedded, so that nginx can execute lua scripts to handle high concurrency and non-blocking requests. In openresty, you can use nginx to directly build srcache_nginx + redis caches, instead of using dynamic languages (QPS can be easily improved)

Let's take a look at the srcache-nginx-module workflow in openresty.

Okay, you don't have to talk about it.

I. Installation

Pcre

Cd/usr/local/src

Ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz wget-c

Tar zxf pcre-8.38.tar.gz

Drizzle7

Cd/usr/local/src/

Wget http://openresty.org/download/drizzle7-2011.07.21.tar.gz

Tar xzvf drizzle-2011.07.21.tar.gz

Cd drizzle-2011.07.21/

./Configure

Make

Make install

Export LD_LIBRARY_PATH =/usr/local/lib: $ LD_LIBRARY_PATH

JIT (Just-In-Time Compiler)

Http://luajit.org/download/LuaJIT-2.0.2.tar.gz wget-c

Tar xzvf LuaJIT-2.0.2.tar.gz

Cd LuaJIT-2.0.2

Make install PREFIX =/usr/local/luajit

Echo "/usr/local/luajit/lib">/etc/ld. so. conf. d/usr_local_luajit_lib.conf

Ldconfig

Export LUAJIT_LIB =/usr/local/luajit/lib

Export LUAJIT_INC =/usr/local/luajit/include/luajit-2.0

Nginx

Cd/usr/local/src

Http://nginx.org/download/nginx-1.9.9.tar.gz wget-c

Git clone https://github.com/simpl/ngx_devel_kit.git

Git clone https://github.com/openresty/set-misc-nginx-module.git

Git clone https://github.com/openresty/memc-nginx-module.git

Git clone https://github.com/openresty/echo-nginx-module.git

Git clone https://github.com/openresty/lua-nginx-module.git

Git clone https://github.com/openresty/srcache-nginx-module.git

Git clone https://github.com/openresty/drizzle-nginx-module.git

Git clone https://github.com/openresty/rds-json-nginx-module.git

Wget http://people.freebsd.org /~ Osa/ngx_http_redis-0.3.7.tar.gz

Tar zxf nginx-1.9.9.tar.gz

Cd nginx-1.9.9

 

./Configure \

-- Prefix =/usr/local/nginx-1.9.9 \

-- Add-module = ../memc-nginx-module \

-- Add-module = ../srcache-nginx-module \

-- Add-module = ../ngx_devel_kit \

-- Add-module = ../ngx_image_thumb \

-- Add-module = ../redis2-nginx-module \

-- Add-module = ../echo-nginx-module \

-- Add-module = ../lua-nginx-module \

-- Add-module = ../set-misc-nginx-module \

-- Add-module = ../ngx_http_redis-0.3.7 \

-- With-pcre = ../pcre-8.38 \

-- With-pcre-jit

Make & make install

Redis

Cd/usr/local

Wget http://download.redis.io/releases/redis-3.0.6.tar.gz

Tar zxf redis-3.0.6.tar.gz

Cd redis-3.0.6

Make

./Src/redis-server &

Configuration File

Daemonize yes

Pidfile/var/run/redis-6379.pid

Port 6379

Bind 127.0.0.1

Timeout 0

Tcp-keepalive 0

Loglevel notice

Logfile stdout

Databases 16

Stop-writes-on-bgsave-error yes

Rdbcompression yes

Rdbchecksum yes

Dbfilename dump. rdb

Slave-serve-stale-data yes

Slave-read-only yes

Repl-disable-tcp-nodelay no

Slave-priority 100

Maxmemory 8096 mb

Maxmemory-policy volatile-ttl

Appendonly no

Appendfsync everysec

No-appendfsync-on-rewrite no

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64 mb

Lua-time-limit 5000

Slowlog-log-slower-than 10000

Slowlog-max-len 128

Hash-max-ziplist-entries 512

Hash-max-ziplist-value 64

List-max-ziplist-entries 512

List-max-ziplist-value 64

Set-max-intset-entries 512

Zset-max-ziplist-entries 128

Zset-max-ziplist-value 64

Activerehashing yes

Client-output-buffer-limit normal 0 0 0

Client-output-buffer-limit slave 256 mb 64 mb 60

Client-output-buffer-limit pubsub 32 mb 8 mb 60

Hz 10

Aof-rewrite-incremental-fsync yes

Simple nginx Configuration

User www;

Worker_processes auto;

 

Error_log logs/error. log info;

 

Pid logs/nginx. pid;

 

 

Events {

Use epoll;

Worker_connections 65536;

}

 

 

Http {

Include mime. types;

Default_type application/octet-stream;

Charset UTF-8;

 

Log_format main '$ remote_addr-$ remote_user [$ time_local] "$ request "'

'$ Status $ body_bytes_sent "$ http_referer "'

'"$ Http_user_agent" "$ http_x_forwarded_for "';

 

Log_format srcache_log '$ remote_addr-$ remote_user [$ time_local] "$ request "'

'"$ Status" $ body_bytes_sent $ request_time $ bytes_sent $ request_length'

'[$ Upstream_response_time] [$ srcache_fetch_status] [$ srcache_store_status] [$ srcache_expire]';

Server_tokens off;

 

Keepalive_timeout 60 20;

Client_header_timeout 3 m;

Client_body_timeout 3 m;

Send_timeout 3 m;

 

Client_header_buffer_size 16 k;

Large_client_header_buffers 4 32 k;

Server_names_hash_max_size 512;

Server_names_hash_bucket_size 64;

 

Sendfile on;

Tcp_nopush on;

Tcp_nodelay on;

 

 

Gzip on;

Gzip_min_length 1 k;

Gzip_buffers 4 16 k;

Gzip_http_version 1.0;

Gzip_comp_level 2;

Gzip_types text/plain application/x-javascript text/css application/xml;

Gzip_vary on;

 

Upstream memcache {

Server 192.168.1.30: 12000;

Keepalive 10;

}

Upstream redis {

Server 127.0.0.1: 6379;

Keepalive 20;

}

 

Server

{

Listen 90 default;

Server_name _;

Return 444;

}

Include vhosts/*. conf;

}

Ii. srcache + memcache

Server {

Listen 8099;

Server_name 192.168.1.30;

Root/data/www;

Index. php index.html index.htm;

Default_type text/plain;

 

Access_log logs/host. access. log main;

 

Location/hello {

Echo "This is a test ";

}

Location =/lua-version {

Content_by_lua'

If jit then

Ngx. say (jit. version)

Else

Ngx. say (_ VERSION)

End

';

}

 

 

Location/memc {

Internal;

Memc_connect_timeout 100 ms;

Memc_send_timeout 100 ms;

Memc_read_timeout 100 ms;

Set $ memc_key $ query_string;

Set $ memc_exptime 300;

Memc_pass memcache;

}

 

Location ~ \. Php $ {

Charset UTF-8;

Default_type text/html;

Set $ key $ uri $ args;

Srcache_fetch GET/memc $ key;

Srcache_store PUT/memc $ key;

Add_header X-Cached-From $ srcache_fetch_status;

Add_header X-Cached-Store $ srcache_store_status;

Fastcgi_pass 127.0.0.1: 9000;

Fastcgi_index index. php;

Include fastcgi_params;

Fastcgi_param SCRIPT_FILENAME $ document_root $ fastcgi_script_name;

}

 

Error_page 500 502 503 x.html;

Location =/50x.html {

Root html;

}

}

First visit

Connection: keep-alive

Content-Encoding: gzip

Content-Type: text/html; charset = UTF-8

Date: Wed, 20 Jan 2016 16:32:32 GMT

Keep-Alive: timeout = 20

Server: nginx

Transfer-Encoding: chunked

Vary: Accept-Encoding

X-Cached-From: <strong> MISS </strong>

X-Cached-Store: STORE

Second access

Connection: keep-alive

Content-Encoding: gzip

Content-Type: text/html; charset = UTF-8

Date: Wed, 20 Jan 2016 16:33:17 GMT

Keep-Alive: timeout = 20

Server: nginx

Transfer-Encoding: chunked

Vary: Accept-Encoding

X-Cached-From: <strong> HIT </strong>

X-Cached-Store: BYPASS

Which access needs can be customized?

Iii. srcache + redis

Redis configuration test

Server {

Listen 9001;

Server_name 192.168.1.30;

Root/data/www;

Index. php index.html index.htm;

Default_type text/plain;

Access_log logs/host. access. log main;

 

Location/testx {

Echo '1 ';

}

 

Location ~ . * \. Php {

Srcache_store_private on;

Srcache_methods GET;

Srcache_response_cache_control off;

 

Set $ key $ uri;

Set_escape_uri $ escaped_key $ key;

Srcache_default_expire 172800;

Srcache_fetch GET/redis_get $ key;

Srcache_store PUT/redis_set key = $ escaped_key & exptime = $ srcache_expire;

 

Add_header X-Cached-From $ srcache_fetch_status;

Set_md5 $ md5key $ key;

Add_header X-md5-key $ md5key;

Add_header X-Cached-Store $ srcache_store_status;

Add_header X-Key $ key;

Add_header X-Query_String $ query_string;

Add_header X-expire $ srcache_expire;

Add_header X-uri $ uri;

Access_log logs/9001-access.log srcache_log;

 

Include fastcgi_params;

Fastcgi_pass 127.0.0.1: 9000;

Fastcgi_index index. php;

Fastcgi_connect_timeout 60;

Fastcgi_send_timeout 180;

Fastcgi_read_timeout 180;

Fastcgi_buffer_size 128 k;

Fastcgi_buffers' 4 256 k;

Fastcgi_busy_buffers_size 256 k;

Fastcgi_temp_file_write_size 256 k;

Fastcgi_intercept_errors on;

Fastcgi_param SCRIPT_FILENAME $ document_root $ fastcgi_script_name;

Fastcgi_split_path_info ^ (. + \. php) (. *) $;

Fastcgi_param PATH_INFO $ fastcgi_path_info;

}

 

Location =/redis_get {

Internal;

Set_md5 $ redis_key $ args;

Redis_pass redis;

}

Location =/show {

Echo $ request_uri;

Echo $ args;

}

 

Location =/redis_set {

Internal;

 

Set_unescape_uri $ exptime $ arg_exptime;

Set_unescape_uri $ key $ arg_key;

Set_md5 $ key;

 

Redis2_query set $ key $ echo_request_body;

Redis2_query expire $ key $ exptime;

Redis2_pass redis;

}

 

Location =/one {

Set $ value 'first ';

Redis2_query set one $ value;

Redis2_pass redis;

}

 

Location =/get {

Set_unescape_uri $ key $ arg_key; # this requires ngx_set_misc

Redis2_query get $ key;

Redis2_pass redis;

}

 

Error_page 500 502 503 x.html;

Location =/50x.html {

Root html;

}

}

Check whether the log cache is hit

Iv. lua

Lua processing stage and scope of use in Nginx

Init_by_lua http

Set_by_lua server, server if, location, location if

Rewrite_by_lua http, server, location, location if

Access_by_lua http, server, location, location if

Content_by_lua location, location if

Header_filter_by_lua http, server, location, location if

Body_filter_by_lua http, server, location, location if

Log_by_lua http, server, location, location if

Timer

Lua code

Ngx. req. read_body () -- explicitly read the req body

Local data = ngx. req. get_body_data ()

If data then

Ngx. say ("body data :")

Ngx. print (data)

Return

End

 

-- Body may get buffered in a temp file:

Local file = ngx. req. get_body_file ()

If file then

Ngx. say ("body is in file", file)

Else

Ngx. say ("no body found ")

End

 

Local res = ngx. location. capture ("/foo/index. php ")

If res then

Ngx. say ("status:", res. status)

Ngx. say ("body :")

Ngx. print (res. body)

End

Nginx Configuration

Location/lua_test {

Content_by_lua_file conf/lua_test.lua;

}

For more Nginx tutorials, see the following:

Deployment of Nginx + MySQL + PHP in CentOS 6.2

Build a WEB server using Nginx

Build a Web server based on Linux6.3 + Nginx1.2 + PHP5 + MySQL5.5

Performance Tuning for Nginx in CentOS 6.3

Configure Nginx to load the ngx_pagespeed module in CentOS 6.3

Install and configure Nginx + Pcre + php-fpm in CentOS 6.4

Nginx installation and configuration instructions

Nginx log filtering using ngx_log_if does not record specific logs

Nginx details: click here
Nginx: click here

This article permanently updates the link address: