Use Openresty to speed up Web pages

Happy New Year ~ ~ ~
The previous article talked about using multilevel caching to reduce access to the database to speed up the page, but there is still no "swish" on the load out of the feeling, want to optimize again, optimize the code or something has reached the limit. Last week, inadvertently saw the openresty, can directly in the Nginx layer directly to the request processing, and do not need to access tomcat every time, especially for their own home page/HTTP/ Www.wenzhihuai.com, because of the broad content of the home page, need to involve too many methods, each time to open the home page has to spend a lot of time.

First, the overall structure

Currently I have a total of three servers:
a:119.23.46.71 (Shenzhen, nginx environment, local REDIS,TOMCAT server)
b:119.29.188.224 (Guangzhou, Tomcat server)
c:47.95.10.139 (Beijing, Tomact server)
In order to reduce the corresponding time in the backend, it was previously used to integrate Ehcache as the first level cache in the application, and Redis as a level two cache. This architecture has a special situation: When Nginx will be the first page of the request distribution to the Beijing node, the response will become extremely slow, the user's request from Shenzhen to Beijing, and then back from Beijing to Shenzhen, just delay will cost 40ms (the best case), because the speed is 1m/s, the worst case, It can take a few seconds to respond to a user's request. Therefore, in order to reduce this extreme situation, the design of this architecture (personal project just a whole, how to come all right).
Because it is a blog class project, the consistency requirement is not high, so simply cache the home page directly to Redis.

Steps:
1. After the request arrives Nginx, Openresty reads the local cache through LUA and, if not, feeds back to the Tomcat cluster.
The 2.tomcat cluster first reads the primary cache Ehcache from its own server and, if there is no hit, proceeds back to the source level two cache.
3. Read level Two cache Redis, if still not hit, back to the MySQL server.

Second, the configuration process 2.1 Openresty

The installation process can refer directly to the official documentation: Http://openresty.org/cn/download.html, the following development libraries need to be installed before installation:

yum install pcre-devel openssl-devel gcc curl

Then compile the installation:

tar -xzvf openresty-VERSION.tar.gzcd openresty-VERSION/./configuremakesudo make install

2.2 Nginx Related Configuration

Openresty comes with the nginx. So, as long as the installation of Openresty, you can directly use Nginx configuration.
The following is only part of the need to see Mynginxconfig

HTTP {include mime.types;    Default_type Application/octet-stream; # need to add LUA-related libraries Lua_package_path "/opt/openresty/lualib/?.    LUA;; "; Lua_package_cpath "/opt/openresty/lualib/?"    so;; ";    ... access_log Logs/access.log main;    Sendfile on;    Keepalive_timeout 65;        Upstream backend {#consistent_hash is not configured hash $uri;        Server 47.95.10.139:8080;        Server 119.23.46.71:8080;    Server 119.29.188.224:8080;        } server {Listen 80;        server_name www.wenzhihuai.com;            # exact Match, open first page when enter location =/{Default_type text/html;            root HTML;            Index index.html index.htm;            ... # Close cache Lua script, debug when dedicated lua_code_cache off;            Content_by_lua_file/opt/lua/hello.lua;        # Don't proxy_pass here, otherwise lua script useless # Proxy_pass http://backend; # If the above is not met, then match the following location/{Default_type Text/html;            root HTML;            Index index.html index.htm;        # Reverse proxy proxy_pass http://backend for requests; }    }    ...}

2.3 Lua Script

The script remembers to be placed in the/opt/lua/hello.lua directory, corresponding to the Nginx configuration, and the need to introduce a Redis module.

LocalRedis= require "Resty.redis"LocalRed=Redis:new()LocalRequest_uri=Ngx.Var.Request_uriif (Request_uri== "/" orRequest_uri== "/index.html")  ThenRed:set_timeout( +) --1 SECRedConnect("119.23.46.71", 6340)    LocalOk,Err=Red:auth("Root")    if  notOk ThenNgx.Say("Failed to connect:",Err)        return    End    --The cached homepage is placed in the key index    LocalResp,Errr=Red:get("Index")    if  notResp Then        return    End    ifResp==Ngx.Null ThenResp= "    End    --If found, the output contentNgx.Print(Resp)RedClose()    returnEndLocalPagenum=Ngx.Req.Get_uri_args()["Pagenum"]--The head needs to be reset because proxy_pass_request_headers off is set in Nginx, that is, the request head is not uploaded to Lua .Ngx.Req.Set_header("Accept", "Text/html,application/xhtml+xml,application/xml;")--When this comes back to Tomcat, accept-encoding defaults to gzip, which means that the data returned is already gzip compressed, and is compressed once again when returned to the user, causing a heap of garbled characters. So set the accept-encoding to empty. Ngx.Req.Set_header("Accept-encoding", "")LocalRespp=Ngx.Location.Capture("/index.do", {Method=Ngx.Http_get,Args= {Pagenum=Pagenum} })--PrintNgx.Print(Respp.Body)return

2.4 Update home to Redis

Every 20 seconds direct access to the back end of the first page crawl, and then stored in the Redis inside, simple rough.

@Controller@SuppressWarnings("Unchecked") Public classTimecontroller {//logger    Private Static FinalLogger Logger = loggerfactory.GetLogger(Timecontroller.class);@Scheduled(cron ="0/20 * * * * *?") Public void Refreshindex()throwsException {String IP = iputils.Getserverip().ReplaceAll("\ n","");if(Regularip.equals(IP)) {String content = Httphelper.getinstance().Get("http://119.29.188.224:8080"); Jedisutil.getinstance().Set("Index", content); }    }}

Problems with the capture error

2018/02/10 18:53:51 [error] 2833#0: *3942 lua subrequests cycle while processing "/index.html", client: 113.108.186.130, server: www.wenzhihuai.com, request: "GET /index.html?pagenum=23 HTTP/1.1", subrequest: "/index.html", host: "www.wenzhihuai.com"2018/02/10 18:53:51 [error] 2833#0: *3942 lua entry thread aborted: runtime error: /opt/lua/hello.lua:25: failed to issue subrequest: -1stack traceback:coroutine 0:    [C]: in function 'capture'    </opt/lua/hello.lua:1>, client: 113.108.186.130, server: www.wenzhihuai.com, request: "GET /index.html?pagenum=23 HTTP/1.1", subrequest: "/index.html", host: "www.wenzhihuai.com"

Open Web page garbled, with curl normal display

Looked for a long time, only to find openresty if the use of Nginx.location.capture is re-requested once, if with a request header and gzip compression is turned on, then the data is the equivalent of being compressed again.

Workaround: Remove the request header
Ngx.req.set_header ("accept-encoding", "" ");

Use Openresty to speed up Web pages