OpenResty (nginx + lua)

OpenResty (nginx + lua)
OpenResty Official Website: http://openresty.org/OpenResty is a nginx and its various third-party modules of a packaged software platform. The most important thing is that it packs lua/luajit so that we can use the lua script for web development. With lua, we can use nginx's asynchronous and non-blocking functions to achieve the use of lua asynchronous and concurrent access to backend MySQL, PostgreSQL, Memcached, Redis and other services. In particular, the unique ngx. location. capture_multi function is impressive. It can greatly reduce the number of http connections of browsers and asynchronously and concurrently access interfaces such as backend Java, PHP, and Python. The OpenResty web architecture can easily surpass Node. js performance and has no restrictions on backend languages. You can use various languages such as Java, PHP, and Python. OpenResty (nginx + lua) can replace the frontend rendering function of node. js. OpenResty (aka. ngx_openresty) is a full-fledged web application server by bundling the standard Nginx core, lots of 3rd-party Nginx modules, as well as most of their external dependencies. by taking advantage of various well-designed Nginx modules, OpenResty restart tively turns the nginx server into a powerful web app server, in which the web developers can use the Lua programming language to script varous existing nginx C modules and Lua modules and construct extremely high-performance web applications that are capable to handle 10 K + connections. openResty aims to run your server-side web app completely in the Nginx server, leveraging Nginx's event model to do non-blocking I/O not only with the HTTP clients, but also with remote backends like MySQL, PostgreSQL, Memcached, and Redis. 1. install OpenResty first install dependency: yum install readline-devel pcre-devel openssl-devel gcc unzip: tar zxvf ngx_openresty-1.9.3.1.tar.gz build a soft connection: ln-s ngx_openresty-1.9.3.1 openresty into Directory: cd openresty Compilation:

./configure \             --with-cc-opt="-I/usr/local/include" \             --with-ld-opt="-L/usr/local/lib" \             --prefix=/opt/openresty ... ...Configuration summary  + using system PCRE library  + using system OpenSSL library  + md5: using OpenSSL library  + sha1: using OpenSSL library  + using system zlib library  nginx path prefix: "/opt/openresty/nginx"  nginx binary file: "/opt/openresty/nginx/sbin/nginx"  nginx configuration prefix: "/opt/openresty/nginx/conf"  nginx configuration file: "/opt/openresty/nginx/conf/nginx.conf"  nginx pid file: "/opt/openresty/nginx/logs/nginx.pid"  nginx error log file: "/opt/openresty/nginx/logs/error.log"  nginx http access log file: "/opt/openresty/nginx/logs/access.log"  nginx http client request body temporary files: "client_body_temp"  nginx http proxy temporary files: "proxy_temp"  nginx http fastcgi temporary files: "fastcgi_temp"  nginx http uwsgi temporary files: "uwsgi_temp"  nginx http scgi temporary files: "scgi_temp"

 

-- Prefix =/opt/openresty specifies the installation directory. If it is not specified, it is installed in the/usr/local/openresty directory by default. Compile and install: make & make install

[root@localhost src]# cd /opt/openresty/[root@localhost openresty]# lsbin  luajit  lualib  nginx

 

You can see four folders in the/opt/openresty directory, including luajit and nginx. Start openresty:/opt/openresty/nginx/sbin/nginx-c/opt/openresty/nginx/conf/nginx. conf-p/opt/openresty/nginx/

[root@localhost src]# ps -elf|grep nginx1 S root      2076     1  0  80   0 - 34999 -      21:24 ?        00:00:00 nginx: master process /opt/openresty/nginx/sbin/nginx -c /opt/openresty/nginx/conf/nginx.conf -p /opt/openresty/nginx/5 S nobody    2077  2076  0  80   0 - 35045 -      21:24 ?        00:00:00 nginx: worker process                                    0 S root      2079  1678  0  80   0 -  1088 -      21:24 pts/1    00:00:00 grep nginx

 

Verification: curl 127.0.0.1 2. content_by_lua and content_by_lua_file how to embed the lua script in nginx. The method is to use the content_by_lua or cotent_by_lua_file command in the nginx configuration file nginx. conf: 1) content_by_lua is generally used in a very simple lua script:

        location /lua {                set $test "hello, world.";                content_by_lua '                        ngx.header.content_type = "text/plain";                        ngx.say(ngx.var.test);                ';        }

 

Access http: // localhost/lua and you can see the hello, world. 2) cotent_by_lua_file output to the page is adapted to complicated lua scripts and is specially placed into a file:

        location /lua2 {            #lua_code_cache off;            content_by_lua_file lua/hello.lua;        }

 

Path relative to/opt/openresty/nginx

[root@localhost lua]# pwd/opt/openresty/nginx/lua[root@localhost lua]# cat hello.luangx.say('hello ngx_lua!!!!');

 

In this example, hello. lua only contains one sentence: ngx. say ('Hello ngx_lua !!!! '); Access/lua2: [root @ localhost lua] # curl localhost/luahello ngx_lua !!!! The access is successful. In nginx. server {...} of the conf file {.....} add lua_code_cache off; you can easily debug the lua script. After modifying the lua script, you do not need to reload nginx. the principle of embedding nginx in openresty into luajit: A luajit virtual machine is embedded in every nginx process to execute the lua script. Nginx handed over the execution of lua script to luajit vm.3. ngx_lua commands and APIs. we mentioned that the lua script embedded in nginx can use content_by_lua and content_by_lua_file, which are actually Directive (Directives ), there are many similar commands, which are the entry for nginx to access lua scripts. Ngx_lua API: The command is the entry for nginx to access lua scripts. So how does the lua script call functions in nginx? It is through the ngx_lua API. The various * _ by_lua and * _ by_lua_file configuration directives serve as always ways to the Lua API within the nginx. conf file. the NGINX Lua API described below can only be called within the user Lua code run in the context of these configuration directives. the API is exposed to Lua in the form of two standard packages ngx and ndk. these packages are in the default global scope within ngx_lua and Re always available within ngx_lua ctictives. In fact, the interactive development of nginx and Lua mainly involves commands and APIs. Of course, there is also the lua script syntax. The command is the entry for nginx to access lua. The API is the function that lua calls nginx, and lua is the script programming language. The command is actually very simple, so it is mainly familiar with the ngx_lua API and Lua syntax. 4. lua access redis lua-resty-redis module: https://github.com/openresty/lua-resty-redis (see for documentation) in nginx. conf Add:

        location /redis_test{            content_by_lua_file lua/redis_test.lua;        }

 

Redis_test.lua content:

[Root @ localhost lua] # cat redis_test.lualocal redis = require "resty. redis "local red = redis: new () red: set_timeout (1000) local OK, err = red: connect (" 127.0.0.1 ", 6379) if not OK then ngx. say ("failed to connect:", err) returnendngx. say ("set result:", OK) local res, err = red: get ("dog") if not res then ngx. say ("failed to get doy:", err) returnendif res = ngx. null then ngx. say ("dog not found. ") returnendngx. say ("dog:", res) [root @ localhost lua] # access: [root @ localhost lua] # curl localhost/redis_testset result: 1dog: an animal [root @ localhost lua] #

 

The access is successful. 5. lua access mysql module of mysql openresty: lua-resty-mysql: https://github.com/openresty/lua-resty-mysql (see for documentation) Add the following configuration in nginx. conf:

    location /mysql_test {            content_by_lua_file lua/mysql_test.lua;        }

 

Mysql_test.lua script content:

[root@localhost lua]# pwd/opt/openresty/nginx/lua[root@localhost lua]# cat mysql_test.lualocal mysql = require "resty.mysql"local db, err = mysql:new()if not db then        ngx.say("failed to instantiate mysql: ", err)        returnenddb:set_timeout(1000)local ok, err, errno, sqlstate = db:connect{        host = "127.0.0.1",        port = 3306,        database = "ngx_lua",        user = "root",        password="digdeep",        max_packet_size = 1024 * 1024}if not ok then        ngx.say("failed to connect: ", err, ": ", errno, " ", sqlstate)        returnendngx.say("connected to mysql.")local res, err, errno, sqlstate = db:query("drop table if exists cats")if not res then        ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".")        returnendres, err, errno, sqlstate = db:query("create table cats " .. "(id int not null primary key auto_increment, "                                        .. "name varchar(30))")if not res then        ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".")        returnendngx.say("table cats created.")res, err, errno, sqlstate = db:query("insert into cats(name) " .. "values (\'Bob\'),(\'\'),(null)")if not res then        ngx.say("bad request: ", err, ": ", errno, ": ", sqlstate, ".")        returnendngx.say(res.affected_rows, " rows inserted into table cats ", "(last insert id: ", res.insert_id, ")")res, err, errno, sqlstate = db:query("select * from cats order by id asc", 10)if not res then        ngx.say("bad result ", err, ": ", errno, ": ", sqlstate, ".")        returnendlocal cjson = require "cjson"ngx.say("result: ", cjson.encode(res))local ok, err = db:set_keepalive(1000, 100)if not ok then        ngx.say("failed to set keepalive: ", err)        returnend

 

Test:

[root@localhost lua]# curl localhost/mysql_testconnected to mysql.table cats created.3 rows inserted into table cats (last insert id: 1)result: [{"name":"Bob","id":1},{"name":"","id":2},{"name":null,"id":3}]

 

Test passed. 5. lua capture and capture_multi (subquery) capture_multi are very powerful functions of openresty. It can greatly reduce the number of http requests sent by front-end browsers, breaking the browser's limit on the number of concurrent requests to the same server, because it can reduce Multiple front-end http requests to only one http request to nginx, then nginx uses the capture_multi feature to initiate multiple asynchronous concurrent requests to the backend, then the results are returned to the front-end. The following is an example: Add the following location configuration in nginx. conf and configure nginx to access php:

        location /capture {            content_by_lua_file lua/capture.lua;            #access_by_lua_file lua/capture.lua;        }        location ~ \.php$ {            root           html;            fastcgi_pass   127.0.0.1:9000;            fastcgi_index  index.php;            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;            include        fastcgi_params;        }

 

The capture. lua code is as follows:

[root@localhost lua]# pwd/opt/openresty/nginx/lua[root@localhost lua]# cat capture.lualocal res1,res2,res3,res4 = ngx.location.capture_multi{        {"/mysql_test", {args="t=1&id=1"}},        {"/redis_test", {args="t=2&id=2"}},        {"/lua", {args="t=3&id=3"}},        {"/index.php", {args="t=3&id=3"}},}ngx.header.content_type="text/plain"ngx.say(res1.body)ngx.say(res2.body)ngx.say(res3.body)ngx.say(res4.truncated)ngx.say(res4.status)ngx.say(res4.header["Set-Cookie"])--ngx.say(res4.body)

 

Index. php code:

[root@localhost html]# pwd/opt/openresty/nginx/html[root@localhost html]# cat index.php<?php        echo phpinfo();?>

 

Access:

[root@localhost html]# curl localhost/captureconnected to mysql.table cats created.3 rows inserted into table cats (last insert id: 1)result: [{"name":"Bob","id":1},{"name":"","id":2},{"name":null,"id":3}]set result: 1dog: an animalhello ngx_lua!!!!false200nil

 

The access is successful. The results of/mysql_test,/redis_test,/lua,/index. php are all output. Note: ngx. location. capture_multi {......} multiple asynchronous concurrent requests in can be nginx. the location (for example,/mysql_test,/redis_test,/lua) configured in conf can also be not the path configured by location, such as index. php is not. Index. php is a simple background php script. Of course, it can also be a background interface implemented by java. 6. the openresty cache lua_shared_dict defines a cache: In the nginx configuration file nginx. the following command is added to the http end of conf: lua_shared_dict ngx_cache 128 m; a memory named ngx_cache with a size of m is defined for caching. Note that the cache is shared by all nginx work processes. Access the cache in the lua script:

local ngx_cache = ngx.shared.ngx_cachelocal value = ngx_cache:get(key)local succ, err, forcible = ngx_cache:set(key, value, exptime)

 

The following is a test. First, add the following content to the nginx. conf server:

        location /cache {            content_by_lua_file lua/cache.lua;        }

 

Then write the cache. lua script:

[root@localhost lua]# cat cache.lualocal redis = require "resty.redis"local red = redis:new()function set_to_cache(key, value, exptime)        if not exptime then                exptime = 0        end        local ngx_cache = ngx.shared.ngx_cache        local succ, err, forcible = ngx_cache:set(key, value, exptime)        return succendfunction get_from_cache(key)        local ngx_cache = ngx.shared.ngx_cache;        local value = ngx_cache:get(key)        if not value then                value = get_from_redis(key)                set_to_cache(key, value)                return value        end        ngx.say("get from cache.")        return valueendfunction get_from_redis(key)        red:set_timeout(1000)        local ok, err = red:connect("127.0.0.1", 6379)        if not ok then                ngx.say("failed to connect: ", err)                return        end        local res, err = red:get(key)        if not res then                ngx.say("failed to get doy: ", err)                return ngx.null        end        ngx.say("get from redis.")        return resendfunction set_to_redis(key, value)        red:set_timeout(1000)        local ok, err = red:connect("127.0.0.1", 6379)        if not ok then                ngx.say("failed to connect: ", err)                return        end        local ok, err = red:set(key, value)        if not ok then                ngx.say("failed to set to redis: ", err)                return        end        return okendset_to_redis('dog', "Bob")local rs = get_from_cache('dog')ngx.say(rs)

 

Test:

[root@localhost ~]# curl localhost/cacheget from redis.Bob[root@localhost ~]# curl localhost/cacheget from cache.Bob[root@localhost ~]# curl localhost/cacheget from cache.Bob

 

It is obtained from redis for the first time and then from the cache every time. You can use AB to test rps (Requests per second): AB-n 1000-c 100-k http: // 127.0.0.1/cache7. solve cache invalidation storm lua-resty-lock cache invalidation storm refers to when the cache expires due to expiration time, this will cause all requests to access the backend redis or mysql, resulting in immediate increase in CPU performance. So the key is that when the cache fails, use lock to ensure that only one thread accesses redis or mysql in the background, and then update the cache. Use the locking and unlocking functions of the lua-resty-lock module. Lua-resty-lock documentation: https://github.com/openresty/lua-resty-lock first in nginx. add the following command to the http end of conf: lua_shared_dict ngx_cache 128 m; # cachelua_shared_dict cache_lock 100 k; # lock for cache and then in nginx. add:

        location /cache_lock {            content_by_lua_file lua/cache_lock.lua;        }

 

Cache_lock.lua code: test:

[root@localhost lua]# curl localhost/cache_lockget from cache.Bob[root@localhost lua]# curl localhost/cache_lockget from cache.Bob

 

7. openresty the nginx execution stage is divided into many stages, so third-party modules can add some processing at a certain stage. Openresty simplifies the process into seven phases: the execution sequence of the seven phases is as follows: set_by_lua: Process Branch judgment, judge the initial variable ha rewrite_by_lua: Use the lua script to implement nginx rewrite access_by_lua: ip access, whether access is permitted. Firewall content_by_lua: the memory generates header_filter_by_lua: filter http header information, add header information body_filter_by_lua: The content is case sensitive, and the content is encrypted log_by_lua: logs can be recorded locally or remotely, but only content_by_lua can be used. All functions are completed at this stage.