Implement online high-performance http interface nginx load tornado back-end lua data

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1622342X4-0.jpg "title =" tornado.jpg "alt =" 101638437.jpg"/>

Recently implemented an http interface ~ Http. The access result is json.

Nginx performs load, web separation, and url forwarding ~

Tornado performs data operations and asynchronously calls lua interfaces ~

Memcached shares sessions (for the sake of not using redis for sessions, on the one hand, it is to avoid kv conflicts with redis, server interruption redundancy, and more importantly, I didn't make it when using redis to store sessions, depressed ....)

Redis implements kv database and Queue Systems

The backend nginx lua is the mysql json display interface in http mode.

All users know about nginx configuration is to use upstream load.

worker_processes 4;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              error_log /var/log/nginx/error.log;pid /var/run/nginx.pid;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              events {    worker_connections 30000;    useepoll;}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              http {    charset utf-8;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  # Enumerate all the Tornado servers here    upstream frontends {        server 127.0.0.1:8000;        server 127.0.0.1:8001;        server 127.0.0.1:8002;        server 127.0.0.1:8003;    }                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  include/path/to/nginx.mime.types;    default_type application/octet-stream;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  access_log /var/log/nginx/access.log;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  keepalive_timeout 65;    proxy_read_timeout 200;    sendfile on;    tcp_nopush on;    tcp_nodelay on;    gzip on;    gzip_min_length 1000;    gzip_proxied any;    gzip_types text/plain text/css text/xml               application/x-javascript application/xml               application/atom+xml text/javascript;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  # Only retry ifthere was a communication error, not a timeout    # on the Tornado server (to avoid propagating "queries of death"    # to all frontends)    proxy_next_upstream error;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  server {        listen 80;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      location ^~ /static/ {            root /path/to/app;            if($query_string) {                expires max;            }        }                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      location / {            proxy_pass_header Server;            proxy_set_header Host $http_host;            proxy_redirect off;            proxy_set_header X-Real-IP $remote_addr;            proxy_set_header X-Scheme $scheme;            proxy_pass http://frontends;        }    }}

For backend nodes, use the nginx check module for monitoring checks ~

Module: https://github.com/cep21/healthcheck_nginx_upstreams## :: ngx_http_healthcheck_module

Upstream myhttpd {server 127.0.0.1: 8800 weight = 5; server 127.0.0.1: 8801 weight = 5; healthcheck_enabled; enable the health check function healthcheck_delay 1000; interval between two checks on the same backend server, in milliseconds. The default value is 1000. Healthcheck_timeout 1000; the timeout time for a health check, in milliseconds. The default value is 2000. Healthcheck_failcount 3; check whether a backend server succeeds or fails several times before determining whether it succeeds or fails, and enable or disable healthcheck_send "GET /. health HTTP/1.0 "; check whether a backend server is successful or failed several times before determining whether it is successful or failed, and enable or disable the server. Only the http 1.0 protocol is supported.

Comparison of uwsgi tornado:

A friend asked me that uwsgi's performance is okay...

In the past, when I used flask, I often used it with uwsgi. However, I tested the stress in many aspects with people in the group. uwsgi was not very stable and often encountered access errors, I can't see anything about debug.

How to enable so many tornado processes?

Use screen,>/dev/null, and nohup? It's so awkward ~

We recommend that you use supervisord to manage messy tornado services and other python Services.

Source http://rfyiamcool.blog.51cto.com/1030776/1301410

Here is the configuration of centos supervisord

Configure a supervisor

yum install supervisorpip install supervisor

Echo_supervisord_conf>/etc/supervisord. conf

Vim/etc/supervisord. conf

[program:tornado8000]command=python /ops/tornado8000.pyprocess_name=%(program_name)sstdout_logfile=/ops/access.logstderr_logfile=/ops/error.log[program:tornado8001]command=python /ops/tornado8001.pyprocess_name=%(program_name)sstdout_logfile=/ops/access.logstderr_logfile=/ops/error.log[program:tornado8002]command=python /ops/tornado8002.pyprocess_name=%(program_name)sstdout_logfile=/ops/access.logstderr_logfile=/ops/error.log[program:tornado8003]command=python /ops/tornado8003.pyprocess_name=%(program_name)sstdout_logfile=/ops/access.logstderr_logfile=/ops/error.log

After saving the configuration, run ~

/usr/bin/supervisord -c /etc/supervisord.conf

After Supervisord is installed, there are two available command lines: supervisor and supervisorctl. The command usage is explained as follows:

• Supervisord: starts Supervisord and starts and manages the processes set in the configuration.

• Supervisorctl stop programxxx, stop a process (programxxx), programxxx is[program:chatdemon]Value configured in. This example is chatdemon.

• Supervisorctl start programxxx to start a process

• Supervisorctl restart programxxx, restart a process

• Supervisorctl stop groupworker:, restart all processes that belong to the group named groupworker (start and restart are the same)

• Supervisorctl stop all, stop all processes, note: start, restart, stop will not load the latest configuration file.

• Supervisorctl reload: load the latest configuration file, stop the original process, start and manage all processes according to the new configuration.

• Supervisorctl update: Start new or modified processes based on the latest configuration file. Processes with no configuration changes will not be affected and restart.

• Note: The process stopped with stop is displayed. It will not be restarted automatically with reload or update.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/16223430S-1.png "title =" 11:47:15 screen .png "alt =" 114739607.png"/>

Then use the decorator to determine the status...

@ Tornado. web. authenticated

The code is here for you to configure your own system.

Https://github.com/mmejia27/tornado-memcached-sessions

For usage of lua, you can refer to my previous blog post and have detailed usage of mysql.

Module code:

Https://github.com/agentzh/lua-resty-mysql

Instance code:

upstream nginxbackend { drizzle_server 127.0.0.1:3306 dbname=nginx password=youpassword user=yourdbname protocol=mysql charset=utf8; drizzle_keepalive max=100 mode=single overflow=reject; }                                                                                                                                                                                                                                                                                                                                                               location = /getuser { internal; set_unescape_uri $user $remote_user; set_unescape_uri $passwd $remote_passwd; set_quote_sql_str $user $user; set_quote_sql_str $passwd $passwd; drizzle_query "select count(*) as count from user where user=$user and passwd=$passwd"; drizzle_pass nginxbackend; rds_json on; }

In short, we need a variety of test pressure to find a suitable framework for our own business.

This article is from "Fengyun, it's her ." Blog, please be sure to keep this source http://rfyiamcool.blog.51cto.com/1030776/1301410