Introduction to Python's approach to deploying Web development

This article mainly introduced Python Deployment Web development program Several methods, has the very good reference value. Let's take a look at the little series.

1, fastcgi, through the Flup module to support, in Nginx corresponding configuration instructions are Fastcgi_pass

2, Http,nginx use Proxy_pass forwarding, this requires the backend appplication must be built to handle high-concurrency HTTP server, in the Python web framework, only select Tornado.

3,Uwsgi, including 4 components:

• UWSGI protocol

• Web server built-in Support protocol module

• Application Server protocol Support module

• Process Control Procedures

Nginx starting from 0.8.4 the built-in support UWSGI protocol, UWSGI protocol is very simple, a 4 byte header+ a body,body can be a lot of protocol package, such as http,cgi (through the header field marked).

Uwsgi is characterized by its own process Control program. It is written in C, using the Natvie function, which is actually similar to SPAWN-FCGI/PHP-FPM. So UWSGI can support a variety of application frameworks, including (PYTHON,LUA,RUBY,ERLANG,GO), etc.

4, Mod_python, this is the Apache built-in module, very heavily dependent on the Mod_python compiled using the Python version, and Apache companion use, not recommended

5, CGI, this is too old, not recommended, and Nginx does not support CGI mode, only with LIGHTTPD or Apache

6, spawn-fcgi, this is fastcgi multi-process management program, LIGHTTPD installation package comes with, and flup effect, the difference is Flup is the Python code-level introduction, SPAWN-FCGI is an external program. SPAWN-FCGI uses a wide range of code to support any language development, Php,python,perl, as long as your code implements the FastCGI interface, it can help you manage your process

7, scgi, the full name is simple Common Gateway Interface, but also the alternative version of CGI, SCGI protocol is very easy, I think and fastcgi almost, just not how to promote, nginx corresponding configuration command is Scgi_pass, If you want to use it, Flup also supports it.

8, Gunicorn, and Uwsgi similar tools, from the Rails deployment Tool (Unicorn) migrated over. But it uses the protocol is WSGI, the full name is Python Web Server Gateway Interface, which is the official standard defined by python2.5 (PEP 333), Root red Miaozheng, and the deployment is relatively simple, gunicorn.org/on the detailed tutorial

9, Mod_wsgi,apache A module, is also supporting the WSGI protocol, code.google.com/p/modwsgi/

Uwsgi

Installing UWSGI

pip install uwsgi

Configure Uwsgi

UWSGI has several configurations available:

1,ini 2,xml 3,json4,yaml

Configuration examples

$ cat Etc/uwsgi.ini [uwsgi]socket = 127.0.0.1:9005chdir =/users/suoning/python_project/trunk/wsgi-file = main.pyprocesses = 4stats = 127.0.0.1:9000daemonize =/tmp/uwsgiserver.logpidfile =/tmp/uwsgi.pidvacuum = Truelog-maxsize = 50000000disable-logging = Truecallable = app$

Detailed configuration parameters:

Common options:

socket: address and port number, for example: socket = 127.0.0.1:50000

processes: Number of open processes

workers: number of open processes, equivalent to processes (the official website says spawn the specified numbers of workers/processes)

chdir: Specify Run directory (chdir to specified directory before apps loading)

wsgi-file: loading Wsgi-file (load. wsgi file)

Stats: on the specified address, turn on the status service (enable the stats server on the specified)

Threads: runs the thread. Because of the Gil's existence, I feel that this sincerity is useless. (Run each worker in prethreaded mode with the specified number of threads)

Master: allow the main process to exist (enable master processes)

daemonize: causes the process to run in the background and hits the log to the specified log file or UDP server (Daemonize uwsgi). In fact, the most common thing is to output the running record to a local file.

log-maxsize: cut the log file in fixed file size (in kilobytes). For example: Log-maxsize = 50000000 is 50M a log file.

Pidfile: Specifies the location of the PID file and records the PID number of the main process.

Vacuum: automatically cleans up the environment when the server exits, removes UNIX socket files and PID files (try to remove all of the generated file/sockets)

disable-logging: Log of request information is not logged. Only errors and UWSGI internal messages to the log are logged. If this is not turned on, there will be a large number of such records in your log:

[pid:347|app:0|req:106/367] 117.116.122.172 () {VARs in 961 bytes} [Thu Jul 7 19:20:56] Post/post = genera Ted 6 Msecs (http/1.1) 2 headers in bytes bytes (1 switches on Core 0)

Configure Nginx

$ cat Etc/nginx/servers/tongbupan.confserver {listen  ; server_name localhost; location/{  include Uwsgi_ params;  Uwsgi_pass 127.0.0.1:9005; } location/webstatic/{  expires 7d;  Add_header Cache-control public;  alias/users/suoning/probject/python_project/webstatic/trunk/; }}$ $ nginx-tnginx:the Configuration file/usr/local/etc/nginx/nginx.conf syntax is oknginx:configuration file/usr/loc Al/etc/nginx/nginx.conf test is successful$ $ nginx-s reload$

Configure Application

Flask Example

... app = Flask (' Pan ') ... if name = = ' main ': # App.run (host= ' 0.0.0.0 ', port=5000) App.run () # Note: Variable app corresponds to uwsgi config file Uwsgi.ini Callable = App

Start Uwsgi

$ $ uwsgi--ini/usr/local/etc/uwsgi.ini[uwsgi] Getting INI configuration from/usr/local/etc/uwsgi.ini$ $ ps-ef|grep UWS   gi11428 1 0 11:40 pm??   0:01.23 Uwsgi--ini/usr/local/etc/uwsgi.ini11432 11428 0 11:40 pm??   0:00.00 Uwsgi--ini/usr/local/etc/uwsgi.ini11433 11428 0 11:40 pm??   0:00.00 Uwsgi--ini/usr/local/etc/uwsgi.ini11434 11428 0 11:40 pm??   0:00.00 Uwsgi--ini/usr/local/etc/uwsgi.ini11435 11428 0 11:40 pm?? 0:00.00 Uwsgi--ini/usr/local/etc/uwsgi.ini11440 69240 0 11:40 pm ttys000 0:00.00 grep uwsgi$ $ lsof-i tcp:9000command PI D USER FD TYPE DEVICE size/off NODE nameuwsgi 11428 suoning 28u IPv4 0x5583e11534d24e73 0t0 TCP Localhost:cslistener ( LISTEN) $$ lsof-i tcp:9005command PID USER FD TYPE DEVICE size/off NODE nameuwsgi 11428 suoning 6u IPv4 0x5583e11535699 E73 0t0 TCP localhost:9005 (LISTEN) uwsgi 11432 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005 (LISTEN) Uwsgi 1 1433 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005 (LISTEN) Uwsgi 11434 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005 (LISTEN) uwsgi 11435 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005 (LISTEN) $

FCGI

Reference: Webpy.org/cookbook/fastcgi-nginx

Configure Nginx

$ cat etc/nginx/servers/pan.confserver {listen; server_name localhost; error_page 500 502 503 504/50x.html;  Location =/50x.html {root HTML,} location/{Fastcgi_param Request_method $request _method;  Fastcgi_param query_string $query _string;  Fastcgi_param Content_Type $content _type;  Fastcgi_param content_length $content _length;  Fastcgi_param Gateway_interface cgi/1.1;  Fastcgi_param server_software nginx/$nginx _version;  Fastcgi_param remote_addr $remote _addr;  Fastcgi_param Remote_port $remote _port;  Fastcgi_param server_addr $server _addr;  Fastcgi_param server_port $server _port;  Fastcgi_param server_name $server _name;  Fastcgi_param server_protocol $server _protocol;  Fastcgi_param script_filename $fastcgi _script_name;  Fastcgi_param path_info $fastcgi _script_name; Fastcgi_pass 127.0.0.1:9005;  } location/webstatic/{expires 7d;  Add_header Cache-control public; alias/users/suoning/probject/python_project/webstatic/trunk/; }}$

Configure Application

Simple example

From flup.server.fcgi import wsgiserverfrom pan import appwsgiserver (app, bindaddress= (host, Port), maxthreads=threads) . Run ()

Production Environment Example

#!/usr/bin/env python#-*-coding:utf-8-*-author = ' suoning ' import sysimport argparsefrom flup.server.fcgi import wsgise Rverfrom Lib.daemon Import daemonfrom pan import appapp_name = ' pan_platform ' app_inst_name = ' 20170501 ' parser = Argparse.a Rgumentparser (description=u ' Run an pan FastCGI server ') parser.add_argument (' command ', type=str, help=u ' command [start]  |stop|restart] ', choices=[' start ', ' Stop ', ' restart ']) parser.add_argument ('-P ', '--port ', type=int, Help=u ' Port of This server ', required=true) parser.add_argument ('-t ', '--threads ', Type=int, default=50, Help=u ' max number of threads ') parser.add_argument ('-host ', '--host ', default= ' 0.0.0.0 ', Help=u ' Listen to the main clause ') class Panplatformdaemon ( Daemon): def run (self): # Run service try:wsgiserver (app, bindaddress= (Args.host, args.port), Maxthreads=args.thre Ads, umask=0111). Run () except:sys.stderr.write (' oops ') def gen_pidfile (port): Return '/var/run/%s_%s_%d.pid '% (A Pp_name, App_inst_name, pORT) If name = = ' main ': args = Parser.parse_args () daemon = Panplatformdaemon (Gen_pidfile (args.port)) if ' start ' = = args.co   Mmand:daemon.start () elif ' stop ' = = Args.command:daemon.stop () elif ' restart ' = = Args.command:daemon.restart () Else: print "Unknown command" Sys.exit (2) sys.exit (0)

Comparison of FASTCGI protocol and HTTP protocol in code deployment

• Although FASTCGI is a binary protocol, it does not conserve resources relative to the HTTP protocol. Binary protocol, can only save the expression of numbers, such as 1234567, with a string to represent the need for 7 byte, with a number is 4 byte, and the string to where all the same

• FastCGI in the transfer of data, in order to be compatible with the CGI protocol, but also with a bunch of CGI environment variables, so compared with the HTTP protocol, with the fastcgi transmission of data does not save, but more

• FastCGI the only advantage is that it is a long connection, the user concurrent 1000 request,fastcgi may be used 10 links to the back end of the appplication, if the HTTP protocol, how much to give, will be back to the end Appplication Initiate 1000 requests

• HTTP proxy forwarding, in the face of ultra-high concurrency in the case of problems, because, TCP stack, port is int16 integer you create a local connect, you need to consume a port, up to 65536. External concurrent hundreds of thousands of requests, port pool dry, your server can only refuse to respond

CGI, FCGI, scgi, WSGI difference

WIKI Links:

Cgi-en.wikipedia.org/wiki/common_gateway_interface
fcgi-en.wikipedia.org/wiki/fcgi
scgi-en.wikipedia.org/wiki/scgi
Wsgi-en.wikipedia.org/wiki/wsgi

Other reference:

helpful.knobs-dials.com/index.php/cgi%2c_fastcgi%2c_scgi%2c_wsgi%2c_servlets_and_such#fastcgi_and_scgi

CGI = Common Gateway Interface

As the name implies, it is an interface specification. This specification details the server Agent running in the Web server, how to obtain and return the parameter names in the server environment context and the HTTP protocol during Web page generation, as you know: Request_method,query_string,content_ Type, and so on. Most Web server programs accept and process HTTP requests as script proxies, returning HTTP pages or responses. These scripts are known as PHP, ASP, JSP, and so on.

FCGI = Fast CGI

It is actually a variant of CGI in a concrete implementation. It is designed to achieve the ultimate goal of improving Web service performance by reducing the communication overhead of the CGI agent and web hosting service programs. This shows that fcgi in the specification and CGI is not different, but the specific implementation of the way to improve: CGI, for each HTTP request, the web Host service program to establish a new process to invoke the server script, the corresponding request; fcgi's approach is, Establishing a stand-alone fcgi process that communicates with the web hosting service process, which, once started, allocates resources, creates threads to respond to HTTP requests, and determines its life cycle, greatly reducing the resource overhead that the system makes in order to create processes. Modern popular Web server programs, such as PHP, ASP. NET, are basically fcgi implementations.

scgi = Simple CGI

It is the product of fcgi after streamlining the data protocol and response process. It is designed to accommodate an increasing number of Ajax-or rest-based HTTP requests and to make faster and more concise responses. and scgi Convention, when the server returns a response to an HTTP protocol request, close the HTTP connection immediately. So it is not difficult to see that scgi is more suitable for the "request-forget" mode of communication advocated by SOA in general sense.

WSGI = Web Server Gateway Interface

This protocol is a patent for the Python language, which defines a set of universally applicable interfaces for communication between the Web service Host program and the HTTP response agent. It was generated because Python programmers noticed that there was a serious coupling between the web framework and the web hosting server program, for example, that some frameworks were designed for Apache's Mod_python. As a result, WSGI defines a very low-level interface. Common Python Web frameworks implement this protocol: such as CherryPy, Django, web.py, Web2py, TurboGears, Tornado, pylons, Bluebream, Google App Engine[dubi Ous–discuss], Trac, Flask, Pyramid, et cetera.