Python----deployment of production environments

Several ways Python deploys the web development program

• FASTCGI, supported by the Flup module, the corresponding configuration command in Nginx is Fastcgi_pass

• Http,nginx uses Proxy_pass forwarding, which requires that the backend appplication must have a built-in HTTP server capable of handling high concurrency, and in Python's web framework, only tornado can be selected.

• Uwsgi, consisting of 4 parts:

  • UWSGI protocol
  • Web server built-in Support protocol module
  • Application Server protocol Support module
  • Process Control Procedures

  Nginx starting from 0.8.4 the built-in support UWSGI protocol, UWSGI protocol is very simple, a 4 byte header+ a body,body can be a lot of protocol package, such as http,cgi (through the header field marked).

  Uwsgi is characterized by its own process Control program. It is written in C, using the Natvie function, which is actually similar to SPAWN-FCGI/PHP-FPM. So UWSGI can support a variety of application frameworks, including (PYTHON,LUA,RUBY,ERLANG,GO), etc.

• Mod_python, this is the Apache built-in module, very heavily dependent on Mod_python compiled using the Python version, and Apache companion use, not recommended

• CGI, this is too old, not recommended, and Nginx does not support CGI mode, only with LIGHTTPD or Apache

• SPAWN-FCGI, this is fastcgi multi-process management program, LIGHTTPD installation package comes with, and flup effect, the difference is Flup is the Python code-level introduction, SPAWN-FCGI is an external program. SPAWN-FCGI uses a wide range of code to support any language development, Php,python,perl, as long as your code implements the FastCGI interface, it can help you manage your process

• SCGI, the full name is simple Common Gateway Interface, but also the alternative version of CGI, SCGI protocol is very easy, I think and fastcgi almost, just not how to promote, nginx corresponding configuration command is Scgi_pass, If you want to use it, Flup also supports it.

• Gunicorn, and Uwsgi similar tools, migrated from the Rails deployment Tool (Unicorn). But the protocol it uses is WSGI, the full name is Python Web Server Gateway Interface, which is the official standard defined at python2.5 (PEP 333), Root red Miaozheng, and deployment is relatively simple, http://gunicorn.org /On a detailed tutorial

• A module of Mod_wsgi,apache, also supports the WSGI protocol, https://code.google.com/p/modwsgi/

UWSGI installation Uwsgi

Pip Install Uwsgi

Configure Uwsgi

UWSGI has several configurations available:

123, JSON4,yaml

Configuration examples

$ cat etc/= 127.0.0.1:9005=/users/suoning/python_project/trunk/wsgi-file =4  = 127.0.0.1:9000=/tmp/=/tmp/= truelog-maxsize = 50000000  Disable-logging == app$

Detailed configuration parameters:

Common options:

socket : Address and port number, for example: socket = 127.0.0.1:50000

processes : number of open processes

workers : Number of open processes, equivalent to processes (the official website says spawn the specified numbers of workers/processes)

chdir : Specify Run directory (chdir to specified directory before apps loading)

wsgi-file : Loading wsgi-file (load. wsgi file)

stats : On the specified address, turn on the status service (enable the stats server on the specified)

Threads : Runs the thread. Because of the Gil's existence, I feel that this sincerity is useless. (Run each worker in prethreaded mode with the specified number of threads)

Master : Allow the main process to exist (enable master processes)

daemonize : Causes the process to run in the background and hits the log to the specified log file or UDP server (Daemonize uwsgi). In fact, the most common thing is to output the running record to a local file.

log-maxsize: cut the log file in fixed file size (in kilobytes). For example: Log-maxsize = 50000000 is 50M a log file.

pidfile : Specifies the location of the PID file and records the PID number of the main process.

Vacuum : Automatically cleans up the environment when the server exits, removes UNIX socket files and PID files (try to remove all of the generated file/sockets)

disable-logging : Log of request information is not logged. Only errors and UWSGI internal messages to the log are logged. If this is not turned on, there will be a large number of such records in your log:

[pid:347|app:0|req:106/367] 117.116.122.172 () {VARs in 961 bytes} [Thu Jul 7 19:20:56] Post/post = Gener Ated bytes in 6 msecs (http/1.1) 2 headers in bytes (1 switches on Core 0)

Configure Nginx

$ cat etc/nginx/servers/pan.conf server {Listen80;    server_name localhost; Location/{include uwsgi_params; Uwsgi_pass127.0.0.1:9005; } Location/webstatic/{expires 7d; Add_header Cache-Control public; Alias/users/suoning/probject/python_project/webstatic/trunk/; }}$ $ nginx-Tnginx:the configuration file/usr/local/etc/nginx/nginx.conf syntax isoknginx:configuration File/usr/local/etc/nginx/nginx.conf Test issuccessful$ $ nginx-s reload$

Configuring the Applicationflask Sample

= Flask ('pan') ... if __name__ ' __main__ ' :     # app.run (host= ' 0.0.0.0 ', port=5000)     app.run ()#

Start Uwsgi

$ $ Uwsgi--ini/usr/local/etc/Uwsgi.ini[uwsgi] Getting INI configuration from/usr/local/etc/uwsgi.ini$ $ ps-ef|grep Uwsgi501 11428 1 0 11:40 pm?? 0:01.23 Uwsgi--ini/usr/local/etc/Uwsgi.ini501 11432 11428 0 11:40 pm?? 0:00.00 Uwsgi--ini/usr/local/etc/Uwsgi.ini501 11433 11428 0 11:40 pm?? 0:00.00 Uwsgi--ini/usr/local/etc/Uwsgi.ini501 11434 11428 0 11:40 pm?? 0:00.00 Uwsgi--ini/usr/local/etc/Uwsgi.ini501 11435 11428 0 11:40 pm?? 0:00.00 Uwsgi--ini/usr/local/etc/Uwsgi.ini501 11440 69240 0 11:40 ttys000 0:00.00grep uwsgi$ $ lsof-I. tcp:9000COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE Nameuwsgi11428 suoning 28u IPv4 0x5583e11534d24e730t0 TCP Localhost:cslistener (LISTEN) $$ lsof-I. tcp:9005COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE Nameuwsgi11428 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005(LISTEN) Uwsgi11432 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005(LISTEN) Uwsgi11433 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005(LISTEN) Uwsgi11434 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005(LISTEN) Uwsgi11435 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost:9005(LISTEN) $

FCGI

Reference: Http://webpy.org/cookbook/fastcgi-nginx

Configure Nginx

$ cat etc/nginx/servers/pan.conf server {Listen80;    server_name localhost; Error_page500 502 503 504/50x.html; Location= /50x.html {root html; } Location/{fastcgi_param request_method $request _method;        Fastcgi_param query_string $query _string;        Fastcgi_param Content_Type $content _type;        Fastcgi_param content_length $content _length; Fastcgi_param Gateway_interface CGI/1.1; Fastcgi_param server_software Nginx/$nginx _version;        Fastcgi_param remote_addr $remote _addr;        Fastcgi_param Remote_port $remote _port;        Fastcgi_param server_addr $server _addr;        Fastcgi_param server_port $server _port;        Fastcgi_param server_name $server _name;        Fastcgi_param server_protocol $server _protocol;        Fastcgi_param script_filename $fastcgi _script_name;        Fastcgi_param path_info $fastcgi _script_name; Fastcgi_pass127.0.0.1:9005; } Location/webstatic/{expires 7d; Add_header Cache-Control public; Alias/users/suoning/probject/python_project/webstatic/trunk/; }}$

Simple example of configuring application

 from Import Wsgiserver  from Import Appwsgiserver (    app,    bindaddress=(host, Port),
    maxthreads=threads). Run ()

Production Environment Example

#!/usr/bin/env python#-*-coding:utf-8-*-__author__='suoning'ImportSYSImportArgparse fromflup.server.fcgiImportWsgiserver fromLib.daemonImportDaemon fromPanImportAppapp_name='Pan_platform'App_inst_name='20170501'Parser= Argparse. Argumentparser (description=u'Run an pan FastCGI server') parser.add_argument ('Command', type=str, Help=u'command [Start|stop|restart]', Choices=['Start','Stop','Restart']) parser.add_argument ('- P','--port', type=int, help=u'Port of this server', required=True) parser.add_argument ('- T','--threads', Type=int, default=50, Help=u'Max Number of threads') parser.add_argument ('-host','--host', default='0.0.0.0', Help=u'Listen to the main clause')classPanplatformdaemon (Daemon):defRun (self):#Running the service        Try: Wsgiserver (app, Bindaddress=(Args.host, Args.port), MaxThreads=args.threads, Umask=0111). Run ()except: Sys.stderr.write ('oops')defGen_pidfile (port):return '/var/run/%s_%s_%d.pid'%(App_name, App_inst_name, Port)if __name__=='__main__': Args=Parser.parse_args () daemon=Panplatformdaemon (Gen_pidfile (args.port))if 'Start'==Args.command:daemon.start ()elif 'Stop'==args.command:daemon.stop ()elif 'Restart'==Args.command:daemon.restart ()Else:        Print "Unknown Command"Sys.exit (2) sys.exit (0)

Comparison of FASTCGI protocol and HTTP protocol in code deployment

• Although FASTCGI is a binary protocol, it does not conserve resources relative to the HTTP protocol. Binary protocol, can only save the expression of numbers, such as 1234567, with a string to represent the need for 7 byte, with a number is 4 byte, and the string to where all the same

• FastCGI in the transfer of data, in order to be compatible with the CGI protocol, but also with a bunch of CGI environment variables, so compared with the HTTP protocol, with the fastcgi transmission of data does not save, but more

• FastCGI the only advantage is that it is a long connection, the user concurrent 1000 request,fastcgi may be used 10 links to the back end of the appplication, if the HTTP protocol, how much to give, will be back to the end Appplication Initiate 1000 requests

• HTTP proxy forwarding, in the face of ultra-high concurrency in the case of problems, because, TCP stack, port is int16 integer you create a local connect, you need to consume a port, up to 65536. External concurrent hundreds of thousands of requests, port pool dry, your server can only refuse to respond

CGI, FCGI, scgi, WSGI difference

WIKI Links:

cgi-http://en.wikipedia.org/wiki/common_gateway_interfacefcgi- http://en.wikipedia.org/wiki/fcgiscgi-  http://en.wikipedia.org/wiki/scgiwsgi- http://en.wikipedia.org/wiki/wsgi other reference:http:/ /helpful.knobs-dials.com/index.php/cgi%2c_fastcgi%2c_scgi%2c_wsgi%2c_servlets_and_such#fastcgi_and_scgi  CGI = Common Gateway interface As the name implies, it is an interface specification. This specification details the server Agent running in the Web server, how to obtain and return the parameter names in the server environment context and the HTTP protocol during Web page generation, as you know: Request_method,query_string,content_ Type, and so on. Most Web server programs accept and process HTTP requests as script proxies, returning HTTP pages or responses. These scripts are known as PHP, ASP, JSP, and so on.  fcgi = Fast CGI It is actually a variant of the CGI in the concrete implementation. It is designed to achieve the ultimate goal of improving Web service performance by reducing the communication overhead of the CGI agent and web hosting service programs. This shows that fcgi in the specification and CGI is not different, but the specific implementation of the way to improve: CGI, for each HTTP request, the web Host service program to establish a new process to invoke the server script, the corresponding request; fcgi's approach is, Establishing a stand-alone fcgi process that communicates with the web hosting service process, which, once started, allocates resources, creates threads to respond to HTTP requests, and determines its life cycle, greatly reducing the resource overhead that the system makes in order to create processes. Modern popular Web server programs, such as PHP, ASP. NET, are basically fcgi implementations.  scgi = Simple CGI It is the product of fcgi after streamlining the data protocol and the response process. It is designed to accommodate an increasing number of Ajax-or rest-based HTTP requests and to make faster and more concise responses. and scgi Convention, when the serverImmediately after returning a response to an HTTP protocol request, close the HTTP connection. So it is not difficult to see that scgi is more suitable for the "request-forget" mode of communication advocated by SOA in general sense.  wsgi = web Server Gateway Interface This protocol is a patent for the Python language, which defines a set of universally applicable interfaces for communication between the WEB Service Host program and the HTTP response agent. It was generated because Python programmers noticed that there was a serious coupling between the web framework and the web hosting server program, for example, that some frameworks were designed for Apache's Mod_python. As a result, WSGI defines a very low-level interface. Common Python Web frameworks implement this protocol: such as  cherrypy, Django, web.py, Web2py, TurboGears, Tornado, pylons, Bluebream, Google App engin E[dubious–discuss], Trac, Flask, Pyramid, et cetera.

Python----deployment of production environments