How to deploy web development in python

This article mainly introduces several methods for deploying web development programs in Python, which has good reference value. Next, let's take a look at it. This article mainly introduces several methods for deploying web development programs in Python, which has good reference value. Let's take a look at it with the small editor.

1. fastcgi, supported by the flup module. the configuration command in nginx is fastcgi_pass.

2. http and nginx use proxy_pass forwarding. Therefore, the backend appplication must have a built-in http server capable of processing high concurrency. in the python web framework, only tornado can be selected.

3,Uwsgi consists of four parts:

• Uwsgi protocol

• Web server built-in support protocol module

• Application Server Protocol support module

• Process Control Program

Nginx has built in support of the uwsgi protocol since 0.8.4. The uwsgi protocol is very simple. a 4-byte header + a body, the body can be a package of many protocols, such as http, cgi and so on (marked by fields in the header ).

Uwsgi features its own process control program. it is written in C language and uses the natvie function, which is similar to spawn-fcgi/php-fpm. Therefore, uwsgi supports a variety of application frameworks, including (python, lua, ruby, erlang, go) and so on.

4. mod_python, which is a built-in module of apache, relies heavily on the python version used for mod_python Compilation. it is used together with apache and is not recommended.

5. cgi. this is too old and is not recommended. nginx does not support cgi. it can only use lighttpd or apache.

6. spawn-fcgi. this is the fastcgi multi-process management program, which is included in the lighttpd installation package. The effect is the same as that of flup. The difference is that flup is python code-level introduction and spawn-fcgi is an external program. Spawn-fcgi is widely used and supports code developed in any language. php, python, and perl can help you manage your processes as long as your code implements the fastcgi interface.

7. scgi: The full name is Simple Common Gateway Interface, which is also an alternative to cgi. The scgi protocol is very Simple. I think it is similar to fastcgi, but it is not widely used, the configuration command corresponding to nginx is scgi_pass. you can use it if you want to use it. flup also supports it.

8. Gunicorn, a tool similar to uwsgi, is transplanted from the rails deployment tool (Unicorn. However, the protocol used is WSGI, which is full name of the Python Web Server Gateway Interface. this is the official standard (PEP 333) defined in python2.5, and the deployment is simple, detailed tutorial on gunicorn.org/

9. mod_wsgi, a module of apache, also supports the WSGI protocol, code.google.com/p/modwsgi/

Uwsgi

Install uwsgi

Pip install uwsgi

Configure uwsgi

Uwsgi has multiple configurations available:

1，ini 2，xml 3，json4，yaml

Configuration Example

$ cat etc/uwsgi.ini [uwsgi]socket = 127.0.0.1:9005chdir = /Users/suoning/python_project/trunk/wsgi-file = main.pyprocesses = 4stats = 127.0.0.1:9000daemonize = /tmp/uwsgiServer.logpidfile = /tmp/uwsgi.pidvacuum = truelog-maxsize = 50000000disable-logging = truecallable = app$

Configuration parameters:

Common options:

Socket:Address and port number, for example, socket = 127.0.0.1: 50000

Processes:Number of processes enabled

Workers:The number of processes enabled is equivalent to processes (the official website says spawn the specified number of workers/processes)

Chdir:Specify the running directory (chdir to specified directory before apps loading)

Wsgi-file:Load wsgi-file (load. wsgi file)

Stats:Enable the stats server on the specified address)

Threads:Running thread. Due to the existence of GIL, I think this is really useless. (Run each worker in prethreaded mode with the specified number of threads)

Master:Enable master process)

Daemonize:Run the process in the background and send the logs to the specified log file or udp server (daemonize uWSGI ). In fact, the most common method is to output the running records to a local file.

Log-maxsize:Cut log files by fixed file size (unit: KB. For example, log-maxsize = 50000000 is a 50 m log file.

Pidfile:Specify the location of the pid file and record the pid number of the main process.

Vacuum:When the server exits, the environment is automatically cleared and the unix socket file and pid file are deleted (try to remove all of the generated file/sockets)

Disable-logging:Logs that do not record request information. Only errors and uWSGI internal messages are logged. If this option is not enabled, a large number of such records will appear in your logs:

[Pid: 347 | app: 0 | req: 106/367] 117.116.122.172 () {52 vars in 961 bytes} [Thu Jul 7 19:20:56 2016] POST/post => generated 65 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 88 bytes (1 switches on core 0)

Configure nginx

$ cat etc/nginx/servers/tongbupan.confserver { listen  80; server_name localhost; location / {  include uwsgi_params;  uwsgi_pass 127.0.0.1:9005; } location /webstatic/ {  expires 7d;  add_header Cache-Control public;  alias /Users/suoning/probject/python_project/webstatic/trunk/; }}$ $ nginx -tnginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is oknginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful$ $ nginx -s reload$

Configure application

Flask example

... App = Flask ('Pan ')... if name = 'main': # app. run (host = '0. 0.0.0 ', port = 5000) app. run () # Note: The variable app corresponds to the uwsgi configuration file uwsgi. callable = app in ini

Start uwsgi

$ Uwsgi -- ini/usr/local/etc/uwsgi. ini [uWSGI] getting INI configuration from/usr/local/etc/uwsgi. ini $ ps-ef | grep uwsgi11428 1 0 PM ??. 23 uwsgi -- ini/usr/local/etc/uwsgi. ini11432 11428 0 PM ??. 00 uwsgi -- ini/usr/local/etc/uwsgi. ini11433 11428 0 PM ??. 00 uwsgi -- ini/usr/local/etc/uwsgi. ini11434 11428 0 PM ??. 00 uwsgi -- ini/usr/local/etc/uwsgi. ini11435 11428 0 PM ?? 0: 00. 00 uwsgi -- ini/usr/local/etc/uwsgi. ini11440 69240 0 ttys000. 00 grep uwsgi $ lsof-I tcp: 9000 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEuwsgi 11428 suoning 28u IPv4 0x5583e11534d24e73 0t0 TCP localhost: cslistener (LISTEN) $ lsof-I tcp: 9005 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEuwsgi 11428 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost: 9005 (LISTEN) uwsgi 11432 suoning 6u IPv4 protocol 0t0 TCP localhost: 9005 (LISTEN) uwsgi 11433 suoning 6u IPv4 protocol 0t0 TCP localhost: 9005 (LISTEN) uwsgi 11434 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost: 9005 (LISTEN) uwsgi 11435 suoning 6u IPv4 0x5583e11535699e73 0t0 TCP localhost: 9005 (LISTEN) $

FCGI

Reference: webpy.org/cookbook/fastcgi-nginx

Configure Nginx

$ cat etc/nginx/servers/pan.confserver { listen  80; server_name localhost; error_page 500 502 503 504 /50x.html; location = /50x.html {  root html; } location / {  fastcgi_param REQUEST_METHOD $request_method;  fastcgi_param QUERY_STRING $query_string;  fastcgi_param CONTENT_TYPE $content_type;  fastcgi_param CONTENT_LENGTH $content_length;  fastcgi_param GATEWAY_INTERFACE CGI/1.1;  fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;  fastcgi_param REMOTE_ADDR $remote_addr;  fastcgi_param REMOTE_PORT $remote_port;  fastcgi_param SERVER_ADDR $server_addr;  fastcgi_param SERVER_PORT $server_port;  fastcgi_param SERVER_NAME $server_name;  fastcgi_param SERVER_PROTOCOL $server_protocol;  fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;  fastcgi_param PATH_INFO $fastcgi_script_name;  fastcgi_pass 127.0.0.1:9005; } location /webstatic/ {  expires 7d;  add_header Cache-Control public;  alias /Users/suoning/probject/python_project/webstatic/trunk/; }}$

Configure application

Simple example

from flup.server.fcgi import WSGIServerfrom pan import appWSGIServer( app, bindAddress=(host, port), maxThreads=threads).run()

Production Environment Example

#! /Usr/bin/env python #-*-coding: UTF-8-*-author = 'suoning' import sysimport argparsefrom flup. server. fcgi import WSGIServerfrom lib. daemon import Daemonfrom pan import appAPP_NAME = 'Pan _ platform' APP _ INST_NAME = '000000' parser = argparse. argumentParser (description = u'run an pan FastCGI server') parser. add_argument ('command', type = str, help = u'command [start | stop | restart] ', choices = ['start', 'stop', 'restart']) parser. add_argument ('-p',' -- port', type = int, help = u 'Port of this server', required = True) parser. add_argument ('-t',' -- threads', type = int, default = 50, help = u 'Max number of threads') parser. add_argument ('-host',' -- host', default = '0. 0.0.0 ', help = u'listen to the main clause') class panPlatformDaemon (Daemon): def run (self): # run the service try: WSGIServer (app, bindAddress = (args. host, args. port), maxThreads = args. threads, umask = 0111 ). run () failed t: sys. stderr. write ('opps') def gen_pidfile (port): return '/var/run/% s _ % d. pid '% (APP_NAME, APP_INST_NAME, port) if name = 'main': args = parser. parse_args () daemon = panPlatformDaemon (gen_pidfile (args. port) if 'Start' = args. command: daemon. start () elif 'stop' = args. command: daemon. stop () elif 'restart' = args. command: daemon. restart () else: print "Unknown command" sys. exit (2) sys. exit (0)

Comparison of fastcgi protocol and http protocol in code deployment

• Although fastcgi is a binary protocol, it does not save resources compared with http. The binary protocol can only save the expression of numbers, for example, 1234567. it requires 7 bytes to use a string, and 4 bytes to use a number. the strings are the same everywhere.

• Fastcgi carries a bunch of cgi environment variables to ensure compatibility with the cgi protocol during data transmission. Therefore, compared with http, fastcgi does not save much for data transmission.

• The unique advantage of fastcgi is that it is a persistent connection, with 1000 concurrent requests. fastcgi may forward 10 links to the backend appplication. if http is used, if the number of requests is less than the limit, 1000 requests will be sent to the backend appplication.

• The http proxy forwarding method may cause problems in the face of ultra-high concurrency, because the port in the tcp protocol stack is an int16 Integer. you need to create a local connect and consume a port, it can be up to 65536. The port pool consumes up to 100,000 external concurrent requests, and your server can only refuse to respond.

Differences between CGI, FCGI, SCGI, and WSGI

WIKI Links:

CGI-en.wikipedia.org/wiki/Common_Gateway_Interface
FCGI-en.wikipedia.org/wiki/Fcgi
SCGI-en.wikipedia.org/wiki/SCGI
WSGI-en.wikipedia.org/wiki/Wsgi

Other reference:

Bytes

CGI = Common Gateway Interface

As the name suggests, it is an interface specification. This specification defines in detail the server proxy program running on the Web server, how to obtain and return the parameter names in the server environment context and HTTP when the Web page is generated, as you are familiar: REQUEST_METHOD, QUERY_STRING, CONTENT_TYPE, and so on. Most Web server programs accept and process HTTP requests as proxies in the form of scripts, and return HTTP pages or responses. These script programs are well-known PHP, ASP, JSP, and so on.

FCGI = Fast CGI

It is actually a variant of CGI in specific implementation. The design idea is to reduce the communication overhead between the CGI proxy program and the Web host service program to improve the Web service performance. It can be seen that FCGI is not different from CGI in terms of specifications, but it is improved in the specific implementation method: the CGI approach is that for each HTTP request, all Web host service programs create new processes to call server scripts and request requests. FCGI establishes an independent FCGI service program process to communicate with the Web host service process, once started, the FCGI service process allocates resources by itself, creates a thread to respond to HTTP requests, and determines its own lifecycle. This greatly reduces the resource overhead made by the system to create a process. Modern popular Web server programs, such as PHP and ASP. Net, are basically implemented by FCGI.

SCGI = Simple CGI

It is the product of FCGI after streamlining data protocols and response processes. It is designed to adapt to more and more HTTP requests based on AJAX or REST, and to make faster and more concise responses. In addition, SCGI stipulates that when the server returns a response to an HTTP request, the HTTP connection is immediately closed. It is not difficult to see that SCGI is more suitable for the "request-forget" communication mode advocated by SOA in the general sense.

WSGI = Web Server Gateway Interface

This protocol is a patent for the Python language and defines a set of universally applicable interfaces for communication between the Web service host program and the HTTP response proxy program. It was generated because Python programmers noticed that there was a serious coupling between the Web framework and the Web host server program. for example, some frameworks were designed for Apache mod_python. Therefore, WSGI defines a very low-level interface. Common Python Web frameworks all implement this protocol, such as CherryPy, Django, and web. py, web2py, TurboGears, Tornado, Pylons, BlueBream, Google App Engine [dubious-discuss], Trac, Flask, Pyramid, and so on.

The above describes how to deploy web development in python. For more information, see other related articles in the first PHP community!