Summary of nine deployment methods for python web programs

Mainstream web servers can be counted in a single slice, such as apache, lighttpd, nginx, and iis.

Application: the application code DB server you write based on a web framework refers to the storage service. mysql is widely used in web development. In recent years, due to the expansion of websites, memcache, redis, such as key-value, is also popular.
The front-end web server has three functions

Efficient processing of static files. web server is developed in c and calls are native functions. IO and file transmission are all optimized accordingly.

As a simple network firewall, you can use some denny ip addresses to control the number of concurrent connections.

Processes highly concurrent short connection requests and forwards requests of thousands of users through dozens of persistent connections on the Intranet. One reason is that the web server is very professional in Processing High concurrency, another reason is that most of the frameworks used by applications do not have the ability to handle high concurrency.

In fact, some web frameworks on the market have built-in support for efficient network libraries such as epoll and kqueue, and are capable of handling high concurrency, such as python tornado and java tomcat, jetty and so on. Some people just remove the front-end web server and run it naked, but it is best not to do so when deploying public network applications, because of the two reasons mentioned above, the network conditions from brower to web server are strange. You cannot imagine it,

Nginx is strongly recommended for web server for three reasons

Excellent performance and stability
Easy to install with fewer dependent packages
The conf file is very easy to configure and is simpler than apache/lighttpd.
There are nine ways to deploy a web program developed by python

Mod_python, which is a built-in module of apache, relies heavily on the python version used for mod_python compilation. It is used together with apache and is not recommended.

Cgi, this is too old, not recommended, and nginx does not support cgi, only lighttpd or apache can be used

Fastcgi, which is the most popular practice currently, is supported by the flup module. The configuration command in nginx is fastcgi_pass.

Spawn-fcgi. This is the fastcgi multi-process management program. It is included in the lighttpd installation package. It has the same effect as flup. The difference is that flup is python code-level introduction and spawn-fcgi is an external program. Spawn-fcgi is widely used and supports code developed in any language. php, python, and perl can help you manage your processes as long as your code implements the fastcgi interface.

Scgi: The full name is Simple Common Gateway Interface, which is also an alternative to cgi. The scgi protocol is very Simple. I think it is similar to fastcgi, but it is not widely used. The configuration command for nginx is scgi_pass, if you want to use it, flup also supports it.

Http and nginx use proxy_pass forwarding. Therefore, the backend appplication must have a built-in http server capable of processing high concurrency. In the python web framework, only tornado can be selected.

Python programmers like to invent the wheel. In addition to being a web framework, tornado can also independently provide high-performance http servers. Therefore, if you use other python frameworks to write code, such as bottle, you can also use import tornado to start a high-performance http server. You can also use http protocol and nginx for deployment. Extended, there are many http servers that can handle high concurrency in the python package, such as gevent, which can also be referenced by other frameworks to support http deployment.

In reality, java is used as a web program. Generally, http and nginx are used in combination. The application server selects tomcat or jetty.

Uwsgi, which consists of four parts,

Uwsgi Protocol
Web server built-in support protocol module
Application Server Protocol Support Module
Process Control Program

Nginx has built in support of the uwsgi protocol since 0.8.4. The uwsgi protocol is very simple. A 4-byte header + a body, the body can be a package of many protocols, such as http, cgi and so on (marked by the field in the header), I once made a small-scale performance comparison test, the results show that uwsgi and fastcgi have no obvious advantages in performance, it may also be because the dataset is small.

Uwsgi features its own process control program. It is written in C language and uses the natvie function, which is similar to spawn-fcgi/php-fpm. Therefore, uwsgi supports a variety of application frameworks, including (python, lua, ruby, erlang, go) and so on.

Gunicorn, a tool similar to uwsgi, is transplanted from the rails Deployment Tool (Unicorn. However, the Protocol used is WSGI, Which is full name of the Python Web Server Gateway Interface. This is the official standard (PEP 333) defined in python2.5, And the deployment is simple, detailed tutorials on http://gunicorn.org/

Mod_wsgi, a module of apache, also supports WSGI, https://code.google.com/p/modwsgi/

Comparison of fastcgi protocol and http protocol in code deployment

Although fastcgi is a binary protocol, it does not save resources compared with http. The binary Protocol can only save the expression of numbers, for example, 1234567. It requires 7 bytes to use a string, and 4 bytes to use a number. The strings are the same everywhere.

Fastcgi carries a bunch of cgi Environment Variables to ensure compatibility with the cgi protocol during data transmission. Therefore, compared with http, fastcgi does not save much for data transmission.

The unique advantage of fastcgi is that it is a persistent connection, with 1000 concurrent requests. fastcgi may forward 10 links to the backend appplication. If http is used, if the number of requests is less than the limit, 1000 requests will be sent to the backend appplication.

The http proxy forwarding method may cause problems in the face of ultra-high concurrency, because the port in the tcp protocol stack is an int16 integer. You need to create a local connect and consume a port, it can be up to 65536. The port pool consumes up to 100,000 external concurrent requests, and your server can only refuse to respond.

Summary

My personal habit is to use fastcgi protocol to deploy python programs, which is simple and easy to use. To select a technical solution, you must select the simplest and most common one. The fastcgi running script of this blog is as follows:

kill - `cat / tmp / django.pid`echo 'restart django....' python . / manage.py runfcgi - - settings = lutaf.settings_r maxchildren =  maxspare = minspare =  method = prefork pidfile = / tmp / django.pid host = 127.0 . 0.1  port = outlog = / tmp / dj.out errlog = / tmp / dj.error

We recommend you try Gunicorn, which is the future development direction.