Python Web world View--web Architecture Overview (for traditional programmers)

The problems faced by traditional Web server

We know that the traditional Web server, a process that opens the socket, listens, comes in response to a request to generate a new process (or thread, or block), and itself continues to listen. This is the network model that most of the people who have seen UNIX network programming are exposed to. However, Unix is too old, the network demand has undergone great changes in recent years, the most important thing is the requirement of concurrency.

The increased requirements for concurrency have also changed the architecture of the native Web server, and there have been different requirements for this word. Because server requests may not be handled by a single machine, one problem that needs to be addressed is the collaborative synchronization problem of multiple server machines.

Requirements for modern Web server

The first is the change of the machine. There are two changes in this machine: increased concurrency and the need for rapid development. Our examples are in Python as an example. Because of the rapid development of the requirements of the iteration, C + + and so on, although the performance is good, but the change is slow, so most of the network system back end is done with Python, wait until stable and then cut into high-performance C + + or C or go backstage. Unfortunately, the Internet has not been very stable ...

Single machine improves concurrency

The traditional Web server model says that if millions of accesses are to be made, would it not be a process that will generate billions of processes? (Linux thread is also process, take up PID), and process management really consumes system resources, which leads to: No matter how powerful the hardware, it will sooner or later be exhausted by concurrent losses. In order to solve this problem, the concept of fiber (co-process) is applied. The process itself is the scheduling entity provided by the operating system for the user, and the specific schedule is completed by the operating system. And the process is the application itself in a single process implementation of the pseudo-processes, multiple threads is the execution path of multiple code, the scheduling of the process by the application itself (the current general practice is that the various processes themselves, such as the use of the yield keyword). Thus, in the process of an operating system to simulate a number of independent parallel execution units, and communication between the process is particularly simple (because it is a process, the process appears to be equivalent to the process of internal function bounce and call). This mechanism can greatly solve the concurrency requirement of single machine.

The Python language provides the keyword for implementing the process: yield. But this is not the complete implementation of the process, however the benefit of Pytho is that the lack of what can be supplemented by the module implementation. Using the yield keyword, Python implemented two more well-known co-module Greenlet and Stackless, two the main difference is that stackless is automatic scheduling between the co-scheduling, and greenlet need to manually give up the implementation of the process of execution rights. Each has its advantages, but it is clear that the simple design of greenlet in fine-grained control coincides with the python yield keyword, giving the user a higher level of entitlement (but also requiring more code).

There are others in the greenlet above the encapsulation implemented gevent. is a network library based on the co-process. Because it is based on the co-process, the biggest feature of this network library is its high concurrency. What is a network library? It is nothing more than encapsulating the socket usage and process threading model. For example, you can dynamically open multiple processes (threads), each running greenlet, or directly to a thread pool. Greenlet defines the method of co-process and switching, but there is no provision for how to switch, under the socket, it is easy to block, Gevent is scheduled to switch the situation. In other words, Greenlet is the method used in the process, and gevent implements the strategy.

Speed up Web development iterationsAs mentioned above, in order to develop speed, the industry has gradually switched to python, everyone expects to switch back to a more efficient language someday, but the schedule will never give you a chance, so you will find that many of the current large sites are directly using Python to work.But the Python language itself can actually improve the speed of development. However, in network applications, Python improves development speed by standardizing network development. The framework he proposes is called WSGI, a Web framework definition standard that defines the application side, the service side, and the middleware interface. Most implementations of this standard are implemented middleware, which is followed by the service side. We understand that Web server is not just an open socket listener, and then processing it? Why should there be a layer of middleware? Because, opening sockets, parsing HTTP packets (HTTPS), maintaining and tracking sessions, cookies, and so on are all common to all HTTP servers, and Python's Wsgi standard is to separate these common operations and define interfaces. All back-end servers are programmed directly using the interface of the WSGI definition (middleware burst), which makes the server programming and the server program itself very simple.
There are many Python libraries that implement this WSGI standard, such as Gunicorn. As defined by this standard, there are also many Python libraries implemented: Bottle, Django, flask, Tornado, and so on. Therefore, according to WSGI logic, generally a machine to start gunicorn as a middleware at the same time, a server (such as bottle) as the actual back-end, the backend of the user program development is to use the interface provided by bottle these libraries to implement. What kind of programming experience is this?

1. @route('/helloworld/:yourwords ', methods=[' GET ', ' POST '])                     #url接口, note the parameter writing format, preceded by a colon that is a parameter  
3. def
5. return  ' Hello world. ' + yourwords

Just a few lines of code define what kind of logical code to execute and return (not all) if it is a URL. Is it very refreshing compared to traditional web backend CGI programming? Completely clean only deal with the business. This is the WSGI standard brought to us.The Gunicorn here can be started independently, but the parallelism is not good, so we can use the above-mentioned gevent highly parallel network library. For these two, Gevent is the method, Gunicorn is the strategy. We can see that through this layer of encapsulation and invocation, each component can choose different implementations under different scenarios to achieve different purposes. This flexibility, like building blocks, is not what traditional web development does.
Concurrency across machines: the Cloud BackendIncreasing concurrency on one machine, providing and implementing standards, while significantly improving the combat capability of a single unit, is not enough for modern network requirements. This requires more than one machine to provide services to the outside. With more than one machine, there is a traditional cloud requirement: when and what is the request to choose which PC to serve? There are load balancing, network proxy, availability, consistency and other implementation requirements.Many of the existing software can meet a variety of services, such as Nginx can be in the network agent can also do load balancing. And do load balancing also have F5, LVS and other special software or hardware. KeepAlive software is responsible for usability, such as RAID and many other scenarios that can provide consistency. These are logically separate, but there is likely to be an inclusive software implementation. The software that is popular at the moment may be out of date at some point. But this demand is not going to change. This also makes up the modern Web server's backend as standard as a firewall.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Python Web world View--web Architecture Overview (for traditional programmers)