Baidu Post Bar 1 billion-magnitude LAMP architecture sharing

As a functional product, sticking to the review is an eternal principle. This feature determines that fast iteration is a key issue to be solved. Rapid iteration is divided into the following parts: development stage, rapid development; testing stage, including quick environment setup, automated testing tools; O & M stage, it includes cluster management technology and automated O & M tools. At the same time, these three aspects of work need a comprehensive solution to link up.

Early posts, as a high-performance community, all features are relatively simple and are developed in C language. The system is reusable, the development and testing efficiency is low, and there is little accumulation in O & M. In order to improve efficiency, we started to try the LAMP architecture. After several years of development, all the posts have been migrated to LAMP. With the rapid expansion of the product scale, over 30 subsystems, more than 150 modules, more than 500 machines, and more than 1 billion traffic, we have accumulated a lot of experience in the LAMP architecture and gradually formed an integrated solution for rapid iteration. As shown in:

The solution consists of the development, testing, and O & M phases. The development phase is further divided into the access layer, business logic layer, and storage layer. This solution supports large-scale online applications and maintains the fast iteration feature. Based on this solution, developers can focus on business logic development, testers can focus on continuous integration, and O & M costs can be greatly reduced.

Development

The development includes the access layer, business logic layer, and storage layer.

The access layer is located between the browser and the backend service. It is used to parse the http protocol and organize it into the corresponding protocol format to complete communication between the client and the server, it also includes attack prevention, page cache, Server Load balancer, and other functions. Web server is its core component. The goal of the access layer is to provide a simple and reliable access layer architecture through a unified solution. After comprehensive research, nginx features versatility, high efficiency, comprehensive functions, and flexible configuration, it is the main force in the future development of webserver. It is determined that the nginx unified access layer is used.

The business logic layer includes the PHP framework, business logic, LIB library, and interaction layer. The business logic layer often contains some development specifications. These specifications are just like laws. We must follow laws and regulations. In our solution, the PHP framework = specification + Library, such as Directory deployment specification, URL specification, and configuration specification. These specifications are implemented through the corresponding library, in order to achieve the goal of law-abiding. The LIB library encapsulates common functions. Based on this solution, developers only need to complete the business logic part to develop applications.

The middle layer, as shown in, is included in the business logic layer and is very important for fast iteration of the business logic layer. The middle layer performs interaction abstraction on the lower layer, and supports various protocols to shield protocol details. The deployment details are blocked by resource positioning. The Server Load balancer improves system stability. The middle layer abstracts interfaces to support service integration, interface adaptation, and public logic. The middle layer first establishes the system-subsystem-Module system, carries on the service conformity, in the figure API-LIB is according to the subsystem division, each Module Interface (MIDL: Module IDL) converts it to a sub-system interface (SIDL: Service IDL). The interface adaptation means that the SERVICE interface is described through SIDL, so that the interface description, interface documentation, and online code are automatically synchronized, greatly improving maintainability, at the same time, metadata specifications are used to ensure the consistency of interfaces throughout the system, greatly improving the ease of use; convergence of Public logic, for public logic, such as Permission Logic, greatly improving the maintainability of convergence.

The storage layer provides a variety of common services and components. The general data storage framework provides a general data storage and access solution. It supports the design and implementation of most data storage modules in a unified design mode. It provides a unified data access interface, shield external data splitting and storage details; achieve good scalability of data storage, and cope with data growth through a common data splitting mode; abstract Common requirements into common services or libraries to simplify design and development.

Based on this solution, to develop an application, you only need to configure the corresponding traffic distribution at the access layer, develop the business logic at the business logic layer, and use the appropriate services at the storage layer or complete the development of the data module based on the framework. It can greatly improve development efficiency and support fast iteration.

Test

In terms of testing, in order to support rapid iteration, it is necessary to increase the degree of automation. The primary factor affecting automation is automatic environment building. Common problems include complex environments, such as complex association relationships, high environmental construction costs, and incomplete environment functions. The benchmark environment can be used to solve this problem. After the project is launched, it is automatically updated from scmpf to the benchmark environment. The testing and development environments are synchronized from the benchmark environment. Based on the benchmark environment, system-level continuous integration is also possible, and a large number of test tools can be concentrated.

O & M

O & M faces many problems: high service migration costs, various rollbacks due to inconsistent environments, unbalanced machine utilization, and low O & M automation. To solve these problems, a PHP system O & M solution is proposed. In terms of environment synchronization, the main issue is Code Synchronization. The O & M specification + monitoring solution is adopted. In terms of performance monitoring, the Request status and Interaction performance can be monitored based on the interaction layer, obtains the machine status based on the scheduling center. In terms of machine scheduling, the scheduling center completes dynamic/semi-automatic machine scheduling. As shown in:

Outlook

The LAMP solution greatly improves efficiency in development, testing, and O & M. In the future, LAMP architecture will require more efforts in standardization and platformization. This automation can be carried out only after the specifications are standardized to improve efficiency. The platform can solidify various specifications and provide automated support.