First, large-scale website Architecture Evolution 1, large-scale website features
- High concurrency, large flow
- Highly Available
- Massive data
- Wide range of users and complex network conditions
- Poor safety environment
- Rapid change of demand, frequent release
- Progressive development
2, large-scale website architecture development history
- File server, database server, Application server detach
- Application server increases local cache, local cache takes precedence, increases distributed cache server
- Use an application server cluster to increase the concurrency of a web site and distribute it uniformly by a load balancer server
- Using Distributed file systems and distributed database systems
- Using CDN Network acceleration and reverse proxy servers: The basic principle is cache, CDN deployed in the network provider's room, is the user when requesting the network service, can be from the nearest network to provide the opportunity room to obtain data. And the reverse proxy is deployed in the central room, when the user's request to the central room, the first to access the reverse proxy server, if the reverse proxy Server cache the resources requested by the user, it will be directly returned to the user, the purpose is to return data to the user as soon as possible, speed up user access, reduce the pressure of the backend server.
- Using NoSQL and search servers
- Business split, according to different products, different business, a site split into different applications, each application independent deployment maintenance, each application through the message middleware or access to the same data storage system to form an interconnected complete system
- Distributed services that extract common business, deploy independently, and provide reusable shared business services for other application servers (can be implemented through Dubbo and zookeeper clusters)
3, the site structure of values and design mistakes
- The choice of architecture is as flexible as the web site needs to respond
- The main driving force is the business development of the website
- Do not blindly follow the structure of the large company design, copy copy
- Website technology is for the business existence, not for technical and technical
- Do not attempt to use technology to solve all problems. Technology is used to solve business problems, and the same business problems can be implemented by business means, by modifying the business structure to solve
Second, large-scale Web site architecture model
1, layered, the system in the horizontal latitude divided into several parts, each part responsible for a single responsibility, through the upper layer of the bottom of the dependency and call to form a complete system. The network of 7 Layer communication protocol, computer hardware, operating system, application software, have adopted the idea of layering. Divides the website system into the application layer, the service layer, the data layer, the three layer structure deploys separately on the different server
- Application layer: Responsible for specific business and view presentation
- Service layer: Provide service support for application layer, such as user Management Service, Shopping cart service
- Data tier: Provides data storage access services such as databases, caches, files, search engines, etc.
2, split. In the longitudinal aspect of the System division, such as the application layer, can be divided into different services, shopping, forums, search, advertising segmentation into different applications, by the independent team responsible. Deployed on different servers.
3, distributed. The purpose of layering and partitioning is to facilitate distributed deployment of the segmented modules and to deploy different modules on different servers. At the same time, the distribution also brings a lot of problems,
- Communication between different servers is heavily dependent on the network
- Maintaining data consistency is difficult
- Web site relies on complexity, development and maintenance difficulties
Common distributed scenarios include the following
- Distributed applications and services.
- Distributed static resources
- Distributed Data and storage
- Distributed computing
- Distributed File System
- Distributed locks
- Distributed configuration. Support website on-line service configuration Live Update
4, cluster. While distributed applications have been deployed separately, for modules that are centrally accessed by users, there is a need for a separate deployment server cluster, in which multiple servers deploy the same application to form a cluster that is shared externally by load balancing devices. It also supports linear scaling and failure transfer in the event of a failure. Even a small number of distributed applications and services will deploy at least two small clusters to improve the availability of the system
5, Cache. Caching is the first way to improve software performance. Large Web sites use a cache design in many ways:
- CDN, or content distribution network. Here cache the site's static resources (less variable data), can be nearest to the fastest return to the user. such as video sites and portals will cache users ' access to hot content in CDN
- Reverse proxy. When the user's request arrives at the datacenter, the first access is the reverse proxy server, where the static resources of the site are cached, without the need to continue to forward the request to the application server, directly back to the user
- Local cache. The hotspot data is cached locally on the application server, and the application can be accessed directly in local memory without having to access the data tier
- Distributed cache. The data is stored in a dedicated distributed cache cluster, and the application server accesses the cached data over network traffic.
The cache also has the problem of cache breakdown, cache avalanche, and cache hotspot not concentrating.
The core principles and case studies of large Web site technology Architecture Reading notes
