"Large Web site technology architecture" notes

Recently, the "large Web site Technology architecture" has been looked at again. In the middle of reading a "computer operating system" textbook, feel the technical structure of large sites have a deeper understanding. Take some notes on the understanding of these two books

The traditional OS (Operator System) has four basic functions:

1. Processor management: The process as the basic unit, the creation and revocation of its

A) Process Control

b) Process synchronization

c) Process Communication

d) Scheduling

1. Memory management

A) memory allocation

b) Memory protection

c) Address mapping

d) Memory expansion

1. Device Management: Complete I/O requests. Increase the utilization of CPU and I/O devices and increase I/O speed. User-friendly I/O devices

A) buffer management

1. I. If a buffer is introduced between I/O and CPU, it can effectively mitigate the contradiction between the two speed mismatches, increase the throughput of the CPU, and improve the throughput of the system.

b) Equipment allocation

c) Device handling: Device processor is also a device driver

1. File Management

A) management of file storage space

b) Directory Management

c) Read/write management and protection of files

The evolutionary process of architecture

First, the original site directly with lamp (linux+apache+mysql+php) deployed on a single server

This can result in:

1. All user requests are processed by this server's multithreading
2. All of the memory is on this server (the memory gets faster than the disk read n times.) It is best to put the data that you often need to read in memory.
3. Processing of requests (processing of data: Data in MySQL is accessed via disk, which must involve I/O processing)
4. All files uploaded by the user are on this server

Sum up:

1. This server must have enough CPU processors to handle user requests from colleagues
2. Must have large enough memory storage space
3. Disk access must be fast enough (for example, solid-state drives)
4. Hard disk must be large enough to store files

In addition, there are some other problems. such as network broadband, etc.

such as my blog server upstream 100k/s, 10 people no one 10k,2m picture will have 20 seconds, if 100 people.

In summary, separate deployment of a single server is required:

1. Request Server (multi-processor. Dedicated to handling user requests)
2. Cache server (memory is large enough.) to store frequently accessed data. If using Redis)
3. Database server (with solid state drive)
4. File server (large enough hard disk space)

By deploying separately. Each server has its own strengths, greatly improving the efficiency

and measuring the superiority of a website often takes into account five points

1. Performance

A) When loading the page, deploy the static file to the CDN third-Party vendor server. To reduce requests to the server for resources.

b) Use the cache. The hit resource returns immediately, reducing the database read operation (the disk reads data slower than the cache n times)

c) distributed. Separate servers for different services (avoid centralization of requests)

d) a distributed cluster. Distribute requests to a cluster server by load Balancing algorithm (typically with Nginx reverse proxy server cluster)

e) Distributed cache. Let all the caches focus on these servers dedicated to caching (such as Redis)

f) separate the reading and writing of the database. (later business bigger and the possibility of sub-library)

g) Use Message Queuing (that is, asynchronous) to add minor events to the message queue without waiting for the returned results to respond immediately to the user. The most typical example is the 12306 order email reminder, which is usually received after a certain period of time. This is to add the mailbox alert function to the message queue

1. Availability of

A) for the user, the site is available for an uninterrupted hour. This is the ideal state, usually 4 9 (99.99%) almost.

b) Any server outage will not cause the site to be unavailable. And the cluster server has the ability to backup each other, any one server downtime will not affect the overall availability of the application or cause data loss

1. Extensibility

A) When the function is added only to increase the open interface, without affecting the original function, to achieve a loose coupling effect

b) Expansion of value-added business. It will not affect other business

c) The main means of Web site shrinkage is event-driven architecture and distributed services

1. Scalability

A) The transition can be smooth when adding or reducing servers to the cluster. The user's request can still be high hit.

Therefore, the cache routing algorithm is very important (in general, it will use the Nginx load balancer.) The polling algorithm is used.

1. Security

A) ability to defend against XSS attacks

b) How the script is injected

c) Send invalid source address TCP packet

"Large Web site technology architecture" notes