Large Web site Technology

Large web site development

Website architecture
Cache and Data consistency
Distributed transactions
Load balancing and high availability
Micro-Service
Message Queuing
Seconds Kill System

Large website Features

High volumes of data and high-availability requirements

Estimation of capacity

Common capacity Estimation: Data volume concurrency bandwidth cpu| mem| DISK

Capacity Assessment steps
1. Assess Total traffic
2. Assessment of average traffic QPS
3. Assess Peak QPS
4. Evaluate the system single-machine limit
5. Calculate capacity

Common Performance test scenarios

Ab
Jmeter
LoadRunner

System load

System Load: The measure of the CPU's busy level how many processes are waiting to be called by the CPU (the length of the process waiting queue)
Load Average: Average system load over time 1,5,15 minutes
Top Uptime W

Definition of Load

Load < 1 single core
Load = 1 Single Core
Load > 1 Single core
Load = Dual Core

Application Server and database separation

How to choose (WebApp and database-to-machine requirements) configuration
Application server needs to handle a lot of logic requires a powerful CPU
A file server needs to store a large number of users to upload files that require a larger hard disk
Database servers require faster disk retrieval and data caching requiring greater memory and hard disk

Application Server Cluster

Assuming that the database server has no pressure to change the application server from one to two or more
Distribute user requests to different servers to increase load capacity

Who forwarded the user's request to a specific application server
What are the algorithms for load-balanced forwarding?
How to maintain session consistency if the user is not the same server each time they access it

Load Balancing

Load Balance is a distributed architecture consideration
Evenly distributed request/data across multiple operating units

Load Balancing Architecture

Client layer reverse proxy Layer Nginx site Layer service layer data layer
Each downstream has multiple upstream calls each upstream evenly accessible downstream is load balanced

How load Balancing Works

HTTP redirection
When an HTTP proxy (such as a browser) requests a URL from the Web server, the Web server can return a new URL through the location tag in the HTTP response header information. This means that the HTTP proxy needs to continue requesting this new URL to complete the automatic jump.
Cons: Throughput throttling
Pros: No additional support required
Scenario: We need to weigh the cost of the transfer request and the cost of handling the actual request, the smaller the latter, the greater the significance of the redirect, such as downloading. You can go to a lot of mirror download site to try, you will find that the basic download has been used to redirect the location.

DNS Load Balancing
DNS is responsible for providing the domain name resolution service, when visiting a site, in fact, the first need through the site domain name of the DNS server to obtain the IP address that the domain name points to, in this process, the DNS server completed the domain name to IP address mapping, the same, so that the mapping can also be one-to-many, this time, The DNS server acts as a load Balancer scheduler.
Use the command: Dig, CN to view DNS configuration
The DNS server can look for a server that is closest to the memory in all available A records.
Cons: DNS record cache update is not timely, policy limitations, do not do health checks
Pros: You can find the nearest server and speed up requests.
Application scenario: Generally we can use when the multi-engine room is deployed.

Reverse Proxy load Balancing
When the user's request arrives at the reverse proxy server (which has arrived at the Web site), the reverse proxy server is forwarded to the specific server based on the algorithm. Common Apache,nginx can act as a reverse proxy server. The reverse proxy Scheduler plays the role of the user and the actual server middleman.
Working at the HTTP layer (layer seven)
Cons: Proxy servers become a bottleneck for performance, especially when uploading large files at once.
Advantages: Simple configuration, rich policy, maintain user session, can be forwarded according to access path.
Application scenario: The request is not high, simple load balancing. Applications with large back-end overhead.

IP load Balancing
Working in the Transport layer (four layers)
Through the operating system black changes sent to the IP packet, the destination address of the packet is modified to the internal actual server address, so that the request forwarding, to achieve load balancing. The NAT mode of the LVS.
Cons: All data in or out of the load machine, network bandwidth becomes a bottleneck.
Advantages: The kernel completes the forwarding, high performance.
Application scenario: High performance requirements, but not high bandwidth requirements. Large bandwidth applications such as video and download are not suitable for use.

Load Balancing on the data link layer
Work on the Data Link layer (second tier)
After the request arrives at the load balancer, it can be forwarded by configuring the same virtual IP and load balancer for all cluster machines, and then by modifying the requested MAC address. Unlike IP load balancing, when the server is finished requesting access, it is returned directly to the customer. Without having to go through the load balancer again. LVS DR (Direct Routing) mode.
Cons: Complex configuration
Advantage: The direct return from the cluster machine improves the export bandwidth.
Application scenario: A load balancing method that is most widely used by large web sites.

Common policies for load balancing

Round robin (Round Robin)
Servers with weaker capabilities lead to the first overloading of weaker servers.

Weighted round robin (Weighted Round Robin)
This algorithm solves the disadvantage of the simple round robin scheduling algorithm: Incoming requests are assigned to servers in the cluster sequentially, but the weights assigned to each server are considered in advance.

Minimum number of connections (Least Connection)
The minimum number of connections algorithm is flexible and intelligent, because the configuration of the backend server is not the same, the processing of the request is fast and slow, it is based on the current connection situation of the back-end server, dynamically select one of the current backlog of the minimum number of servers to handle the current request, as far as possible to improve the efficiency of the backend services will be responsible for a reasonable diversion to each server.

Weighted minimum connection (Weighted Least Connection)
If the server has different resource capacities, then the "weighted least connections" approach is more appropriate: Consider the number of connections while also weighing the administrator's customized weights based on server conditions.

Source IP hashes (source IP hash)
This is done by generating a hash of the requested source IP and using the hash value to find the correct real server. This means that the same server is always the same for the same host.

Random
The random algorithm of the system, according to the list size value of the back-end server to randomly select one of the servers to access. The actual effect is close to the polling result.

Common load Balancing scenarios

Hardware
Common hardware has more expensive NetScaler, F5, Radware and array and other commercial load balancer, the advantage is that there is a professional maintenance team maintenance, high performance. The disadvantage is that it is expensive, so there is no need for a smaller network service for the time being.

Software
Nginx (1w), HAProxy (5-10w), LVS (more than 100,000) F5 big-IP (million)

How load balancing maintains session sessions

1. Assign the same user's requests in one session to a fixed server. Common load Balancing algorithm Ip_hash
2.session data Centralized storage session data storage is the use of database or cache to store session data, the implementation of the session and application server decoupling
3. Use cookies instead of session.

Database read/write separation

One master multi-slave, read-write separation, redundant read-more library

However, with the increase of traffic, the load of the database is increasing slowly. Most of the internet business is "read more write less" scenario, database level, read performance is often a bottleneck. For example, the industry usually adopts the database schema of "one master, multiple slave, read and write separation, redundant reading library" to improve the reading performance of the database.

MySQL Replication principle

1.master record changes to binary log (these are called binary logs events, binary log event)
2.slave Copy the binary log events of master to its trunk logs (relay log)
3.slave Redo the event in the trunk log, which will change the data reflecting its own.

Replication cannot extend write operations

In the previous example, the reason is that it is not possible to extend a write operation as easily as an extended read operation. Therefore, the replication function of MySQL is only used to extend the read operation, unable to extend the write operation.

The only way to increase the write load is to divide the database into tables

DB master-Slave Architecture Consistency Optimization method

Semi-synchronous replication
The system first writes to the Db-master
Write Master library, etc. master-slave synchronization complete, write the main library request to return
Read from the library, read the latest data (if the read request is completed first, write the request to complete, read "then" the latest

Write script to force read the main library, you can use the database agent to implement
Reads the subsequent read operations from the unified script from the main library. You can configure automatic database proxy implementation. such as Db_proxy, etc.

Cache Record Write Key method
Write a key on a library, record it in the cache, and set the cache timeout for experience master and slave sync time, such as 500ms
Modify Database
Read the library to the cache first to see, corresponding to the library key has no relevant data if the cache hit, there is relevant data, indicating that the key has just occurred on the write operation, the need to route the request to the main library to read the latest data
If the cache miss, stating that there has not been a write operation on this key recently, the request is routed to the slave library to continue the read and write separation

Use search engines and caches to ease the stress of reading libraries

Database to do reading library, often to fuzzy search, even if the separation of reading and writing, the problem has not been solved.
Take our trading website as an example, often based on the title of the product to find the corresponding product. For this kind of demand, we are usually implemented by the like function, but the cost of this method is very great. At this point we can use the search engine inverted index to complete.
Caching is a common technique for improving the read performance of a system, and we often use caching to optimize for applications where there are few read-write scenarios.

Popular search Engines

Lucene is more like an SDK. There is a complete API family and the corresponding implementation. You can use these to implement advanced queries in your own applications (based on inverted index technology). is a set of information retrieval toolkits, but does not include search engine systems, so you still need to focus on things like data acquisition, parsing, and word segmentation when using Lucene.
SOLR is based on Lucene. It's closer to what we recognize as a search engine system, a search engine service that allows your app to use search services with a variety of APIs, without the need to couple search logic into the app.
Elasticsearch is also based on Lucene. But it's more performance than SOLR in real-time search, and it's a little simpler to use. At present, more and more users, recommended to use.
Sphinx is an SQL-based full-text search engine that can be combined with mysql,postgresql for full-text searching, providing more specialized search capabilities than the database itself, making it easier for applications to achieve specialized full-text indexing

Cache-to-database consistency update cache VS retire Cache

The cost of updating the cache is minimal, and we should be more inclined to update the cache to ensure a higher cache hit ratio
If the balance is calculated from very complex data, it tends to delete the cache.
The elimination of cache operation is simple, and the side effects are only increased once the cache miss, recommended as a common way of processing.

Operate the database first vs cache First

When a write operation occurs, it is assumed that the elimination of the cache as a common way of handling the cache faces two choices:
Write the database first, and then delete the cache.
Delete the cache first, and then write the database.
For an operation that is not guaranteed to be transactional, it must involve the question "which task to do first, which task to do after".
The direction to solve this problem is: if there is inconsistency, who first do the business impact less, who first execute.

Write the database first, and then retire the cache:
The first step to write the database operation succeeds, the second step to eliminate the cache fails, the DB is the new data, the cache is the old data, the data inconsistent.

First retire the cache, then write the database
The first step is to retire the cache successfully, the second write database fails, and only one cache miss is raised.

A consistent hash algorithm for caching

Split

Vertical split: Splits the contents of the original table into multiple tables by function or business, or a library is split into multiple libraries.

Horizontal split: Stores data of the same type in multiple tables with the same structure.
A good table key is usually the primary key for a very important core entity in the database. such as the user ID.
user_id% 5 = 0
user_id% 5 = 1

Issues with cross-Library joins

Before splitting, the data required for many lists and detail pages in the system can be done through SQL join. After splitting, the database may be distributed on different instances and on different hosts, and the join becomes cumbersome.

1. Global Tables
Some of the tables that are likely to be dependent on all modules in the system are saved in each library.

2. Field Redundancy
"Seller ID" is saved in the "order Form", and the seller's "Name" field is also redundant, so there is no need to query the "Seller user table" When querying the order details.

3. Data synchronization
The Tbl_a table in a library and the Tbl_b association in the B Library can be timed to synchronize the specified table.

If the join is complex, you first need to consider whether the design of the library is reasonable, and then consider using Hadoop or spark to solve it.

Vertical teardown, cross-Library transaction issues

CAP: Consistency, availability, and partitioning fault tolerance
Implementation of distributed transactions: two phase commit
The two-phase commit protocol (two-phase commit,2pc) is often used to implement distributed transactions.

Generally divided into the Transaction Coordinator TC and resource management si two roles, where the resource management is a specific database.

1. The application initiates a start request to the TC;
2. Launch to all Si

How to guarantee the high availability of the system

1. Single point is often the largest system of high availability of the risk and enemy, should try to avoid a single point in the system design process.
2. The principle of high-availability assurance is "redundancy". Only a single point, hang up the service will be affected, if there is redundant backup, hang up and other backup can be overhead.
3. The core guidelines for highly available architecture design are: redundancy.
4. With redundancy, it is not enough, each failure requires manual intervention to restore the system will inevitably increase the non-service practice. Therefore, it is often through "automatic failover" to achieve high availability of the system.

High-availability and load-balanced software solutions

Both LVS and Haproxy are implemented as load balancing functions.
Both keepalived and heartbeat are highly available and avoid single points of failure.
Keepalived and Heartbeat simple differences:
Keepalived configuration is simple, heartbeat powerful configuration rich.
Both LVS and keepalived work by detecting VRRP packets, so the keepalived is better suited to the LVS.
Heartbeat feature-rich for business use.
Choose heartbeat in a business that keepalived cannot achieve.
The usual collocation methods in daily work are:
lvs+ keepalived and Haproxy+heartbeat

Message Queuing

What are the common application scenarios for Message Queuing?
The main thing is: application of decoupling, asynchronous operation, traffic cutting front.

Asynchronous operation

After writing the registration information to the database, send the registration message and then send the registration text message. When all of the above three tasks are complete, return to the client.

The user's response time is equivalent to the time the registration information was written to the database, which is 50 milliseconds. Registering messages, sending text messages to the message queue, returning directly, so the message queue is written quickly and can be ignored, so the user's response time may be 50 milliseconds

Application decoupling

After the user orders, the order system needs to notify the inventory system. The traditional approach is that the order system calls the interface of the inventory system

After decoupling, if the inventory system is not used properly when the order is being given. Also does not affect the normal order, because after placing orders, the order system to write to the message queue will no longer care about other follow-up operations

Traffic reduction

Traffic clipping is also a common scenario in Message Queuing, and is generally used extensively in the second or group robbery activities.

The second kill activity, generally because the traffic is too large, causing the traffic to burst, the application hangs off. In order to solve this problem, it is generally necessary to join the message queue in the application front.
The number of people who can control the activity;
Can relieve high-flow crushing applications in short time

Common Message Queuing software

RabbitMQ
An open source message queue, written in Erlang, itself supports a lot of protocols: Amqp,xmpp, SMTP, STOMP, and because of this, it's very heavyweight and better suited for enterprise-class development.

Redis
Redis is a NoSQL database based on Key-value, which is very active in development and maintenance. Although it is a Key-value database storage system, it natively supports MQ functionality, so it can be used as a lightweight queue service.

ZeroMQ
Known as the fastest Message Queuing system, especially for large-throughput demand scenarios. ZMQ can implement advanced/complex queues that rabbitmq are not good at, but developers need to assemble multiple technical frameworks themselves, and technical complexity is a challenge to the success of this MQ application. ZEROMQ only provides a non-persistent queue, which means that if the down machine, the data will be lost. Of these, Twitter uses zeromq as the stream of data by default in storm.

ActiveMQ
Similar to ZEROMQ, it is able to implement queues with agent and peer-to-peer technology. It is also similar to RABBITMQ, where a small amount of code makes it possible to implement high-level scenarios efficiently.

Kafka/jafka
is a high performance cross-language distributed Publish/subscribe Message Queuing system.
Fast persistence for message persistence under O (1) system overhead;
High throughput, on a common server can achieve the 10w/s throughput rate;
Fully distributed system, Broker, Producer, consumer are native automatically support distributed,
Automatic implementation of complex equalization, support Hadoop data parallel loading, for the same as Hadoop, the same log data and offline analysis system, but also require real-time processing constraints, this is a feasible solution.

Cache

Caching is an important component of a large web site, addressing the performance issues of hot data access in high concurrency and big data scenarios. For scenarios where read and write less.
From our use point of view, mainly include:
CDN Cache
Reverse Proxy Cache
Distributed cache

CDN Cache

CDN primarily addresses caching of data to the nearest user location, generally caching static resource files (pages, scripts, pictures, videos, files, etc.).
The basic idea is to avoid the bottleneck and link of the Internet which may affect the speed and stability of data transmission, and make the content transmission faster and more stable.

Reverse Proxy Cache

The reverse proxy is located in the application server room and handles all requests to the Web server.
Reverse proxies generally cache static resources, and dynamic resources are forwarded to the application server for processing. The common cache application server has varnish,ngnix,squid.

Proxy Cache Comparison
Varnish and squid are professional cache services,
Nginx needs third-party module support;
Varnish uses the memory cache, avoids frequently exchanging files in memory, disk, performance is higher than squid;
Varnish because it is memory cache, so small files such as css,js, small picture What support is great, the back end of the persistent cache can be used squid or ATS;
Squid function full and large, suitable for a variety of static file cache, usually hang a haproxy or nginx in front of the load balancer run multiple instances;
Nginx adopts the third-party module Ncache to do the buffering, the performance basically achieves the varnish, generally uses as the reverse proxy, may realize the simple cache.

Distributed cache

Distributed cache, mainly refers to cache users frequently access the data cache, the data source for the database. Generally play a role in hot data access and mitigating database stress

Distributed cache

Redis
Redis is an open source (BSD-licensed), memory-based, multi-data structure storage system. Can be used as database, cache, and message middleware. Supports multiple types of data structures such as strings (strings), hashes (hashes), lists (lists), collections (sets), ordered collections (sorted sets) and range queries, bitmaps, hyperloglogs and geospatial (Geospa tial) index RADIUS query.

Cluster scenarios:
Twemproxy
Codis
The official recently launched Redis Cluster

Comparison of Memcache and Redis
1 data structure: Memcache only supports key value storage, Redis supports more data types, such as key value,hash,list,set,zset;
2 Multi-Threading: Memcache support Multithreading, Redis support single-threaded, CPU utilization memcache better than redis;
3 Persistence: Memcache does not support persistence, Redis supports persistence;
4 Memory Utilization: Memcache High, Redis low (using compression in case of higher than memcache);
5 Expiration policy: memcache after expiration, do not delete the cache, will cause the next time to fetch data data problems, Redis has a dedicated thread, clear cache data;

Seconds Kill System

Features: Only one copy of the inventory, everyone will read and write the data in a concentrated time. But few people succeed in the end.

Idea: try to intercept the request as much as possible upstream of the system.

1 Browser Layer request interception
JS level, limiting the user to submit only one request within X seconds.
2 site layer (webServer) request blocking and page caching
The same UID, limit the frequency of access, do page caching, in x seconds to reach the site layer of the request, all return to the same page.
3 Service Layer (SOA) request interception and data caching
For the write request, make the request queue, each time only through a limited write request to the data layer, if both successfully put down a batch, if the inventory is not enough then the queue of write requests all returned "sold out"
For read requests, the cache to resist, whether it is memcached or Redis, stand-alone 10w per second should be no problem.
4 The data layer processes the real order request

Large Web site Technology