Summary of "billion-level traffic website Architecture Core Technology"

Nginx Back-end node Health Check

There are three main ways to achieve this:
1. The Ngx_http_proxy_module module and the Ngx_http_upstream_module module, which are self-bringing, are inert detection.
Ngx_http_proxy_module:proxy_connect_timeout (timeout for connection to backend server), Proxy_read_timeout (timeout for reading responses from back-end servers), and Proxy_next_ Upstream (in which case a failed request should be sent to the next backend server)
Ngx_http_upstream_module:max_fails (number of failed attempts), Fail_timeout (how long to fail, and no more requests within this time).
2. Nginx_upstream_check_module Module (Taobao development module), belongs to the active detection. The health state of the background server can be detected at intervals and the status of the server is flagged. It also provides a path to view the health status of each server in real time.
3. Direct use of Taobao Open source Tengine, with the active detection of the health status of the module, active detection.the process of Tomcat processing requestsThe connector port monitor hears the request and resolves the request based on the HTTP protocol. The parsed HTTP message is transformed into the HttpServletRequest and httpservletreponse corresponding to the Servlet API by the adornment mode. Through Layer container engine, host, context finally to the service method of the business servlet we wrote. The Business method service, which processes the related business logic, writes the corresponding response to the response, and returns the Tomat container component. Tomcat the processing thread closes the response stream response and returns the response content to the client. Tomcat the processing thread is freed and then used for processing the next request.asynchronous characteristics of servlet3.0Advantage: A synchronized servlet thread is blocked until the processing completes the return response before it is released, and the asynchronous servlet thread forwards the actual business processing to the worker thread, then releases the servlet thread immediately or puts it back into the pool, so the throughput can be increased relative to the synchronization servlet. But there is no improvement in performance. Steps:
Servletrequest.startasync turns on Asynchrony, gets an instance of Asynccontext. Asynccontext provides a way for the ServletRequest and Servletresponse objects to reference, then releases the servlet thread, but does not close the corresponding response stream, so we can continue processing in the business code. The request is then forwarded to a thread in the business thread pool for processing, and after processing is finished, call Asynccontext. Complete (), write the response back into the referenced asynccontext response stream, and close the response stream to finish the request processing. Note: Asynccontext is not a true asynchronous output, but a synchronous output, but frees the server-side thread to use. When using Asynccontext, they are waiting for the output to be synchronized for the browser, but for the server side, the thread that handles the request does not have the card waiting for it, and the current processing is converted to the thread pool. In SPRINGMVC, the controller returns a callable or Defferredresult, at which time the servlet container thread has been released, SPRINGMVC is to process the actual business by invoking a thread in the pool with the taskexecutor thread pool.Guava Frame

In contrast to the collection tool classes provided by Apache Commons Collection, Goolgle's guava provides a more powerful collection of tool classes, in addition to the set of operations, including files, strings, and other operational tool classes.
1. Create a view of the collection and set the read-only operation.
2. Transform, filter, etc. the data in the collection.
3. Verify the parameters (non-empty, length, etc.).
4. Intersection, difference set, and set of different sets.
5. Read operation of the file.
6. The segmentation, merging and other operations of the string.
7. Multimap advanced operations on the Map collection.
.................Power-equal design of distributed systemIdempotent: Refers to a single and multiple requests for a resource to the system impact is consistent. In an HTTP request, get, put, delete are idempotent, but post is not idempotent. The method of guaranteeing idempotent:
Go to the table, each time the interface is called according to the request address + parameters to generate a unique key, as the ID of the Redo table, and then save. No error indicates success, error indicates duplicate request. State machine idempotent, such as order status of processing, the general existence of a finite state machine, if the state machine is already in the next state, this time a request for a state change, in theory is not able to change. The power of the finite state machine is ensured. token mechanism to prevent pages from repeating submissions. It is generally the case that the form data is submitted to the server before the Token,token is placed in the cache and has an expiration time, then submits the data to the background and verifies that the token is consistent with the past tokens, while removing tokens.protobuf binary transfer formatDefinition: PROTOBUF is the Google Open source binary transfer format. Use: To define a structured message format according to a certain syntax, and then to a command-line tool, the tool automatically generates related classes, and can easily invoke related methods for serialization and deserialization. Performance comparison: The size of the PROTOBUF serialization is JSON one-tenth, XML One-twentieth, Java comes with binary serialization of One-tenth, so the performance is very high. And Facebook's open source, thrift, has a higher performance. Suitable for the scene:
You need to interact with other systems and are sensitive to message size. Small data for the occasion. Big Data doesn't fit.Java out -of-heap memoryThere are several ways that Java stores objects locally: heap memory, out-of-heap memory, and disk. Out-of-heap memory can be set by-xx:maxdirectmemorysize, and defaults to heap memory, if not set. From the Java.nio. Advantages relative to heap memory:
Avoids the garbage collection GC's work. Reduces memory replication at IO, does not require a copy of the heap memory buffer into direct memory before it is written to the socket. Create (Take Netty as an example): The Netty used by the out-of-heap memory is the Directbytebuffer class for Java NiO. The first amount is applied to the bits class, according to a global variable totalcapacity (the total size of the memory outside the heap), determine if there is sufficient quota allocation, the authorization call Sun.misc.Unsafe allocates memory, returns the memory base address. The quota is not sufficient, then the GC is then allocated, if the garbage collected 100ms memory is not enough, then throw an Oom exception. Recycling: External memory is only recycled when full GC is performed, and can be invoked proactively to trigger System.GC, but there is uncertainty. So generally take the initiative to remove the Sun.misc.Cleaner from the Directbytebuffer, and then call its clean () method. Applicable scenario: not suitable for storing very complex objects, generally simple objects or flat structure is more suitable.Cache Avalanche, cache traversal, cache concurrency, cache warming, cache algorithmCache avalanche: Either the data is not loaded into the cache, or the cache is invalidated at the same time, causing all requests to look up the database, causing the database CPU and memory load to be too high or even down. Solution Ideas:
Lock count (that is, limit the number of concurrent, you can use Semphore) or a certain number of queues to avoid a large number of requests concurrent to the database when the cache fails. But this approach reduces throughput. Analyze user behavior and then distribute the failure time evenly. or a random number of 1-5 minutes on the basis of the expiration time. If a cache server is down, consider the backup. Cache penetration: Refers to the user query data, in the database is not, naturally in the cache will not have. This causes the user to query, in the cache can not be found, each time to go to the database query. Solution Ideas:
If the query database is also empty, set a default value directly to the cache, so that the second time in the buffer gets the value, and does not continue to access the database. Set an expiration time or replace the value in the cache when there is a value. You can set some formatting rules for the key and then filter out the non-conforming key before querying. Cache Concurrency: If the site has high concurrent access, if a cache fails, multiple processes can query the db at the same time, setting the cache, and if the concurrency is really large, this can also cause the db to be too stressed and the cache frequently updated. Solution Ideas:
Lock the cache query, if key does not exist, lock, then check the db into the cache, and then unlock the other process if the lock is found to wait, and then after the unlock to return data or into the DB query. Cache warm-up: The goal is to load the data into the cache before the system goes live. Solution Ideas:
If the amount of data is small, it will be loaded directly when the system starts. Write a simple cache warm-up program yourself. Cache algorithm:
FIFO algorithm: First Out, FIFO. Principle: When a data is first entered into the cache, it should be eliminated first. In other words, when the cache is full, the first data to enter the cache should be eliminated. LFU algorithm: Least frequently used, the least frequently used algorithm. LRU algorithm: Least Recently used, the least recently used algorithm. The difference between LRU and LFU. The LFU algorithm selects the least-used data item based on the number of times the data item is used, which is determined by the difference in usage times. LRU is determined by the difference in usage time.the persistence mechanism of RedisRedis mainly provides two persistence mechanisms: Rdb and aof. RDB: Enabled by default, snapshots of in-memory data are taken to disk at a specified time, creating a Dump.rdb file that is then restored to memory at Redis startup. Redis creates fork () a sub-process, writes the data to a temporary file, ends the persistence process, replaces the last snapshot file with the temporary file, and then exits the child process. It is important to note that each snapshot persistence is the full write memory data to disk once, not incremental only synchronous change data, so if the data volume is large, and the write operation is frequent, will inevitably cause a lot of disk I/O operations, will seriously affect performance, and the last persisted data may be lost. AOF: Each write is recorded as a log (the read operation is not logged), only the file is appended, but the file is not overwritten, and Redis starts all the way through the log to complete the recovery of the data. Including Flushdb will also be executed. There are two main ways to trigger this: write-writing, time-per-second write (and data loss). Because aof use append way, so the file will be bigger and larger, in response to this problem, a new rewrite mechanism, that is, when the log file is large to a certain extent, will fork out a new process to traverse the process memory data, each record corresponding to a set statement, write to the temporary file, Then replace it with the old log file (similar to how the RDB works). The default trigger is triggered when the aof file size is one times the size of the last rewrite and the file is larger than 64M. When both modes are turned on, data recovery Redis will prefer aof recovery. In general, simply use an RDB that is turned on by default, because it is easier to make a database backup than Aof,rdb, and restores the dataset much faster. The ability to turn on persistent caching has a certain effect on performance, especially when the set memory is full, and it drops to hundreds of reqs/s. So if it's just for caching, you can turn the persistence off.thread pool Executorservice offWhen the JVM shuts down or restarts, pay attention to actively shutting down the thread pool, otherwise it may cause the Java process to remain in the OS. To close the process:
Shutdown, as a notification function, cannot continue to append new tasks to the thread pool. Awaittermination, returns True at the end of all tasks at the specified time, otherwise returns false, which returns false, which means that the thread is still not completed. Shutdownnow, issuing interrupted to all executing threads to terminate the running of the thread.HttpClient4.3 Connection PoolUse the basic steps:
Create the Connection Manager Poolinghttpclientconnectionmanager. Sets the total number of concurrent connections maxtotal and the number of single-routed connections Maxperroute. Sets the configuration of the socket. Sets the configuration of the HTTP connection. Sets the configuration for request requests. Set the retry policy, which can be overridden. Create a httpclient, set up the manager created by the above steps, request related configuration, retry policy, and so on.
Depending on the different attributes, different executor actuators are created at this time. HttpClient uses the responsibility chain pattern, and all executor implement the Execute () method of the Clientexecchain interface, each of which holds the next executor reference to be executed. This creates a executor chain of execution that requests to be passed on this chain. Create a GET request and reset the request parameters, overriding the default. Executes the request. Release the connection back to the connection pool, close the connection, and close the connection manager. Basic structure:
Connection pool entity PoolentryContains the managedhttpclientconnection connection, is actually a httpclient connection, the real connection will bind a socket to transmit the HTTP message. Route routing information for the connection. Connection survival time interval information. Linkedlist<poolentry> avaiable, store the available connections. After use, all reusable connections are placed back into the available list header, and then the connection is taken precedence from the available list header to iterate through the available connections. The use of LinkedList is based on the characteristics of its queue, which can be inserted and deleted at the first and the end of the team. Into the available linked list is AddFirst () into the head, gets when the connection is iterated from the head, so that you can get to the latest link in the list of connections, it is more than the expiration time (this policy can be as far as possible to ensure that the obtained connection is not expired, Getting a connection from the end of the queue can be done as much as possible before the connection expires, but the risk of getting an expired connection is large, and deleting the available linked list starts at the end of the queue, removing the connection that is most likely to expire. Hashset<poolentry> leased, which stores the leased connection. Maxtotal limits the size of the sum of leased collections and available queues in the outer httpconnpool, with no separate limits on the size of leased and available. Linkedlist<poolentryfuture> pending, store the future of the thread waiting to get the connection. When getting a connection from a pool, if the available linked list does not have an out-of-the-box connection, and the current route or connection pool has reached the maximum limit, the connection cannot be created, and the entire connection pool is not blocked, but the future of the current thread used to get the connection is placed at the end of the pending list , then the current thread calls await (), releases the held lock, and waits for the wake. When a connection is released back to the connection pool, it gets the future from the pending-Link header and wakes its thread to continue acquiring the connection, so that it is FIFO-first. Precautions
Cause the TCP connection is too high: because the connection pool after the close, and will not really release TCP and put back into the pool, so when the new more than the httpclient, the concurrency of a high, it is possible to cause the TCP link high, it will wait until the timeout to release itself. Because HttpClient is thread-safe under multiple threads, in a multithreaded environment, you should only use a global singleton httpclient, and use Multithreadhttpconnectionmanager to manage connection. Be aware that the maximum number of connections per route is set (the route refers to an address) and the default value is 2. Raises the issue of frequent throw-Timeout exceptions: It is possible that the maximum number of connections for a route is not set, and that the timeout setting for reading data is too large. Source Code Analysis Reference: HttpClient 4.3 connection pool parameter configuration and source code interpretationco-process (fiber) FiberThe concept of the process: a user-defined lightweight thread, in fact, is a single-threaded, specified to execute the entire function into a part and then go out to perform other, and so on, the next update frame to continue to execute. The advantage is that without the overhead of thread context switching, full development of the single CPU capacity, low resource consumption, suitable for high concurrent I/O. The disadvantage is also obvious, is that there is no way to take advantage of multi-CPU. Framework: Quasar, the Scheduler uses Forkjoinpool to dispatch these fiber. The fiber scheduler Fiberscheduler is an efficient, work-stealing, multi-threaded scheduler. Scenario: Service A usually needs to invoke other services, but other services are heavily delayed when concurrency is high.
You can start with the httpclient connection pool + thread pool, but if the delay in invoking the service is too high or timed out, the throughput of service A will be particularly poor. The main reason is generally a link by a thread to handle, is blocked, so the number of thread pool is limited, the throughput certainly does not go. And when all threads are I/O blocked, CPU resources are wasted, and the CPU keeps doing useless context switching. At this time, you can consider the co-process to replace. Reference article: http://blog.csdn.net/qq910894904/article/details/41699541,
http://blog.csdn.net/hj7jay/article/details/51980038,
http://blog.csdn.net/zdy0_2004/article/details/51323583forkjoinpool thread PoolThe Fork/join framework is a framework that JAVA7 provides for parallel execution of tasks, a framework that divides large tasks into small tasks, and ultimately summarizes the results of a large task after each small task. The work-stealing (work-stealing) algorithm is the most important feature of the Fork/join framework. Usually a thread will correspond to a task queue, when processing a faster thread to finish their task, will be stealing another slow processing of the corresponding task, there will be two threads processing a queue at the same time, so the task queue generally use a double-ended queue, The stolen task thread will always execute from the head of the double-ended queue, while the thread that steals the task will always carry the task from the tail of the double-ended queue. The advantage is to make full use of threads for parallel computing and reduce the competition between threads.NiO principleAll IO events are handled by a dedicated thread and are responsible for distribution. Event-driven: events are triggered when they arrive, rather than synchronized to monitor events. In the asynchronous event-driven model, an operation that causes blocking is translated into an asynchronous operation, and the main thread initiates this asynchronous operation and processes the result of the asynchronous operation. Nginx is the model used. Thread communication: Threads communicate through wait, notify, and so on. Ensure that each context switch is meaningful. Reduce unnecessary thread switching. The server and the client each maintain a management channel object, which we call selector, which detects events on one or more channels (channel). We take the server as an example, if a read event is registered on the server's selector, at some point the client sends some data to the server, blocking I/O then calls the Read () method to block the data, and the service side of NiO adds a read event to the selector. The service-side processing thread polls the selector and, if an event of interest is found when accessing selector, handles these events and, if no event of interest arrives, the processing thread blocks until the event of interest arrives.