Hystrix Thread Isolation technology parsing-thread pool (RPM)

Meet Hystrix

Hystrix is a fault-tolerant framework for Netflix open source, which includes common methods of fault tolerance: thread isolation, semaphore isolation, downgrade strategies, and fuse technology.
Under high concurrent access, the stability of the service dependent on the system has a great impact on the system, there are many factors that are not controllable, such as slow network connection, sudden and busy resource, temporary unavailability, service offline, etc. If we want to build a stable and reliable distributed system, we must have such a fault-tolerant method.
This article mainly discusses the thread isolation technology.

Why do you want to do thread isolation

For example, we now have 3 business calls to query orders, query products, query users, and these three business requests are dependent on third-party services-order services, commodity services, User Services. Three of the services are called through RPC. When the query order service, if the thread is blocked, this time there are a large number of query order requests come over, then the number of threads in the container will continue to increase the direct CPU resources exhausted to 100%, the entire service is not available, the cluster environment is an avalanche. Such as

The order service is not available. png

：

The entire Tomcat container is not available. How Pnghystrix is threaded through thread pooling

Hystrix through the command mode, each type of business request is encapsulated into the corresponding command request, such as query order, Order command, query goods----goods command, query user-user command. Each type of command corresponds to a thread pool. The created thread pool is placed into the concurrenthashmap, such as querying the order:

final static ConcurrentHashMap<String, HystrixThreadPool> threadPools = new ConcurrentHashMap<String, HystrixThreadPool>();threadPools.put(“hystrix-order”, new HystrixThreadPoolDefault(threadPoolKey, propertiesBuilder));

When a second query order is requested, the thread pool can be obtained directly from the map. Specific processes such as:

Hystrix thread execution procedures and asynchrony. png

To create a thread in the thread pool method, view the source code as follows:

Public Threadpoolexecutor Getthreadpool (final Hystrixthreadpoolkey Threadpoolkey, hystrixproperty<integer> Corepoolsize, hystrixproperty<integer> maximumpoolsize, hystrixproperty<integer> keepAliveTime,    Timeunit unit, blockingqueue<runnable> workQueue) {threadfactory threadfactory = null; if (! Platformspecific.isappenginestandardenvironment ()) {threadfactory = new Threadfactory () {protected Fin            Al Atomicinteger threadnumber = new Atomicinteger (0); @Override public Thread Newthread (Runnable r) {Thread thread = new Thread (R, "hystrix-" + Threa                Dpoolkey.name () + "-" + threadnumber.incrementandget ());                Thread.setdaemon (TRUE);            return thread;    }        };    } else {threadfactory = Platformspecific.getappenginethreadfactory ();    } final int dynamiccoresize = Corepoolsize.get ();    Final int dynamicmaximumsize = Maximumpoolsize.get (); if (dynamiccoresize > Dynamicmaximumsize) {logger.error ("Hystrix ThreadPool configuration at startup for:" + threadpoolkey.name (  ) + "is trying to set coresize =" + Dynamiccoresize + "and maximumsize =" + Dynamicmaximumsize + ". Maximum size would be set to "+ Dynamiccoresize +", the coresize value, since it must is equal to or great        Er than the coresize value ");    return new Threadpoolexecutor (Dynamiccoresize, Dynamiccoresize, Keepalivetime.get (), unit, WorkQueue, threadfactory); } else {return new Threadpoolexecutor (Dynamiccoresize, Dynamicmaximumsize, Keepalivetime.get (), unit, WorkQueue, T    Hreadfactory); }}

There are four ways to execute the command, directly look at the official document (Https://github.com/Netflix/Hystrix/wiki/How-it-Works), the specific differences are as follows:

• Execute (): Executes run () in a synchronous blocking manner. After calling execute (), Hystrix creates a new thread to run running (), and the calling program is blocked at the Execute () call until run () is complete.

• Queue (): Executes run () in an asynchronous, non-blocking manner. The call to queue () directly returns a future object, while Hystrix creates a new thread running run (), and the caller gets the return result of run () through Future.get (), and Future.get () is blocked.

• Observe (): Execute Run ()/construct () before event registration. The first step is to call observe () to automatically trigger the execution of Run ()/construct () before the event is registered (if the inherited Hystrixcommand,hystrix will create a new thread non-clogging execution run () If the inherited Hystrixobservablecommand, will be blocked by the calling program thread to execute construct ()), the second step is to return from observe () after calling the program call SUBSCRIBE () Completion event registration, if run ()/ Construct () execution succeeds triggers onnext () and oncompleted () and triggers onerror () If an exception is executed.

• Toobservable (): Executes run ()/construct () after the event is registered. The first step is to call Toobservable () to return an Observable<string> object immediately before the event is registered, and the second step calls subscribe () to automatically trigger execution run ()/construct () After the event registration is completed. (If the inherited Hystrixcommand,hystrix will create a new thread non-clogging execution run (), the calling program does not have to wait for run (); If you inherit Hystrixobservablecommand, Construct () will be blocked by the calling program thread, and the calling program waits for construct () to continue to go down), and if Run ()/construct () executes successfully it triggers OnNext () and oncompleted (), Trigger OnError If an exception is executed ()
  Note:
  Execute () and queue () are in Hystrixcommand, observe () and toobservable () are in Hystrixobservablecommand. From the bottom of the implementation, Hystrixcommand is actually using observable implementation (see Hystrix Source code, you can find a lot of use of Rxjava), although it only returns a single result. The Hystrixcommand queue method actually calls the Toobservable (). toblocking (). Tofuture (), and the Execute method actually calls the queue (). get ().

How to apply to the actual code

Package Myhystrix.threadpool;import Com.netflix.hystrix.*;import Org.junit.test;import java.util.List;import java.util.concurrent.future;/** * Created by Wangxindong on 2017/8/4.    */public class Getordercommand extends hystrixcommand<list> {orderservice orderservice; Public Getordercommand (String name) {Super (Setter.withgroupkey (HystrixCommandGroupKey.Factory.asKey (" Threadpooltestgroup ")). Andcommandkey (HystrixCommandKey.Factory.asKey (" Testcommandkey ")). An                        Dthreadpoolkey (HystrixThreadPoolKey.Factory.asKey (name)). Andcommandpropertiesdefaults (                Hystrixcommandproperties.setter (). Withexecutiontimeoutinmilliseconds (5000)                                ). Andthreadpoolpropertiesdefaults (Hystrixthreadpoolproperties.setter ()   . Withmaxqueuesize (10)//configure queue size. Withcoresize (2)//Configure line constructor thread count             )        );    } @Override protected List run () throws Exception {return orderservice.getorderlist (); } public static class UnitTest {@Test public void Testgetorder () {//New Getordercommand ("Hyst            Rix-order "). Execute ();        future<list> Future =new Getordercommand ("Hystrix-order"). Queue (); }    }}

Summarize

The thread that executes the dependent code is detached from the requesting thread (such as the Tomcat thread), and the request thread is free to control the time of departure, which is what we typically call asynchronous programming, Hystrix is asynchronous programming with Rxjava. By setting the thread pool size to control the number of concurrent accesses, the service can be denied when the thread is saturated, preventing dependency problems from spreading.

Thread isolation. png

Advantages of thread Isolation:
[1]: The application will be fully protected and will not affect other parts of the application, even if the thread pool of one of the services it relies on is full.
[2]: We give the application to introduce a new low-risk client lib, if a problem occurs, also in this lib, and will not affect other content, so we can boldly introduce the new Lib library.
[3]: When a dependent failed service returns to normal, the application will immediately return to normal performance.
[4]: If some of our application parameters are misconfigured, the health of the thread pool will be displayed quickly, such as latency, timeouts, rejections, and so on. The parameter configuration of correcting errors can also be handled by real-time execution of dynamic properties.
[5]: If the performance of the service changes, which requires adjustment, such as increasing or reducing the time-out period, change the number of retries, the thread pool indicator dynamic properties can be modified, and does not affect other call requests.
[6]: In addition to the isolation advantages, the Hystrix has a dedicated thread pool that provides built-in concurrency features that allow you to build asynchronous appearance patterns on top of synchronous calls, which makes it easy to do asynchronous programming (Hystrix introduces the Rxjava asynchronous framework).

Although the thread pool provides threading isolation, our client-side code must also have a timeout setting that cannot be blocked indefinitely so that the thread pool is saturated.

Disadvantages of thread Isolation:
[1]: The main disadvantage of the thread pool is that it increases the cost of computing, and each business request (packaged as a command), when executed, involves request queuing, scheduling, and context switching. But within Netflix, the thread-isolation overhead was considered small enough to have no significant cost or performance impact.

The Netflix API processes billion Hystrix Command executions per day using thread isolation. Each API instance have 40+ thread-pools with 5–20 threads in each (most is set to 10).
The Netflix API uses thread isolation to process 1 billion hystrix command executions per day. Each API instance has more than 40 thread pools, with 5-20 threads in each thread pool (most of which are set to 10).

For services that do not rely on network access, such as relying only on memory caching, this is not appropriate for the thread pool isolation technique, but for semaphore isolation, which is described later in the article.

So we can safely use Hystrix's thread isolation technology to prevent an avalanche of this horrible deadly online failure.

Reprint please indicate the source, and attach the link http://www.jianshu.com/p/df1525d58c20

Resources:
Https://github.com/Netflix/Hystrix/wiki

New House Book
Links: Http://www.jianshu.com/p/df1525d58c20
Source: Pinterest
Copyright belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please specify the source.

Hystrix Thread Isolation Technology in addition to the thread pool, there is another way: semaphores.

The difference between the thread pool and the semaphore

In the article "Hystrix Thread Isolation technology parsing-thread pool" Finally, we talked about the disadvantage of the thread pool, when we rely on the service is very low latency, such as access to memory cache, there is no need to use the thread pool, so the cost is not worth it, but it is recommended to use the semaphore this way. The following diagram illustrates the main differences between thread pool isolation and semaphore isolation: The thread pool mode is not the same thread for the business request thread and the threads that are executing the dependent service, and the traffic request thread and the thread that executes the dependent service are the same thread in the semaphore mode

The difference between the semaphore and the thread pool. How PNG uses semaphores to isolate threads

Setting the property execution.isolation.strategy to SEMAPHORE, like this executionisolationstrategy.semaphore, hystrix use semaphores instead of the default thread pool for isolation.

public class CommandUsingSemaphoreIsolation extends HystrixCommand<String> {    private final int id;    public CommandUsingSemaphoreIsolation(int id) {        super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))                // since we‘re doing work in the run() method that doesn‘t involve network traffic                // and executes very fast with low risk we choose SEMAPHORE isolation                .andCommandPropertiesDefaults(HystrixCommandProperties.Setter()                        .withExecutionIsolationStrategy(ExecutionIsolationStrategy.SEMAPHORE)));        this.id = id;    }    @Override    protected String run() {        // a real implementation would retrieve data from in memory data structure        // or some other similar non-network involved work        return "ValueFromHashMap_" + id;    }}

Summarize

The amount of semaphore isolation is limited by the total number of concurrent calls, each time the request is made, and the thread that invokes the dependent service is the same thread, so if the remote RPC call is not involved (there is no network overhead) The semaphore is used to isolate it, which is lighter and less expensive.

Reprint please indicate the source, and attach the link http://www.jianshu.com/p/af8dc67e5238

Reference: Https://github.com/Netflix/Hystrix/wiki

New House Book
Links: http://www.jianshu.com/p/af8dc67e5238
Source: Pinterest
Copyright belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please specify the source.

Hystrix Thread Isolation technology parsing-thread pool (RPM)