Traditional project using Hystrix to realize service isolation of hotspot interface

During this time, we need to add an interface to the data system I am currently responsible for, mainly to record the user's behavior. Front-end docking project mainly has the company's Pc,wap,weixin,app, and so on, each end about two or so load. Because at present my project is mainly internal-oriented, responsible for data operations related content, is a single project. If the various access points on the line do not limit, the instantaneous large number of concurrent entry will inevitably lead to the current project crashes, other functions can not be used properly.

1. Demand Analysis

Through the pre-demand analysis, the current online system can not be limited to current processing, so the final solution to the problem is to start from the interface.

Currently, I have two implementations of interface processing:

1. You can use MQ to implement the error peaks of messages and send messages to the MQ server.
2. Implement an isolation limit on the interface to avoid the impact of the current interface on other functions.

In fact, I think the best implementation is the first, you can guarantee the accurate delivery of messages, to avoid the use of concurrent resources. However, because of the limitations of the company conditions temporarily can not add middleware, so can only be modified on the existing system, finally can only use the second method.

2, Hystrix's brief introduction

The official definition is not said, here is simply my understanding. Hystrix as a circuit breaker is mainly to achieve fault-tolerant service protection, in simple terms, service isolation, service degradation, service fuse, service limit flow these items.

For a common example, when you "snapped" a product, it often pops up [network error, please try again. ] Prompt, is this time a real network problem? Display is not. In this case, the interface is actually degraded processing, when the number of degraded or proportional to a certain condition, the circuit breaker will be opened directly, then the access will be directly degraded, and will not determine whether the downgrade. To achieve fault-tolerant protection of the service and user-friendly hints. The detailed process can be seen.

The fault-tolerant protection of general hystrix is used on the client side, that is the caller, in MicroServices. And I am actually used in the provider, mainly the front desk of the project I can not control, only on the interface to think about the method. At present, the main purpose of my use of hystrix is to realize the isolation and current limit of the service, but the degradation and the fuse is not special concern, of course, the actual use to combine the scene.

Because the generic tomcat is a thread pool of 150 threads by default, if there are too many requests for a single hotspot interface, there are other features that do not have thread availability or even direct program crashes. There are two main types of service isolation in hystrix, which is commonly used to isolate the thread pool and establish a separate thread pool on the hotspot interface to avoid the influence on the main program. The other is the way the semaphore is used, not too many scenes. The difference between the two is actually an increase in the cost of the system, one directly limits the total number of threads of concurrency, the cost is smaller.

Hystrix Service invocation Logic diagram

3. Application implementation in the project

This article is the application in the traditional spring project, the correlation configuration and the dependence in the springboot are slightly different.

Maven dependencies:

     <!--Hystrix -        <Dependency>            <groupId>Com.netflix.hystrix</groupId>            <Artifactid>Hystrix-core</Artifactid>            <version>1.5.9</version>        </Dependency>        <Dependency>            <groupId>Com.netflix.hystrix</groupId>            <Artifactid>Hystrix-metrics-event-stream</Artifactid>            <version>1.5.9</version>        </Dependency>        <Dependency>            <groupId>Com.netflix.hystrix</groupId>            <Artifactid>Hystrix-javanica</Artifactid>            <version>1.5.9</version>        </Dependency>

To configure Hystrix slice information in spring's configuration file

    <id= "Hystrixaspect"  class= " Com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect "></Bean  >    </>

The main is to open the annotated AOP scan, here we can see in the source code of this class to realize

We can see that we mainly scan the relevant annotations of hystrix through this kind of slice to achieve the code of Hystrix related logic before interface processing.

/*** Provide customer behavior interface **/@Controller @requestmapping (value= "/test") Public classBehaviorcontroller {Logger Logger= Logger.getlogger (Behaviorcontroller.class); @AutowiredPrivateBehaviorservice Behaviorservice; @RequestMapping (Value= "/addbehavior", method = Requestmethod.post,produces = "Application/json;charset=utf-8") @ResponseBody @HystrixCommand (Fallbackmethod= "fallback", threadpoolproperties ={@HystrixProperty (name= "Coresize", value = "a"), @HystrixProperty (name = "Maxqueuesize", value = "100"), @HystrixProperty (name= "Queuesizerejectionthreshold", value = "20")}, Commandproperties={@HystrixProperty (name= "Execution.isolation.thread.timeoutInMilliseconds", value = "30000"), @HystrixProperty (name= "Circuitbreaker.requestvolumethreshold", value = "20")      })     Publicstring Addbehavior (@RequestBody string parms) {//Business Logic Implementation            returnresult; }     Publicstring Fallback (@RequestBody string parms) {Logger.info ("Fallback"); //the implementation of the failure        returnresult; }}    

Attention:
The requested interface must be public,fallback for degraded interface logic, either private or public.

However, it is important to note that the return value and parameters of the fallback method must be the same as the request method .

It is also necessary to say that The fallback method is entered when the request fails, is rejected, timed out, or the circuit breaker is open, but entering the fallback method does not mean that the circuit breaker has been opened.

4. Introduction of common parameters

Parameters	Describe	Default value
Execution.isolation.strategy	Isolation policy, with thread and semaphore Thread-it executes on a separate thread, and concurrent requests are limited by the number of threads in the thread pool SEMAPHORE-it executes on the calling thread, and the concurrent request is limited by the semaphore count	Using the thread mode by default, the following scenarios can use the semaphore mode: Just want to control the concurrency level External methods have been thread-isolated Call a local method or a very high-reliability, time-consuming method (such as Medis)
Execution.isolation.thread.timeoutInMilliseconds	Timeout period	Default value: 1000 In thread mode, the time-out is reached and can be interrupted In semaphore mode, it waits for the execution to complete before deciding whether to timeout Setting the standard: Have retry,99meantime+avg meantime No retry,99.5meantime.
Execution.timeout.enabled	Hystrixcommand.run () execution should have timed out.	Default value: True
Fallback.isolation.semaphore.maxConcurrentRequests	Sets the maximum number of concurrent requests that are allowed to execute the fallback method when used	Default value: 10
Circuitbreaker.requestvolumethreshold	Sets the minimum number of requests for a circuit breaker to fuse in a scrolling time window	Default value: 20 scrolling window Default 10s, that is, 20 failed requests in 10s, the fuse is turned on
Coresize	Sets the number of core threads executing the command thread pool.	Default value: 10
Maxqueuesize	Sets the number of core threads executing the command thread pool.	Default value: 1 When set to-1, the thread pool uses the Synchronousqueue implementation queue, otherwise the queue implemented by the Linkedblockingqueue will be used
Queuesizerejectionthreshold	Set a deny threshold for a queue	Default value: 5 When this parameter is set, the request can be rejected even if the queue does not reach the maximum value. Note: This property does not take effect when the Maxqueuesize property is-1

It is also important to note that thefallback property maxconcurrentrequests, when the request reaches the maximum concurrency, the subsequent request will be rejected and thrown an exception (because it has no subsequent fallback can be called), The exception information is generally com.netflix.hystrix.exception.HystrixRuntimeException:xxxxxxx fallback execution rejected.

See the official documentation for more parameters:github.com/netflix/hystrix/wiki/configuration

Also attached to a netizen translated documents:78611225

5. Interface Test

There are many ways to test concurrent interfaces, you can write code, or you can use Apache batch and JMeter tools. To test this interface, take the usual jmeter as an example.

Set environment variables: jmeter_home   D:\apache-jmeter-3.0CLASSPATH     %jmeter_home%\lib\ext\apachejmeter_core.jar;%jmeter_ Home%\lib\jorphan.jar;%jmeter_home%\lib/logkit-2.0.jar;

New Thread Group:

To set concurrency parameters:

The first parameter is the number of threads, the second parameter is the start time, and the third parameter is the number of requests. Take the above configuration as an example to start 50 threads in 1 seconds, one request at a time per thread.

Add an HTTP request

This sets the path of the request, parameters, and so on.

Setting the request header information varies from person to person

My settings: Content-type:application/json

Table View Results

View results:

This can be based on the performance of the machine test, in my interface, for example, when the number of concurrent set is 100, there is basically no demotion processing, when the number of concurrent more than 100, there will be a partial request to enter the degraded interface.

6. Interface Monitoring

In the actual project, we often need to monitor our interface and project situation, Hystrix already for us to consider. Hystrix provides near real-time monitoring, Hystrix will record all the execution information about Hystrixcommand in real time, including how many requests executed per second, how many successes, how many failures, etc., more indicators see: github.com/ Netflix/hystrix/wiki/metrics-and-monitoring

Add the following configuration in Web. xml:

    <!--For Hystrix -    <servlet>        <Display-name>Hystrixmetricsstreamservlet</Display-name>        <Servlet-name>Hystrixmetricsstreamservlet</Servlet-name>        <Servlet-class>Com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet</Servlet-class>    </servlet>    <servlet-mapping>        <Servlet-name>Hystrixmetricsstreamservlet</Servlet-name>        <Url-pattern>/hystrix.stream</Url-pattern>    </servlet-mapping>

Launch the app. To access the http://hostname:port/project name/hystrix.stream, you can see the following information

This is the most native monitoring usage, and you can integrate an additional Dashboard application provided by Hystrix. Dashboard is a standalone application that we can deploy independently. If you need to monitor the entire Hystrix cluster, you need to use the Turbine app. Turbine is also a service for Netflix open source. But there is an obvious disadvantage of using hystrix Stream and Turbine, which is the inability to view historical monitoring data. To solve this problem, we need to implement it ourselves.

Monitoring of the relevant is not the focus of this article, not many introduced, we can Baidu under the Hystrix in the Micro-service project monitoring implementation, here online articles a lot, basically and the traditional project no difference, we can refer to the implementation.