Boosting performance with asynchronous servlet

After the publication of this article, I received a lot of feedback. Based on these feedback, we have updated the examples in this article to make it easier for readers to understand and grasp, and if you find errors and omissions, we hope to provide feedback to help us improve.

This paper introduces the corresponding performance optimization solution for a common problem in webapp today. Today's web programs are no longer just passively waiting for browser requests, and they also communicate with each other. Typical scenarios include live chat, real-time auctions, etc.-the background program spends most of its time with the browser in an idle state and waits for an event to be triggered.

These applications raise a new class of problems, especially in the case of high loads. The resulting conditions include thread starvation, affecting user experience, request timeouts, and so on.

Based on the practice of this type of application under high load, I will introduce a simple solution. After the Servlet 3.0 became mainstream, it was a truly simple, standardized and elegant solution.

Before demonstrating a specific solution, let's start by understanding what's going on. Please look at the code:

@WebServlet(urlPatterns = "/BlockingServlet")public class BlockingServlet extends HttpServlet {  protected void doGet(HttpServletRequest request, HttpServletResponse response) {    waitForData();    writeResponse(response, "OK");  }  public static void waitForData() {    try {      Thread.sleep(ThreadLocalRandom.current().nextInt(2000));    } catch (InterruptedException e) {      e.printStackTrace();    }  }}

The scenario represented by this servlet is as follows:

• Certain events occur every 2 seconds, such as update of quote information, arrival of chat information, etc.
• The end user requests a listener for certain events.
• The thread is temporarily blocked until the next event is received.
• When an event is received, the response information is processed and sent to the client

Here's an explanation of the waiting scene. Our system triggers an external event every 2 seconds. When a user request is received, it needs to wait for a period of time, between 0 and 2000 milliseconds, until the next event occurs. To demonstrate the need, here is a call Thread.sleep() to simulate a random wait time. Each request waits about 1 seconds on average.

Now, you might think this is a very common servlet. In most cases, this is true-the code is not wrong, but it will be overwhelmed if the system faces a lot of concurrent load.

To simulate this load, I created a simple test with JMeter, started 2000 threads, and executed 10 requests per thread to perform a system stress test.

The requested URI is /BlockedServlet , deployed under the Tomcat 8.0.30 default configuration, and the test results are as follows:

• Average response time: 9,492 ms
• Minimum response time: 205 ms
• Maximum response time: 11,368 ms
• Throughput: 195 Requests/sec

TOMCAT is configured with 200 worker threads by default, plus the simulated workload (avg. thread hibernation), which explains the throughput data well-200 threads should be able to complete 200 execution cycles per second, averaging about 1 seconds. But there are some context switching costs, so the throughput is 195 requests per second, very much in line with our expectations.

For 99.9% of applications, this throughput data also looks normal. But look at the maximum response time, and the average response time, and you'll see that the problem is too serious. The worst case scenario is that the client needs 11 seconds to get a response, which is expected to be 2 seconds, which is unfriendly to the user.

Let's look at another implementation that uses the asynchronous nature of Servlet 3.0:

  @WebServlet (asyncsupported = true, value = "/asyncservlet") public class Asyncservlet extends HttpServlet {Pro tected void doget (HttpServletRequest request, httpservletresponse response) throws Servletexception, IOException {AddT  Owaitinglist (Request.startasync ());  } private static Scheduledexecutorservice Executorservice = Executors.newscheduledthreadpool (1);  static {executorservice.scheduleatfixedrate (asyncservlet::newevent, 0, 2, timeunit.seconds);    } private static void Newevent () {ArrayList clients = new arraylist<> (Queue.size ());    Queue.drainto (clients);      Clients.parallelstream (). ForEach ((Asynccontext ac), {servletutil.writeresponse (Ac.getresponse (), "OK");    Ac.complete ();  });  } private static final blockingqueue queue = new arrayblockingqueue<> (20000);  public static void Addtowaitinglist (Asynccontext c) {Queue.add (c); }}

The code above is a little bit more complicated, so let me first reveal the performance of this scenario: the Response delay (latency) is only 1/5 of the original; The throughput (throughput-wise) also increased 5 times times. Seeing this result, you definitely want to learn more about the second option.

The servlet doGet approach looks very simple. There are two places worth mentioning:

One is to declare the servlet, and to support asynchronous method invocations:

@WebServlet(asyncSupported = true, value = "/AsyncServlet")

The second is addToWaitingList the details of the method:

  public static void addToWaitingList(AsyncContext c) {    queue.add(c);  }

In it, the entire request is processed with only one line of code, and the Asynccontext instance is added to the queue. The asynccontext contains the request and response objects provided by the container, which we can use to respond to user requests. So incoming requests are waiting for notification--either a quote update event in a monitored auction group, or a next group chat message. It is important to note that after the Asynccontext is added to the queue, the servlet container's thread is finished doget Operation, and then release it, you can accept another new request.

Now, the system notification arrives every 2 seconds, of course, this part we implemented through the static block of scheduled events, every 2 seconds to execute the Newevent method. When a notification arrives, all waiting requests in the queue are handled by the same worker thread and send a response message. This time, the code does not block hundreds of threads to wait for an external event notification, but is implemented in a more concise and straightforward way, placing the request of interest in a group, which is processed in batches by a single thread.

The result needless to say, the same configuration, the same test, the Tomcat 8.0.30 server ran out the following results:

• Average response time: 1,875 ms
• Minimum response time: 356 ms
• Maximum response time: 2,326 ms
• Throughput: 939 Requests/sec

Although the examples are constructed manually, similar performance improvements are common in the real world.

Now, don't be anxious to refactor all the servlets into asynchronous Servlets. Because of this scenario, there will be a lot of performance gains for tasks that meet certain characteristics, such as chat rooms, or auction price reminders. For operations that require the underlying database to be requested, there is probably no performance improvement. So, as before, I must reiterate my favorite performance tuning advice--weigh the whole thing down and don't take it for granted.

But if it does fit the scenario, then I congratulate you! Not only can it significantly improve throughput and latency, but it can also perform well under a lot of concurrency pressures to avoid possible thread starvation issues.

Another important message is that the processing of asynchronous requests has finally been standardized. Application servers that are compatible with Servlet 3.0-such as Tomcat 7+, JBoss 6, or Jetty 8+---support this scenario. No longer need to be trapped in solutions that are coupled to specific platforms, such as Weblogic FutureResponseServlet .

Original link: https://plumbr.eu/blog/java/how-to-use-asynchronous-servlets-to-improve-performance

Translators: Anchor Http://blog.csdn.net/renfufei

Translation time: December 08, 2016

Boosting performance with asynchronous servlet