Thoughts on multithreading Systems

Problem background

Concurrency is a performance bottleneck of multi-threaded systems. The direct factor restricting the system concurrency is the number of threads. When resources are limited, the number of threads has a critical value. Once the threshold value is exceeded, the system performance will decrease as the number of threads increases. Therefore, the server usually sets a maximum number of threads, which is the maximum number of concurrent threads.

The above situation brings about a performance problem: it is possible that other resources in the system are still available, but the concurrency reaches the maximum value, so that the system cannot process more tasks. If we change the system to a single thread, the number of threads will disappear and the system performance will be improved to a higher level. Then the problem arises. What kind of system can be operated by a single thread?

Blocking-> non-blocking

In blocking mode, a single thread is not feasible. A single thread executes functions one by one, and only one function returns can it execute the next one. If the function is blocked, the entire system stops waiting for the blocking to end. The result of this execution method is that the system performance has suffered a devastating blow.

Change the blocking mode to non-blocking mode, and the task can be executed in a single-threaded environment. In blocking mode, the function can return only when a condition is met. In non-blocking mode, tasks with waiting conditions are put into a waiting queue, and then return directly without waiting conditions.

The system has two types of threads: start thread and callback thread. The start thread executes tasks one by one, and the next task is executed after a task is returned. Of course, the returned tasks may not be completed, but are placed in the waiting queue. The callback thread regularly checks the waiting condition. Once the condition is set, it extracts a task from the waiting queue and calls its callback function.

Real-time task

Real-time tasks cannot run in a single-threaded environment. Real-time tasks are characterized by running and output simultaneously. A player is a typical real-time task that outputs the video and audio while executing it. The criterion for determining a real-time task is whether the task execution result can be output together at the end of the task. For the player, the above conditions are obviously not met.

Servlet execution features a one-time response. Although the servlet may operate the database and call the WebService interface during execution, the delay of these operations until the end of the servlet does not affect. Therefore, servlet is a non-real-time task.