Java multithreaded-concurrent programming model

The following is transferred from http://ifeve.com/%E5%B9%B6%E5%8F%91%E7%BC%96%E7%A8%8B%E6%A8%A1%E5%9E%8B/:

Concurrent systems can be implemented with multiple concurrent programming models. The concurrency model specifies how threads in the system can collaborate to complete the jobs assigned to them. Different concurrency models split jobs in different ways, while collaboration and interaction between threads are not the same. This concurrency model tutorial will be a more in-depth introduction to the current (2015, the writing time) of several popular concurrency models.

The similarity between concurrency model and distributed system

The concurrency model described in this article is similar to many of the architectures used in distributed systems. The threads in the concurrent system can communicate with each other. Processes can also communicate with each other in distributed systems (processes may be in different machines). There are many similar characteristics between threads and processes. This is why many concurrent models are often similar to various distributed system architectures.

Of course, distributed systems also face additional challenges in dealing with network failures, remote hosts, or process outages. But concurrent systems running on giant servers can also encounter similar problems, such as a CPU failure, a failure of a NIC, or a disk corruption. Although the probability of failure may be low, it may still be possible in theory.

Because the concurrency model is similar to Distributed system architecture, they can often learn from each other's ideas. For example, a model that assigns jobs to workers (threads) is generally similar to a load-balancing system in a distributed system. Similarly, they have similarities in error handling techniques such as logging, failover, idempotent, and so on.

"Note: idempotent, a power-like operation is characterized by its arbitrary multiple execution of the impact is the same as the impact of a single execution"

One, parallel workers

The first concurrency model is what I call the parallel worker model. The incoming jobs are assigned to different workers. Shows the parallel worker model:

In the parallel worker model, the delegate (Delegator) assigns the incoming job to different workers. Each worker completes the entire task. Workers work in parallel on different threads, possibly even on different CPUs.

If a parallel worker model is implemented in a car factory, each car will be produced by a worker. The workers will get the production specifications of the car and take care of all the work from start to finish.

In Java applications, the parallel worker model is the most common concurrency model (even if it is transitioning). Many of the concurrency utilities in the Java.util.concurrent package are designed for this model. You can also see traces of this model in the Java Enterprise (EE) application Server design.

Advantages of the parallel worker model

The advantage of the parallel worker pattern is that it's easy to understand. You only need to add more workers to improve the degree of parallelism of the system.

For example, if you're working on a web crawler, try grabbing a certain number of pages with a different number of workers, and then see how many workers consume the shortest amount of time (which means the highest performance). Because the web crawler is an IO-intensive job, the end result is probably that each CPU or core in your computer is assigned several threads. Allocating only one thread per CPU can be a bit less because the CPU will be idle for a lot of time while waiting for the data to be downloaded.

Disadvantages of the parallel worker model

While the parallel worker model looks simple, it hides some drawbacks. In the following chapters I will analyze some of the most obvious weaknesses.

Shared state can be complex

In practical applications, the parallel worker model may be much more complex than the previous described situation. Shared workers often need to access some shared data, either in-memory or in a shared database. Shows how the parallel worker model has become complex:

Some sharing states are under a communication mechanism such as the job queue. But there are some shared states that are business data, data caching, database connection pooling, and so on.

Once the shared state dives into the parallel worker model, the situation becomes complicated. The thread needs to access the shared data in some way to ensure that the modification of one thread is visible to other threads (data modifications need to be synchronized to main memory, not just the data in the cache of the CPU that executes the thread). Threads need to avoid the concurrency of the state, deadlocks, and many other shared states.

In addition, waiting for the shared data structure to be accessed will lose some parallelism between the threads waiting for each other. Many concurrent data structures are blocked, meaning that only one or a few threads can access them at any one time. This results in a competitive state on these shared data structures. When executing code that requires access to the shared data Structure section, high competition basically results in some degree of serialization at execution time.

Today's non-blocking concurrency algorithms may reduce competition and improve performance, but the implementation of nonblocking algorithms is more difficult.

A data structure that can be persisted is another option. At the time of modification, the persisted data structure always protects its previous version from being affected. Therefore, if multiple threads point to the same persisted data structure, and one of the threads is modified, the thread that makes the modification gets a reference to the new structure. All other threads keep a reference to the old structure, and the old structure is not modified and thus guarantees consistency. Scala programming consists of several persisted data structures.

Note: The persistent data structure here is not a persistent storage, but a data structure, such as the string class in Java, and the Copyonwritearraylist class, which can refer to the "

While persistent data structures can be elegant in resolving concurrent modifications to shared data structures, the persistence of data structures is often unsatisfactory.

For example, a persistent linked list requires inserting a new node in the head and returning a reference to the newly added node (which points to the remainder of the list). All other localities still retain the first node before the list, and for these threads the list is still changed. They cannot see the newly added element.

This list of persistent lists is implemented using a chain table. Unfortunately, the list does not perform well on modern hardware. Each element in the list is a separate object that can be spread throughout the computer's memory. Modern CPUs are able to perform sequential accesses more quickly, so you can use arrays on modern hardware to implement lists to achieve higher performance. Arrays can store data sequentially. The CPU cache is able to load a chunk of the array at a time to cache, and once the CPU is loaded, the data in the cache can be accessed directly. This is unlikely to be possible for a linked list of elements scattered in RAM.

Non-state workers

Shared state can be modified by other threads in the system. So workers have to reread the state every time they need to make sure that the latest copy is accessible every time, regardless of whether the shared state is in memory or in an external database. Workers cannot save this state internally (but can be reread every time they need it) called stateless.

Rereading the data you need each time will result in slower speeds, especially when the state is stored in an external database.

The order of the tasks is indeterminate

Another disadvantage of the parallel worker pattern is that the job execution order is indeterminate. There is no guarantee which job will be executed first or last. Job A may have been assigned a worker before job B, but job B might have been executed before job a.

This non-deterministic nature of the parallel worker pattern makes it difficult to infer the state of a system at any particular point in time. This also makes it more difficult (if not impossible) to ensure that a job is executed before other jobs.

Second, the pipeline model

The second concurrency model we call the pipelining concurrency model. I chose this name only to match the metaphor of "parallel worker". Other developers may choose other titles (such as reactor systems, or event-driven systems) based on the platform or community. Represents a pipelined concurrency model:

Similar to the workers on the production line in the factory to organize workers. Each worker is responsible for only part of the work in the job. When you have completed this part of your work, the worker forwards the job to the next worker. Each worker runs in its own thread and does not share state with other workers. Sometimes also become a no-shared parallel model.

Typically, non-blocking IO is used to design systems that use pipelining concurrency models. Non-blocking IO means that once a worker starts an IO operation (such as reading a file or reading data from a network connection), the worker does not wait until the end of the IO operation. IO operations are slow, so it is a waste of CPU time waiting for the IO operation to end. At this point the CPU can do some other things. When the IO operation is complete, the results of the IO operation (such as the read-out data or the state of the data being written) are passed on to the next worker.

With non-blocking IO, you can use IO operations to determine the boundaries between workers. The worker runs as much as possible until it encounters and initiates an IO operation. Then hand over the control of the operation. When the IO operation is complete, the next worker on the pipeline continues to operate until it also encounters and initiates an IO operation.

In practical applications, the operation may not be carried out along a single pipeline. Because most systems can perform multiple jobs, the job flows from one worker to another depends on the work that the job needs to do. In practice, there may be multiple different virtual pipelines running concurrently. This is the reality of the operation in the pipeline system can move the situation:

Jobs may even be forwarded to more than one worker on concurrent processing. For example, a job might be forwarded to the job executor and the job logger at the same time. Illustrates how three pipelines can be completed by forwarding jobs to the same worker (the last worker in the middle pipeline):

The pipeline is sometimes more complicated than this situation.

Reactors, event-driven systems

Systems that use pipelining concurrency models are sometimes referred to as reactor systems or event-driven systems. The workers in the system respond to events that occur within the system, and these events may also come from the outside world or from other workers. The event can be an incoming HTTP request, or a file can be successfully loaded into memory medium. There are a lot of interesting reactor/event-driven platforms that can be used in writing this article, and there will be more in the near future. The more popular seems to be these few:

• Vert.x
• AKKa
• node. JS (JavaScript)

I personally think Vert.x is quite interesting (especially for those of me who use JAVA/JVM)

Actors and channels

Actors and channels are two similar pipelined (or reactor/event-driven) models.

In the actor model, each worker is called an actor. The actor can send and process messages directly between them asynchronously. Actors can be used to implement one or more job-processing lines as described earlier in this article. The actor model is given:

In the channel model, workers do not communicate directly with each other. Instead, they publish their own messages (events) in different channels. Other workers can listen to messages on these channels, and the sender does not need to know who is listening. The channel model is given:

In writing this article, the channel model seemed more flexible to me. A worker does not need to know who is working on a later assembly line. Just know which channel the job (or message, etc.) needs to be forwarded to. Listeners on the channel are free to subscribe or unsubscribe, without affecting the workers who send messages to this channel. This allows for a loose coupling between workers.

Advantages of pipelining Models

Compared to the parallel worker model, the pipelining concurrency model has several advantages, and in the following chapters I'll cover several of the biggest benefits.

No need to share a state

There is no need to share state between workers, which means that there is no need to consider all the concurrency problems resulting from concurrent access to shared objects. This makes it very easy to implement a worker. When implementing a worker it is as if a single thread is working-basically a single-threaded implementation.

Workers with status

When workers know that no other thread can modify their data, the worker can become stateful. For stateful, I mean, they can save the data they need to manipulate in memory, just in the end to rewrite it back to the external storage system. As a result, stateful workers tend to have higher performance than stateless workers.

Better hardware integration (Hardware conformity)

Single-threaded code often has a better advantage when it comes to consolidating the underlying hardware. First, it is often possible to create more optimized data structures and algorithms when the code can be determined to execute only in single-threaded mode.

Second, as described earlier, a single-threaded threads state can cache data in memory. While the data is cached in memory, it also means that the data is likely to be cached in the CPU that executes the thread. This makes accessing the cached data faster.

I say hardware consolidation means code written in some way that can naturally benefit from the workings of the underlying hardware. Some developers call it mechanical sympathy. I prefer the term hardware integration, because the computer has very few mechanical parts and can metaphor "better match better", and I think the word "conform" is very good when compared to the meaning of the word "sympathy" in the context of sympathy. Of course, it's a bit of a nit-picking, just use your favorite terminology.

Reasonable Order of operations

The concurrency system based on pipeline concurrency model is possible to ensure the order of the jobs in some way. The orderly nature of the job makes it easier to roll out the state of the system at a particular point in time. Further, you can write all the jobs you arrive to the log. Once a part of the system is hung up, the log can be used to rebuild the state of the system from the beginning. Jobs are written to the log in a specific order, and in this order as a guaranteed job order. Shows a possible design:

It is not easy to implement a guaranteed job sequence, but it is often feasible. If you can, it will greatly simplify some tasks, such as backup, data recovery, data replication, etc., which can be done through log files.

Disadvantages of pipelining Models

The biggest disadvantage of the pipelining concurrency model is that job execution is often distributed across multiple workers, and therefore distributed across multiple classes in a project. This makes it difficult to track exactly what code is being executed by a job.

Again, this makes coding difficult. Sometimes the worker's code is written in the form of callback processing. If too many callback handlers are embedded in the code, the so-called Callback Inferno (callback Hell) phenomenon often occurs. The so-called callback of hell means that it becomes very difficult to trace what the code did during the callback and to ensure that each callback only accesses the data it needs.

Using the parallel worker model can simplify this problem. You can open the code of the worker and read the executed code beautifully from beginning to end. Of course, the code for the parallel worker pattern may also be distributed in different classes, but it is often easy to parse the order of execution from the code.

Third, functional parallelism (functional Parallelism)

The third concurrency model is a function-parallel model, which is also a more recent (2015) discussion of a more model. The basic idea of function-parallel is to implement the program by function call. Functions can be thought of as "proxies" or "actors", and functions can send messages to each other like pipelining models (aka reactors or event-driven systems), agents. A function calls another function, which is similar to sending a message.

Functions pass parameters by copying, so no entity can manipulate the data except the receive function. This is necessary to avoid the race of sharing data. It also makes the execution of a function similar to an atomic operation. The execution of each function call is independent of the invocation of any other function.

Once each function call can be executed independently, they can be distributed across different CPUs. This also means that the algorithm can be executed in parallel on multiple processors using a function-based implementation.

The forkandjoinpool contained in the Java.util.concurrent package in Java7 can help us implement something similar to functional parallelism. In Java8, parallel streams can be used to help us iterate large collections in parallel. Remember that some developers have criticized Forkandjoinpool (you can see the link to criticism in my forkandjoinpool tutorial).

The hardest part of functional parallelism is determining the function call that needs to be parallelized. Cross-CPU coordination function calls require a certain amount of overhead. A function completes a unit of work that needs to reach a certain size to compensate for this overhead. If a function call is very small, parallelizing it may be slower than single-threaded, single-CPU execution.

I personally think (probably not quite right) that you can use a reactor or event-driven model to implement an algorithm that functions as a parallel approach to the decomposition of the work. Using the event-driven model gives you more precise control over how parallelization is implemented (my point of view).

In addition, the cost of reconciling a task when it is split to multiple CPUs is only meaningful if the task is the only task currently being performed by the program. However, if the current system is performing several other tasks (such as a Web server, a database server, or many other similar systems), it makes no sense to parallelize a single task. No matter what other CPUs in the computer are busy with other tasks, there's no reason to disrupt them with a slow, functional parallel task. Using the pipelining (reactor) concurrency model can be a bit better because it is less expensive (sequential execution in single-threaded mode) and better integrated with the underlying hardware.

Is it best to use that concurrency model?

So, what kind of concurrency model is better?

Usually, the answer depends on what your system intends to do. If your job itself is parallel, independent, and not necessarily shared, you might use a parallel worker model to implement your system. Although many jobs are not naturally parallel and independent. For this type of system, I believe that the use of pipelining concurrency model can better play its advantages, but also more than the parallel worker model advantage.

You don't even have to write the infrastructure of all pipelining models yourself. A modern platform like Vert.x has achieved a lot for you. I will also go to explore how to design my next project and make it run on an excellent platform like Vert.x. I feel that Java EE has no advantage.

Java multithreaded-concurrent programming model