Thread granularity division, depth placement, and thread granularity in the thread pool
Abstract: multi-threaded task processing is very helpful for improving performance. The thread pool provided in Java also facilitates the implementation of multi-threaded tasks. It is easy to use, and if you use it incorrectly, the code will be messy. So how can I use it correctly, as shown in the following example? Have you got this skill?
Key words: multithreading Thread Pool Database Algorithm
Solve the problem: how to correctly use the thread pool.
As we all know, thread pools are very common in Java, and using them is also the most basic skill. However, we need to summarize how to use the thread pool more reasonably and conveniently.
The following describes the basic usage of the thread pool.
ExecutorService jobPool = Executors.newFixedThreadPool(10);while(true){Job_anqi job_anqi = new GetData_anqi();job_anqi.setParm(parm);jobPool.submit(job_anqi);}From the code above, we can see that there are many jobs that we don't want to complete in order and there is no relationship between them. Therefore, "distributed jobs", mapReduce? It's still too early. It's too easy to get involved. Let's finish the single-host multi-task first. Therefore, we have created a thread pool that constantly fills in the job in the loop. This thread pool is not large and can only accommodate 10 threads to run simultaneously, after other threads are put in, they have to wait in line honestly. Of course, this is just a simple Demo. Although it contains such a thing as "passing parameters to various threads", it is more complicated to get the return values of various thread results in the main thread ".
ExecutorService executorService = Executors. newCachedThreadPool (); List <Future <String> resultList = newArrayList <Future <String> (); // create 10 tasks and execute for (int I = 0; I <10; I ++) {// use ExecutorService to execute Callable tasks, and save the results in the future variable Future <String> future = executorService. submit (newTaskWithResult (I); // store the task execution result to resultList in the List. add (future);} executorService. shutdown (); // traverse the task result for (Future <String> fs: resultList) {try {System. out. println (fs. get (); // print the execution results of each thread (task), and occasionally throw an exception} catch (InterruptedException e) {e. printStackTrace ();} catch (ExecutionException e) {executorService. shutdownNow (); // when an exception occurs to a thread, disable thread pool e. printStackTrace (); return ;}}
The above Code not only uses the multi-thread method to obtain the result value, but also has another function: When executing a job, if it encounters an exception, all other threads running on this thread pool will be closed together. This is sometimes useful. So far, the above discussions are still focused on how to use the thread pool. However, the current problem is not how to use the thread pool, it is too low, no. The question is how to control the depth of a job and how to determine the depth of a job. When you have many tasks to be placed in the thread pool, they may have different organizational forms and different implementation methods. For example: 1. Threads can be divided by day, one thread can be started every day, and several threads can be left in the same day. 2. At the same time, threads can also be divided by data, each data form starts a thread, and how many data forms are used to open multiple threads. 3. Furthermore, threads are simply divided by granularity, I package every one thousand data batches into one thread, regardless of the day, regardless of the data form, but each thousand pieces of data is a thread to run; 4, or even, you can set a thread pool in the thread pool, every day, one thread opens a thread for each data form in the daily data form, and then each thousand pieces of data is run by one thread. This is a complex scenario. So how should we choose which thread division method to use? Let's analyze it slowly. Solution 3 is the most frequently remembered thread pool division method. This method divides the thread granularity to the finest, And the thread pool is placed in the deepest and most convenient position. Of course, theoretically, solution 3 will be executed in the same way as other solutions. After all the tasks are executed, the same results can be obtained as those in other solutions. However, once we have special requirements, it is not practical. If you need to print logs or record the execution Status of programs, for example, if you want to record the execution of tasks on the day by day, it is very suitable to use solution 1. If the performance is not good yet, you can also select solution 4. Because in both solution 1 and solution 4, you can know when the task of the day is completed. If solution 2 is used, the daily task execution status is unclear, so the record cannot be implemented. solution 3 is not implemented because its thread pool is too deep. To record the execution status of the task, you need to control the thread pool. It is impossible to submit all the tasks to the thread pool. The following is the most common code segment that monitors whether all threads in the thread pool have been executed.
ExecutorService jobPool = Executors. newFixedThreadPool (10); while (true) {Job_anqi job_anqi = newGetData_anqi (); job_anqi.setParm (parm); jobPool. submit (job_anqi);} pool. shutdown (); try {while (! JobPool. isTerminated () {Thread. sleep (1000) ;}} catch (InterruptedException e) {e. printStackTrace ();} logger.info ("angel wang finished all the work, Good job! ");
We can see that after all the jobs are submitted, we need to initiate a while loop to listen to this thread pool and ask every second if all the threads are running. After receiving a positive response, you can say, "angel wang has finished all the work, Good job! . To sum up, it is very important to create a thread pool. It directly determines the thread granularity, and you need to control the thread granularity. PS: java. SQL. Connection is non-thread-safe, and operations on the same Connection instance cannot be performed between threads. Otherwise, operations on the database will be lost. However, you should also note that if you need to start multiple threads to operate on the same database, each thread must use its own Connection instance to connect to and operate the database. If no restrictions are imposed, therefore, the number of connections established for the database will be too large, resulting in an unsuccessful connection. Therefore, you can either configure the maximum number of connections that can be created in the database connection pool, or control the number of created threads. It is much easier to configure the connection pool. This is also one of the meanings of the database connection pool, such as Druid, DBCP, and C3P0. I recently changed my city, changed my job, and joined Internet companies. I encountered many problems that I had never encountered before, technical, management, and communication ....... After all, it's really difficult for a family to learn. However, it is difficult for us to hide the water and solve any problems.
From Wang Anqi
