Java: Parallel Programming and synchronous use

• Know that Java can be used under the Java.util.concurrent package

Countdownlatch

Executorservice

Future

Callable

It is very simple to implement parallel programming and to synchronize with parallel threads.
Implementation principle:

1. Countdownlatch counts the number of parallel thread finishes and provides an await () method that waits for all parallel threads to complete, or specifies the maximum wait time.
2. Executorservice provides execute (callable) execution threading method, and a submit (callable) commit thread is provided.
3. The future accepts the return value of the Callable<v> interface (executable thread) and accepts the Executors.submit (callable<v>) return value. and Future<v> provides get () to retrieve the parameters returned by the parallel sub-thread, and can also specify an expiration time for get.

Think of Concurrent, you can think of C #, namespace System.collection,concurrent, which provides some thread-safe collection classes under that namespace.

How to use:

Define the executable thread class:

 Public classUploadfiletotaskImplementsCallable<uploadfiletotaskresult> {    Private FinalTask_uploadfiletotaskitem TaskItem; Private FinalLog log = Loghelper.getinstance (Importmain.class); Private FinalCountdownlatch threadssignal; Private FinalHdfsutil Hdfsutil =NewHdfsutil (); Private Final StaticString Hadoop_hdfs_path =Hdfsconfiguration.gethdfsurl ();  PublicUploadfiletotask (Countdownlatch threadssignal, Task_uploadfiletotaskitem taskItem) { This. taskitem=TaskItem;  This. threadssignal=threadssignal; } @Override PublicUploadfiletotaskresult Call ()throwsException {String area=Taskitem.getarea (); String filegeneratedate=taskitem.getfilegeneratedate (); String manufacturer=Taskitem.getmanufacturer (); String Enodebid=taskitem.getenodebid (); String FilePath=Taskitem.getfilepath (); FileType FileType=Taskitem.getfiletype (); TaskStatus TaskStatus=taskstatus.success; //It is not certain that the filesystem is thread-safe, so it is initialized at each thread. Configuration conf =NewConfiguration (); Path Dstpath=NewPath (Hadoop_hdfs_path); FileSystem HDFs=dstpath.getfilesystem (conf); //Core Code ... //uploading an Mr File//Upload Signal file//If the file path is not empty, start uploading files to HDFs        if(Uploadfilepath.length () >0){            if(!Hdfsutil.uploadfiletohdfs (HDFs, FilePath, Uploadfilepath)) {TaskStatus=Taskstatus.fail; }} taskgroupinfo Taskgroupinfo=NewTaskgroupinfo ();        Taskgroupinfo.setarea (area);        Taskgroupinfo.setmanufacturer (manufacturer);        Taskgroupinfo.setfilegeneratedate (filegeneratedate);        Taskgroupinfo.setfiletype (FileType); String Key= String.Format ("%s,%s,%s,%s", Taskgroupinfo.getarea (), Taskgroupinfo.getmanufacturer (), Taskgroupinfo.getfilegeneratedate (),        String.valueof (Taskgroupinfo.getfiletype (). GetValue ())); Uploadfiletotaskresult result=NewUploadfiletotaskresult (); //padding return valueResult.setstatus (TaskStatus);        Result.settaskgroupinfo (Taskgroupinfo);        Result.settaskgroupkey (key);        Result.settaskoid (Taskitem.getoid ()); System.out.println ("Task ID:" + taskitem.getoid () + ">>>> Thread Name:" + Thread.CurrentThread (). GetName () + "end. and "+ threadssignal.getcount () +" Threads "); //You must wait until the core processing logic is complete before you can subtract 1         This. Threadssignal.countdown (); returnresult; }}

• Implementing the parallel thread synchronization core code:

            //get current node with execute taskarraylist<task_uploadfiletotaskitem> TaskItems = Uploadfiletotaskitemdao.gettopntodotaskitems ( This. Computenode.getid (), Configuration.gettaskcount ());            //Bulk Modify task status is processing state (doing). Log.info ("Start:>>>>>>batch Modify task status (doing) >>>>>>");Log.info ("Over:>>>>>>batch Modify task status (doing) >>>>>>"); //Batch upload tasks (upload files to)Log.info ("Start:>>>>>>each Process task (upload to) >>>>>>");
Countdownlatch threadssignal=NewCountdownlatch (Taskitems.size ()); Executorservice Executor=Executors.newfixedthreadpool (Taskitems.size ()); List<Future<UploadFileToTaskResult>> resultlazyitems=NewArraylist<future<uploadfiletotaskresult>>();  for(Task_uploadfiletotaskitem taskitem:taskitems) {//using the future storage sub-thread to return results after execution, you must use Get () after all child threads have completed; //If you use Get () here, it will cause waiting to be synchronized. future<uploadfiletotaskresult> future = Executor.submit (NewUploadfiletotask (Threadssignal,taskitem));            Resultlazyitems.add (future); }            //wait for all parallel child thread tasks to complete. threadssignal.await ();
Executor.shutdown (); //is not to terminate the running of the thread, but to disallow the addition of new tasks in this executor log.info ("Over:>>>>>>each Process Task (upload to) >>>>>>"); //Bulk Modify task processing statusMap<string, taskgroupinfo> taskgroupitems=NewHashmap<string, taskgroupinfo>(); Map<integer, taskstatus> successtaskitems =NewHashmap<integer, taskstatus>(); Map<integer, taskstatus> failtaskitems =NewHashmap<integer, taskstatus>();  for(future<uploadfiletotaskresult>Future:resultlazyitems) {Uploadfiletotaskresult result=Future.get (); if(!Taskgroupitems.containskey (Result.gettaskgroupkey ()))                {Taskgroupitems.put (Result.gettaskgroupkey (), Result.gettaskgroupinfo ()); }                if(Result.getstatus () = =taskstatus.success)                {Successtaskitems.put (Result.gettaskoid (), Result.getstatus ()); }Else{failtaskitems.put (Result.gettaskoid (), Result.getstatus ()); }            }

• Resources:

http://blog.csdn.net/wangmuming/article/details/19832865

Http://www.importnew.com/21312.html

Java: Parallel Programming and synchronous use