concurrency algorithm for file directory traversal

Problem: Calculate the size of the file under the specified directory.

This is a very simple question, just do a recursive on the line, the order algorithm:

     Public LongGetFileSize (Finalfile file) {        if(File.isfile ()) {returnfile.length (); }        Else{            LongTotal = 0; File files[]=File.listfiles (); if(Files = =NULL)return0;  for(File f:files) { Total+=GetFileSize (f); }            returnTotal ; }    }

Very simple, a recursive implementation, so now we think about the concurrency algorithm

Concurrency ideas: Each time a recursive operation, each time a thread to calculate the current directory file size, and then perform a recursive operation

Concurrency code:

Private LongGETFILESIZEBF (FinalExecutorservice Service,FinalFile file)throwsinterruptedexception, executionexception, timeoutexception{if(File.isfile ())returnfile.length (); Else{            FinalFile files[] =File.listfiles (); if(Files = =NULL)                return0; Else{                LongTotal = 0; Finallist<future<long>> tasks =NewArraylist<future<long>>();  for(FinalFile F:Files) {Tasks.add (Service.submit (NewCallable<long>() {@Override PublicLong Call ()throwsException {//TODO auto-generated Method stub
Recursively, the file size of the directory is returned
                            returnGETFILESIZEBF (service, F);                }                    })); }                 for(FinalFuture<long>fl:tasks) { Total+=Fl.get (); }                returnTotal ; }        }    }

There seems to be no problem, we come to the actual test;

We see, call get from the future to fetch data from the time, and no set timeout, the actual operation found that when the folder directory structure is simple, the directory tree is relatively shallow when it can run out, but the structure of the directory tree complex, ran for a long time or run out of results.

Analysis: Each thread will open a new thread to calculate the size of the sub-directory, this time, the thread will go to wait, waiting for the return of the child thread, this time there will be a situation, if the structure of the directory tree is complex, then many threads will enter the waiting state, waiting for the return value of the child thread, However, these parent threads still occupy the thread pool, but the child threads do not request a thread pool to execute, and this time they enter a deadlock .

Such as:

Optimization strategy: 1. Since the poolsize of the thread pool is not enough, then we will increase the size of poolsize, well, now we are facing a directory structure is not so complex, by simply adding poolsize can also be done, but very complex words will not be able to.

2. The reason we are creating

Concurrency algorithm for file directory traversal