flume-ng-1.4.0 spooling Source method adds support for recursive detection of directories

Because Flume Spooldir does not support recursive detection of subdirectory files, and the business needs, so modify the source code, recompile

Code modification Reference from: http://blog.csdn.net/yangbutao/article/details/8835563

In 1.4, however, the Spoolingfilelinereader class has not been modified, but apache-flume-1.4.0-src\flume-ng-core\src\main\java\org\apache\flume\ Client\avro\reliablespoolingfileeventreader.java

and the variable directory should be changed to SpoolDirectory

1     /*2 * @author admln3      * 4 * @date April 8, 2015 morning 9:37:205      */6     Private voidListdirfiles (list<file>files, File dir, filefilter filter) {7File[] Childs =Dir.listfiles (filter);8          for(inti =0; i < childs.length; i++) {9             if(Childs[i].isfile ()) {Ten Files.add (Childs[i]); One}Else { A                 if(Childs[i].isdirectory ()) { - listdirfiles (Files, childs[i], filter); -                 } the             } -         } -     } -  +     /** - * Find and open the oldest file in the chosen directory. If or more + * files are equally old, the file name with lower lexicographical value is A * returned. If The directory is empty, this would return an absent option. at      */ -     PrivateOptional<fileinfo>GetNextFile () { -         /*Filter to exclude finished or hidden files*/ -FileFilter filter =NewFileFilter () { -              PublicBoolean Accept (File candidate) { -String FileName =candidate.getname (); in                 if((Candidate.isdirectory ()) -||(Filename.endswith (completedsuffix)) to|| (Filename.startswith (".")) +||Ignorepattern.matcher (FileName). Matches ()) { -                     return false; the                 } *                 return true; $             }Panax Notoginseng         }; -         //list<file> candidatefiles = the         //Arrays.aslist (Spooldirectory.listfiles (filter)); +List<file> Candidatefiles =NewArraylist<file>(); AListdirfiles (Candidatefiles, spooldirectory, filter);

When you recompile, you can refer to: http://www.iteblog.com/archives/1032
Compile command:

MVN install-phadoop-2 -dskiptests-dtar

It's going to be a lot of different jar packs, wasting time.

Already compiled: HTTP://PAN.BAIDU.COM/S/1HQGV9VM g4w6

flume-ng-1.4.0 spooling Source method adds support for recursive detection of directories