Some New Discovery about Nutch

1 Nutch common is '

bin/nutch crawl <urlDir> [-dir d] [-threads n] [-depth i] [-topN]  ',

 

   nutch will generate segment for each depth ,and topN means each layer will collect topN urls. Generally each layer has one single segment,it depends on maxNumSegments(1 is the default value) in Generator.java,so we may find 'fetcher.fetch(segs[0]... '( from Crawl.java)  just use the first value of  segment array for each layer.

And I also find a variant maxCount , In my mind MaxCount is the maximum number of links from the same host can be stored in each segment ,if the value is -1 that means no  limit here.