Java版本的BloomFilter (布隆過濾器)

一般比較常見的應用是字串去重..也就是採集網址去重.防止重複採集

Java代碼
 

1. BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream("D:\\Users\\caiqing\\workspace\\CQ\\library\\dictionary-utf8.TXT"),"UTF-8")) ;
     
2.         String str = null ;   
3.         System.out.println("begin");   
4.         long start = System.currentTimeMillis() ;   
5.         while((str=br.readLine())!=null){   
6.             if(bf.containsAndAdd(str)){   
7.                 System.out.println("containsAndAdd:"+str);   
8.             }   
9.         }   
10.            
11.         br.close() ;   
12.            
13.         br = new BufferedReader(new InputStreamReader(new FileInputStream("D:\\Users\\caiqing\\workspace\\CQ\\library\\dictionary-utf8.TXT"),"UTF-8")) ;
      
14.             System.out.println("begin-find");   
15.             start = System.currentTimeMillis() ;   
16.             while((str=br.readLine())!=null){   
17.                 if(!bf.contains(str)){   
18.                     System.out.println("contains:"+str);   
19.                 }   
20.             }   
21.                
22.         System.out.println(System.currentTimeMillis()-start);   
23.         br.close() ;