a simple implementation of inverted indexes
Inverted index is a commonly used search engine algorithm, mainly used to achieve full text searching, the establishment of keywords and documents in the mapping relationship, a lot of powerful functions are built on this basis, about inverted index detailed description can see Wikipedia. The following is done according to your own ideas, just to understand the workings of this data structure. TODO: If you want to search the full occurrence of a sentence, such as "What is it" can be searched for these words and then see the same file in the continuous location of the results can be, set operation.
package Mythought.invertedindex;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
public class Invertedindex { //key word <----> doc file
Private map<string, set<string>> indexs =
new hashmap<string, set< String>> (); //This is assumed to be small files Public
void addfile (string fileName, string content) {string[] words = content. Split ("");For
(
int i = 0; I < words. length; I+ +) {String Word = words[i]; //Only record first appeared position set<string> Wordindex = indexs. Get (word);
if (wordindex = =
null) { Wordindex =
new hashset<string> (); indexs. Put (word, wordindex); } wordindex. Add ("+fileName + ", " + I+")"); } } Public
void addfile (String fileName)
throws exception{BufferedReader br =
new bufferedreader (
new filereader (fileName));
try {StringBuilder SB =
new StringBuilder ();String line = br. ReadLine ();while (line!
=
null) { sb. Append (line); sb. Append (""); //per line direct connection Line = br. ReadLine (); }This
. AddFile (fileName, sb. toString ());}
Catch(Exception e) { e. Printstacktrace ();}
finally { BR. Close (); } }Public
set<string> search (String keyword) {set<string> Results = indexs. Get (keyword);
return
new hashset<string> (results); }Public
static
void main (string[] args)
throws exception{ invertedindex Test =
new invertedindex (); test. AddFile ("File1", "Hello Fuck World Todotodo"); test. AddFile ("File2", "go to get it if you want It"); test. AddFile ("C:/data/hello.txt");System. Out. println (test. Search ("it"));
System. println (test. Search ("You"));
System. Out. println (test. Search ("Vonzhou"));
}}
Operation Result:[(file2,7), (file2,3)][(file2,5)][(c:/data/hello.txt,4)]
Reference: 1.http://en.wikipedia.org/wiki/inverted_index
A simple implementation of inverted indexes
