Simple implementation of inverted indexes and simple implementation of inverted Indexes

Source: Internet
Author: User

Simple implementation of inverted indexes and simple implementation of inverted Indexes
Simple implementation of inverted index
Inverted index is a common algorithm in search engines. It is mainly used to implement full text searching and to establish mappings between keywords and documents. Many powerful functions are built on this basis, for a detailed description of Inverted Index, see Wikipedia. The following is based on your own ideas, just to understand the operation of this data structure. Todo: If you want to search for a complete sentence such as "what is it", you can search for these words separately and then see the result of the continuous location of the same file. This is a set operation.
Package mythought. invertedindex;Import java. io. BufferedReader;Import java. io. FileReader;Import java. util. HashMap;Import java. util. HashSet;Import java. util. Map;Import java. util. Set;Public class InvertedIndex {// Key word <----> doc filePrivateMap <String, Set <String> indexs =NewHashMap <String, Set <String> (); // It is assumed that all files are small files.Public VoidAddFile (String fileName, String content) {String [] words = content. split ("");For(IntI = 0; I <words. length; I ++) {String word = words [I]; // only record first appeared position Set <String> wordIndex = indexs. get (word );If(WordIndex =Null) {WordIndex =NewHashSet <String> (); indexs. put (word, wordIndex);} wordIndex. add ("(" + fileName + "," + I + ")");}}Public VoidAddFile (String fileName)ThrowsException {BufferedReader br =NewBufferedReader (NewFileReader (fileName ));Try{StringBuilder sb =NewStringBuilder (); String line = br. readLine ();While(Line! =Null) {Sb. append (line); sb. append (""); // each line is directly connected to line = br. readLine ();}This. AddFile (fileName, sb. toString ());}Catch(Exception e) {e. printStackTrace ();}Finally{Br. close ();}}PublicSet <String> search (String keyword) {Set <String> results = indexs. get (keyword );Return NewHashSet <String> (results );}Public Static VoidMain (String [] args)ThrowsException {InvertedIndex test =NewInvertedIndex (); test. addFile ("file1", "hello fuck world todotodo"); test. addFile ("file2", "go to get it if you want it"); test. addFile ("C:/data/hello.txt"); System.Out. Println (test. search ("it"); System.Out. Println (test. search ("you"); System.Out. Println (test. search ("vonzhou "));}}
Running result: [(file2, 7), (file2, 3)] [(file2, 5)] [(C:/data/hello.txt, 4)]
Reference: 1.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.