Hubble. net search engine Analysis 1

As I only search for this knowledge, I only analyze Hubble. Net's internal search knowledge. I am a hobbyist, so the level is limited. I hope you can give me more advice.

Note: I analyzed the code of version 0 [1]. 7.2.0-Src.

 

1. Hubble. Net File Information

Hubble. Net is used with databases to build a search system. Therefore, Hubble. Net only records the index information of the document, and the content of the document is recorded in the database. The Hubble. Net document information is as follows:

1. *. hdx file: records the index information corresponding to the index word. For example, the index word, the position and length in the corresponding. idx file.

Data format: size | postion | length | word size contains the size of postion, length, and word

2. *. idx file: records the information of index words in the document. Such as the Document ID, the number of times the current index word appears in the document (count), its value and Data. equal length, record the location list (Data) of the current index word in the document, and record the document's Rank value.

Data format: Data. Length | DocId | Count | Rank | Data

Note:. hdx and. idx appear in pairs .. Idx records the index word information in the document, such as the number, location, and Rank .. Hdx records the location and length of index words and index information in the idx file. File Name structure: {d: 7} + Serial.

3. payload. DB file: records the table data column information, document ID, and the number of index words in each column. The data column information includes the field. Name, field. tabindex, field. datatype, field. datalength, and field. indextype of each column. Data records the number of index words in each column.

Data format: field. name + field. tabindex + field. datatype + field. datalength + field. indextype (UNICODE MD5 encryption) | Document ID | data (number of indexed words) | .....

Note: The number of indexed words equals to the number of words in each field. Different value methods vary based on the field index type. The untokenized type stores the data value itself, and the tokenized type stores the number of indexed words after data value segmentation.

4. tableinfo. xml file: Stores Table configuration information. Such as table name, connectionstring, database table name (dbtablename), data column information (fields), _ dbadaptertypename, _ forcecollectcount (number of documents for creating index cache), _ indexonly (whether the index is read-only), _ querycacheenabled (whether query cache is supported), _ querycachetimeout (timeout), and _ sqlforcreate (create an SQL statement ). Use XML serialization to store table configuration information.

5. Delete. DB files: stores the list of deleted documents.

Data format: docid | docid ....