Why hash is rarely used for large-scale data processing?

This article is reproduced in the set version of smth, made by Daniel pennyliang. I felt that the writing was really good, so I took it shamelessly.

 

First of all, from the database perspective, we know that the basic structure of MySQL is generally B + tree. Why isn't hash commonly used? From several aspects.

Generally, the data storage method can be abstracted.
Primay key, candiate key1, candidate key2,...-> value {field1, field2 ,...}

If we only support
Select field1 from table where primay key = xxx
Hash can be used for this purpose, and it may be better.

But what if it is such a query?
Select field1 from table where primay key> XX and key <YY
Obviously, it is difficult to deal with it by using a normal hash. We need to enumerate all the keys from xx to YY and try them one by one.
For example, when two tables overlap, using hash will also hurt a lot of unnecessary calculations.

Therefore, an issue is raised, and order preserving (Order preserved) is required. The hash function can be used as a perfect hash function to implement Order preserving. This is a brilliant discussion in the upcoming book mg, furthermore, a perfect hash function can only be used for static sets for long-term use, such as static dictionaries.

The B + tree solves this problem well. The values are on the leaf nodes and are sorted. Therefore, it is easy to process such queries, however, the B + tree requires the overhead of non-leaf nodes. There is a certain cost for insertion and deletion.

If you do not need to insert or delete an index at a time, the simplest method is to directly sort the index, that is, we can convert the previous structure.
Primary Key Value
Key1 V1
Key2 v2
Key3 v3
......
Keyn VN
Each key is sorted, which is very important. The other candidate keys are also similar. Each index is represented by an index file.

If the value is huge, the offset of the value on the hard disk can be stored here (in Linux, you can also directly write the device and directly use the device offset to represent the data location)

Sorting creates good conditions for optimization. For example, we need to perform a query like this:
Select table1_key, table2_key, table1_value from Table1, Table2
Where table1_key = table2_key
Because the two tables are sorted by key, so that the two ordered indexes can be fast, only O (min (length (Table1), length (table2) is required )) time complexity.

The sorting method is also very advantageous in processing duplicate keys. It only puts the location of the same key together, And the hash also needs to handle this situation (note that this is not a conflict, for example, if the student name is used as the candidate index, the same name may occur ).

In addition, the sorting method is the most cost-effective, with almost no waste.