Hbase partition split source code after reading

With the increase in data volume, more and more data is available for a single partition serve, and the performance will gradually get worse. At this time, partition split will be launched.

Hbase's partition split can be implemented on the client/hbaseadmin through the command line. in Java, its parameters include region and split pointer (the list partition in cmdele is more customizable and allows you to specify the PK list for partition, but generally it is not so fine)

After the hregionserver receives the partition split request, compactsplitthread. requestsplit will enable the thread to try to split (compaction and split look very similar and have a common entrance, but I think the two can be directly Combined ?), The specific process is as follows:

1. The client sends the split request to the region server.
2. After the region server receives the split request, it first checks the validity and selects the split point
A) Check whether split is allowed: if the file contains the reference file (meaning that the split has just occurred), re-split is not allowed.
B) If split is allowed, select the split point: select the store file with the largest size (equivalent to the youchao file), and then select its middle-key. The middle-key is not read once, instead, you can find it directly from the index of the file. items [length/2]
C) ensure that the status of the parent region is not close or closing, and that the split point is valid (min (key) <split point <max (key ))
3. Start the split process after checking validity and selecting the split point:
A) create an entry for Ephemeral on zookeeper to indicate that parent region is being splitting.
B) create a temporary split dir in the parent region folder.
C) Disable parent region (all memory stores (memory files) will be flushed and wait until the end of active compaction). From now on, parent region will not be available.
D) offline parent region from the local server. Each region server maintains a list of valid region. In this step, the parent region is removed from the list.
E) split all store files (youchao file). This step creates a reference file for each file. The reference file consists of two parts:
I. The first part is the path of the source file, and the second part is the reference of the new reference file to reference the source file split key and to reference the first or second part;
II. For example, if the source file is Table1/storefile.11 and the split point is key1, the split file may be Table1/storefile.11.bottom. key1,

Table1/storefile.11.up. key1 indicates that after storefile.11 is cut from key1, the two referenced files reference the lower half and upper half of the source file respectively.
F) create a child region:
I. set various attributes. For example, divide the access metrics of parent region to child region, half of each person.
II. move the Temporary Folder generated in the preceding parent folder (which contains the file reference for parent region) to the table directory. Now, at the directory level, child region is equal to parent region.
G) write the parent region split information to the system meta server and write the child region name together (the split state is persistent at the meta level)
H) open two child region nodes respectively. The steps are as follows:
I. Write the child region information to the meta server.
Ii. Load all store files and replay log
III. if the reference file is included, compaction (similar to merge) is performed until compact is completed for all reference files. here we can see that the parent region file will be split and written to each child regions.
(I) switch the status of the parent region from splitting to split. zookeeper will notify the master to start processing the split event, and the master will start offline parent region and online child regions.
J) after the worker waits for the master to complete processing, it is confirmed that all child regions are online and the split ends.

The entire split process can be divided into two major steps: one is to record the parent region information split into child regions into Meta; second, compaction is performed before child regions online until all reference files are filtered, so that the parent region folder can be deleted.

However, it divides all the store files into two copies. I think the write volume is large. In fact, parent region can be retained, and only one daughter region can be generated at a time, the invalid data in the parent region can be filtered out during major compaction.