Hbase authoritative guide: storefile Merge (compaction)

????????? Hbase needs to merge these small storefiles into a relatively large storefile when necessary to prevent too many small files from being flushed to the disk menstore to ensure query efficiency, this process is called compaction. In hbase, there are two types of compaction: minor? Compaction and maj

????????? Hbase ensures query efficiency to prevent too many small files (menstore flushed to disk, hbase needs to merge these small store files into a relatively large store file when necessary. This process is called compaction. In hbase, there are two types of compaction: minor? Compaction and maj

????????? Hbase ensures query efficiency to prevent too many small files (menstore flushed to disk, hbase needs to merge these small store files into a relatively large store file when necessary. This process is called compaction. In hbase, there are two types of compaction: minor? Compaction and major compaction.

????????? The major compaction function combines all the store files into one. The possible conditions for triggering the major compaction include the major_compact command, the majorCompact () API, and the region server Automatic Running (related parameter: hbase. hregion. by default, majoucompaction is 24 hours and hbase. hregion. majorcompaction. jetter defaults to 0.2 to prevent the region server from performing major compaction at the same time ). Hbase. hregion. majorcompaction. the jetter parameter is used for hbase. hregion. the value specified by majoucompaction is floating. If both parameters are default values 24 and 19.2, the final value used by major compact is ~ The value range is 28.8.

???????? The running mechanism of minor compaction must be more complex. It is determined by several parameters:

?????????????????? Hbase. hstore. compaction. min: The default value is 3, indicating that minor compaction starts only when at least three store files meet the conditions are required.

?????????????????? The default value of hbase. hstore. compaction. max is 10, indicating that up to 10 store files can be selected in one minor compaction.

?????????????????? Hbase. hstore. compaction. min. size indicates that the store file with the file size smaller than this value must be added to the store file of minor compaction.

?????????????????? Hbase. hstore. compaction. max. size indicates that store files whose file size is greater than this value will be excluded by minor compaction.

?????????????????? Hbase. hstore. compaction. ratio sorts store files by file age (older to younger). minor compaction is always selected from the older store file if the file size is smaller than hbase. hstore. compaction. if the sum of the max store file sizes is multiplied by the ratio value, the store file will also be added to the minor compaction.

??????? If you do not understand the minor compaction process, you can see the source code about minor compaction in hbase, or: http://blog.csdn.net/macyang/article/details/6423078

Author: azhao_dn posted on 16:05:52 Original article link

Read: 25 comments: 0 view comments

Original article address: hbase authoritative guide: store file Merge (compaction). Thank you for sharing it.