Transferred from: http://www.cnblogs.com/nexiyi/p/hbase_config_94.html
The goal is to look at the configuration of the production environment and the default configuration.
Hbase.hregion.max.filesize:100ghbase.regionserver.hlog.blocksize:512mhbase.regionserver.maxlogs:32
.............
Version: 0.94-cdh4.2.1hbase-site.xml configuration Hbase.tmp.dir
- The local file system TMP directory, generally configured as local mode settings, but it is best to set up, because many files will be set by default to the following
- On-line configuration
< property > < name >hbase.tmp.dir</ name > < value >/mnt/dfs/11/hbase/hbase-tmp</ value > </ property > |
- Default value:
${java.io.tmpdir}/hbase-${user.name} |
Write to the system's/tmp directory
Hbase.rootdir
- All Regionserver shared Directories in the HBase cluster, used to persist hbase data, are typically set up in HDFs file directories, such as Hdfs://namenode.example.org:9000/hbase
- On-line configuration
< property > < name >hbase.rootdir</ name > < value >hdfs://mycluster/hbase</ value > </ property > |
- Default value:
hbase.cluster.distributed
- Cluster mode, distributed or standalone mode, if set to False, the hbase process and the zookeeper process are in the same JVM process.
- True on-line configuration
- Default value: False
Hbase.zookeeper.quorum
- Zookeeper the URL configuration of the cluster, separated by commas (,) between multiple hosts
- On-line configuration
< property > < name >hbase.zookeeper.quorum</ name > < value >inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org</ value > </ property > |
- Default value: localhost
Hbase.zookeeper.property.dataDir
- The configuration in the Zookeeper zoo.conf. Where the snapshot is stored
- On-line configuration:/home/hadoop/zookeeperdata
- Default value: ${hbase.tmp.dir}/zookeeper
Zookeeper.session.timeout
- Client-to-ZK connection time-out
- On-line configuration: 1200000 (20min)
- Default value: 180000 (3min)
Hbase.zookeeper.property.tickTime
- The time interval between client side and ZK sending heartbeat
- On-line configuration: 6000 (6s)
- Default value: 6000
Hbase.security.authentication
- HBase cluster security authentication mechanism, the current version only supports Kerberos security authentication.
- On-line configuration: Kerberos
- Default value: Empty
Hbase.security.authorization
- Whether HBase opens a security authorization mechanism
- On-line configuration: True
- Default value: False
Hbase.regionserver.kerberos.principal
- Regionserver principal name for Kerberos authentication (consists of three parts: service or user name, instance name, and domain name)
- On-line configuration: Hbase/[email protected]xxx.xxx.com
- Default: None
Hbase.regionserver.keytab.file
- Regionserver keytab file path
- On-line configuration:/home/hadoop/etc/conf/hbase.keytab
- Default value: None
Hbase.master.kerberos.principal
- Principal name of Kerberos authentication for master (consisting of three parts: service or user name, instance name, and domain name)
- On-line configuration: Hbase/[email protected]xxx.xxx.com
- Default: None
Hbase.master.keytab.file
- Master keytab file path
- On-line configuration:/home/hadoop/etc/conf/hbase.keytab
- Default value: None
Hbase.regionserver.handler.count
- Regionserver the number of threads processing IO requests
- On-line configuration: 50
- Default configuration: 10
Hbase.regionserver.global.memstore.upperLimit
- Regionserver process block Flush trigger condition: Memstore of all region on the node reaches upperlimit*heapsize
- On-line configuration: 0.45
- Default configuration: 0.4
Hbase.regionserver.global.memstore.lowerLimit
- A condition for the regionserver process to trigger flush: the sum of all region Memstore on the node reaches lowerlimit*heapsize
- On-line configuration: 0.4
- Default configuration: 0.35
Hbase.client.write.buffer
- Client writes buffer, setting AutoFlush to False when client writes full buffer to flush
- On-line configuration: 8388608 (8M)
- Default configuration: 2097152 (2M)
Hbase.hregion.max.filesize
- The region size of a single columnfamily, and if you follow the Constantsizeregionsplitpolicy policy, the value set above is automatically split
- On-line configuration: 107374182400 (100G)
- Default configuration: 21474836480 (20G)
Hbase.hregion.memstore.block.multiplier
- More than multiples of memstore size reach this value then block all write requests, self-protection
- On-line configuration: 8 (the memory is large enough to be properly resized, this situation requires the client to make adjustments)
- Default configuration: 2
Hbase.hregion.memstore.flush.size
- Memstore size, when this value is reached, it is flush to the external memory device
- On-line configuration: 104857600 (100M)
- Default value: 134217728 (128M)
Hbase.hregion.memstore.mslab.enabled
- Whether to turn on the Mslab scheme to reduce the full GC due to memory fragmentation and improve overall performance
- On-line configuration: True
- Default configuration: True
Hbase.regionserver.maxlogs
- Regionserver Number of Hlog
- On-line configuration: 128
- Default configuration: 32
Hbase.regionserver.hlog.blocksize
- Hlog size limit, reach this value block, roll off
- On-line configuration: 536870912 (512M)
- Default configuration: Block size for HDFS configuration
Hbase.hstore.compaction.min
- Minimum number of storefiles to enter the minor compact queue
- On-line configuration: 10
- Default configuration: 3
Hbase.hstore.compaction.max
- The maximum number of files in a single minor compact
- On-line configuration: 30
- Default configuration: 10
Hbase.hstore.blockingStoreFiles
- When the number of storefile in a region reaches this value, Block writes, waiting for the compact
- On-line configuration: 100 (production environment can be set very large)
- Default configuration: 7
Hbase.hstore.blockingWaitTime
- Block's wait Time
- On-line configuration: 90000 (90s)
- Default configuration: 90000 (90s)
Hbase.hregion.majorcompaction
- Cycles that trigger the major compact
- On-line configuration: 0 (switch off the major compact)
- Default configuration: 86400000 (1d)
Hbase.regionserver.thread.compaction.large
- Large the number of threads in the compact thread pool
- On-line configuration: 5
- Default configuration: 1
Hbase.regionserver.thread.compaction.small
- Small the number of threads in the compact thread pool
- On-line configuration: 5
- Default configuration: 1
Hbase.regionserver.thread.compaction.throttle
- Compact (major and minor) requests to enter the critical point of the large and small compact thread pool
- On-line configuration: 10737418240 (10G)
- Default configuration: 2 * this.minfilestocompact * this.region.memstoreFlushSize
Hbase.hstore.compaction.max.size
- Minor Compact queue storefile file Max size
- On-line configuration: 21474836480 (20G)
- Default configuration: Long.max_value
Hbase.rpc.timeout
- RPC Request Timeout time
- On-line configuration: 300000 (5min)
- Default configuration: 60000 (10s)
Hbase.regionserver.region.split.policy
- Split Operation Default Policy
- On-line configuration: Org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy (take the old strategy, own control split)
- Default configuration: Org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy (if the region does not reach MaxFileSize, if FileSize reaches Regio Ncount * Regioncount * flushsize split operation)
Hbase.regionserver.regionSplitLimit
- Maximum number of region on a single regionserver
- On-line configuration: 150
- Default configuration: 2147483647
HBASE-ENV.SH configuration Specifies the system operating environment
export JAVA_HOME= /usr/lib/jvm/java-6-sun/ #JDK HOME export HBASE_HOME= /home/hadoop/cdh4/hbase-0 .94.2-cdh4.2.1 # HBase 安装目录 export HBASE_LOG_DIR= /mnt/dfs/11/hbase/hbase-logs #日志输出路径 |
Tuning JVM Parameters
export HBASE_OPTS="-verbose:gc -XX:+PrintGCDetails -Xloggc:${HBASE_LOG_DIR} /hbase-gc .log -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime \ -server -Xmx20480m -Xms20480m -Xmn10240m -Xss256k -XX:SurvivorRatio=4 -XX:MaxPermSize=256m -XX:MaxTenuringThreshold=15 \ -XX:ParallelGCThreads=16 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:CMSFullGCsBeforeCompaction=5 -XX:+UseCMSCompactAtFullCollection \ -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSMaxAbortablePrecleanTime=5000 \ " |
HBase parameter Configuration and description