An explanation of HBase configuration items

Hbase.tmp.dir: Temporary directory for the local file system, default isJAva io tm pd ir / hb as e− Java.io.tmpdir/hbase−{user.name};

Hbase.rootdir:hbase Persistent directory, is shared by all regionserver, default ${hbase.tmp.dir}/hbase, generally set to hdfs://namenode.example.org:9000/hbase similar, with fully qualified name;

Hbase.cluster.distributed:hbase cluster mode operation or not flag, default is False, open need to set to True,false when starting HBase will run HBase and ZK in a JVM;

Hbase.zookeeper.quorum: Important also must be set, start ZK server list, comma separated, cluster mode must be set, default is Localhost,hbase client also need to set this value to access ZK;

Hbase.local.dir: Local file system is used in local storage directory, default ${hbase.tmp.dir}/local/;

Hbase.master.port:hbase Master bound port, default is 60000;

Hbase.master.info.port:hbase master Web interface port, default is 60010, set to 1 can disable UI;

The binding address of the Hbase.master.info.bindAddress:master Web interface, default is 0.0.0.0;

Hbase.master.logcleaner.plugins: The list of plugins for the cleanup log, comma delimited, logcleanerdelegate that are called by Logservice, can be customized, executed sequentially, clean up Wal and Hlog; defaultorg.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner

Hbase.master.logcleaner.ttl:HLog the longest time to live in the. Oldlogdir directory, which is expired by the master thread collection, which defaults to 600000;

Hbase.master.hfilecleaner.plugins:HFile List of cleanup plug-ins, comma separated, called by Hfileservice, can be customized, default Org.apache.hadoop.hbase.master.cleaner.TimeTo Livehfilecleaner

Hbase.master.catalog.timeout:Catalog Janitor time-out from master to Meta, we know that this janitor is timed to go to the Meta Scan Table directory to decide to reclaim useless regions, Default is 600000;

Fail.fast.expired.active.master: If master expires, then no need to recover from ZK, direct termination, default is false;

The DNS interface of Hbase.master.dns.interface:master, which provides IP to the interface, defaults to default;

The DNS hostname or IP used by the hbase.master.dns.nameserver:master is default;

Hbase.regionserver.port:regionserver bound port, default is 60020;

Hbase.regionserver.info.port:regionserver Web Interface port,-1 Cancel interface, default is 60030;

Hbase.regionserver.info.bindAddress:regionserver Web binding, default is 0.0.0.0;

Hbase.regionserver.info.port.auto:master or regionserver whether to automatically search the bound port, the default is false;

Number of RPC listener on Hbase.regionserver.handler.count:regionserver, http://kenwublog.com/ Hbase-performance-tuning This configuration is called the number of IO threads, in fact, the same, that is, in the Regionserver last processing RPC handler, the default is 30;

Hbase.regionserver.msginterval:regionserver the interval of messages to master, default 3000 milliseconds;

Hbase.regionserver.optionallogflushinterval: If there is not enough entry to trigger the synchronization, then after this interval hlog will be synchronized to HDFs, the default is 1000 milliseconds;

Hbase.regionserver.regionSplitLimit:regionsplit the maximum limit, the default is max_int=2147483647, after setting this limit, when the limit is reached, the region split will not be carried out;

Hbase.regionserver.logroll.period: No matter how many versions, directly roll off the commit log cycle, that is, a fixed period of time, the roll expires, the default is 3600000 milliseconds;

Hbase.regionserver.logroll.errors.tolerated: The number of acceptable Wal shutdown errors that will trigger the server to terminate when it arrives; set to 0 then stop the region when Wal writer does log rolling failure Server, default is 2;

HBASE.REGIONSERVER.HLOG.READER.IMPL:HLOG file Reader Implementation class, Default is Org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader;

Hbase.regionserver.hlog.writer.impl:HLog file Writer's implementation class, Default is Org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter;

Hbase.regionserver.global.memstore.upperLimit:memstore the upper limit in regionserver memory, when the new update is blocked and flush is forced to write, the default is 0.4 is 40 of the heap memory %; The blocking state continues to the capacity of all memstore of the regionserver to reach Hbase.regionserver.global.memstore.lowerLimit;

Hbase.regionserver.global.memstore.lowerLimit:memstore the maximum limit in regionserver memory, Flush is forced to write when it arrives, the default is 0.38 equivalent to 38% of the memory capacity;

Hbase.regionserver.optionalcacheflushinterval: An edit version in memory cache length, the default 3600000 milliseconds, set to 0, the automatic flush is forbidden;

Hbase.regionserver.catalog.timeout:regionserver Catalog Janitor Access meta time-out, default is 600000;

Hbase.regionserver.dns.interface: Similar to master ~ ~ Not speaking

Hbase.regionserver.dns.nameserver: Similar to master

Zookeeper.session.timeout: This is a worth to say the configuration, first ZK client to use, HBase use ZK client contact overall, but also used to start a ZK server, as the maxsessiontimeout of ZK, Overall is regionserver and ZK key parameters, if the connection timeout, master will be re-balance,regionserver will also be removed from the cluster list, the default is 90000; one problem is if ZK is maintained by HBase itself, Then the parameter as a regionserver connection is a value, if ZK is in another cluster, then ZK's own maxsessiontimeout parameter will take precedence over this parameter of HBase, when the time-out may occur with different problems;

Zookeeper.znode.parent:znode Store Root region address, default is Root-region-server;

ACL of Zookeeper.znode.acl.parent:root Znode, default ACL;

Hbase.zookeeper.dns.interface:zk DNS interface, default defaults;

HBASE.ZOOKEEPER.DNS.NAMESERVER:ZK DNS service address, default;

HBASE.ZOOKEEPER.PEERPORT:ZK communication port between peer, default is 2888;

HBASE.ZOOKEEPER.LEADERPORT:ZK Select leader Communication port, default is 3888;

HBASE.ZOOKEEPER.USEMULTI:ZK supports multiple update, requires ZK to be above 3.4 version, default is false;

Hbase.config.read.zookeeper.config: Let Hbaseconfig to read ZK's config, default false, also do not support open, this function is very funny ~ ~ Personal views;

HBASE.ZOOKEEPER.PROPERTY.INITLIMIT:ZK configuration, synchronization of the number of properties limit, default 10 ~ ~ No use;

HBASE.ZOOKEEPER.PROPERTY.SYNCLIMIT:ZK configuration, the number of times each request is synchronized, the default 5;

HBASE.ZOOKEEPER.PROPERTY.DATADIR:ZK configuration, snapshot storage directory, the default is ${hbase.tmp.dir}/zookeeper;

HBASE.ZOOKEEPER.PROPERTY.CLIENTPORT:ZK configuration, Client connection ZK port, default 2181;

HBASE.ZOOKEEPER.PROPERTY.MAXCLIENTCNXNS:ZK configuration, allows access to ZK maximum number of concurrent connections limit, by IP assignment, default 300;

Hbase.client.write.buffer:htable client Write buffer size, the default is 2097152BYTE, this buffer is to write the temporary storage of data, set large, waste client and server storage, set small, if write more data, too much RPC brings network overhead, official A side-to-end storage cost evaluation calculation is: Hbase.client.write.buffer*hbase.regionserver.handler.count, the service side of the RS processing handler number is also critical;

Hbase.client.pause:pause duration, the length of time to pause on hbase when get or other operation fail is dropped, default is 100;

Hbase.client.retries.number: The number of retries occurred when the operation fail, combined with the previous indicator to control the total retry time, the default is 35;

Hbase.client.max.total.tasks: The maximum number of concurrent tasks a htable instance can submit to a cluster, default is 100;

Hbase.client.max.perserver.tasks: The maximum number of concurrent tasks that a htable instance submits to a regionserver, default is 5;

Hbase.client.max.perregion.tasks: The maximum number of connections a client connects to a region, in other words, when you have so many connections in region, the new operation is not sent until the operation is complete, the default is 1;

Hbase.client.scanner.caching: When doing the next operation of scanner (if the local client does not find) the number of cached data rows, the value of the setting also needs to trade off, the cache is much faster, but eat memory, less cache will require more pull data, It is important to note that if the time difference of two calls is greater than scanner's timeout, do not set the value, default is 100;

Hbase.client.keyvalue.maxsize: A keyvalue instance of the maximum size, which is the storage file in a entry capacity limit, reasonable setting this value can control Regionserver Split,split will not remove keyvalue, so the Ke The size of the Yvalue is set to a scale fraction of regionserver size (can be removed) is a good choice, the default is 10485760;

Hbase.client.scanner.timeout.period: Combined with just caching, scanner time-out, the default is 60000 milliseconds;

Hbase.client.localityCheck.threadPoolSize: Do localitycheck thread pool size, default is 2;

Hbase.bulkload.retries.number: The maximum number of retries to do bulk load, the default is 0, which means to retry continuously;

Hbase.balancer.period:Master Run balancer cycle, default is 300000 milliseconds;

Hbase.regions.slop: If there is more than average+ (Average*slop) of the region of Regionserver, then rebalance, the default is 0.2;

Hbase.server.thread.wakefrequency: The sleep time of the service thread, default 10000 milliseconds, such as log roller;

Hbase.server.versionfile.writeattempts: The number of retries to write version file before exiting, default 3, the interval of each attempt is controlled by the previous parameter;

Hbase.hregion.memstore.flush.size:Memstore Write disk flush threshold, more than this size is flush, default is 134217728;

Hbase.hregion.preclose.flush.size: If the size of the memstore of a region equals or exceeds the amount of this parameter, when the region is closed (place close flag), flush before the region closes, and the default size is 52428 80;

Hbase.hregion.memstore.block.multiplier: If the size of Memstore satisfies the hbase.hregion.block.memstore * Hbase.hregion.flush.size a byte, then block update, this configuration can avoid unnecessary long time split or compact, or even oome, default is 2;

hbase.hregion.memstore.mslab.enabled: memstore-local Allocation Buffer is turned on, this configuration avoids heap memory fragmentation in the case of High writes, Can reduce the Stop-the-world GC frequency in large heap cases, the default is true;

The maximum size of the hbase.hregion.max.filesize:HStoreFile, in other words, when any one of the column families in a region exceeds this size, the region is split and the default is 10737418240;

Hbase.hregion.majorcompaction: The time period of the major compact for all hstorefile in a region, the default is 604800000 milliseconds (7 days);

Hbase.hregion.majorcompaction.jitter:major compaction of the occurrence of jitter range, so easy to understand, that is, the last parameter is not a strict cycle, there will be a jitter, this parameter is the ratio of jitter, Default is 0.5;

Hbase.hstore.compactionThreshold: A Hstore storage hstorefile threshold value, above which all hstorefile will be written to a new hstore, need to balance the trade-offs, the default is 3;

Hbase.hstore.blockingStoreFiles: A hstore storage hstorefile block update threshold, exceeding this threshold, hstore compaction, until finished to allow update, the default is 10;

Hbase.hstore.blockingWaitTime: A more powerful configuration, with the last parameter, when the Hstore block update, more than this time limit, blocking cancellation, even if the compaction is not completed, update will not be blocked, the default is 90000 milliseconds;

Hbase.hstore.compaction.max: The maximum number of hstorefile per minor compaction, default is 10;

Hbase.hstore.compaction.kv.max: The maximum number of keyvalue allowed at flushing or compacting, if there is a large keyvalue or oome, configure a small value, if the number of rows and small is configured large value, the default is 10;

Hbase.storescanner.parallel.seek.threads: If the thread pool size is found in parallel, the default is 10;

Hfile.block.cache.size: A configuration scale, allowing the maximum heap of the corresponding proportion of memory as the hfile and Hstorefile block cache, the default is 0.4, that is, 40%, set to 0 disable this ratio, do not recommend this;

Hfile.block.index.cacheonwrite: Allows the put no root (non-root) multi-level index block to the block cache when the index is written, the default is false;

Hfile.index.block.max.size: In the multi-level index tree structure, if any layer of block index reached this configuration size, block write, and replace the new block, the default is 131072;

Hfile.format.version: The hfile format version of the new file, set to one to test backward compatibility, default is 2;

Hfile.block.bloom.cacheonwrite: For the inline block of the combined Bron filter to open the Cache-on-write, the default is false;

Io.storefile.bloom.block.size: The size of a single piece (chunk) of a combined fabric filter, this value is an approximation value, the default is 131072;

Hbase.rs.cacheblocksonwrite: If a block cache is written when a hfile block is completed, the default is false;

Hbase.rpc.server.engine:hbase do the RPC server scheduling management class, Implement self-org.apache.hadoop.ipc.RpcServerEngine, default is Org.apache.hadoop.hbase.ipc.ProtobufRpcServerEngine;

Hbase.rpc.timeout:Hbase the time-out period when the client initiates a remote call, uses ping to confirm the connection, but eventually throws a timeoutexception with a default value of 60000;

Hbase.rpc.shortoperation.timeout: Another version of Hbase.rpc.timeout, which controls the time-out period for short operations, such as the time-out limit for the operation of Region Server reporting master can be set to small, This is beneficial to master's failover, the default is 10000;

Hbase.ipc.client.tcpnodelay: The default is true, which is to set no delay when the TCP socket is connected;

Hbase.master.keytab.file:kerberos keytab file full path name, used to do log for Hmaster, no default value;

Hbase.master.kerberos.principal: The principal name of Kerberos is required to run the Hmaster process, which is the value of this name, such as: hbase/[email Protected];

Hbase.regionserver.keytab.file:kerberos keytab file full path name, used to do log for Hregionserver, no default value;

Hbase.regionserver.kerberos.principal: The principal name of Kerberos is required to run the Hregionserver process, which is the value of this name, as in the form: hbase/[ Email protected];

Hadoop.policy.file:RPC server to do permissions authentication required security policy configuration file, after HBase security is turned on, the default is Habse-policy.xml;

Hbase.superuser:Hbase security power on the super-user Configuration, a series of comma-separated user or group;

Hbase.auth.key.update.interval:Hbase security Open After Server update authentication key time interval: default is 86400000 milliseconds;

Hbase.auth.token.max.lifetime:Hbase security, after the authentication token after the life cycle, the default is 604800000 milliseconds;

Hbase.ipc.client.fallback-to-simple-auth-allowed:client when using a secure connection to link to a non-secure server, the server prompts the client to switch to SASL Simple authentication mode (non-secure), If set to true, client agrees to switch to a non-secure connection and, if False, exits the connection;

Hbase.coprocessor.region.classes: comma-delimited list of coprocessores that will be loaded on all the default tables. After you have implemented a coprocessor, add it to the classpath of HBase and join the fully qualified name. can also be delayed loading, specified by Htabledescriptor;

Hbase.rest.port:Hbase Rest server port, default is 8080;

Hbase.rest.readonly: Defines the mode that the rest server starts, in two ways, false: All HTTP methods will be passed-get/put/post/delete,true: only GET method OK. The default value is false;

Hbase.rest.threads.max:REST the maximum number of threads for the server thread pool, China the new request will be automatically queued, restricting this configuration can control the amount of memory on the server, prevent oom, the default is 100;

Hbase.rest.threads.min: Similar to the same, the minimum number of threads, in order to ensure the Server service state, the default is 2;

Hbase.rest.support.proxyuser: Enable rest server to support Proxy-user mode, default is false;

Hbase.defaults.for.version.skip: Whether to skip the Hbase.defaults.for.version check, the default is false;

Hbase.coprocessor.master.classes: Coprocessors loaded by the hmaster process, comma delimited, all implemented Org.apache.hadoop.hbase.coprocessor.MasterObserver, with C Oprocessor similar, add classpath and fully qualified name;

Hbase.coprocessor.abortonerror: If the coprocessor load fails or the initialization fails or the Throwable object is thrown, the host exits. Setting to False will allow the system to continue running, but the state of the coprocessor will be inconsistent, so the general debug will be set to False, which is true by default;

Hbase.online.schema.update.enable: Set True to allow online schema changes, which by default is true;

Hbase.table.lock.enable: Set to True to allow the ZK lock table when schema changes, the lock table can organize concurrent schema changes resulting in inconsistent table state, the default is true;

Hbase.thrift.minWorkerThreads: The core size of the thread pool, after reaching the magnitude configured here, the new thread will be created when the new connection is created, the default is 16;

Hbase.thrift.maxWorkerThreads: As the name implies, the maximum number of threads, reached this number, the server began to drop the connection, the default is 1000;

Hbase.thrift.maxQueuedRequests:Thrift the maximum number of connection queues, if the line Cheng, first cache the request in this queue, the cache upper limit is the configuration, the default is 1000;

The maximum limit for table pool on the Hbase.thrift.htablepool.size.max:Thrift server, default is 1000;

Hbase.offheapcache.percentage:JVM parameter-xx:maxdirectmemorysize percent value, default is 0, that is, do not open out-of-heap allocation;

Hbase.data.umask.enable: After opening, the file will be regionserver write the relevant settings, the default is false does not open;

Hbase.data.umask: When the above configuration is turned on, the permissions of the file are Umask, the default is 000;

Hbase.metrics.showTableName: If the table name prefix is displayed for each indicator, the default is true;

Hbase.metrics.exposeOperationTimes: Whether to make a report about the operation in the use of the time dimension of the indicator, such as Get PUT DELETE increment, etc., by default is true;

Hbase.snapshot.enabled: Whether the snapshot is allowed to be used, stored and cloned, the default is true;

Hbase.snapshot.restore.take.failsafe.snapshot: In the restore process, if the failure is enabled snapshot substitution, the successful deletion of snapshot, the default turn on true;

Hbase.snapshot.restore.failsafe.name: Just said the process of snapshot name, the default is Hbase-failsafe-{snapshot.name}-{restore.timestamp};

Hbase.server.compactchecker.interval.multiplier: Check whether the time interval of the compact is required, typically when the compact is triggered after, for example, Memstore flush or other events But sometimes a different compact strategy is required, so periodically check the interval =hbase.server.compactchecker.interval.multiplier * Hbase.server.thread.wakefrequency, default 1000;

Hbase.lease.recovery.timeout: In the DFS lease timeout period, timeout is discarded, default is 900000;

Hbase.lease.recovery.dfs.timeout:dfs the time-out period of the restore lease call, which is 64000 by default;

An explanation of HBase configuration items