Deploy Hbase in the Hadoop cluster and enable kerberos
System: LXC-CentOS6.3 x86_64
Hadoop version: cdh5.0.1 (manmual installation, cloudera-manager not installed)
Existing Cluster Environment: node * 6; jdk1.7.0 _ 55; zookeeper and hdfs (HA) installed), yarn, historyserver, and httpfs, and kerberos is enabled (kdc is deployed on a node in the cluster ).
Package to be installed: All nodes> yum install hbase master node> yum install hbase-master hbase-thrift hbase-rest slave node> yum install hbase-regionserver
The configuration file to be modified is as follows:
Main configuration file and content:/etc/hbase/conf/hbase-site.xml
<Configuration>
<Property>
<Name> hbase. cluster. distributed </name>
<Value> true </value>
</Property>
<Property>
<Name> hbase. rootdir </name>
<Value> hdfs: // irs-hdp-HA/hbase </value>
</Property>
<Property>
<Name> hbase. rest. port </name>
<Value> 60050 </value>
</Property>
<Property>
<Name> hbase. zookeeper. quorum </name>
<Value> cdh01.hypers.com, cdh02.hypers.com, cdh03.hypers.com </value>
</Property>
<Property>
<Name> hbase. replication </name>
<Value> true </value>
</Property>
<Property>
<Name> hbase. security. authorization </name>
<Value> true </value>
</Property>
<Property>
<Name> hbase. coprocessor. master. classes </name>
<Value> org. apache. hadoop. hbase. security. access. AccessController </value>
</Property>
<Property>
<Name> hbase. coprocessor. region. classes </name>
<Value> org. apache. hadoop. hbase. security. token. TokenProvider, org. apache. hadoop. hbase. security. access. AccessController </value>
</Property>
<Property>
<Name> hbase. security. authentication </name>
<Value> kerberos </value>
</Property>
<Property>
<Name> hbase. rpc. engine </name>
<Value> org. apache. hadoop. hbase. ipc. SecureRpcEngine </value>
</Property>
<Property>
<Name> hbase. regionserver. kerberos. principal </name>
<Value> hbase/_ HOST@HADOOP.COM </value>
</Property>
<Property>
<Name> hbase. regionserver. keytab. file </name>
<Value>/etc/hadoop/conf/hbase. keytab </value>
</Property>
<Property>
<Name> hbase. master. kerberos. principal </name>
<Value> hbase/_ HOST@HADOOP.COM </value>
</Property>
<Property>
<Name> hbase. master. keytab. file </name>
<Value>/etc/hadoop/conf/hbase. keytab </value>
</Property>
<Property>
<Name> hbase. thrift. keytab. file </name>
<Value>/etc/hadoop/conf/hbase. keytab </value>
</Property>
<Property>
<Name> hbase. thrift. kerberos. principal </name>
<Value> hbase/_ HOST@HADOOP.COM </value>
</Property>
<Property>
<Name> hbase. rest. keytab. file </name>
<Value>/etc/hadoop/conf/hbase. keytab </value>
</Property>
<Property>
<Name> hbase. rest. kerberos. principal </name>
<Value> hbase/_ HOST@HADOOP.COM </value>
</Property>
<Property>
<Name> hbase. rest. authentication. type </name>
<Value> kerberos </value>
</Property>
<Property>
<Name> hbase. rest. authentication. kerberos. principal </name>
<Value> HTTP/_ HOST@HADOOP.COM </value>
</Property>
<Property>
<Name> hbase. rest. authentication. kerberos. keytab </name>
<Value>/etc/hadoop/conf/hbase. keytab </value>
</Property>
</Configuration>
Security related configuration file/etc/hbase/conf/zk-jaas.conf
Client {
Com. sun. security. auth. module. Krb5LoginModule required
UseKeyTab = true
UseTicketCache = false
KeyTab = "/etc/hadoop/conf/hbase. keytab"
Principal = "hbase/cdh01.hypers.com@HADOOP.COM ";
};
Hbase-related environment variable file:/etc/hbase/conf/hbase-env.sh
Export HBASE_OPTS = "-XX: + UseConcMarkSweepGC"
Export HBASE_OPTS = "$ HBASE_OPTS-Djava. security. auth. login. config =/etc/hbase/conf/zk-jaas.conf"
Export HBASE_MANAGES_ZK = false
Zookeeper configuration file (only the last two rows are appended to hbase configuration):/usr/lib/zookeeper/conf/zoo. cfg
MaxClientCnxns = 50
TickTime = 2000
InitLimit = 5
SyncLimit = 2
DataDir =/var/lib/zookeeper
ClientPort = 2181
Server.1 = cdh01.hypers.com: 2888: 3888
Server.2 = cdh02.hypers.com: 2888: 3888
Server.3 = cdh03.hypers.com: 2888: 3888
AuthProvider.1 = org. apache. zookeeper. server. auth. SASLAuthenticationProvider
JaasLoginRenew = 3600000
Kerberos. removeHostFromPrincipal = true
Kerberos. removeRealmFromPrincipal = true
------------------------------------------------------- End -----------------------------------------------------
Hadoop + HBase cloud storage creation summary PDF
Regionserver startup failed due to inconsistent time between HBase nodes
Hadoop + ZooKeeper + HBase cluster configuration
Hadoop cluster Installation & HBase lab environment setup
HBase cluster configuration based on Hadoop cluster'
Hadoop installation and deployment notes-HBase full distribution mode installation
Detailed tutorial on creating HBase environment for standalone Edition
Reference documentation (hortonworks will be short for hdp; cloudera is cdh ):
1. Create a system template. Because I found the centos6.5 template in openvz, we tried to keep it consistent with production in the test environment, so we should use CentOS6.3, note automatically according to the official documentation, create a template, the link is as follows: http://openvz.org/Creating_a_CentOS_6_Template
2. Install cluster reference cdh official documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/CDH5-Installation-Guide.html
3. A large number of hortonworks documents have been referenced in the integration phase of kerberos and hadoop, because the cloudera documents give the author the feeling that these are OK (this is simply a mad heart ...), the specific link is as follows:
Kerberos configuration link for hortonworks: http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.1.0/bk_ambari_security/content/index.html
Kerberos configuration link for cloudera: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/CDH5-Security-Guide.html
Kerberos official site: http://web.mit.edu/kerberos/krb5-latest/doc/
4. Please refer to the cloudera document and the hbase official document (because cdh does not mention the relevant configuration during thrift and rest installation, causing startup failure ..). The links are as follows:
Cdh documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hbase_installation.html
Cdh hbase and kerberos deployment integration documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/cdh5sg_hbase_security.html
Hbase official documentation: http://hbase.apache.org/book/security.html