Deploy Hbase in the Hadoop cluster and enable kerberos

Deploy Hbase in the Hadoop cluster and enable kerberos

System: LXC-CentOS6.3 x86_64

Hadoop version: cdh5.0.1 (manmual installation, cloudera-manager not installed)

Existing Cluster Environment: node * 6; jdk1.7.0 _ 55; zookeeper and hdfs (HA) installed), yarn, historyserver, and httpfs, and kerberos is enabled (kdc is deployed on a node in the cluster ).

Package to be installed: All nodes> yum install hbase master node> yum install hbase-master hbase-thrift hbase-rest slave node> yum install hbase-regionserver

The configuration file to be modified is as follows:

Main configuration file and content:/etc/hbase/conf/hbase-site.xml

<Configuration>

<Property>

<Name> hbase. cluster. distributed </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. rootdir </name>

<Value> hdfs: // irs-hdp-HA/hbase </value>

</Property>

<Property>

<Name> hbase. rest. port </name>

<Value> 60050 </value>

</Property>

<Property>

<Name> hbase. zookeeper. quorum </name>

<Value> cdh01.hypers.com, cdh02.hypers.com, cdh03.hypers.com </value>

</Property>

<Property>

<Name> hbase. replication </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. security. authorization </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. coprocessor. master. classes </name>

<Value> org. apache. hadoop. hbase. security. access. AccessController </value>

</Property>

<Property>

<Name> hbase. coprocessor. region. classes </name>

<Value> org. apache. hadoop. hbase. security. token. TokenProvider, org. apache. hadoop. hbase. security. access. AccessController </value>

</Property>

<Property>

<Name> hbase. security. authentication </name>

<Value> kerberos </value>

</Property>

<Property>

<Name> hbase. rpc. engine </name>

<Value> org. apache. hadoop. hbase. ipc. SecureRpcEngine </value>

</Property>

<Property>

<Name> hbase. regionserver. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. regionserver. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. master. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. master. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. thrift. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. thrift. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. rest. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. authentication. type </name>

<Value> kerberos </value>

</Property>

<Property>

<Name> hbase. rest. authentication. kerberos. principal </name>

<Value> HTTP/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. authentication. kerberos. keytab </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

</Configuration>

Security related configuration file/etc/hbase/conf/zk-jaas.conf

Client {

Com. sun. security. auth. module. Krb5LoginModule required

UseKeyTab = true

UseTicketCache = false

KeyTab = "/etc/hadoop/conf/hbase. keytab"

Principal = "hbase/cdh01.hypers.com@HADOOP.COM ";

};

Hbase-related environment variable file:/etc/hbase/conf/hbase-env.sh

Export HBASE_OPTS = "-XX: + UseConcMarkSweepGC"

Export HBASE_OPTS = "$ HBASE_OPTS-Djava. security. auth. login. config =/etc/hbase/conf/zk-jaas.conf"

Export HBASE_MANAGES_ZK = false

Zookeeper configuration file (only the last two rows are appended to hbase configuration):/usr/lib/zookeeper/conf/zoo. cfg

MaxClientCnxns = 50

TickTime = 2000

InitLimit = 5

SyncLimit = 2

DataDir =/var/lib/zookeeper

ClientPort = 2181

Server.1 = cdh01.hypers.com: 2888: 3888

Server.2 = cdh02.hypers.com: 2888: 3888

Server.3 = cdh03.hypers.com: 2888: 3888

AuthProvider.1 = org. apache. zookeeper. server. auth. SASLAuthenticationProvider

JaasLoginRenew = 3600000

Kerberos. removeHostFromPrincipal = true

Kerberos. removeRealmFromPrincipal = true

------------------------------------------------------- End -----------------------------------------------------

Hadoop + HBase cloud storage creation summary PDF

Regionserver startup failed due to inconsistent time between HBase nodes

Hadoop + ZooKeeper + HBase cluster configuration

Hadoop cluster Installation & HBase lab environment setup

HBase cluster configuration based on Hadoop cluster'

Hadoop installation and deployment notes-HBase full distribution mode installation

Detailed tutorial on creating HBase environment for standalone Edition

Reference documentation (hortonworks will be short for hdp; cloudera is cdh ):

1. Create a system template. Because I found the centos6.5 template in openvz, we tried to keep it consistent with production in the test environment, so we should use CentOS6.3, note automatically according to the official documentation, create a template, the link is as follows: http://openvz.org/Creating_a_CentOS_6_Template

2. Install cluster reference cdh official documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/CDH5-Installation-Guide.html

3. A large number of hortonworks documents have been referenced in the integration phase of kerberos and hadoop, because the cloudera documents give the author the feeling that these are OK (this is simply a mad heart ...), the specific link is as follows:

Kerberos configuration link for hortonworks: http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.1.0/bk_ambari_security/content/index.html

Kerberos configuration link for cloudera: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/CDH5-Security-Guide.html

Kerberos official site: http://web.mit.edu/kerberos/krb5-latest/doc/

4. Please refer to the cloudera document and the hbase official document (because cdh does not mention the relevant configuration during thrift and rest installation, causing startup failure ..). The links are as follows:

Cdh documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hbase_installation.html

Cdh hbase and kerberos deployment integration documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/cdh5sg_hbase_security.html

Hbase official documentation: http://hbase.apache.org/book/security.html