Deploy Hbase in the Hadoop cluster and enable kerberos

Source: Internet
Author: User
Tags hortonworks

Deploy Hbase in the Hadoop cluster and enable kerberos

System: LXC-CentOS6.3 x86_64

Hadoop version: cdh5.0.1 (manmual installation, cloudera-manager not installed)

Existing Cluster Environment: node * 6; jdk1.7.0 _ 55; zookeeper and hdfs (HA) installed), yarn, historyserver, and httpfs, and kerberos is enabled (kdc is deployed on a node in the cluster ).

Package to be installed: All nodes> yum install hbase master node> yum install hbase-master hbase-thrift hbase-rest slave node> yum install hbase-regionserver

The configuration file to be modified is as follows:

Main configuration file and content:/etc/hbase/conf/hbase-site.xml


<Configuration>

<Property>

<Name> hbase. cluster. distributed </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. rootdir </name>

<Value> hdfs: // irs-hdp-HA/hbase </value>

</Property>

<Property>

<Name> hbase. rest. port </name>

<Value> 60050 </value>

</Property>

<Property>

<Name> hbase. zookeeper. quorum </name>

<Value> cdh01.hypers.com, cdh02.hypers.com, cdh03.hypers.com </value>

</Property>

<Property>

<Name> hbase. replication </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. security. authorization </name>

<Value> true </value>

</Property>

<Property>

<Name> hbase. coprocessor. master. classes </name>

<Value> org. apache. hadoop. hbase. security. access. AccessController </value>

</Property>

<Property>

<Name> hbase. coprocessor. region. classes </name>

<Value> org. apache. hadoop. hbase. security. token. TokenProvider, org. apache. hadoop. hbase. security. access. AccessController </value>

</Property>

<Property>

<Name> hbase. security. authentication </name>

<Value> kerberos </value>

</Property>

<Property>

<Name> hbase. rpc. engine </name>

<Value> org. apache. hadoop. hbase. ipc. SecureRpcEngine </value>

</Property>

<Property>

<Name> hbase. regionserver. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. regionserver. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. master. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. master. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. thrift. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. thrift. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. keytab. file </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

<Property>

<Name> hbase. rest. kerberos. principal </name>

<Value> hbase/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. authentication. type </name>

<Value> kerberos </value>

</Property>

<Property>

<Name> hbase. rest. authentication. kerberos. principal </name>

<Value> HTTP/_ HOST@HADOOP.COM </value>

</Property>

<Property>

<Name> hbase. rest. authentication. kerberos. keytab </name>

<Value>/etc/hadoop/conf/hbase. keytab </value>

</Property>

</Configuration>

Security related configuration file/etc/hbase/conf/zk-jaas.conf

Client {

Com. sun. security. auth. module. Krb5LoginModule required

UseKeyTab = true

UseTicketCache = false

KeyTab = "/etc/hadoop/conf/hbase. keytab"

Principal = "hbase/cdh01.hypers.com@HADOOP.COM ";

};

Hbase-related environment variable file:/etc/hbase/conf/hbase-env.sh

Export HBASE_OPTS = "-XX: + UseConcMarkSweepGC"

Export HBASE_OPTS = "$ HBASE_OPTS-Djava. security. auth. login. config =/etc/hbase/conf/zk-jaas.conf"

Export HBASE_MANAGES_ZK = false

Zookeeper configuration file (only the last two rows are appended to hbase configuration):/usr/lib/zookeeper/conf/zoo. cfg

MaxClientCnxns = 50

TickTime = 2000

InitLimit = 5

SyncLimit = 2

DataDir =/var/lib/zookeeper

ClientPort = 2181

Server.1 = cdh01.hypers.com: 2888: 3888

Server.2 = cdh02.hypers.com: 2888: 3888

Server.3 = cdh03.hypers.com: 2888: 3888

AuthProvider.1 = org. apache. zookeeper. server. auth. SASLAuthenticationProvider

JaasLoginRenew = 3600000

Kerberos. removeHostFromPrincipal = true

Kerberos. removeRealmFromPrincipal = true

------------------------------------------------------- End -----------------------------------------------------

Hadoop + HBase cloud storage creation summary PDF

Regionserver startup failed due to inconsistent time between HBase nodes

Hadoop + ZooKeeper + HBase cluster configuration

Hadoop cluster Installation & HBase lab environment setup

HBase cluster configuration based on Hadoop cluster'

Hadoop installation and deployment notes-HBase full distribution mode installation

Detailed tutorial on creating HBase environment for standalone Edition


Reference documentation (hortonworks will be short for hdp; cloudera is cdh ):


1. Create a system template. Because I found the centos6.5 template in openvz, we tried to keep it consistent with production in the test environment, so we should use CentOS6.3, note automatically according to the official documentation, create a template, the link is as follows: http://openvz.org/Creating_a_CentOS_6_Template

2. Install cluster reference cdh official documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/CDH5-Installation-Guide.html

3. A large number of hortonworks documents have been referenced in the integration phase of kerberos and hadoop, because the cloudera documents give the author the feeling that these are OK (this is simply a mad heart ...), the specific link is as follows:


Kerberos configuration link for hortonworks: http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.1.0/bk_ambari_security/content/index.html

Kerberos configuration link for cloudera: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/CDH5-Security-Guide.html

Kerberos official site: http://web.mit.edu/kerberos/krb5-latest/doc/

4. Please refer to the cloudera document and the hbase official document (because cdh does not mention the relevant configuration during thrift and rest installation, causing startup failure ..). The links are as follows:

Cdh documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hbase_installation.html

Cdh hbase and kerberos deployment integration documentation: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/cdh5sg_hbase_security.html

Hbase official documentation: http://hbase.apache.org/book/security.html

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.