Apache Hadoop configuration Kerberos Guide

Apache Hadoop configuration Kerberos Guide

Generally, the security of a Hadoop cluster is guaranteed using kerberos. After Kerberos is enabled, you must perform authentication. After verification, you can use the GRANT/REVOKE statement to control role-based access. This article describes how to configure kerberos in a CDH cluster.

1. KDC installation and configuration script

The script install_kerberos.sh can complete all the installation configurations and corresponding parameter configurations of the kerberos server.

#! /Bin/bash

#
Echo "ready to install and config kerberos"

# Run with root permission
Cd/root
Chmod 755/etc
Chmod 755/etc/hadoop

# Install the kerberos component
Yum install-y krb5-server
Yum install-y openldap-clients
Yum-y install krb5-workstation

# Update the domain name and host name in the configuration file

# Setting domain names
Sed-I. orig's/EXAMPLE. COM/CLOUDERA/G'/etc/krb5.conf
# Set the kerberos server host name
Sed-I. m1's/kerberos.example.com/quickstart.cloudera/g'/etc/krb5.conf
# Set the domain name to cloudera
Sed-I. m2's/example.com/cloudera/g'/etc/krb5.conf

# Download JDK from the official Oracle website to the root directory

Mkdir jce
Cd jce
Unzip ../UnlimitedJCEPolicyJDK7.zip
# Save the original jar file for backup
Cp/usr/java/jdk1.7.0 _ 67-cloudera/jre/lib/security/local_policy.jar local_policy.jar.orig
Cp/usr/java/jdk1.7.0 _ 67-cloudera/jre/lib/security/US_export_policy.jar US_export_policy.jar.orig

# Copy the new jar package
Cp/root/jce/UnlimitedJCEPolicy/local_policy.jar/usr/java/jdk1.7.0 _ 67-cloudera/jre/lib/security/local_policy.jar
Cp/root/jce/UnlimitedJCEPolicy/US_export_policy.jar/usr/java/jdk1.7.0 _ 67-cloudera/jre/lib/security/US_export_policy.jar

# Create a kerberos Database
Echo suggested password is cloudera

Kdb5_util create-s

# Update kdc. conf
Sed-I. orig's/EXAMPLE. COM/CLOUDERA/G'/var/kerberos/krb5kdc/kdc. conf
# Add a line to the ticket File
Sed-I. m1 '/dict_file/a max_life = 1d'/var/kerberos/krb5kdc/kdc. conf
# Increase the maximum number of files that can be created
Sed-I. m2 '/dict_file/a max_renewable_life = 7d'/var/kerberos/krb5kdc/kdc. conf
# Indent the new two rows
Sed-I. m3's/^ max _/'/var/kerberos/krb5kdc/kdc. conf

# Admin permission is required for the acl File
Sed-I's/EXAMPLE. COM/CLOUDERA/'/var/kerberos/krb5kdc/kadm5.acl

# Modifying the kdc. conf file
Sed-I. m3 '/supported_enctypes/a default_principal_flags = + renewable, + forwardable'/var/kerberos/krb5kdc/kdc. conf
# Set indentation
Sed-I. m4's/^ default_principal_flags/'/var/kerberos/krb5kdc/kdc. conf

# Start krb5kdc and kadmin services
Service krb5kdc start
Service kadmin start

Kadmin. local & lt; # cloudera-scm/admin@YOUR-LOCAL-REALM.COM

# Add an admin user to the Cluster
Kadmin. local & lt;

2. Cloudera Manager Kerberos Configuration

After running the script, we now have a running Kerberos server which can be used to protect Hadoop clusters. ClouderaManager encapsulates most of the work. We only need to configure some parameter values.

Access http: // <your. cm. manager>: 7180 in a browser and log on to Cloudera Manager.

Go to management> Security> Kerberos, as shown in

Click "enable Kerberos ".

All the four options in the list are completed by the running script. Then select "continue ".

The Kerberos wizard needs to know the configuration item value of the script. You must enter the following options:

• KDC server host: <your_kdc_server>
• Kerberos security: <your_safe_realm>
• Kerberos encryption type: aes256-cts-hmac-sha1-96

Click "continue ".

Select Manage krb5.conf through Cloudera Manager. Click continue

Enter the created user name and password. Click continue. A message is displayed, indicating that the cluster is successfully restarted. After the restart, we built a hadoop cluster that uses kerberos for security protection.

3. Cluster Usage After kerberos is started

If kerberos is not enabled, hdfs users are hdfs superusers. You can use su hdfs to obtain permissions for hdfs operations. Use the following method after kerberos is Enabled:

Kinit hdfs @ HADOOP

Use hdfs command for verification

Hadoop fs-mkdir/kongc
Hadoop fs-rmdir/kongc

Make Kerberos token invalid

Kdestroy

A running Kerberos cluster has been created!

Hadoop2.3-HA high-availability cluster environment construction

Hadoop project-Cloudera 5.10.1 (CDH) installation and deployment based on CentOS7

Detailed explanation of Hadoop2.7.2 cluster construction (high availability)

Use Ambari to deploy a Hadoop cluster (build an intranet HDP source)

Hadoop cluster installation in Ubuntu 14.04

Install Hadoop 2.7.2 In CentOS 6.7

Build a distributed Hadoop-2.7.3 cluster on Ubuntu 16.04

Build a Hadoop 2.6.4 distributed Cluster Environment in CentOS 7

Hadoop2.7.3 + Spark2.1.0 fully distributed cluster Construction Process