Hadoop permission management)

For example, hadoop access control is divided into two levels. servicelevel authorization is a system level, which is used to control whether a specified service can be accessed. For example, whether a user/group can submit a job to a cluster, it is the most basic access control, and takes precedence over File Permission verification and mapred queue permission verification. Access Control on Job Queues controls the permissions of the mapred queue on the Job Scheduling Policy layer. Dfspermmision User Controls file permissions. In the current version, the user/group information connected to the hadoop cluster depends on the client environment, that is, the user name and group name obtained by 'whoam' and 'bash-C groupup' on the client host, there is no UID or GID. If there is a user group in the user group list that is the same as the user group configured in the cluster, this group has the permission.

1. ConfigurationService level authorization

Modify core-site.xml

<Property><Name>Hadoop. Security. Authorization</Name><Value>True</Value></Property>
 

If hadoop. Security. Authorization = true, servicelevel authorization is enabled. If it is false, no authentication is performed. All Users Have Full permissions. (To modify this configuration, restart hadoop)

Service levelauthorization has nine configurable attributes. Each attribute can specify users or user groups with corresponding access permissions. The nine ACL properties are as follows (hadoop-policy.xml ):

The nine attributes are configured in the same way, as shown in the following code:

Each user can be configured with multiple users, separated by commas (,). multiple user groups can be configured. "," is used between groups. users and groups are separated by spaces, if only the group is available, a space is reserved before the group, for example:

<Property><Name>Security. Job. Submission. Protocol. ACL</Name><Value>Alice, bobgroup1, group2</Value></Property>
 

 

By default, these nine attributes are not available to any users or groups.

The configuration file can be dynamically loaded using the following command:

(1) Update namenode attributes: Bin/hadoop dfsadmin-refreshserviceacl

(2) Update jobtracker attributes: Bin/hadoopmradmin-refreshserviceacl

 

2. Scheduler configuration

Modify mapred-site.xml

<Property><Name>Mapred. jobtracker. taskscheduler</Name><Value>Org. Apache. hadoop. mapred. capacitytaskscheduler</Value></Property>
 

 

To enable access contol onjob queues, You must select a scheduler that supports multi-queue management. Therefore, mapred. jobtracker. taskscheduler can only be capacitytaskscheduler or fairscheduler.

Configure the queue in the mapred-site.xml, such:

 
<Property><Name>Mapred. queue. Names</Name><Value>Default, hadoop, stat, query</Value></Property>
 

 

3. Access contol on jobqueues Configuration

Access contol on Job Queues switch on the mapred-site.xml, as shown below:

 
<Property><Name>Mapred. ACLS. Enabled</Name><Value>True</Value></Property>
 

Mapred. ACLS. Enabled = true enabled, false disabled.

The specific ACL attributes are in the mapred-queue-acl.xml, such:

 

<Property><Name>Mapred. queue. Stat. ACL-submit-job</Name><Value>User1, user2 group1, group2</Value></Property>
 

Indicates user1, user2, and group1. group2 can submit a job to stat queue.

4. DFS permission Configuration

Modify hdfs-site.xml

<Property><Name>Dfspermission</Name><Value>True</Value></Property>
 

DFS. Permission: whether to enable File Permission verification. If it is set to true, read/write permission verification is not performed if it is set to false. (Note: DFS. if permission is enabled or not, the DFS permission information will not be changed and will be lost. chown, chgrp, and chmod operations will always verify the permission. dfspermission information is only in namenode and is not associated with blocks in danode)

Use chown, chgrp, and chmod to modify the owner, group, and permissions of the file/directory.

 

Supplement: Job ACL

The default job ACL is configured in the mapred-site.xml as follows:

 <  Property  >      <  Name  > Mapreduce. Job. ACL-View-job </  Name  >      <  Value  > User1 </ Value  >    </  Property  >    <  Property  >      <  Name  > Mapreduce. Job. ACL-Modify-job </  Name  >      <  Value  > User1 </  Value  >    </  Property  >   

By default, user1 has the permission to view and modify jobs. 

The job submitter can specify the values of mapreduce. Job. ACL-View-job and mapreduce. Job. ACL-Modify-job. The value specified during submission will overwrite the default value.

Job submitter, superuser, Cluster Administrator (mapreduce. Cluster. Administrators), and jobqueue administrator always have this permission.

 

From http://blog.csdn.net/cheersu/article/details/8080162