Hive0.13 permission bug fix

Recently, the online hive has been upgraded to 0.13, and many problems have been encountered. In the preceding permissions, hive. Security. Authorization. createtable. Owner. Grants is set in hive0.13, and the table created by the user is not authorized. After analyzing the source code and debugging, find the RC and fix it, and record it below.

1. first, perform the table creation test in hive0.11 and hive0.13 respectively. By viewing the metadata in the database, it is found that if the owner parameter is set in hive0.11 after the table is created, the user will have all permissions for this table (for details, you can analyze the db_privs, tbl_privs tables, and hive. security. authorization. createtable. owner. grants is set to all), but hive0.13 is empty.

2. Pass

hive -hiveconf hive.root.logger=DEBUG,console

After comparing the logs, we found that at hive0.11, the created table attributes had permission settings.

In hive0.11, this is empty.

Hive0.11: 14/07/16 17:05:39 debug hive. log: DDL: struct ericni4 {string a} 14/07/16 17:05:39 debug lazy. lazysimpleserde: Org. apache. hadoop. hive. serde2.lazy. lazysimpleserde initialized with: columnnames = [a] columntypes = [String] separator = [[[email protected] nullstring = \ n lastcolumntakesrest = false14/07/16 17:05:39 info MetaStore. hivemetastore: 0: create_table: Table (tablename: ericni4, dbname: temp, Owner: ericn2, createtime: 1405501539, lastaccesstime: 0, retention: 0, SD: storagedescriptor (Cols: [fieldschema (Name: A, type: String, comment: NULL)], Location: NULL, inputformat: Org. apache. hadoop. mapred. sequencefileinputformat, outputformat: Org. apache. hadoop. hive. QL. io. hivesequencefileoutputformat, compressed: false, numbuckets:-1, serdeinfo: serdeinfo (Name: NULL, serializationlib: Org. apache. hadoop. hive. serde2.lazy. lazysimpleserde, parameters: {serialization. format = 1}), bucketcols: [], sortcols: [], parameters: {}, skewedinfo: skewedinfo (skewedcolnames: [], skewedcolvalues: [], Region: {}), storedassubdirectories: false), partitionkeys: [], parameters: {}, vieworiginaltext: NULL, viewexpandedtext: NULL, tabletype: managed_table, privileges: Sums: {ericn2= [privilegegrantinfo (privilege: All, createtime:-1, grantor: ericn2, grantortype: User, grantoption: True)]}, groupprivileges: NULL, roleprivileges: NULL )) # hive0.11 add this setting when creating a table hive0.1314/07/16 17:10:07 debug hive. log: DDL: struct ericni4 {string a} 14/07/16 17:10:07 debug lazy. lazysimpleserde: Org. apache. hadoop. hive. serde2.lazy. lazysimpleserde initialized with: columnnames = [a] columntypes = [String] separator = [[[email protected] nullstring = \ n lastcolumntakesrest = false14/07/16 17:10:07 info MetaStore. hivemetastore: 0: create_table: Table (tablename: ericni4, dbname: temp, Owner: ericni1, createtime: 1405501807, lastaccesstime: 0, retention: 0, SD: storagedescriptor (Cols: [fieldschema (Name: A, type: String, comment: NULL)], Location: NULL, inputformat: Org. apache. hadoop. mapred. sequencefileinputformat, outputformat: Org. apache. hadoop. hive. QL. io. hivesequencefileoutputformat, compressed: false, numbuckets:-1, serdeinfo: serdeinfo (Name: NULL, serializationlib: Org. apache. hadoop. hive. serde2.lazy. lazysimpleserde, parameters: {serialization. format = 1}), bucketcols: [], sortcols: [], parameters: {}, skewedinfo: skewedinfo (skewedcolnames: [], skewedcolvalues: [], Region: {}), storedassubdirectories: false), partitionkeys: [], parameters: {}, vieworiginaltext: NULL, viewexpandedtext: NULL, tabletype: managed_table)

3. Will the parameter not take effect? Comment out this setting in hive0.13 for comparison. It is found that the attribute of create_table has not changed, indicating that this parameter does not take effect at least in the current 0.13 environment. In 0.11, there were changes.

4. In hive0.11, we set an incorrect value to obtain the stack call information:

Obtain the call stack of 0.11:

        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.checkPrivilege(CreateTableAutomaticGrant.java:118)        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.getGrantorInfoList(CreateTableAutomaticGrant.java:97)        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.create(CreateTableAutomaticGrant.java:52)        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:275)            at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:278)        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670)        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

Analyze the source code

In hive0.11, when the session is initialized, the start method of sessionstate is called, and the createtablegrants attribute is directly set based on the createtableautomaticgrant class.

Sessionstate. start: Try {startss. authenticator = hiveutils. getauthenticator (startss. getconf (), hiveconf. confvars. hive_authenticator_manager); startss. authorizer = hiveutils. getauthorizeprovidermanager (startss. getconf (), hiveconf. confvars. hive_authorization_manager, startss. authenticator); startss. createtablegrants = createtableautomaticgrant. create (startss. getconf (); // set the owner's related permissions} catch (hiveexception e) {Throw new runtimeexception (E );}

Call the hive. createtable method when creating a table.

(public void createTable(Table tbl, boolean ifNotExists))

  public CreateTableAutomaticGrant getCreateTableGrants() {    return createTableGrants;  }

In hive0.13, even if an error is set, no error is reported. The call relationship is changed through the debug source code. The getcreatetablegrants method of sessionstate calls the setupauth method.

Public createtableautomaticgrant getcreatetablegrants () {setupauth (); // call the setupauth method to set createtablegrants return createtablegrants ;}

The setupauth method is as follows:

Private void setupauth () {If (authenticator! = NULL) {// auth has been initialized return;} Try {authenticator = hiveutils. getauthenticator (Conf, hiveconf. confvars. hive_authenticator_manager); authenticator. setsessionstate (this); Authorizer = hiveutils. getauthorizeprovidermanager (Conf, hiveconf. confvars. hive_authorization_manager, authenticator, true); If (Authorizer = NULL) {// The permission is generated only when the value of authorizer is null. // if it was null, the new Authorization plugin must be specified in // config hiveauthorizerfactory authorizerfactory = hiveutils. getauthorizerfactory (Conf, hiveconf. confvars. hive_authorization_manager); authorizerv2 = authorizerfactory. createhiveauthorizer (New hivemetastoreclientfactoryimpl (), Conf, Authenticator); authorizerv2.applyauthorizationconfigpolicy (CONF); // create the create table grants with new config creat Etablegrants = createtableautomaticgrant. create (CONF) ;}} catch (hiveexception e) {Throw new runtimeexception (E);} If (log. isdebugenabled () {object authorizationclass = getauthorizationmode () = authorizationmode. v1? Getauthorizer (): getauthorizerv2 (); log. debug ("session is using authorization class" + authorizationclass. getclass ();} return ;}

You can see that when the value of authorizer is null, the operation with the production ower permission (createtableautomaticgrant. Create (CONF) will be available ))

The value of authorizer is obtained by the hive. Security. Authorization. Manager parameter. In hive0.13, this parameter has the default value:

HIVE_AUTHORIZATION_MANAGER("hive.security.authorization.manager",      "org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider"),

In this way, the setupauth method does not use createtableautomaticgrant. Create (CONF) to set the owner's permissions. RC finds the permissions. Fix is also relatively simple.

Try {authenticator = hiveutils. getauthenticator (Conf, hiveconf. confvars. hive_authenticator_manager); authenticator. setsessionstate (this); Authorizer = hiveutils. getauthorizeprovidermanager (Conf, hiveconf. confvars. hive_authorization_manager, authenticator, true); createtablegrants = createtableautomaticgrant. create (CONF); // whether the authorizer is null or not, set the relevant permission if (Authorizer = NULL ){.......

After the bug fix, use the wrong parameter. The stack is as follows, which indicates that it takes effect:

14/07/18 12:33:38 WARN session.SessionState: authenticator is null ,return,auth has been initialized14/07/18 12:33:38 WARN session.CreateTableAutomaticGrant: Privilege is nullFAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Privilege alldddd is not found.14/07/18 12:33:38 ERROR ql.Driver: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Privilege alldddd is not found.java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Privilege alldddd is not found.        at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:410)        at org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:979)        at org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:990)        at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:508)        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:461)        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:322)        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:975)        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1040)        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359)        at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456)        at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466)        at org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:502)        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Privilege alldddd is not found.        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.validatePrivilege(CreateTableAutomaticGrant.java:129)        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.getGrantorInfoList(CreateTableAutomaticGrant.java:105)        at org.apache.hadoop.hive.ql.session.CreateTableAutomaticGrant.create(CreateTableAutomaticGrant.java:57)        at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:392)        ... 24 more

This article from the "Food light blog" blog, please be sure to keep this source http://caiguangguang.blog.51cto.com/1652935/1440128

Hive0.13 permission bug fix