HBase quota mechanism Source code reading

Related patch:https://issues.apache.org/jira/browse/hbase-11598
REVIEW board:https://reviews.apache.org/r/23981 Quota setting and application process

1. The client receives quotasetting via Admin.setquota sending serialized quotasetting information to the server master and writes the data to the Hbase:quota table

2.regionserver through a chore timed task, every once in a while to get the configuration in Hbase:quota, put in Quotacache

3. When a user read and write operation, in the Rsrpcservice to get the size of the request, see whether the threshold (Checkquota method), to reach the threshold to throw an exception, otherwise it will be cumulative

Both the master and Regionserver end have corresponding manager classes that manage quota related operations (Setquota and read-write Hbase:quota,checkquota) related classes Client Settings Quota ConnectionManager: Add Method Setquota (Rpccontroller controller, setquotarequest request) Admin.java: interface classes, mainly involving methods Setquota (quotasettings), Getquotaretriever (Quotafilter) Hbaseadmin.java: Add Method Setquota (quotasettings) Primary user admin Settings quota,getquotaretriever (final quotafilter filter) Quota Information Lookup quotasettings: abstract class, Username,tablename,namespace,get Quotatype, and Setquotarequest.builder to generate serialization data Quotasettingsfactory: A factory class that generates quotasettings instances, The actual type of quotasetting returned is the throttlesettings or QUOTAGLOBALSSETTINGSBYPASS,RB code that ultimately passes the Admin.setquotas parameter is the corresponding method return value of this class Serializes quota information ProtobufUtil.java:toTimeUnit, Toprototimeunit, Tothrottletype, Toquotascope, Toprotoquotascope, and so on quota information serialization operations, Timeunit set up in the time unit client query quota settings

Quotafilter:list_quota time, through the expression filter needed quota

Quotascope: Enumeration classes, cluster cluster-level quota (multiple machine requests added), machine machine-level quota Hmaster End Quotatype: Enumeration class throttle and Global_bypass throttlesettings:quotasetting Implementation class, the most commonly used threshold processing logic Hbase.proto: Added enumeration class Timeunit (Time unit) Master.proto: Added the following
Setquotarequest:user_name,
User_group,
Namespace
TABLE_NAME,
Remove_all,
Bypass_globals,
Throttlerequest,
and Setquota (setquotarequest) returns (Setquotaresponse);Quota.proto:quotascope,timedquota,throttletype (threshold type, read-write operation quantity, read-write operation data volume, Request_number. Request_size. ) Basemasterandregionobserver.java: Interface class: Pre/post set User/table/namespace Quota defines some actions to be performed before and after the Quota is set, such as when the ACL is opened To check for admin permissions Basemasterobserver.java: Ditto
_Masterobserver.java: Ditto HMaster.java:MasterQuotaManager Quotamanager; class addition method for RPC MasterRpcServices.java:Master-end processing Setquota (Rpccontroller C, setquotarequest req) MasterQuotaManager.java:master-End Management quota classes, Administrator settings quota (Admin.setquota) –> Masterrpcservice calls the Masterquotamanager Setquota method, the manager invokes Quotautil additions and deletions to the Hbase:quota table method, and creates the Hbase:quota table Quotatableutil: When setting up quota, the Hbase:quota table is manipulated by this class Regionserver End: quotacache:regionserver Cache Quota set on the class, which has a chore implementation, timed from the Hbase:quota table to get quota settings
Chore:hbase A class that does a timed task, applying a timeoutmonitor,connectioncache in the Cleaner,balancerchore,clusterstatuschore,cleanerchore , Compactionchecker, etc.quotaretriever: Scan traversal quota set class, Query Quota_table_name table, and put the results into the cache inside Quotalimiter: interface class, implementation class is Timebasedlimiter.java Factory class of QuotaLimiterFactory.java:QuotaLimiter used in the QuotaState.java:QuotaCache. The Checkquota method of this class is called after the rpcservice of the DefaultOperationQuota.java:regionserver is accepted, and the important method 1:checkquota the Estimateconsume method to compute The consumption of this request, judge whether exceed the threshold value, the important method 2:close, inside Call the Limiter.addoperationsize, add up the operation value, for next request judgment Noopoperationquota.java: An empty Operationquota implementation that uses an instance of this Operationquota type without opening the quota mechanism Noopquotalimiter.java: An empty Quotalimiter implementation that uses an instance of this Quotalimiter type without opening the quota mechanism call this manager in RegionServerQuotaManager.java:RSRpcService to get the Operationquota object, Execute Manager.checkquota (inside again call Operationquota.checkquota) get operation Finish, execute Quota.addgetresult, Finally, call Quota.close to add the value of this operation to the accumulator Ratelimiter.java: Class that is actually calculated to exceed the threshold TimeBasedLimiter.java:QuotaLimiter Implementation, encapsulation of the number of read and write, read and write, request times, request the amount of Ratelimiter exception when Quotaexceededexception:quota exceeded threshold when the Throttlingexception:checkquota is abnormal, it throws throttlingexception and inherits the Quotaexceededexception