HBase createTable server-side source code analysis, hbasecreatetable

HBase createTable server-side source code analysis, hbasecreatetable

All HBase request calls are implemented through the RPC mechanism. After the RPCServer listens to the request, it will parse the request content and then call the actual method on the server based on the parsing method and parameters, this is also a classic practice of the remote proxy mode. The createTable request is implemented in HMaster, but the actual table creation process is in the CreateTableHandler class, next, we will analyze the table creation process in HBase in detail.

1. createTable Implementation of HMaster

The following code shows the createTable Process Code in HMaster:

 public void createTable(HTableDescriptor hTableDescriptor,                            byte[][] splitKeys) throws IOException {        if (isStopped()) {            throw new MasterNotRunningException();        }String namespace = hTableDescriptor.getTableName().getNamespaceAsString();        ensureNamespaceExists(namespace);        HRegionInfo[] newRegions = getHRegionInfos(hTableDescriptor, splitKeys);        checkInitialized();        sanityCheckTableDescriptor(hTableDescriptor);        if (cpHost != null) {            cpHost.preCreateTable(hTableDescriptor, newRegions);        }        LOG.info(getClientIdAuditPrefix() + " create " + hTableDescriptor);        this.service.submit(new CreateTableHandler(this,                this.fileSystemManager, hTableDescriptor, conf,                newRegions, this).prepare());        if (cpHost != null) {            cpHost.postCreateTable(hTableDescriptor, newRegions);        }    }

Before creating a table, perform the following operations:
1. Check whether the Master is running normally.
2. ChecknamespaceExist?
3.HRegionInfoThe class contains the HRegion-related information getHRegionInfos (). The function obtains the HRegion information corresponding to the table according to the splitKeys and table description information.

It is necessary to explain hereHRegion,HRegionStores table data information. It contains all columns of each row. One table contains one or more hregion, and each hregion contains multipleHStores, Each HStores contains a part of the Rows and Columns of the corresponding part.
Each HRegion contains a [startKey, endKey) to identify the range of the row it stores.TableNameAndkey rangeUnique

4. Check whether the Master node has been initialized.
5. Check whether the table information meets the requirements
6. Create a table. The table creation process is as follows:

1. cpHost.preCreateTable(hTableDescriptor, newRegions);2. submit(new CreateTableHandler(this,            this.fileSystemManager, hTableDescriptor, conf,            newRegions, this)3.cpHost.postCreateTable(hTableDescriptor, newRegions);

Steps 1 and 3 are hook functions reserved for the coprocessor so that application developers can dynamically add new functions.
Next, we will analyze the table creation operations in step 2.

2. Table creation implementation in CreateTableHandler

In fact, in the codethis.service.submit(new CreateTableHandler(this,
this.fileSystemManager, hTableDescriptor, conf,
newRegions, this).prepare());The calling process is divided into two parts: prepare and submit. Let's take a look at the work of prepare ().

2.1 preparations before table creation: prepare () Analysis

  public CreateTableHandler prepare()      throws NotAllMetaRegionsOnlineException, TableExistsException, IOException {    // Need hbase:meta availability to create a table    try {      if (server.getMetaTableLocator().waitMetaRegionLocation(          server.getZooKeeper(), timeout) == null) {        throw new NotAllMetaRegionsOnlineException();      }      // If we are creating the table in service to an RPC request, record the      // active user for later, so proper permissions will be applied to the      // new table by the AccessController if it is active      if (RequestContext.isInRequestContext()) {        this.activeUser = RequestContext.getRequestUser();      } else {        this.activeUser = UserProvider.instantiate(conf).getCurrent();      }    }    //acquire the table write lock, blocking. Make sure that it is released.    this.tableLock.acquire();    boolean success = false;    try {      TableName tableName = this.hTableDescriptor.getTableName();      if (MetaTableAccessor.tableExists(this.server.getConnection(), tableName)) {        throw new TableExistsException(tableName);      }      checkAndSetEnablingTable(assignmentManager, tableName);      success = true;    } finally {      if (!success) {        releaseTableLock();      }    }    return this;  }

The main work here is as follows:
1. Obtain hbase: meta information. meta is a special table of hbase. It stores information about the RegionServer on HBase and its distribution.
2. Check and record the permissions of the user who created the table.
3. Get the table write lock
4. Check whether the table exists. If yes, an exception is thrown.
5. To prevent multiple threads from initiating the creation of the same table, you can set the table enable before the table creation fails, so that other threads can no longer create the table.

The preparation before table creation ends here. analyze the specific table creation process

2.2 table creation implementation handleCreateTable Process Analysis

HandleCreateTable mainly involves three things: 1. Create a table on the disk, 2. meta table, and 3. Assign the corresponding regionserver to the newly created table. The Code is as follows:
This piece of code is divided into eight small steps, we analyze one by one,

1. Create a table Descriptor

// 1. Create Table DescriptorPath tempTableDir = FSUtils.getTableDir(tempdir, tableName);new FSTableDescriptors(this.conf).createTableDescriptorForTableDirectory(tempTableDir, this.hTableDescriptor, false);Path tableDir = FSUtils.getTableDir(fileSystemManager.getRootDir(), tableName);

First, create a temporary folder, then create the corresponding file table descriptor, and finally create the path of the table in the file system,

HBase table data corresponds to a folder in the file system, which is the table name.

2. Create Regions

// 2. Create RegionsList<HRegionInfo> regionInfos = handleCreateHdfsRegions(tempdir, tableName);

Create an in-disk data structure for the table. The HRegion storing the table data is created internally and the hregioninfo information is returned.

3. Move the Temporary Folder in step 1 to the root directory of HBase. If hregions is successfully created, continue with the following steps:

if (regionInfos != null && regionInfos.size() > 0) {// 4. Add regions to META    addRegionsToMeta(regionInfos);// 5. Add replicas if needed    regionInfos = addReplicas(hTableDescriptor, regionInfos);// 6. Trigger immediate assignment of the regions in round-robin fashion    ModifyRegionUtils.assignRegions(assignmentManager, regionInfos);}

// 7. Set table enabled flag up in zk.try {        assignmentManager.getTableStateManager().setTableState(tableName,        ZooKeeperProtos.Table.State.ENABLED);    } catch (CoordinatedStateException e) {      throw new IOException("Unable to ensure that " + tableName + " will be" +        " enabled because of a ZooKeeper issue", e);    }

8. Follow the new tabledescripter cache

// 8. Update the tabledescriptor cache.((HMaster) this.server).getTableDescriptors().get(tableName);

So far, the source code analysis of the database table creation process on the server end is complete.