Precautions for using htable and htablepool

Both htable and htablepool are part of hbase client APIs. They can be used to perform crud operations on hbase tables. The following is a summary of the precautions during the use of the two Based on the application in the project.

Htable

Htable is a Java API object for communication between the hbase client and the hbase server. The client can perform crud operations (add, delete, modify, and query) with the htable object and the server ). It is easy to create:

Configuration conf = HBaseConfiguration.create();HTable table = new HTable(conf, "tablename");//TODO CRUD Operation……

Precautions for using htable:

1.Htable avoidanceObject creation overhead

This is because the client needs to perform a series of operations after creating an htable object: Check. meta. the table confirms whether the hbase table with the specified name exists and whether the table is valid. The whole time overhead is heavy and may take several seconds, therefore, it is best to create the required htable object at one time when the program starts. If Java API is used, it is generally created in the constructor and reused directly after the program starts.

2.HtableThe object is not thread-safe.

Htable objects are not thread-safe for the client to read and write data. Therefore, when multithreading, you must create and reuse an htable object for each thread. Do not share htable objects with different objects, especially when the Client Auto flash is set to false, data inconsistency may occur due to the existence of local write buffer.

3.HtableSharing Configuration between objects

The htable object shares the configuration object, which has the following advantages:

• Shared zookeeper connection: each client needs to establish a connection with zookeeper to query the user's table regions location. This information can be cached and shared after the connection is established;
• Share public resources: the client needs to use zookeeper to find-root-and. meta. table, which requires network transmission overhead. After the client caches these public resources, it can reduce the subsequent network transmission overhead and speed up the search process.

Therefore, compared with the following method:

HTable table1 = new HTable("table1");HTable table2 = new HTable("table2");

The following method is more effective:

Configuration conf = HBaseConfiguration.create();HTable table1 = new HTable(conf, "table1");HTable table2 = new HTable(conf, "table2");

Note: even high-load multi-threaded programs do not find any performance problems caused by shared configuration. If not, try not to share the configuration.

Htablepool

Htablepool can solve the thread insecurity problem of htable. By maintaining a fixed number of htable objects, htable resource objects can be reused during program running.

Configuration conf = HBaseConfiguration.create();HTablePool pool = new HTablePool(conf, 10);

1. htablepool can automatically create htable objects, which is completely transparent to clients and can avoid concurrent data modification between multiple threads.

2. htable objects in the htablepool are connected by public configuration, which can reduce network overhead.

Htablepool is easy to use: before each operation, use the gettable method of htablepool to obtain an htable object, and then perform put/get/scan/delete operations, finally, the htable object is put back into the htablepool through the puttable method of htablepool.

The following is a simple example of using htablepool:

public void createUser(String username, String firstName, String lastName, String email, String password, String roles) throws IOException {　　HTable table = rm.getTable(UserTable.NAME);　　Put put = new Put(Bytes.toBytes(username));　　put.add(UserTable.DATA_FAMILY, UserTable.FIRSTNAME,　　Bytes.toBytes(firstName));　　put.add(UserTable.DATA_FAMILY, UserTable.LASTNAME,　　　　Bytes.toBytes(lastName));　　put.add(UserTable.DATA_FAMILY, UserTable.EMAIL, Bytes.toBytes(email));　　put.add(UserTable.DATA_FAMILY, UserTable.CREDENTIALS,　　　　Bytes.toBytes(password));　　put.add(UserTable.DATA_FAMILY, UserTable.ROLES, Bytes.toBytes(roles));　　table.put(put);　　table.flushCommits();　　rm.putTable(table);}

The actual performance of multi-threaded htablepool needs to be obtained through actual testing.

 

Article transferred from: http://www.cnblogs.com/panfeng412/archive/2012/07/11/htable-and-htablepool-apply-notes.html

Precautions for using htable and htablepool