Spring for hadoop--working with HBase (i)

The recent use of Web projects in Hadoop,hbase, a summary of the use of spring for Hadoop (SHDP), mainly uses some of the encapsulated HBase modules provided in SHDP. This blog will be written in two parts:

1. Basic understanding of spring for hadoop--working with HBase (combined with official documentation and own experience)
2. Use cases in the project

1. Basic understanding of spring for hadoop--working with HBase

SHDP has integrated hbase and spring, and developers can easily manipulate hbase with this framework. (You can associate the database with Spring+hibernate.)

SHDP provides a configuration file to set HBase through Hbase-configuration: for example:

<!---<configuration-ref= "Hadoopcconfiguration "/>

The above declaration makes it easier to create configrution objects for HBase, in addition to support for managing HBase connections: When application context is closed, All open connections in HBase can be adjusted through the Stop-proxy and Delete-connetcion properties.

<!---<stop-proxy= "false"  Delete-connection= "true">  foo=bar  property=value</  hdp:hbase-configuration>

Alternatively, you can let clients connect to HBase by specifying the port number of ZK

<!---
<zk-quorum= "${hbase.host}"  zk-port= "${hbase.port}"> 

Of course, you can also add configurations for this profile by introducing the properties of other profiles, such as:

<properties-ref= "Some-props-bean"  properties-location  = "Classpath:/conf/testing/hbase.properties"/>

Support for DAO

 SHDP through  org.springframework.data.hadoop.hbase  The package provides hbase support for the DAO, via  hbasetemplate     and several other callback functions such as  tablecallback  ,    rowmapper    ,  resultsextractor    query, scanner preparation and analysis of some results, greatly improving the efficiency of development  

  The core of this DAO is  hbasetemplate    a high-level abstract class that interacts with HBase, the use of which requires the configuration of HBase to be set, once it is successfully set hbasetemplate  is thread-safe and can be reused by multiple instances at the same time (  frequent Web projects that access HBase are very important!!! )  

//default HBase configuration < hdp:hbase-configuration /> //Wire hbase configuration (using default name ' Hbaseconfiguration ') into the template <  ID= "Htemplate"  class= " Org.springframework.data.hadoop.hbase.HbaseTemplate "  p:configuration-ref=" Hbaseconfiguration "/>

HbaseTemplateIt also provides a general callback function for the execution logic of the table or for the extraction of the results and rows.

//writing to ' MyTable ' Template.execute ("MyTable", new TableCallback<Object>  () {  @Overridepublic Object dointable (htable table) throws Throwable {    put p = new put (Bytes.tobytes ("so Merow "));    P.add (Bytes.tobytes ("Somecolumn"), Bytes.tobytes ("Somequalifier"), Bytes.tobytes ("Avalue"));    Table.put (p);    return null;  }});

This code shows the use of TableCallback, which completes the lookup of the table and the cleanup of resources without the explicit processing of the user, noting that when using the callback function, HBase Exceptions thrown by the API will be automatically captured and converted to spring DAO's exception, and cleanup of resources will be explicitly called . In addition , Hbasetemplate provides a packaged method for some common operations that users can use directly without having to write their own callback functions .

//Read each row from ' MyTable ' List < String > rows = Template.find ("MyTable", "Somecolumn", New RowMapper<String> () {  @Override public    String maprow (Result result,int rowNum) throws exception{    return Result.tostring ();  }));

(the above two pieces of code are familiar with spring and Hibernater can easily associate the spring with the Hibernatecallback and Hibernatertemplate) besides HbaseTemplate , org.springframework.data.hadoop.hbase HbaseInterceptor类 through 和 HbaseSynchronizationManager The hbaseinterceptor class supports the automatic binding of hbase tables to the current thread, that is, every DAO action class executed on HBase will be Wrapper, so once you find that the table you are using will be bound to the current thread, then you will not need to look up the table again when you use the table (it also achieveshbase Connection Pool effect), after the call is finished, the table will be closed by itself.  Reference: Spring for Hadoop official documentation(I will summarize the actual combat part of the project, including the source code of this part and the considerations in the development)

Spring for hadoop--working with HBase (i)