1. Environment Preparation:
Maven
Eclipse
Java
Spring version 3..2.9
2. Maven pom.xml Configuration
12345678910111213141516 |
<!-- Spring hadoop -->
<
dependency
>
<
groupId
>org.apache.hbase</
groupId
>
<
artifactId
>hbase-client</
artifactId
>
<
version
>0.96.1.1-hadoop2</
version
>
</
dependency
>
<
dependency
>
<
groupId
>org.springframework.data</
groupId
>
<
artifactId
>spring-data-jpa</
artifactId
>
<
version
>1.6.0.RELEASE</
version
>
</
dependency
>
<
dependency
>
<
groupId
>org.springframework.data</
groupId
>
<
artifactId
>spring-data-hadoop</
artifactId
>
<
version
>2.0.2.RELEASE</
version
>
</
dependency
>
|
3. Spring and Hadoop, hbase related configuration files
12345678910111213141516171819 |
<?
xml version="1.0" encoding="UTF-8"?>
<
beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:util="http://www.springframework.org/schema/util"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
xmlns:mongo="http://www.springframework.org/schema/data/mongo"
<span style="color: #ff0000;">xmlns:hdp="http://www.springframework.org/schema/hadoop"</
span
>
xmlns:cache="http://www.springframework.org/schema/cache" xmlns:c="http://www.springframework.org/schema/c"
xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="
http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd
http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-mongo.xsd
<
span style="color: #ff0000;">http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd</
span
>
http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
|
Where red is the spring Hadoop XML namespace configuration.
Hadoop hbase The relevant configuration files are as follows:
<!--default Properties-->
"${delete-connection}" zk-quorum="${ Hbase.zookeeper.quorum} "zk-port=" ${hbase.zookeeper.property.clientport} "/>
the corresponding properties as follows:
hbase.zookeeper.property.clientport=2181
hbase.zookeeper.quorum=192.98.8.224
hbase.master=192.98.8.224:600000
fs.default.name=hdfs://192.98.8.224:8010
Delete-connection=true
#hive JDBC URL
Hive.url=jdbc:hive://192.98.8.224:10000/default
Spring Hbasetemplate The configuration is as follows:
<bean id="hbasetemplate" class="Org.springframework.data.hadoop.hbase.HbaseTemplate"> <property name="Configuration" ref="hbaseconfiguration" />
</bean>
Hbasetemplate Using code Examples:
1234567891011 |
Tile t = hbaseTemplate.get(
"GW_TILES"
,
"0_1_1"
,
new RowMapper<Tile>() {
@Override
public
Tile mapRow(Result result,
int
rowNum)
throws
Exception {
// TODO Auto-generated method stub
Tile t =
new
Tile();
t.setData(result.getValue(
"T"
.getBytes(),
"key"
.getBytes()));
return
t;
}
});
|
hbasetemplate Introduction to Common methods:
Hbasetemplate.get ("Gw_tiles", "0_1_1", new rowmapper are commonly used for queries, as shown in the following example:
1234567891011 |
Tile t = hbaseTemplate.get(
"GW_TILES"
,
"0_1_1"
,
new
RowMapper<Tile>() {
@Override
public
Tile mapRow(Result result,
int
rowNum)
throws Exception {
// TODO Auto-generated method stub
Tile t =
new
Tile();
t.setData(result.getValue(
"T"
.getBytes(),
"key"
.getBytes()));
return
t;
}
});
|
Hbasetemplate.execute (Dataidentifier, new tablecallback is commonly used for update operations, as shown in the following example:
12345678910111213141516 |
return
hbaseTemplate.execute(dataIdentifier,
new
TableCallback<Boolean>() {
@Override
public
Boolean doInTable(HTableInterface table)
throws
Throwable {
// TODO Auto-generated method stub
boolean
flag =
false
;
try
{
Delete delete =
new
Delete(key.getBytes());
table.delete(delete);
flag =
true
;
}
catch
(Exception e){
e.printStackTrace();
}
return
flag;
}
});
|
Note : Spring Hbasetemplate is a powerful package for hbase interfaces, with common functionality that uses its powerful interfaces, while complex functions can also use hbase native interfaces such as: Htableinterface, Result and so on. Its class methods are as follows:
At the same time, Hbasetemplate encapsulates the HBase connection pool, and its creation and release is automatically managed through configuration.
Spring Hadoop access to HBase Getting Started