Alex's Hadoop Rookie Tutorial: 8th Lesson The Java invocation method of Hbase

Statement

• This article is based on CentOS 6.x + CDH 5.x
• In this example, Hbase is installed in cluster mode
• This article is based on maven3.5+ and Eclipse 4.3
• After the tutorial, we must look at the following

We do not build hbase to use the shell to check the data, we are writing HBase-based applications, so learning how to use Java to invoke HBase is a required course. Setting up the project open Eclipse to build a Maven project, archetype selected QuickStart, Project Artifactid and GroupId casually
Modify the Pom.xml to modify the JDK to 1.6+, and introduce the Hadoop related jar package

<project xmlns= "http://maven.apache.org/POM/4.0.0" xmlns:xsi= "Http://www.w3.org/2001/XMLSchema-instance" xsi: schemalocation= "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" >< Modelversion>4.0.0</modelversion><groupid>org.crazycake</groupid><artifactid> Playhbase</artifactid><version>0.0.1-snapshot</version><packaging>jar</packaging ><name>playhbase</name><url>http://maven.apache.org</url><properties>< Project.build.sourceencoding>utf-8</project.build.sourceencoding></properties><resources> <resource><directory>${basedir}/conf</directory><filtering>false</filtering>< Includes><include>hbase-site.xml</include></includes></resource></resources> <dependencies><dependency><groupid>junit</groupid><artifactid>junit</ artifactid><version>3.8.1</Version><scope>test</scope></dependency><dependency><groupid>org.apache.hbase </groupId><artifactId>hbase-client</artifactId><version>0.98.4-hadoop2</version> </dependency></dependencies><build><plugins><plugin><artifactId> Maven-compiler-plugin</artifactid><version>2.0.2</version><configuration><source >1.6</source><target>1.6</target><encoding>UTF-8</encoding><optimise> True</optimise><compilerargument>-nowarn</compilerargument></configuration></plugin ><plugin><groupid>org.apache.maven.plugins</groupid><artifactid>maven-shade-plugin </artifactId><version>2.3</version><configuration><transformers>< transformerimplementation= "Org.apache.maven.plugins.shade.resource.ApacheLicenseResourceTransformer" ></ Transformer></transformers></configuration><executions><execution><phase>package</phase><goals><goal >shade</goal></goals></execution></executions></plugin></plugins></ Build></project>

• In addition to the introduction of HBase jar package, also introduced a MAVEN plug-in called Maven-shade-plugin, this plug-in can prevent "certificate duplication problem", duplicate certificate file will cause hdinsight cluster running error.
• A resource is also added to the configuration, which refers to a configuration file Hbase-site.xml, where the connection information for HBase is written.

Create a configuration file to create a src/main/resources folder and add it to the source folder and create a Hbase-site.xml file within the folder that contains

<?xml version= "1.0"? ><?xml-stylesheet type= "text/xsl" href= "configuration.xsl"?><!--/** * Copyright The Apache Software Foundation * * Licensed to the Apache Software Foundation (ASF) under one * or more contributor L  Icense agreements.  See the NOTICE file * Distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * under the Apache License, Version 2.0 (The * "License");  You are not a use of this file except in compliance * with the License. Obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * unless required by applicab Le law or agreed into writing, software * Distributed under the License is distributed on a "as is" BASIS, * without WAR Ranties or CONDITIONS of any KIND, either express OR implied. * See the License for the specific language governing permissions and * limitations under the License. */--><configuration> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name    >hbase.zookeeper.quorum</name> <value>host1,host2</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> </configuration>

This host1 and host2 is you install zookeeper machine, because I only installed two machines, so only host1 and host2, under normal circumstances is at least 3, and is an odd number of growth

To create a table and insert data create a class called Createtable.java This example from: http://azure.microsoft.com/en-us/documentation/articles/ Hdinsight-hbase-build-java-maven/?rnd=1 I did the translation

Package Org.crazycake.playhbase;import Java.io.ioexception;import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.hbase.hbaseconfiguration;import Org.apache.hadoop.hbase.hcolumndescriptor;import Org.apache.hadoop.hbase.htabledescriptor;import Org.apache.hadoop.hbase.masternotrunningexception;import Org.apache.hadoop.hbase.tablename;import Org.apache.hadoop.hbase.zookeeperconnectionexception;import Org.apache.hadoop.hbase.client.hbaseadmin;import Org.apache.hadoop.hbase.client.htable;import Org.apache.hadoop.hbase.client.put;import Org.apache.hadoop.hbase.util.bytes;public class CreateTable {public static void Main (string[] args) throws Masternotrunningexception, Zookeeperconnectionexception, IOException { Configuration config = hbaseconfiguration.create ();//This side of the annotation is the dynamic setting of zookeeper parameters, if you do not hbase-site.xml or want to dynamically change// Can be set in dynamic mode////Config.set ("Hbase.zookeeper.quorum",//"Zookeepernode0,zookeepernode1,zookeepernode2    "); Config.set ("Hbase.zookeeper.prOperty.clientport "," 2181 ");        Config.set ("hbase.cluster.distributed", "true");//Use a configuration file to create an Admin object hbaseadmin admin = new hbaseadmin (config);        CREATE TABLE Htabledescriptor tabledescriptor = new Htabledescriptor (tablename.valueof ("people"));    Create 2 column cluster tabledescriptor.addfamily (new Hcolumndescriptor ("name"));        Tabledescriptor.addfamily (New Hcolumndescriptor ("ContactInfo"));        Admin.createtable (Tabledescriptor); Let's get some data in there. string[][] people = {{"1", "Marcel", "Haddad", "[email protected]"}, {"2", "Frankli  N "," Holtz "," [email protected] "}, {" 3 "," Dwayne "," McKee "," [email protected] "}, {" 4 "," Rae ", "Schroeder", "[email protected]"}, {"5", "Rosalie", "Burton", "[email protected]"}, {"6", "Gab    Riela "," Ingram "," [Email protected] "}};        htable table = new htable (config, "people");  Insert this data into the table for (int i = 0; i< people.length; i++) {//The first column does Rowkey    Put person = new put (Bytes.tobytes (people[i][0)); Put the Marcel in the first field of the Name column to Person.add (bytes.tobytes ("name"), Bytes.tobytes ("first"), Bytes.tobytes (people[i      ][1]));      Person.add (bytes.tobytes ("name"), Bytes.tobytes ("Last"), Bytes.tobytes (people[i][2]));      Person.add (Bytes.tobytes ("ContactInfo"), Bytes.tobytes ("email"), bytes.tobytes (people[i][3]));    Table.put (person);    }//finally remember to submit and close the table table.flushcommits (); Table.close ();}}

Note:Before running, use the hbase shell to connect, and then run the list command to see if it is normal, if it is not normal to use JPS to see if HBase and Hadoop services are started up, the start is given. Otherwise, Java has run out of error, do not know where the wrong code on the blind cause of wasting time
When everything is ready, run the code! If your code for a long time card master, don't be silly, go to the HBase deployment machine to look at the logs:

tail-200f  /var/log/hbase/hbase-hbase-master-host1.localdomain.log

Look at the results after the run is finished

HBase (main):003:0> scan ' People ' ROW Column+cell  1 Column=contactinfo:email, timestamp=1421338694666, [email protected] 1 Column=name:first, timestamp=1421338694666, value =marcel 1 Column=name:last, timestamp=1421338694666, VA Lue=haddad 2 Column=contactinfo:email, timestamp=14213 38694932, [email protected] 2 Column=name:first, timestamp=142133869493 2, Value=franklin 2 column=name:last, timestamp=142133869 4932, Value=holtz 3 Column=contactinfo:email, Timesta mp=1421338694977, [email protected] 3 Column=name:first, timestamp=1421338694977, Value=dwayne 3 Column=name:last, timestamp=1421338694977, Value=mck EE 4 column=contactinfo:email, timestamp=142133869503  4, [email protected] 4 Column=name:first, timestamp=1421338695034, Value=rae 4 Column=name:last, timestamp=14213386950 Value=schroeder, 5 column=contactinfo:email, timestamp =1421338695054, [email protected] 5 Column=name:first, timestamp=142133 8695054, Value=rosalie 5 column=name:last, timestamp=142 1338695054, Value=burton                                       6 Column=contactinfo:email, timestamp=1421338695076, [email protected] 6 Column=name:first, timestamp=1421338695076, value=g Abriela 6 Column=name:last, timestamp=1421338695076, Valu E=ingram 6 row (s) in 0.3910 seconds

Search by email to create a Searchbyemail class

Package Org.crazycake.playhbase;import Java.io.ioexception;import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.hbase.hbaseconfiguration;import Org.apache.hadoop.hbase.client.htable;import Org.apache.hadoop.hbase.client.result;import Org.apache.hadoop.hbase.client.resultscanner;import Org.apache.hadoop.hbase.client.scan;import Org.apache.hadoop.hbase.filter.comparefilter.compareop;import Org.apache.hadoop.hbase.filter.regexstringcomparator;import Org.apache.hadoop.hbase.filter.singlecolumnvaluefilter;import org.apache.hadoop.hbase.util.bytes;/** * According to email To search for users * @author Alexxiyang (Https://github.com/alexxiyang) * */public class Searchbyemail {public static void main (string[ ] args) throws IOException {//create config configuration config = hbaseconfiguration.create ();//Open tables htable table = new Htable (confi G, "people");//define a series of columns to be used and columns//define the column cluster byte[] contactfamily = bytes.tobytes ("ContactInfo");//Column byte[] Emailqualifier = Bytes.tobytes ("email");//column cluster byte[] namefamily = Bytes.tobytes ("NAme ");//Column byte[] firstnamequalifier = Bytes.tobytes (" First "); byte[] Lastnamequalifier = Bytes.tobytes (" last ");// Create a comparer for a regular expression regexstringcomparator emailfilter = new Regexstringcomparator ("[email protected]");//Create a filter, Pass this regular comparator in singlecolumnvaluefilter filter = new Singlecolumnvaluefilter (contactfamily, Emailqualifier, Compareop.equal, Emailfilter);//Create a Scan object scan scan = new scan ();//pass filter in Scan.setfilter (filter);//Start query, and get the result resultscanner results = Table.getscanner (scan);//Traverse results Print data for (result result:results) {String id = new String (resul T.getrow ()); byte[] Firstnameobj = Result.getvalue (namefamily, firstnamequalifier); String firstName = new string (firstnameobj); byte[] Lastnameobj = Result.getvalue (namefamily, lastnamequalifier); String lastName = new string (lastnameobj); System.out.println (FirstName + "" + LastName + "-ID:" + ID); byte[] Emailobj = Result.getvalue (contactfamily, emailqual Ifier); string email = new string (emailobj); System.out.println (FirstName + "" + LastName + "-" + email + "-ID:" + ID);} Close result Results.close ();//Close Table Table.close ();}}

Run results

Rosalie Burton-id:5rosalie Burton-[email protected]-Id:5

Delete Table Build Deletetable class

Package Org.crazycake.playhbase;import Java.io.ioexception;import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.hbase.hbaseconfiguration;import org.apache.hadoop.hbase.client.hbaseadmin;/** * Delete Table * @author Alexxiyang (Https://github.com/alexxiyang) * */public class Deletetable {public static void main (string[] args) throws IOE Xception {//create config configuration config = hbaseconfiguration.create ();//build adminhbaseadmin admin = new Hbaseadmin (config) ;//First disable table, then deleteadmin.disabletable ("people"); Admin.deletetable ("People");}}

Go to HBase to check the results

HBase (main):004:0> listtable                                                                                                                         employee                                                                                                                      employee2                                                                                                                     student                                                                                                                       users                                                                                                                         4 row (s) in 4.8460 seconds= > ["Employee", "Employee2", "Student", "users"]

People table is gone.
There was a little episode in the middle of my problem that wasted my day: At first I took this tutorial http://azure.microsoft.com/en-us/documentation/articles/ Hdinsight-hbase-build-java-maven/?rnd=1 do when found I ran the code found long time card master, I went to the hbase-master machine to look at the log, no exception. Again to regionserver on the machine to see the log, no exception.
Then I think there may be a problem with hbase, I first stop the program with the HBase Shell to try, built a table, inserted a data, all normal

HBase (main):002:0> create ' users ', ' info ' 0 row (s) in 36.5110 seconds=> hbase::table-usershbase (main):003:0> Li                                                                                                                         Sttable                                                                                                                      Employee                                                                                                                     Employee2                                                                                                                       Student                                                                                                                         Users 4 row (s) in 0.4520 seconds=> ["Employee", "Employee2", "Student", "Users"]hbase (main):004:0> p                              UT ' users ', 1, ' Info:name ', ' Ted ' 0 row (s) in 0.8350 secondshbase (main):005:0> scan ' users ' row            Column+cell                                                                       1 Column=info:name, T imestamp=1421252020520, value=ted 1 row (s) in 0.3140 seconds

This problem is likely to be on the zookeeper, because your Java API is not directly interacting with HBase, is the first through zookeeper interaction, so I went to see the zookeeper log, I use tail monitoring zookeeper log

Tail-200f/var/log/zookeeper/zookeeper.log

Then I run Java code, look at zookeeper have not reported anything unusual, but let me down, zookeeper nothing to report
Using Telnet to connect two zookeeper machines is OK, no problem found.
I have no choice but to use the most earthy method, direct debugging to the source code inside, found the card in the detection of zookeeper is available code
Here comes a localhost Lenovo to what I saw on the HBase Management Interface (HOST1:60010)
It is estimated that this localhost has a problem, causing the Java program to detect 2181 of this machine, so it will definitely get stuck.
Then I went to check my host1 on the Hbase-site.xml file, found that there is no configuration hbase.zookeeper.quorum This parameter, so I configure this parameter, and then restart Hbase-master, and then access host1:60010
This is more presentable, and then I go to run Java code, found the card owner! Breakpoint to checkifbasenodeavailable Discovery acquired configuration or localhost:2181! It seems that the problem is the parsing of the configuration file. Resumes parsing of the breakpoint configuration file. Then I found that the program read the configuration file is not Hbase-site.xml but zoo.cfg, this is what?! I looked at the official document "HBase will load the zoo.cfg inside of the configuration, the hbase-site.xml inside of the cover out.", lying trough! In this case, I will zookeeper configuration file zoo.cfg simply copy to the Conf folder, try again. It's still not working. Then I suddenly thought that this hbase-site.xml did not add to the compilation, how can I be read by Java? So I decisively ignore the tutorial, I set up the resources folder, and put this file into this directory, and then run it!
Resources

• Use the Maven to build Java applications this use HBase with HDInsight (Hadoop)

Alex's Hadoop Rookie Tutorial: 8th Lesson The Java invocation method of Hbase