HBase coprocessor Combat

First, the coprocessor introduction

Definition: HBase provides a mechanism (framework) for computing some of the logic of the user on the data store side and the HBase server. The coprocessor allows users to run their own code on the HBase server.

Category: System coprocessor, table coprocessor

Observer, equivalent to a trigger inside a relational database

Endpoint, similar to stored procedures

Second, OBserver

The design intent of the observer is to allow the user to overload the Upcall method of the coprocessor framework by inserting code, and the specific event-triggering callback method is executed by the core code of HBase. Can be divided into Regionobserver, Regionserverobserver, Masterobserver, Walobserver

Regionobserver provides client-side data manipulation Events hooks: Get, Put, Delete, scan, etc.

Regionserverobserver specializes in dealing with some of the events on Regionserver

Masterobserver provides ddl-type of operation Hooks. such as creating, deleting, modifying data tables, etc.

Walobserver provides Wal-related operations

These interfaces can be used in the same place at the same time, in different priority order. Users can implement the complex HBase function layer arbitrarily based on the coprocessor. HBase has a number of events that can trigger the observer method, which is integrated in the HBase API from the HBase0.92 version. However, these APIs may vary for a variety of reasons

Changes, different versions of the interface changes relatively large. The Regionobserver works as shown in.

Third, Endpoint

The terminal is the interface of the dynamic RPC plug-in, and its implementation code is installed on the server side, thus being able to wake up via HBase RPC. The Client class library provides a very convenient way to invoke these dynamic interfaces, which can invoke a terminal at any time, and their implementation code will be executed remotely by the target region, and the result will be returned to the terminal. Users can combine the

Add new features to hbase with these powerful plug-in interfaces.

Endpoint service-Side authoring

1. Create a Endpoint.proto file, generate a Java file

2. Option Java_package = "Edu.endpoint";
4. Option Java_outer_classname = "Sum";
6. Option java_generic_services = true;
8. Option Java_generate_equals_and_hash = true;
10. option optimize_for = speed;
12. Message Sumrequest {
14. Required String family = 1;
16. Required String column = 2;
18. }
20. Message Sumresponse {
22. Required Int64 sum = 1 [default = 0];
24. }
28. Service Sumservice {
30. RPC Getsum (sumrequest)
32. Returns (Sumresponse);
34. }

2. Generate the Java code for the. Proto file.

Protoc endpoint.proto–java_out=./

Copy the generated. java files to the source code package file for the Java project that corresponds to eclipse. Add all dependent library files under HBase lib.

3. Writing the service-side Sumendpoint.java class

2. Package edu.endpoint;
6. Import java.io.IOException;
8. Import java.util.ArrayList;
10. Import java.util.List;
14. Import Org.apache.hadoop.hbase.Coprocessor;
16. Import org.apache.hadoop.hbase.CoprocessorEnvironment;
18. Import Org.apache.hadoop.hbase.client.Scan;
20. Import org.apache.hadoop.hbase.coprocessor.CoprocessorException;
22. Import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
24. Import Org.apache.hadoop.hbase.regionserver.InternalScanner;
26. Import org.apache.hadoop.hbase.util.Bytes;
30. Import Com.google.protobuf.RpcCallback;
32. Import Com.google.protobuf.RpcController;
34. Import Com.google.protobuf.Service;
38. Import Edu.endpoint.Sum.SumRequest;
40. Import Edu.endpoint.Sum.SumResponse;
42. Import Edu.endpoint.Sum.SumService;
46. Public class Sumendpoint extends Sumservice implements coprocessor,coprocessorservice{
48. private regioncoprocessorenvironment env;
52. public void Getsum (Rpccontroller controller,sumrequest request,rpccallback<sumresponse> done) Throws ioexception{
54. Scan scan = new Scan ();
56. Scan.addfamily (Bytes.tobytes (request.getfamily ()));
58. Scan.addcolumn (Bytes.tobytes (request.getfamily ()), Bytes.tobytes (Request.getcolumn ()));
60. Sumresponse response = null;
62. Internalscanner scanner = null;
64. try{
66. Scanner = Env.getregion (). Getscanner (scan);
68. list<cell> results = new arraylist<cell> ();
70. Boolean hasmore = false;
72. Long sum = 0;
74. Do {
76. Hasmore = Scanner.next (results);
78. For (Cell cell:results) {
80. Sum + = Long.parselong (new String (Cellutil.clonevalue (cell)));
82. }
84. Results.clear ();
86. } while (Hasmore);
88. Response = Sumresponse.newbuilder (). Setsum (sum). Build ();
90. }catch (IOException e) {
92. Responseconverter.setcontrollerexception (controller,e);
94. }finally {
96. if (scanner!=null) {
98. try {
100. Scanner.close ();
102. } catch (IOException e) {
104. //TODO auto-generated catch block
106. E.printstacktrace ();
108. }
110. }
114. }
116. Done.run (response);
118. }
120. Public Service GetService () {
122. return this ;
124. }
126. @Override
128. public void Start (Coprocessorenvironment env) throws IOException {
130. if (env instanceof regioncoprocessorenvironment) {
132. this.env = (regioncoprocessorenvironment) env;
134. }Else {
136. throw New Coprocessorexception ("No load Region");
138. }
140. }
142. @Override
144. public void Stop (coprocessorenvironment arg0) throws IOException {
146. }
148. }

HBase coprocessor Combat