HBase entry 4-storage

The previous articles described how to install HBase, how to use Hbase commands and APIs, and how to optimize HBase, this article titled introduction to HBase 4 describes how to put HBase data on HDFS. At present, I am an absolute newbie to HBase. please correct me if I have any mistakes in this article. Thank you. OK.

The previous articles described how to install HBase, how to use Hbase commands and APIs, and how to optimize HBase, this article titled introduction to HBase 4 describes how to put HBase data on HDFS. At present, I am an absolute newbie to HBase. please correct me if I have any mistakes in this article. Thank you. OK.

The previous articles described how to install HBase, how to use Hbase commands and APIs, and how to optimize HBase, this article titled introduction to HBase 4 describes how to put HBase data on HDFS. At present, I am an absolute newbie to HBase. please correct me if I have any mistakes in this article. Thank you.

OK, proceed with the question .........

A table created in HBase can be distributed across multiple Hregion, that is, a table can be split into multiple parts, each of which is called an Hregion. Each Hregion stores a certain segment of continuous data in a table. Each Hregion block in the large table created by the user is maintained by the Hregion server. Accessing the Hregion block is performed through the Hregion server, an Hregion block corresponds to an Hregion server, and a complete table can be saved on multiple Hregion instances. The relationship between the HRegion Server and Region is one-to-many. Each HRegion is physically divided into three parts: Hmemcache, Hlog, and HStore ).
The above relationships are in my mind ,:

1. Relationships between HRegionServer, HRegion, Hmemcache, Hlog, and HStore ,:

2. Distribution of data in the HBase table and HRegionServer ,:

HBase read data
HBase preferentially reads data from HMemcache. If data in Hstore is not retrieved, HBase improves data reading performance.

HBase write data
HBase writes data to HMemcache and Hlog, and HMemcache creates a Cache. Hlog synchronizes transaction logs of Hmemcache and Hstore. When Flush Cache is initiated, data is persisted to Hstore and HMemecache is cleared.

When the client accesses the data, it uses Hmaster. Each Hregion server maintains a persistent connection with the Hmaster server, which is the manager of the HBase distributed system, his main task is to tell each Hregion server which Hregion it wants to maintain. All user data can be stored in the Hadoop distributed file system. If the master server Hmaster crashes, the entire system will be invalid. Next I will consider how to solve the Hmaster SPFO problem. This problem is a bit similar to the Hadoop SPFO problem. Only one NameNode maintains the global DataNode. Once HDFS crashes, some people also say that Heartbeat is used to solve this problem, but I always want to find other solutions. There is always a way to spend more time.

Yesterday in the hadoop-0.21.0, hbase-0.20.6 environment tossing for a long time, has been reported error, error information is as follows:
Exception in thread "main" java. io. IOException: Call to localhost/serv6: 9000 failed on local exception: java. io. EOFException
10/11/10 15:34:34 ERROR master. HMaster: Can not start master
Java. lang. reflect. InvocationTargetException
At sun. reflect. NativeConstructorAccessorImpl. newInstance0 (Native Method)
At sun. reflect. nativeconstruct%cessorimpl. newInstance (nativeconstruct%cessorimpl. java: 39)
At sun. reflect. delegatingconstruct%cessorimpl. newInstance (delegatingconstruct%cessorimpl. java: 27)
At java. lang. reflect. Constructor. newInstance (Constructor. java: 513)
At org. apache. hadoop. hbase. master. HMaster. doMain (HMaster. java: 1233)
At org. apache. hadoop. hbase. master. HMaster. main (HMaster. java: 1274)

It is depressing that HDFS cannot be connected to or connected to HMaster.
Let me think about it. I think it is possible that the exception of java. io. EOFException is caused by inconsistent RPC protocol formats? That is to say, the version of the server and client is inconsistent? After changing a HDFS server end, everything is good, it is indeed a version of the problem, the last use of hadoop-0.20.2 with hbase-0.20.6 is relatively stable.
Final effect:

To view the larger image, click here for some text instructions:
1. hadoop version 0.20.2,
2. hbase version 0.20.6,
3. Create a table tab1 in hbase and exit the hbase shell environment,
4. Run the hadoop command to check whether the file system contains a new tab1 directory,
The preceding figure shows that HBase runs in the Distributed File System Apache HDFS.

Related Articles:
Hbase entry 6-communication between MySQL (RDBMS) and HBase
Lily-distributed search based on HBase
MySQL migration tool to Hive/HBase
HBase entry 5 (cluster)-load splitting and forwarding failure
Hive entry 3-Integration of Hive and HBase
HBase entry 4
HBase entry 3
Introduction to HBase 2-Examples of Java operations on HBase
HBase Basics
Hbase-based distributed messaging (IM) system-JABase

-End-