Hive,pig,hbase Difference and Contact

After looking at the Hadoop ecosystem for a few days, I was confused about hive,pig,hbase, and when I looked at it, I found that I was not the only one who was confused, such as the question that a bird asked for a post, when the use of Hbase and when to use Hive. .... Consult Google, now summarized as follows:

Pig

Pig is a data flow language used to quickly and easily handle huge data. Pig consists of two parts: Pig Interface,pig Latin. Pig can handle HDFs and hbase data very easily, and like hive, Pig can handle what it needs to do very efficiently, and can save a lot of labor and time by directly manipulating pig queries.

When you want to do some conversion on your data, and you don't want to write mapreduce jobs, you can use pig.

Hive

Originated in the role of facebook,hive as a data warehouse in Hadoop. Build on top of the Hadoop cluster and manipulate the SQL-like interface for data stored on the Hadoop cluster. You can do select,join with HIVEQL, and so on.

If you have data warehousing requirements and you're good at writing SQL and don't want to write mapreduce jobs, you can use hive instead.

HBase

HBase runs on HDFs as a column-oriented database, andHDFs lacks the read and write operations that HBase does. HBase is modeled on Google bigtable and stored as a key-value pair. The goal of the project is to quickly locate and access the required data within billions of rows of data in the host.

HBase is a database, a NoSQL database that provides the ability to read and write like other databases, Hadoop does not meet real-time needs, and HBase is ready to meet. If you need real-time access to some data, put it into hbase.

You can use Hadoop as a static data warehouse, HBase as the data store, and the data that makes some things change.

Pig VS Hive

Hive is more suitable for data warehouse tasks, and hive is used primarily for static structures and for work that requires frequent analysis. The similarity between hive and SQL makes it an ideal intersection for Hadoop to be combined with other BI tools.

Pig gives developers more flexibility in large data sets and allows for the development of concise scripts to transform data streams for embedding into larger applications.

Pig is relatively lightweight compared to hive, and its main advantage is that it can drastically reduce the amount of code compared to using Hadoop Java APIs directly. Because of this, pig still attracts a lot of software developers.

Both hive and pig can be used in combination with hbase, and hive and pig also provide high-level language support for HBase, making data statistics processing on hbase very simple

Hive VS HBase

Hive is a batch system built on top of Hadoop to reduce the work of mapreduce jobs, and HBase is a project to support the shortcomings of Hadoop for real-time operations.

Imagine you are operating Rmdb database, if it is a full table scan, use Hive+hadoop, if it is indexed access, use Hbase+hadoop.

The hive query is that mapreduce jobs can be more than 5 minutes to hours, hbase is very efficient and certainly more efficient than hive.

Reference:

Hadoop Toolbox:when to use what

How does Hive compare to HBase?

Comparing Pig Latin and SQL for constructing Data processing pipelines

When to use Pig Latin versus Hive SQL?