Hive, Hasee, pig differences. docx

Hive

originated in Facebook,hive plays the role of Data Warehouse in Hadoop. Build on top of the Hadoop cluster and manipulate the SQL-like interface for data stored on the Hadoop cluster . You can do select,join with HiveQL, and so on.

If you have the data warehouse requirements and you are good at writing SQL and do not want to write MapReduce jobs can be replaced with Hive .

1,hive is the SQL language, the way to operate the HDFs file system through the database , in order to simplify programming, the underlying calculation is MapReduce.

2,hive is a row-oriented storage database.

3,hive itself does not store and calculate data, it is completely dependent on HDFS and MapReduce,the table in hive pure logic.

4,hive for maintenance purposes only, really look very slow.

Hbase and hive its bottom-up is distributed by MapReduce .

Hbase

HBase runs on HDFs as a column-oriented database ,andHDFs lacks the read and write operations thatHBase does . HBase is modeled on Google BigTable and stored as a key-value pair. The goal of the project is to quickly locate and access the required data within billions of rows of data in the host.

HBase is a database, a NOSQL database that provides the ability to read and write like other databases,Hadoop does not meet real-time needs, andHBase is ready to meet. If you need real-time access to some data, put it into HBase.

You can use as a static data warehouse,HBase acts as a data store and places data that can be changed by some operations.

1,HBase for the query, it through the organization of all the machines inside the node, to provide a large deposit Hash table

2,hbase is not a relational database, but a column-oriented distributed database developed on HDFS and does not support SQL.

3,hbase is a physical table, not a logical table, provides a large memory hash table, the search engine through it to store indexes, convenient query operation.

4,hbase is a column store.

Pig

Pigpig is a data flow language used to quickly and easily handle huge data. Pig consists of two parts:pig interface,pig Latin. Pig can handle HDFS and HBase data very easily , and like Hive, pig can handle its needs very efficiently by directly manipulating Pig queries can save a lot of labor and time.

when you want to make some conversions on your data and don't want to write MapReduce jobs can use Pig.

What is the faster the same amount of data,hive and hbase ?

if the amount of data conforms to hbase memory requirements, HBase is sure to be faster (because it is physical memory).

in excess of hbase requirements,hive may be better.

Pig VS Hive

Hive is more suitable for data warehouse tasks,and hive is used primarily for static structures and for work that requires frequent analysis. the similarity between Hive and SQL makes it an ideal intersection for Hadoop to be combined with other BI tools.

Pig gives developers more flexibility in large data sets and allows for the development of concise scripts to transform data streams for embedding into larger applications.

Pig is relatively lightweight compared to Hive , and its main advantage is that it can drastically reduce the amount of code compared to using Hadoop Java APIs directly. Because of this,Pig still attracts a lot of software developers.

Both hive and pig can be used in combinationwith HBase, and Hive and pig also provide high-level language support for HBase, making the hbase It 's very easy to do data statistics processing on

Hive VS Hbase

Hive is a batch system built on top of Hadoop to reduce the work of MapReduce jobs, andHBase is to support the process of making up Hadoop Items that are defective for real-time operations .

Imagine that you are operating RMDB Database, if it is a full table scan, use hive+hadoop, if it is indexed access, use hbase+hadoop .

The hive query is that MapReduce jobs can be more than 5 minutes to hours,HBase is very efficient and certainly more efficient than Hive.

Hive, Hasee, pig differences. docx