Hadoop, though powerful, is not everything.

With the continuous expansion of Hadoop application, many people fall into the blind worship of it, think it can solve all problems. While Hadoop is a great distributed large http://www.aliyun.com/zixun/aggregation/14206.html "> Data Computing Framework, Hadoop is not omnipotent." For example, the following scenarios are not suitable for using Hadoop:

1. Low Latency Data access

Hadoop does not apply to data access that requires real-time queries and low latency. The fact that a database can reduce latency and rapid response through index logging is simply not a substitute for Hadoop. But if you really want to replace a real-time database, you can try hbase to realize real-time reading and writing in the database.

2. Structured data

Hadoop does not apply to structured data, but it works well for semi-structured and unstructured data. Unlike Hadoop and RDBMS, distributed storage is generally used, so there is a problem with latency when querying.

3, when the amount of data is not big

How much data does Hadoop normally apply to? The answer is: TB or PB. Using Hadoop is no good when your data is only dozens of GB. According to the needs of the enterprise selective use of Hadoop, do not blindly follow the trend. Hadoop is powerful. However, before using Hadoop or large data, companies should first identify their goals and decide whether to choose the right tool.

4, a large number of small files

Small files refer to files that have a much smaller size than the HDFS block size (default 64M). If you store a large number of small files in the HDFs, each file corresponds to a block, then you will consume namenode large amounts of memory to hold the information of these blocks. If the size of small files is larger, it will be beyond the current level of computer hardware to meet the limit.

5. Too many writes and file updates

HDFs is a number of ways to read. Hadoop has no way to support when there are too many file update requirements.

6, MapReduce may not be the best choice

MapReduce is a simple parallel programming model. is a powerful tool for large data parallel computing, but many computational tasks, work and algorithms are inherently unsuitable for using mapreduce framework.

If you let data share in MapReduce, you can do this:

Iteration: Run multiple MapReduce jobs, the previous MapReduce output as input to the next MapReduce.

Share state information: But do not share information in memory, as each mapreduce is run on a single JVM.