Big Date Storage

Hadoop is more suitable for solving big data problems, and relies heavily on its big data storage system, namely HDFS and big data processing system. For MapReduce, we know a few questions.

MapReduce in Hadoop is a simple software framework based on which an application can run on a large cluster of thousands of commercial machines and process terabytes of data in parallel with a reliable fault tolerance.

MapReduce is a programming model for parallel computing of large-scale data sets (greater than 1TB) to solve the computational problems of massive data.

This article briefly describes the execution steps and workflow of the mapreduce programming model in the form of graphics, which is simple and easy to understand.

Earlier we used HDFS for related operations, and we also understood the principles and mechanisms of HDFS. With a distributed file system, how do we handle files? This is the second component of Hadoop-MapReduce.

Hadoop MapReduce is a programming model for data processing that is simple but powerful enough to be designed for parallel processing of big data.

In MapReduce, shuffle is more like the inverse process of shuffling, which refers to "disrupting" the random output of the map end according to the specified rules into data with certain rules so that the reduce end can receive and process it.