Background of hadoop2.0

Differences between hadoop and hadoop Ecosystem

Hadoop: SuitableBig DataDistributed StorageAndDistributed ComputingOfPlatform, Which corresponds to HDFS and mapreduce in hadoop1.x;

Hadoop ecosystem: a huge concept. hadoop is one of the most important and fundamental components; each subsystem in the ecosystem is only responsible for solving a specific problem domain (or even narrower). It is not an all-around system but a small and refined multiple small systems;

 

Problems in hadoop1.x

In hadoop1.x, HDFS and mapreduceHigh availability and scalabilityAnd so on:

HDFS problems:

1. namenode existsSingle point of failure, Affecting the real use of the production environment;

2. namenodeHigh Pressure, AndLimited Memory, Affecting system scalability;

High pressure: because there is only one namenode, all requests must pass through it;

Memory limit: because there is only one namenode, all metadata information must be stored on namenode;

Mapreduce problems:

1. jobtrackerSingle point of failure;

2. jobtracker accessHigh Pressure, Affecting system scalability;

3. It is difficult to support computing frameworks other than mapreduce (such as spark and storm );

 

Based on the above problems in hadoop1.x, hadoop2 was generated;