Spark Rdd Flex 7 Point

1. Automatic seamless switching between disk data and memory

2, based on lineage of high-efficient fault tolerance, nth error, will be executed from the beginning of n-1

3. Task failure will be retried a specific number of times

4, the stage failure will automatically make a specific number of retries, and only run the calculation of the failed data shards

5, checkpoint (similar to the archive in a single game) and presist, persistent cache

6, data scheduling flexibility, DAG task is not related to resource management

7, data fragmentation of the high elasticity, repartition,1w a large, into a small 10W, 10W small to become 1W large.

Spark Rdd Flex 7 Point