Parquet + Spark SQL

Mass data storage is recommended to replace files on HDFs with parquet Columnstore

The following two articles explain the use of parquet Columnstore to store data, mainly to improve query performance, and storage compression

Parquet in Spark SQL uses best practices and code combat http://blog.csdn.net/sundujing/article/details/51438306
How-to: Convert text in Spark to parquet to improve performance http://www.ibm.com/developerworks/cn/analytics/blog/ba-parquet-for-spark-sql/ Index.html

Offline: Spark SQL Kafka, HDFS---Parquet
Real-time: Spark SQL (ML, GRAPHX, etc.), Kafka, spark streaming, parquet

Parquet + Spark SQL