System design of Big Data system

Big data has become a growing trend, and more and more companies are involved. Recently engaged in the design and development of big Data systems and recommendations of the engine work, a few articles two series of their own experience record,

Share your big data experiences with everyone.

The whole platform includes monitoring system, log Analysis system and Recommender system. According to the data business step divides, divides into the collection, the cleaning, the storage, the analysis and the service. The entire data flow is as follows:

The actual project is somewhat different from this flowchart. Flume used for front-end log collection.

System design of Big Data system