Spark Version Customization Course-1th lesson

Spark Version Customization Course-1th lesson

1. Learn this course to improve spark yourself, or add functionality to spark. Adding features that some officials do not provide, this course hopes to have some of the top spark experts in place to extend or customize spark to meet the needs of society as a whole.
2, through the first three lessons can be a thorough understanding of spark streaming
3. Why spark Streaming as a starting point to customize it?
#spark最开始并没有streaming等其他框架, the first is very original spark core, to do their own source code custom version, to streaming as a point of entry, through the study of this framework, you can grasp the source of spark power.
#众多框架中选择spark streaming, first spark SQL involves too much syntax parsing and is less suitable for research. Sparkr not mature enough, figure calculation spark GRAPHX in the recent release of several releases and there is no improvement, it is likely GRAPHX has developed to the end, so it is not suitable. In addition, there are too many mathematical-related algorithms involved in graph calculation. The machine learning framework involves too much mathematical knowledge and is not a good choice.
#2015年是spark最火的一年, is also the most fiery year of streaming processing. In the first half of 2016, the survey found that spark was the highest paid in big data and found that streaming was Spark's most attractive skill.
4, spark streaming charm?
#流式计算: In the age of stream processing, all data, if not streamed, are invalid data. Social development will confirm this passage
#流式处理是一个能立即给出反馈, can use the results of all other calculations, into machine learning, SQL and so on. Stream processing can be seamlessly integrated with other frameworks and features are super-powerful.
#精通spark Streaming, very high salary.
#spark所有程序框架中, spark streaming is the most problematic framework. The most tuned frame, mastering the spark streaming, can elevate the spark's skills to a new level.
#streaming很像spark One application on the core, other frameworks, such as machine learning, generally just encapsulate the vector and implement the corresponding algorithm. and spark streaming need to constantly perceive the change of data, the process of perceptual processing. Straming is the best reference for very complex applications.
#任何技术都有控制点, similar to the martial arts in the lone nine swords, the reason is very strong, is because the lone nine sword brilliant in advance to anticipate the opponent's next moves. Spark streaming is similar to predicting the next flow of data in advance. Of course, spark core is similar to an easy-to-rib warp, while mastering both is a very invincible thing.
5, spark streaming similar to the Dragon Cave in Spark, mastered the Dragon Cave is easy to grasp the dragon vein

Spark Version Customization Course-1th lesson