This is a note that reads the code of the Sparkcontext class. When reading this class, the main task is to figure out how Sparkcontext is constructed, and the initialization of Java and C # class is put in one method, and Scala's main constructor code is almost scattered in the Sparkcontext class, This requires us to organize it, in order to have a structured reading work.
A few of the main things that are done in Sparkcontext's main constructor are:
1, all kinds of analytic conf (this I do not have a unified collection, really too much, if you want to see, directly to the Sparkcontext class search "conf")
2, the creation of sparkenv, and his internal components (Listenerbus, metrics system, Blockmanager, etc.) initialization, start
(Listenerbus, Metrics system profile affixed http://blog.csdn.net/allwefantasy/article/details/50449464)
3. Creation and start of TaskScheduler, Dagscheduler, Sparkui
On the Code Note map: (Sparkcontext main constructor code extraction)
Sparkcontext main constructor Code extraction