Spark Kernel unveils -08-spark web monitoring page

Source: Internet
Author: User

You can see the initialization UI code in Sparkcontext:

//Initialize the Spark UIPrivate[Spark]ValUI: Option[sparkui] =if(conf. Getboolean ("Spark.ui.enabled", true)) {Some(Sparkui.Createliveui( This, conf, Listenerbus, Jobprogresslistener,      Env. SecurityManager,AppName))  }Else{//For tests, does not enable the UI    None}//Bind the UI before starting the Task Scheduler to communicate//The bound port to the cluster manager properlyUI. foreach (_.bind ())
To Create a Sparkui object:
defCreateliveui(Sc:sparkcontext,    Conf:sparkconf,    Listenerbus:sparklistenerbus,    Jobprogresslistener:jobprogresslistener,    Securitymanager:securitymanager,    AppName:String): Sparkui = {Create(Some(SC), Conf, Listenerbus, SecurityManager, AppName,    Jobprogresslistener =Some(Jobprogresslistener))}

Enter the Create method:


Enter Sparkui:


Sparkui inherited the WebUI:


The WebUI class defines the Def initialize () method


Sparkui's Initialize () implementation method:


Above code Analysis: Add tab page of page

1, Jobstab:


2, Stagestab:


3, Storagetab:


4, Environmenttab:


5, Executorstab:


6, Createstatichandler:


7, Createredirecthandler:


After executing the Initialize () method, we go back to the Sparkcontext initialize the spark UI:


Bind Port:


As you can see from the code above, the local jettyserver with port number 4040 is started.



The above Code analysis:

1. Create contexthandlercollection and set handlers to Contexthandlercollection

2. Add a filter:



3. Create a jetty server and bind the port number and create a queuedthreadpool that is set to the server:


4. Start jetty server and throw an exception if startup fails:


5, try to focus on several times to start the Startserviceonport, if the boot fails, replace the new port number (rule is 1+oldport), restart:





After the successful launch, the information will be stored in the ServerInfo:



Spark Kernel unveils -08-spark web monitoring page

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.