Spark Source Learning (15)---Application registration mechanism analysis __spark Source code Analysis

First look at the definition of master, inherited from actor, description is a thread.

Private[spark] class Master (
    host:string,
    port:int,
    webuiport:int,
    val securitymgr: SecurityManager,
    Val conf:sparkconf) extends Actor with actorlogreceive and Logging with leaderelectable
  {

Find a request to process Appliction registration:

    Case RegisterApplication (description) => {if
      (state = = Recoverystate.standby) {
        //If the current is Standbymaster, So application to request registration do nothing
        //Ignore, don ' t send response
      } else {
        //active status
        loginfo ("Registering app" + Description.name)
        ///Use description to create Appliction object
        val app = createapplication (description, Sender)
        // Register appliction, add Applictioninfo to the cache, and speak appliction queue
        registerapplication (APP)
        loginfo ("Registered App "+ Description.name +" with ID + app.id)
        //Use persistence engine to persist appliction information
        persistenceengine.addapplication (app
        //actor: The Clientactor
        //Send message to the Sparkdeployschedulerbackend appclint. Note that the method is registeredapplication,registered rather than register
        sender! Registeredapplication (App.id, MasterUrl)
        schedule ()
      }
    }

The above calls the createappliction, inside the parameter descrption encapsulates the appliction need resource information, core,mem and so on.

Private[spark] class Applicationdescription (
    Val name:string,
    Val maxcores:option[int],
    Val Memoryperslave:int,
    Val command:command,
    var appuiurl:string,
    val eventlogdir:option[string] = None,
    //Short name of compression codec used when writing event logs, if any (e.g. Lzf)
    Val Eventlogcodec:option[stri NG] = None)
  extends Serializable {

Or the Createappliction method:

def createapplication (Desc:applicationdescription, driver:actorref): ApplicationInfo = {
    val now = System.currenttimemillis ()
    val date = new Date
    new ApplicationInfo (now, Newapplicationid (date), DESC, Date, driver, defaultcores)
  }

Using time, Descrption creates a Applictioninfo object. Returns paid to the App object. Then call the registration method: RegisterApplication (APP).

  def registerapplication (app:applicationinfo): unit = {
    //First get Dirver address
    val appaddress = app.driver.path.address if
    (Addresstoapp.contains (appaddress)) {
      //If the driver address already exists and is considered a duplicate registration, then return
      directly Loginfo ("Attempted to re-register application in same address:" + appaddress)
      return
    }

    Applicationmetricssystem.registersource (App.appsource)
    //------------Add driver information to the memory cache
    //hashset, Go to heavy app
    apps + + app
    //hashmap
    idtoapp (app.id) = App
    //hashmap
    actortoapp (app.driver) = App
    Addresstoapp (appaddress) = App
    //------------Add app to the cache queue waiting to be scheduled, Waitingapps is a thread queue for Arraybuffer,fifo
    Waitingapps = App
  }