看看spark的源碼

標籤：.sh   div   style   過程   return   false   site   targe   seq   

1. IntelliJ IDEA安裝plugins,加入scala外掛程式和SBT外掛程式
2. 下載spark的原始碼， https://github.com/apache/spark，可以使用git下載或者下載zip包
3. 匯入項目，開啟 IntelliJ 之後選擇 Import Project，而後選擇 Spark 原始碼，並將其匯入為 SBT 項目，在之後的過程中 SBT 過程將會自動進行，下載作為依賴的 jar 包。
4. 原始碼已經準備好了。開幹吧

   private def schedule(): Unit = {    if (state != RecoveryState.ALIVE) {      return    }    // Drivers take strict precedence over executors    val shuffledAliveWorkers = Random.shuffle(workers.toSeq.filter(_.state == WorkerState.ALIVE))    val numWorkersAlive = shuffledAliveWorkers.size    var curPos = 0    for (driver <- waitingDrivers.toList) { // iterate over a copy of waitingDrivers      // We assign workers to each waiting driver in a round-robin fashion. For each driver, we      // start from the last worker that was assigned a driver, and continue onwards until we have      // explored all alive workers.      var launched = false      var numWorkersVisited = 0      while (numWorkersVisited < numWorkersAlive && !launched) {        val worker = shuffledAliveWorkers(curPos)        numWorkersVisited += 1        if (worker.memoryFree >= driver.desc.mem && worker.coresFree >= driver.desc.cores) {          launchDriver(worker, driver)          waitingDrivers -= driver          launched = true        }        curPos = (curPos + 1) % numWorkersAlive      }    }    startExecutorsOnWorkers()  }

    

 

看看spark的源碼