Spark Job scheduling mode __ Spark

Jobs that users submit through different threads can run concurrently, but are subject to resource constraints. Job to the dispatch pool (pool) To request resources, the dispatch pool will be based on the project configuration, decide which scheduling mode to use.
FIFO mode by default, the Spark Scheduler Dispatches job execution in FIFO (first-in first Out) mode. Each job is cut into multiple stage. The first job takes all available resources, and then the second job gets the remaining resources. By the way, if the first job does not occupy all the resources, the second job can also continue to acquire the remaining resources so that multiple jobs can run in parallel. If the first job is large and consumes all resources, the second job needs to wait for the first task to complete, free up the free resources, and then request and assign the job. If the same job has different stage, the earlier stage is executed first. Fair mode in Fair shared mode, spark allocates resources for tasks in a polling (round robin) way between multiple jobs, and all tasks have roughly equal priority to share the resources of the cluster. This means that when a long task is being executed, a short task can still be assigned to a resource, submitted and executed, and given a good response time. This doesn't have to wait for the long task to finish just like before. This scheduling pattern is ideal for multi-user scenarios.