Spark Kernel uncover -02-spark cluster overview

Spark Cluster preview:

Official documentation for the spark cluster is described below, which is a typical master-slave structure:

Official documentation provides detailed guidance on some of the key points in the spark cluster:

The definition of its worker is as follows:

It is important to note that the spark driver cluster needs to be in the same network environment as the spark cluster, because Sparkcontext instances in driver need to send tasks to different worker Node's executor and accepts some executor's execution result information, in general, in the enterprise actual production environment driver The machine is the configuration often is quite good, especially its CPU processing ability to be very formidable.

Spark Kernel uncover -02-spark cluster overview