[Original] briefly describes how to run a program in spark cluster mode

The premise of this article is that scala, sbt, and spark have been correctly installed. Briefly describe the steps to mount the program to the cluster for running: 1. Build the sbt standard project structure: Where :~ /Build. the sbt file is used to configure the basic information of the project (project name, organization name, project version, scala version used, or dependency packages required for re-configuring some projects); project/build. the version of sbt you want to use in the properties file configuration; project/plugins. the sbt file is used to add the required plug-ins to the project; project/Build. scala files are some complex advanced configuration for the project; For detailed sbt installation configuration practical see blog: http://www.cnblogs.com/vincent-hv/p/3309805.html 2, to the corresponding directory to write the program, spark program must create a SparkContext instance. SparkContext ("master", "projectName", "SPARK_HOME", "yourProject. jar path") 3. After the sbt compile command compiles the program correctly, the sbt package command packs the program. The default path for storing the packaged jar files is: project root directory/target/scala-xx.xx.xx/your-project-name_xx.xx.xx-xx.jar 4. Add the packaged jar price to SPAK_CLASSPATH (in linux There are multiple ways to change environment variables depending on the scope of action, here only my configuration method: spark root directory conf/spark-env.sh file add SPARK_CLASSPATH: xxxxxxxx) 5, After configuring the environment variables can be used in the spark root directory. the/run script runs your program, for example :. /run spark. examples. sparkPi spark: // masterIP: port Note: if the program involves IO operations, the default path is SPARK_HOME. How to modify this path remains to be studied