Building the Spark development environment with IntelliJ idea

Recently started to study spark, the development environment has a variety of, because of the habit of using STS Maven project, but according to a lot of data on the way to try to not succeed, it may be my environmental problems, but also in Scala's own eclipse, but not too accustomed to hear that support is not very good, A lot of questions; After consulting some people, the idea has a better support for Scala. So try to use idea to build the spark development environment;

1. Installing the JDK

2. Install idea the latest version, currently is idea 14, the community free version can be;

3. Install Scala plugin, run idea, first run, you can directly pop up the tab box to select "Configure", "plugins", if already used in the project page, "File", "Settings", " Plugins "or CTRL + ALT + S, enter" Scala "and install the plugin

4. Build Spark development environment

4.1 Download Spark's jar package, select the appropriate spark version on the download page, package Type here Select spark1.2.0, and pre-build for Hadoop 2.4 to download Spark-1.2.0-bin-hadoop2.4.tgz, decompression, Lib under the Spark-assembly-1.2.0-hadoop2.4.0.jar is required;

4.2 New Scala project, "File", "New Project", select Scala,scala,next name, Finish

4.3 Importing spark packages

On the project page "File", "Project Structure", "Libraries", click "+", select "Java", Find the Spark-assembly-1.2.0-hadoop2.4.0.jar import so that you can write a Scala program for Spark (example with empty fill)

4.4 Program Export

If you run the program on Spark,

Building the Spark development environment with IntelliJ idea