Modify the spark2.1 source in eclipse

First of all, this is the EoE I reproduced on the writing is very good on the excerpt ...
Reference:
http://cn.soulmachine.me/blog/20130611/
Http://scala-ide.org/download/current.html

1. Install Scala
2. Install SBT
3. Install the Scala IDE http://scala-ide.org/download/current.html (note the version matching issue with the eclipse and Scala IDE, as described on the webpage)
4. Download Spark source code: Git Clone Git://github.com/apache/spark.git
Http://spark.apache.org/downloads.html
5. Start SBT:
Under Windows: CMD--CD $spark--SBT

Linux under: CD $spark--SBT/STB

6. Generate Eclipse Project files: Enter eclipse at the SBT prompt
7. Import the generated project files with Eclipse and add the jar packages under the lib_managed directory
The first step to download the source code

Sv
n Download, download address:
Spark:http://svn.igniterealtime.org/svn/repos/spark/trunk
Openfire:http://svn.igniterealtime.org/svn/repos/openfire/trunk
Smack:http://svn.igniterealtime.org/svn/repos/smack/trunk

Create new folder, name spark-"right svn-" export spark looks like 70M
Second step Eclipse import spark Source

1. Open Eclipse and select File--new--project--java Project. Enter the project name spark,contents Select "Create Project from Existiing source" and add the folder where the spark files are located. Click "Finish" to finish.

2, after the completion of the introduction of new projects, Apple, Linux, battleships errors, online to find that "after the completion of the Spark project, will generate a compilation error, delete the wrong folder", try not to delete a package there is no more errors, But the project icon there is also a wrong flag, compile can be completed but not up, in the online no other method to solve, so choose to delete the error file, compile completed.

Steps:

1, click Window--show view--ant;

2. Right-click the Ant panel and select Add Buildfiles;

3, expand the Spark--build folder, select Build.xml, click "OK";

4. In the Ant panel, expand Spark, double click "Release[default";

5, wait a period of time, will prompt "Build successful".

6. Create the target folder, and when you recompile, delete this folder manually.

Note: If "Build successful" is not displayed, double-click "Release[default" again.

3. Operation Settings

1. Click Run--run Configurations

2. Click "Main"
Name fill in: Spark

Main Class:org.jivesoftware.launcher.Startup (Startup Class)
Tick stop in Main
3. Click classpath tab
Select User Entries to make advanced. The button becomes available. Click the Advanced button. In the pop-up Options window, select Add Folders, then click OK, select the Spark::src::resources folder in the Folders Selection window and click OK.
Select the Common tab and tick the box in front of Debug,run. Click Apply, and then click Close.

Solution Two:
Configurations for developing debugging environments under eclipse the various tools used in this section are: windows10+eclipsejavaee4.4.2+scala2.10.4+sbt0.13.8+maven3.3.3, the spark version tested is 2.1.0.
1. Configure IDE: Select menu item help->installnewsoftware, add site http://download.scala-ide.org/sdk/lithium/e44/scala211/stable/ Site, select Install Scalaideforeclipse and scalaideplugins. For Standard Edition Eclipse, you will also need to install a separate MAVEN plugin. For ease of configuration, you can also use Scala's officially provided scalaide that have all dependencies packaged. In particular, because there are some errors in the project itself, please temporarily turn off the project->buildautomatically function to save time.

2. Download the Spark source code: Create an empty directory and execute the following statement: Git clone https://github.com/apache/spark.git In addition to using GIT directives, you can also download packaged source code from the GitHub page of Spark.

3. Convert the source code into an Eclipse project: go to the source root directory and execute the following statement: Sbteclipse. All the jar packages required by Spark are downloaded during SBT execution, so this step can take a long time. Some of these jar packages need to be downloaded using a network proxy method.

4. Import the project to eclipse: Select the menu item File->import, and select General->existingprojectsintoworkspace, the root path of the project to select the root path of the source code, import all the items (a total of 25).

5. Modify Scala Version: Enter Preference->scala->installations to add Scala2.10.4 installed on the machine (select Lib directory). Since this version of Spark (1.4.0) was written in a Scala2.10.4 environment, you need to modify the Scala version of the project used in Eclipse. To do this: Select all items, right-click Scala->setthescalainstallation and select the appropriate Scala version.

6. Add Scalalibrary to the Old-deps project: Right-click the Old-deps project and select Scala->addscalalibrarytobuildpath.

7. Maveninstall to generate the class required for Spark-streaming-flume-sink: first copy the Scalastyle-config.xml file from the source code root to the Spark-streaming-flume-sink project root, and then the E Open the project in Clipse, right-select the Pom.xml file and select Runas->maveninstall.

8. Modify Spark-sql and Spark-hive package error: Because the source code package settings have errors, you need to move the class files to the correct package for the Spark-sql project, select src/test/ Test.org.apache.spark.sql in Java and all classes in the Test.org.apache.spark.sql.sources package, right-click Refactor->move, Move to Org.apache.spark.sql and Org.apache.spark.sql.sources package. For spark-hive projects, select src/test/respectively Test.org.apache.spark.sql.hive in Java, and all classes in the Test.org.apache.spark.sql.hive.execution package, are moved to Org.apache.spark.sql.hive and Org.ap Ache.spark.sql.hive.execution package.

9. Compile all projects: Open the project->buildautomatically feature and wait for all projects to compile successfully.

10. Check if the installation was successful: Copy the Src->main->resources->org folder from the core project to target->scala-2.10->classes in the examples project. Then execute the ORG.APACHE.SPARK.EXAMPLES.SPARKPI program in the examples project and set its JVM parameter to-dspark.master=local