spark1.4 Windows Local debugging Environment Building summary
Version 1.scala
SCALA-2.10.4 Official recommendation
scala-2.11.7 "Not recommended, non-SBT project, load after need"
Version 2.spark
Spark-1.4.0-bin-hadoop2.6.tgz
3.hadoop
Version 3.1
Hadoop-2.6.0.tar.gz
3.2 Environment variables
hadoop_home=e:/ysg.tools/spark/hadoop-2.6.0
Or
System.setproperty ("Hadoop.home.dir", "E:\ysg.tools\spark\hadoop-2.6.0");
3.3winutils.exe
winutils.exe拷贝至spark/hadoop-2.6.0/bin
File
Http://files.cnblogs.com/files/yjmyzz/hadoop2.6%28x64%29V0.2.zip
4.idea
New NO-SBT Project
libraties 增加 scala sdk spark-1.4.0-bin-hadoop2.6\lib\spark-assembly-1.4.0-hadoop2.6.0.jar
Spark.test.iml
Load spark-assembly-1.4.0-hadoop2.6.0 First
Re-load scala-sdk-2.11.7
<?xml version="1.0" encoding="UTF-8"?><module type="JAVA_MODULE" version="4"> <component name="NewModuleRootManager" inherit-compiler-output="true"> <exclude-output /> <content url="file://$MODULE_DIR$"> <sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" /> </content> <orderEntry type="inheritedJdk" /> <orderEntry type="sourceFolder" forTests="false" /> <orderEntry type="library" name="spark-assembly-1.4.0-hadoop2.6.0" level="project" /> <orderEntry type="library" name="scala-sdk-2.11.7" level="project" /> </component></module>
spark1.4 Windows Local debugging Environment Building summary