9th Lesson Notes-Thorough combat INTELLIJ Spark program development under IDEA

The development of Spark program under the idea of complete combat IntelliJ
Download IntelliJ Idea
Download git
Spark source download: Git clone git://github.com/apache/spark.git
Import Maven Project

IntelliJ Idea Launch Wizard sacal download need to download, this is idea download, and the system layer is not the same
4. Specify jdk1.8.x and Scala2.10.4
5.file->project stucture To set up project Lib core is to add spark jar
6. Add Spark jar dependency, spark1.6.0-bin-hadop2.6.tgz Spark-assemblyu-1.6.0-hadoop2.6.0.jar

7. Create a Scala project and choose the Scala approach

Copy the code execution on eclipse yesterday, normal

AD Click Ranking
Rank processing: Wordcountordered=pairs.reducebykey (_+_). Map (pair=> (pair._2,pair._1)). Sortbykey (FALSE). Map (Pair=> ( pair._1,pair._2))
Reducebykey the same key totals, key and value are sorted after swapping, and then swapped back.
50 minutes

9th Lesson Notes-Thorough combat INTELLIJ Spark program development under IDEA