Ubuntu under Sprak (IDE) WordCount Example

First, enter the IDE interface

CD ~/downloads/idea/bin

idea.sh

Ii. Building a Scala project

Step 1 : Import spark-hadoop corresponding package, select "File" –> "Project Structure" –> "Libraries", select "+" to import Spark-hadoop corresponding package:

Click "OK" to confirm:

Click "OK":

When idea is done, we'll find that Spark's jar package is imported into our project:

Step two, write Scala code implementation wordcount

Import Sparkcontext

Import Sparkcontext._
Object Wordcount {
def main (args:array[string]) {
Val sc = new Sparkcontext ("local", "Simple App", "HOME\\YUANQIN\\DOWNLOADS\\SPARK-0.9.0-INCUBATING-BIN-HADOOP1", NULL)
List ("Lib/spark-assembly_2.10-0.9.0-incubating-hadoop1.0.4.jar")
Val logFile = Sc.textfile ("Hdfs://master:9000/user/root/input/file.txt")//should be some file on your system
Val Count=logfile.flatmap (line = Line.split ("")). Map (Word = = (word,1)). Reducebykey (_+_)
println (count)
Count.saveastextfile ("home\\yuanqin\\output\\2014")

}
}

Step three, right-click Run as ....

Ubuntu under Sprak (IDE) WordCount Example