Build Jar (SBT)

Encountered in Project

Log File & Project file

Build.sbt

startup.sh

> SH startup.sh

Example

. SBT

Name: ="Simple Project"//project nameversion:="1.0"//project versionscalaversion:="2.11.8"//scala versionlibrarydependencies+="Org.apache.spark"%%"Spark-core"%"2.1.0"librarydependencies+="Org.apache.spark"%"spark-streaming_2.11"%"2.1.0"librarydependencies+="Org.apache.spark"%"spark-streaming-kafka-0-8_2.11"%"2.1.0"librarydependencies+="org.json4s"%%"Json4s-jackson"%"3.2.11"
AllDependencies should be aware of the versioncompatibility

Sh

" Org.apache.spark.examples.streaming.KafkaWordCount " /usr/local/spark/mycode/kafka/target/scala-2.11/simple-project_2.11-1.0.jar 127.0.0.1:2181 1 Sex 1

Parameters:

--driver-class-path all jars in spark and all jars in Kafka

--class project file main where class + generated jar is located

127.0.0.1:2181:zookeeper-server Kafka-server Ip+port

1 sex 1:consumer group;topics; number of consumers

REFERENCE:
http://dblab.xmu.edu.cn

Build Jar (SBT)