[streamingpro]開發環境搭建

streamingpro 環境準備 csdn_common

git clone到本地,然後執行mvn install安裝到本地倉庫

➜  csdn_common git:(master) mvn install[INFO] Building jar: /Users/doctorq/Documents/Developer/git-project/csdn_common/target/csdn-common-1.5-SNAPSHOT-javadoc.jar[INFO][INFO] --- maven-install-plugin:2.4:install (default-install) @ csdn-common ---[INFO] Installing /Users/doctorq/Documents/Developer/git-project/csdn_common/target/csdn-common-1.5-SNAPSHOT.jar to /Users/doctorq/.m2/repository/net/csdn/csdn-common/1.5-SNAPSHOT/csdn-common-1.5-SNAPSHOT.jar[INFO] Installing /Users/doctorq/Documents/Developer/git-project/csdn_common/pom.xml to /Users/doctorq/.m2/repository/net/csdn/csdn-common/1.5-SNAPSHOT/csdn-common-1.5-SNAPSHOT.pom[INFO] Installing /Users/doctorq/Documents/Developer/git-project/csdn_common/target/csdn-common-1.5-SNAPSHOT-sources.jar to /Users/doctorq/.m2/repository/net/csdn/csdn-common/1.5-SNAPSHOT/csdn-common-1.5-SNAPSHOT-sources.jar[INFO] Installing /Users/doctorq/Documents/Developer/git-project/csdn_common/target/csdn-common-1.5-SNAPSHOT-javadoc.jar to /Users/doctorq/.m2/repository/net/csdn/csdn-common/1.5-SNAPSHOT/csdn-common-1.5-SNAPSHOT-javadoc.jar[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESS[INFO] ------------------------------------------------------------------------[INFO] Total time: 47.729 s[INFO] Finished at: 2016-10-20T18:37:23+08:00[INFO] Final Memory: 30M/310M[INFO] ------------------------------------------------------------------------

active_orm

git clone到本地,然後執行mvn install安裝到本地倉庫 ServiceframeworkDispatcher

git clone到本地,然後執行mvn install安裝到本地倉庫 ServiceFramework

git clone到本地，然後執行mvn install安裝到本地倉庫 streamingpro

mvn clean package看執行結果，解決相關錯誤。

比如測試代碼中含有以下錯誤

[ERROR] /Users/doctorq/Documents/Developer/git-project/streamingpro/src/test/java/org/apache/spark/streaming/BasicStreamingOperation.scala:23: error: value clear is not a member of object serviceframework.dispatcher.StrategyDispatcher[INFO]         StrategyDispatcher.clear[INFO]                            ^[ERROR] /Users/doctorq/Documents/Developer/git-project/streamingpro/src/test/java/org/apache/spark/streaming/BasicStreamingOperation.scala:40: error: value clear is not a member of object serviceframework.dispatcher.StrategyDispatcher[INFO]         StrategyDispatcher.clear

測試代碼，我直接注釋了.如果mvn clean package執行成功的話會產生streamingpro-0.4.1-SNAPSHOT-online-1.6.1.jar 匯入到ide

以maven項目方式匯入到IDEA 指定不同的profile

mvn clean package -Pdev以dev配置打包

打全量包

mvn clean package -Pdev -Pbuild-distr 版本提升

因為我的環境spark為2.0.1，scala是2.11.8,所以我做了jar包的版本修改

pom.xml 運行

Spark Streaming + Spark SQL 實現配置化ETL流程 continus

下面就可以開始讀源碼+二次開發了。感謝[祝威廉]同學(https://github.com/allwefantasy)開源這個項目