1. Knime Analytics Installation
Download the appropriate version from the official website https://www.knime.com/downloads
Unzip the downloaded installation package on the installation path https://www.knime.com/installation-0
is the Welcome page after the Knime launch
Do I need to install knime in Knime to be mutual with spark set XXX? Extension for Apache Spark. and install the Spark Job Server on the Hadoop cluster boundary node or node capable of performing spark-submit. The architecture diagram is as follows:
2, Knime? Extension for Apache Spark installation
In Knime Analytics, click File->install knime Extensions Select Knime Big Data Extensions, click Next to install.
3. SPARK JOB Server Installation
The following steps take CentOS 6.5 + CDH 5.7 as an example
3. 1 Download Spark Job server
$ wget http://download.knime.org/store/3.5/spark-job-server-0.6.2.3-KNIME_cdh-5.7.tar.gz
3. 2 Login as root or su root
3. 3 Installation
# Linkname=spark-job-server # useradd-d/opt/${linkname}/-m-r-s/bin/false spark-job-server # su-l-C "HDFs dfs-mkdir-p/user/spark-job-server; HDFs dfs-chown-r Spark-job-server/user/spark-job-server "HDFs # CP Spark-job-server-0.6.2.3-knime_cdh-5.7.tar.gz/opt # cd/opt # TAR-XVF Spark-job-server-0.6.2.3-knime_cdh-5.7.tar.gz # ln-s spark-job-server-0.6.2.3-knime_cdh-5.7 ${linkname} # chown-r Spark-job-server:spark-job-server ${linkname} spark-job-server-0.6.2.3-knime_cdh-5.7 |
3. 4 Boot up
# Ln-s/opt/${linkname}/spark-job-server-init.d/etc/init.d/${linkname} #chkconfig--levels 2345 ${linkname} on |
3. 5 Edit environment.conf
Set Master, for example
master = "spark://ifrebdplatform1:7077"
Set default settings for Spark contexts:context-settings
3. 6 Editing Settings settings.sh
Set Spark_home, this example is correct by default, do not change
Set Log_dir, if not the default set of directories
3. 7 Edit Log4j-server.properties According to your preference
3. 8 Starting the Spark job server
/etc/init.d/${linkname} start
3. 9 Add create Spark context node test link in Knime
Right-click the Create Spark context node and click Execute Execution
Right-click the Create Spark context node and click Spark Context to view the results
To be Continued ...
Build Spark machine learning model with Knime 1: Development environment Building