[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 4) (7)

Step 4: build and test the spark development environment through spark ide

 Step 1: Import the package corresponding to spark-hadoop, select "file"> "project structure"> "Libraries", and select "+" to import the package corresponding to spark-hadoop:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4A/2E/wKioL1QjvFuijA0cAAGYTLtlqJ8564.jpg "style =" float: none; "Title =" 1.png" alt = "wkiol1qjvfuija0caagytlqj8564.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4A/2D/wKiom1QjvDmxXkngAAEfnxZDTbQ384.jpg "style =" float: none; "Title =" 2.png" alt = "wkiom1qjvdmxxkngaaefnxzdtbq384.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4A/2E/wKioL1QjvF_hAf5-AAD9ETir8sU678.jpg "style =" float: none; "Title =" 3.png" alt = "wKioL1QjvF_hAf5-AAD9ETir8sU678.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4A/2D/wKiom1QjvDyhMQg4AAD_yXweVAg544.jpg "style =" float: none; "Title =" 4.png" alt = "wkiom1qjvdyhmqg4aad_yxwevag544.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4A/2E/wKioL1QjvGKQQQiNAAEZywO5Oqg613.jpg "style =" float: none; "Title =" 5.png" alt = "wkiol1qjvgkqqqinaaezywo5oqg613.jpg"/>

Click "OK" to confirm:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4A/2D/wKiom1QjvD-C1oYkAAEaIGrfTRY643.jpg "style =" float: none; "Title =" 6.png" alt = "wKiom1QjvD-C1oYkAAEaIGrfTRY643.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4A/2E/wKioL1QjvGOS_eEPAACXyI8m0C0378.jpg "style =" float: none; "Title =" 7.png" alt = "wkiol1qjvgos_eepaacxyi8m0c0378.jpg"/>

Click "OK ":

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4A/2D/wKiom1QjvEGBMHvpAAEn5IY6bYY697.jpg "style =" float: none; "Title =" 8.png" alt = "wkiom1qjvegbmhvpaaen5iy6byy697.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4A/2E/wKioL1QjvGbAXApSAAErYJJt3I4478.jpg "style =" float: none; "Title =" 9.png" alt = "wkiol1qjvgbaxapsaaeryjjt3i4478.jpg"/>

After idea is completed, we will find that the spark jar package is imported into our project:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4A/2D/wKiom1QjvETC0p_rAAEAa4s3PKE272.jpg "style =" float: none; "Title =" 10.png" alt = "wkiom1qjvetc0p_raaeaa4s3pke272.jpg"/>

Step 2: Develop the first spark program. Open the examples directory that comes with spark:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4A/2E/wKioL1QjvHfyKPkoAAGYG6f_eD0851.jpg "style =" float: none; "Title =" 11.png" alt = "wkiol1qjvhfykpkoaagyg6f_ed0851.jpg"/>

At this time, we found that there were a lot of internal files, which were examples provided by spark for me.

Create a Scala object named sparkpi under SRC of our first Scala project:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4A/2D/wKiom1QjvFWgT0B9AAC52Yrx6kQ464.jpg "style =" float: none; "Title =" 12.png" alt = "wkiom1qjvfwgt0b9aac52yrx6kq464.jpg"/>

Open the sparkpi file under examples of spark:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4A/2E/wKioL1QjvHyBFnfrAAFAmnvKvSM596.jpg "style =" float: none; "Title =" 13.png" alt = "wkiol1qjvhybfnfraafamnvkvsm596.jpg"/>

We copy the content of this article directly to the sparkpi created in idea:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4A/2D/wKiom1QjvFmza2gxAAGYZKmKyHw387.jpg "style =" float: none; "Title =" 14.png" alt = "wkiom1qjvfmza2gxaagyzkmkyhw387.jpg"/>

This article is from the spark Asia Pacific Research Institute blog, please be sure to keep this source http://rockyspark.blog.51cto.com/2229525/1558103

[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 4) (7)