[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 5)

We can see two datanode in the console. Click "Live nodes" to view its information:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4C/D0/wKioL1RFumygVjlQAAMMlGo7TeQ441.jpg "style =" float: none; "Title =" 41.png" alt = "wkiol1rfumygvjlqaammlgo7teq441.jpg"/>

We can see two datanode nodes sparkworker1 and sparkworker2 in the console. This is exactly what we expected!

Step 3: Start the yarn Cluster

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4C/CF/wKiom1RFuiGDyH-3AAIp9pQNXSU618.jpg "style =" float: none; "Title =" 42.png" alt = "wKiom1RFuiGDyH-3AAIp9pQNXSU618.jpg"/>

Using the JPS command, we can find that the ResourceManager process is started on the sparkmaster machine:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4C/D0/wKioL1RFumzyaZAhAADjTIg9u1M469.jpg "style =" float: none; "Title =" 43.png" alt = "wkiol1rfumzyazahaadjtig9u1m469.jpg"/>

The nodemanager process is started on sparkworker1 and sparkworker2 respectively:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4C/CF/wKiom1RFuiGyTe56AADwMfS_S5w258.jpg "style =" float: none; "Title =" 44.png" alt = "wkiom1rfuigyte56aadwmfs_s5w258.jpg"/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4C/D0/wKioL1RFumyTZf57AADpVL5Qz7o803.jpg "style =" float: none; "Title =" 45.png" alt = "wkiol1rfumytzf57aadpvl5qz7o803.jpg"/>

Access http: // sparkmaster: 8088 on sparkmaster. You can view the ResourceManager running status on the Web console:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4C/CF/wKiom1RFuiLSr1NBAAHrPAcsKis749.jpg "style =" float: none; "Title =" 46.png" alt = "wkiom1rfuilsr1nbaahrpacskis749.jpg"/>

Access http: // sparkworker1: 8042 on sparkmaster. You can view the nodemanager running status on sparkworker1 on the Web console:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4C/D0/wKioL1RFum3z_yc8AAKO8x5k0vA175.jpg "style =" float: none; "Title =" 47.png" alt = "wkiol1rfum3z_yc8aako8x5k0va175.jpg"/>

Access http: // sparkworker2: 8042 on sparkmaster. You can view the nodemanager running status on sparkworker2 on the Web console:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4C/CF/wKiom1RFuiKyoNlfAALlgeb1TgQ404.jpg "style =" float: none; "Title =" 48.png" alt = "wkiom1rfuikyonlfaallgeb1tgq404.jpg"/>

Next, use mr-jobhistory-daemon.sh to start jobhistory Server:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4C/D0/wKioL1RFum3gmV-tAAEAGK9JgLU703.jpg "style =" float: none; "Title =" 49.png" alt = "wKioL1RFum3gmV-tAAEAGK9JgLU703.jpg"/>

After startup, you can view the task execution history in jobhistory on the Web Console through http: // sparkmaster: 19888:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4C/CF/wKiom1RFuiKSHZoNAAIdmOkNxtY983.jpg "style =" float: none; "Title =" 50.png" alt = "wkiom1rfuikshzonaaidmoknxty983.jpg"/>

This article is from the spark Asia Pacific Research Institute blog, please be sure to keep this source http://rockyspark.blog.51cto.com/2229525/1566160

[Spark Asia Pacific Research Institute Series] the path to spark practice-Chapter 1 building a spark cluster (step 5)