Multi-node configuration for Linux Enterprise-hadoop

I. Configuring the Environment

redhat6.5

Server1 172.25.29.1 Hadoop Master NFS

Server2 172.25.29.2 NFS

Server3 172.25.29.3 NFS

Server4 172.25.29.4 NFS

Server5 172.25.29.5 High Availability standby NFS

Two. Hadoop single-point configuration

1. Installing Hadoop and JDK

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/280119253ea3e12c306a6a854659aaa3.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-05-12.png "alt=" 280119253ea3e12c306a6a854659aaa3.png "/>

Run with Hadoop users

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/d303c71135a29dc7a37d52f1cd306498.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-06-33.png "alt=" D303c71135a29dc7a37d52f1cd306498.png "/>

2. Configure the operating environment for Hadoop on Java

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/09cb7e59f71da918f0ca675ba9681079.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-08-46.png "alt=" 09cb7e59f71da918f0ca675ba9681079.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/0b94156e628756e63656273097461bb0.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-08-04.png "alt=" 0b94156e628756e63656273097461bb0.png "/>

3. Algorithm module test

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/934fbc3b4262f21d93c2ea67c8a09422.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-10-53.png "alt=" 934fbc3b4262f21d93c2ea67c8a09422.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/e1733804f89f021feda1af13881e0306.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-14-05.png "alt=" E1733804f89f021feda1af13881e0306.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/04554a45579067d5a7f8fe198c9231f0.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-14-18.png "alt=" 04554a45579067d5a7f8fe198c9231f0.png "/>

Output-generated data

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/6c4a1166ed02002823b432e92d08dc4f.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-27-44.png "alt=" 6c4a1166ed02002823b432e92d08dc4f.png "/>

4. Configure single point Hadoop

650) this.width=650; "src=" Https://s3.51cto.com/oss/201710/25/4becdb2870a1b45b5c878fe899e1d5a9.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-30-54.png "alt=" 4becdb2870a1b45b5c878fe899e1d5a9.png "/>

650) this.width=650; "src=" Https://s3.51cto.com/oss/201710/25/22e84e812e3cec3c05f0be979cf720ee.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-30-15.png "alt=" 22e84e812e3cec3c05f0be979cf720ee.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/b6866cd32e1440855e68018aebe3acf9.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-30-47.png "alt=" B6866cd32e1440855e68018aebe3acf9.png "/>

5. Set the user password for Hadoop and set up the encryption between the hosts

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/5a7f6977929a309ae329703901d737ce.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-32-05.png "alt=" 5a7f6977929a309ae329703901d737ce.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/ffcb3d48474134ccdf7ae5a2e363c16f.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-33-34.png "alt=" Ffcb3d48474134ccdf7ae5a2e363c16f.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/70aa88bb18d7dd936c352c34a798ec0b.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-33-39.png "alt=" 70aa88bb18d7dd936c352c34a798ec0b.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/9173d41f3df113e892889af7e1cf8792.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-34-05.png "alt=" 9173d41f3df113e892889af7e1cf8792.png "/>

6. Set storage node to 1

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/c92693474d205645ff228f6d24bde45a.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-35-39.png "alt=" C92693474d205645ff228f6d24bde45a.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/b6f6ba6a24361333574c4c0196242cdb.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-35-09.png "alt=" B6f6ba6a24361333574c4c0196242cdb.png "/>

7. Format HDFs

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/397339421feeae3fbaf67d2ac4d92cf9.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-40-55.png "alt=" 397339421feeae3fbaf67d2ac4d92cf9.png "/>

8. Start Hadoop

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/3177fda9f90b47fa7c7ea35ec648b63d.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-42-18.png "alt=" 3177fda9f90b47fa7c7ea35ec648b63d.png "/>

9. Set the run path of the Java command JPS to view the process

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/7e5814b639db1714e005547b66e4b286.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-43-04.png "alt=" 7e5814b639db1714e005547b66e4b286.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/a7eaf0a54f6d9db9ee4d2901d1e5a18b.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-43-54.png "alt=" A7eaf0a54f6d9db9ee4d2901d1e5a18b.png "/>

10. View node run status, test

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/58e593b128916f822a34898471aa143b.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-44-28.png "alt=" 58e593b128916f822a34898471aa143b.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/8733b89ce5acb113c08305890feedd2e.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-47-24.png "alt=" 8733b89ce5acb113c08305890feedd2e.png "/>

11. View results in text mode

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/444cb31ce4d57b8c3dbb22090b9d3fe4.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-48-42.png "alt=" 444cb31ce4d57b8c3dbb22090b9d3fe4.png "/>

Create a Directory

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/7f8d5904bef19c68935225413c7d4849.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-49-50.png "alt=" 7f8d5904bef19c68935225413c7d4849.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/5393b3699740093595f4d28ff8a010db.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-53-36.png "alt=" 5393b3699740093595f4d28ff8a010db.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/9b71aa069a02a04567f0fcfc7a56345a.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-07-55.png "alt=" 9b71aa069a02a04567f0fcfc7a56345a.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/cd10aee684cec903d514127a1370e6b3.png "style=" float: none; "Title=" screenshot from 2017-10-24 14-56-43.png "alt=" Cd10aee684cec903d514127a1370e6b3.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/948165d6ad89b8c3cd4a862809e2e33a.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-09-03.png "alt=" 948165d6ad89b8c3cd4a862809e2e33a.png "/>

Four. Multi-node Hadoop configuration

1.server1 Configuring NFS Services

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/179092d9a5493c3d410a915ecc335cc4.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-12-36.png "alt=" 179092d9a5493c3d410a915ecc335cc4.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/eb5c5ea2b20b9b3f763543850434f5d8.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-14-47.png "alt=" Eb5c5ea2b20b9b3f763543850434f5d8.png "/>

650) this.width=650; "src=" Https://s1.51cto.com/oss/201710/25/6a662892bdc18b4e344ae475fa9efb00.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-15-40.png "alt=" 6a662892bdc18b4e344ae475fa9efb00.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/93dcf30bb84be6c8c34e72a0040520f9.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-15-53.png "alt=" 93dcf30bb84be6c8c34e72a0040520f9.png "/>

2. Install NFS on Server2 to Server5, and create a new Hadoop user, and start the Rpcbind

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/ba8c8f816f8a406493fe2b1021762aed.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-16-31.png "alt=" Ba8c8f816f8a406493fe2b1021762aed.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/oss/201710/25/104ba509f0e3897d0f9cbbb6c2e17b61.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-21-52.png "alt=" 104ba509f0e3897d0f9cbbb6c2e17b61.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/d2a7dd7dbad374ce8f16d101f10ad9b6.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-21-59.png "alt=" D2a7dd7dbad374ce8f16d101f10ad9b6.png "/>

3. Establish a secret between the hosts

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/8beee3f55d8ba18766279f4f626990bd.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-23-32.png "alt=" 8beee3f55d8ba18766279f4f626990bd.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/b38e699b90895dd0a7bf2a63d09fddea.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-23-46.png "alt=" B38e699b90895dd0a7bf2a63d09fddea.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/b26aa6d60d9fb40813843504d7fbf5b8.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-24-45.png "alt=" B26aa6d60d9fb40813843504d7fbf5b8.png "/>

4.hadoop cluster configuration

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/90d733014b90a7e3078a0cc395e4faa3.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-26-35.png "alt=" 90d733014b90a7e3078a0cc395e4faa3.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/738916de261033547cd6f9e76ecb574a.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-25-19.png "alt=" 738916de261033547cd6f9e76ecb574a.png "/>

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/b515247afdca2965af7db3c95258dd25.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-26-18.png "alt=" B515247afdca2965af7db3c95258dd25.png "/>

5. Format HDFs

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/114ea4c287a89838e2ed3fe1d773f4d8.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-27-22.png "alt=" 114ea4c287a89838e2ed3fe1d773f4d8.png "/>

650) this.width=650; "src=" Https://s1.51cto.com/oss/201710/25/f9111ec6ad645564034e81c654350457.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-27-30.png "alt=" F9111ec6ad645564034e81c654350457.png "/>

6. Start the Hadoop cluster

650) this.width=650; "src=" Https://s1.51cto.com/oss/201710/25/504f87797d4078ad9532e8cce11d544b.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-28-33.png "alt=" 504f87797d4078ad9532e8cce11d544b.png "/>

View node 2,3 and status

650) this.width=650; "src=" Https://s1.51cto.com/oss/201710/25/bbda4c2ddd9b3a35103a1c78fdc408ed.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-29-05.png "alt=" Bbda4c2ddd9b3a35103a1c78fdc408ed.png "/>

650) this.width=650; "src=" Https://s1.51cto.com/oss/201710/25/ab791ae8151972a0db3ad18c8f847b83.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-29-21.png "alt=" Ab791ae8151972a0db3ad18c8f847b83.png "/>

7. Login to Web 172.25.29.1:50070

650) this.width=650; "src=" Https://s2.51cto.com/oss/201710/25/67f800d58d9043d683847c8c2f680616.png "style=" float: none; "Title=" screenshot from 2017-10-24 15-29-57.png "alt=" 67f800d58d9043d683847c8c2f680616.png "/>

8. Start the Cluster Management resource module

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/9b5da16f9de56410f05d823d144f0068.png "style=" float: none; "Title=" screenshot from 2017-10-24 16-02-42.png "alt=" 9b5da16f9de56410f05d823d144f0068.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/4da67337a581f327afa2701df9c7e90c.png "style=" float: none; "Title=" screenshot from 2017-10-24 16-02-48.png "alt=" 4da67337a581f327afa2701df9c7e90c.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/4074ec199f9ea69294457c688432a5c0.png "style=" float: none; "Title=" screenshot from 2017-10-24 16-03-08.png "alt=" 4074ec199f9ea69294457c688432a5c0.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/oss/201710/25/b1657ea804355d92f0bc64ffa9473d13.png "style=" float: none; "Title=" screenshot from 2017-10-24 16-03-25.png "alt=" B1657ea804355d92f0bc64ffa9473d13.png "/>

Multi-node configuration for Linux Enterprise-hadoop