CENTOS7 mounting hadoop-cdh-2.5 on Mesos

1. Environmental statement

Master 192.168.0.223 Mesos-master

Slave 192.168.0.225 Mesos-salve

2. Environmental preparedness

Shutting down the firewall

Turn off SELinux

Two machines to modify the host name Master/slave

Set up hosts, can parse each other

3.master and slave Configure SSH Trust

Hadoop users are configured to trust each other, because Hadoop starts with Hadoop users

Master yum-y Install Sshpass Ssh-keygen All-way return ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected]slave yum-y Insta ll Sshpass Ssh-keygen all the way return ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected] test SSH host, do not prompt for password is OK

4. Installing the JDK

Tar zxvf jdk-8u65-linux-x64.tar.gzmv jdk1.8.0_65/usr/jdk

4.1 Setting environment variables

Export Java_home=/usr/jdkexport jre_home=/usr/jdk/jreexport classpath=.: $CLASSPATH: $JAVA _home/lib: $JRE _home/ Libexport path= $PATH: $JAVA _home/bin: $JRE _home/bin execution Source/etc/profile

4.2 Testing the JDK

Java-version #出现版本信息

5. Install Mesos master-slave, see other blogs

The installation is complete and a libmesos.so file is generated under/usr/local/lib

6. Installing and configuring Hadoop

Master and slave

Tar zxvf hadoop-2.5.0-cdh5.4.8.tar.gzmv hadoop-2.5.0-cdh5.4.8/usr/hadoopcd/usr/hadoopmkdir-p tmpcd/usr/hadoop/mv Bi N bin-mapreduce2/ln-s Bin-mapreduce1 binmv example Example-mapreduce2 ln-s example-mapreduce1 examplecd etc/mv Hadoo P Hadoop-mapreduce2ln-s Hadoop-mapreduce1 Hadoop

7. Add a Hadoop environment variable

Vim/etc/profile Export Hadoop_home=/usr/hadoop export path= $PATH: $HADOOP _home: $HADOOP _home/binsource/etc/profile

8. Get the jar package for Hadoop on Mesos

Yum-y install maven openjdk-7-jdk git git clone cd Hadoop MVN package #获取jar包, the jar will be under target

9. Place the acquired jar package in the Hadoop installation directory

Master and slave

CP hadoop/target/hadoop-mesos-0.1.0.jar/usr/hadoop/share/hadoop/common/lib/

10. Configure Hadoop on Mesos

Master and slave

Vim/usr/hadoop/etc/hadoop/mapred.site.xml<property> <name>mapred.job.tracker</name> <value >localhost:9001</value></property><property> <name>mapred.jobtracker.taskscheduler </name> <value>org.apache.hadoop.mapred.mesosscheduler</value></property><property > <name>mapred.mesos.taskScheduler</name> <value> Org.apache.hadoop.mapred.jobqueuetaskscheduler</value></property><property> <name>  Mapred.mesos.master</name> <value>zk://192.168.0.223</value></property><property> <name>mapred.mesos.executor.uri</name> <value>hdfs://localhost:9000/ Hadoop-2.5.0-cdh5.2.0.tar.gz</value></property>

11. Give Hadoop user rights

Master and slave

Chown-r Hadoop:hadoop/usr/hadoop

12. Start Jobtracker on master, connect Mesos

Su Hadoop mesos_native_library=/usr/local/lib/libmesos.so Hadoop jobtracker

13. Testing

Input 192.168.0.223:5050 See if there's Hadoop in the frame.

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M02/75/BC/wKioL1ZBp7XjCu16AAB2UmudooI647.png "title=" eyw@0ro6agl]8) r%v0{ao_2.png "alt=" Wkiol1zbp7xjcu16aab2umudooi647.png "/>

This article is from the "Ming Linux blog" blog, be sure to keep this source http://zhaijunming5.blog.51cto.com/10668883/1711486

CENTOS7 mounting hadoop-cdh-2.5 on Mesos