Create a small Hadoop cluster for learning

I recently learned about hadoop and want to try the true distribution after running the standalone and pseudo distribution modes. So I found several idle PCs to prepare a small cluster. These machines are all Delloptiplex745755. 1. Install the basic system to find a machine to install Ubuntu11.04, choose the server kernel, then install the sun-java-6-jdk, establish h

I recently learned about hadoop and want to try the true distribution after running the standalone and pseudo distribution modes. So I found several idle PCs to make a small one.ClusterThese machines are all obsolete Dell optiplEx745/755.

1. Install the basic system

Find a machine to install Ubuntu 11.04, select the server kernel, and then installSuN-java-6-jdk, build hadoop users, download hadoop set environment variables, For details, see the official documentation. After a machine is installed, use clonezilla to create a system image. Other machines can directly use clonezilla to reimage through ssh.

Summary:

1.1 after Ubuntu 11.04 is started, the monitor sleep.TrL + alt + F1 activation display, initially thought it was a problem with the video card, 10.10 does not seem to have this problem.

1.2 an error occurs in the Ubuntu partner source. If the sun-java-6 path cannot be found, you can directly install it on composer-I.

1.3 The machine with the clonezilla image must allow the root user to log on. Otherwise, a permission error is displayed. sudoPasswdRoot: Disable sudo passwd-l root.

1.4 There is a problem with the machine network after the image,IfconfigThe IP address is 127.0.0.1, And the ifup eth0 error is SIOCSIFADDR: No such device. Google found that/etc/udev/rules. d/70-persistent-net.rules recorded the MAC address of the original machine. Note the MAC address record and restart it.CpObtain the ip address.

1.5 The dhcp server cannot record the IP address and Host Name of the linux server, in/etc/dhcp/dhclient. it is useless to add send host-name "your-host-name" in conf, which may be not supported by the dhcp server. I tried dhcp.CdNo. Final decisionStatIc ip, hope IT doesn't bother me.

1.6 set the static IP address to the hosts file of several machines, sudo vi scp: // root @ hostname/etc/hosts. If it is not sudo, yank does not seem to support p, in addition, you must enable root on the remote machine to save the file.

PS. hadoop also runs well with openjdk, so you don't have to bother looking for Sun, http://wiki.apache.org/hadoop/hadoopjavaversions.