Configure Hadoop on a single machine in Linux

Ubuntu:

1. install JAVA.

First install java. Because OpenJDK is installed in Ubuntu by default, You can uninstall it first. Enter the command on the terminal: sudo apt-get purge openjdk *.

1. Download JDK for linux from the sun homepage. I downloaded jdk-6u37-linux-i586.bin. : Click to open

2. Switch to the root user and the directory to be installed. My installation directory is/usr/lib/jvm/java. Copy the downloaded file to this directory. And use the command: chmod a + x jdk-6u37-linux-i586.bin to set this File Permission to executable type.

3. Start installation. Enter:./jdk-6u37-linux-i586.bin in the terminal, then the installation process is performed. During installation, you will be prompted to press Enter to continue.

4. After the installation is complete, Done will appear. Indicates that the java environment has been installed. The installation path is the current directory/usr/java. Of course, you can select another location.

5. After the installation is complete, an error occurs when you enter java directly on the terminal. In this case, you also need to configure environment variables. If only the export command is used, the current shell of the detachment works. If you switch to shell or restart, the system will still fail. You can select the configuration. bashr file or/etc/profile. The latter modifies the system configuration file, which is valid for all users.

6. Use vim to open the/etc/profile file. Add the following content at the end:

1. Export JAVA_HOME =/usr/lib/jvm/java/jdk1.6.0 _ 37
2. Export JRE_HOME =/usr/lib/jvm/java/jdk1.6.0 _ 37/jre
3. Export CLASSPATH =.: $ JAVA_HOME/lib: $ JRE_HOME/lib: $ CLASSPATH
4. Export PATH = $ JAVA_HOME/bin: $ JRE_HOME/bin: $ PATH

export JAVA_HOME=/usr/lib/jvm/java/jdk1.6.0_37export JRE_HOME=/usr/lib/jvm/java/jdk1.6.0_37/jreexport CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATHexport PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

(Note: Do not make any mistakes. Otherwise, you may be unable to enter the system after the reboot. If the input is incorrect, there is only a blank page after the boot, press ctrl + Alt + F1 to enter the tty1 command line interface, enter the command: sudo vim/etc/profile to check whether the configuration has been wrong, correct the configuration, and restart. Note that the export command = has no space on both sides .)

 

7. Save and restart the computer.

(Note: If you have information on the Internet, you can use source to update it. Enter the command source/etc/profile on the terminal. According to my test, this method will be valid only on one terminal. If a new terminal is opened, the java configuration will be invalid unless the source command is updated again .)

8. Run the env command to view the values of environment variables. If the content of each variable is the same as that configured previously, the configuration is successful. You can also run the java-version command. If java version "1.6.0 _ 37" is output, the configuration is correct.

2. Create a Hadoop group and a Hadoop user.

1. Create a hadoop user group: sudo addgroup hadoop

2. Create a hadoop User: sudo adduser-ingroup hadoop hadoopusr. You are required to enter the password and user information. When entering user information, you can directly contact Enter (indicating the default value ). Enter y.

3. add permissions to the newly created hadoop user hadoopusr to open the/etc/sudoers file. Run sudo gedit/etc/sudoers. Grant the root user the same permissions to hadoopusr. Add the following information at the end of the file:

1. Root ALL = (ALL: ALL) ALL
2. Hadoopusr ALL = (ALL: ALL) ALL

root    ALL=(ALL:ALL)  ALLhadoopusr  ALL=(ALL:ALL)  ALL

 

3. Install the ssh service

Ssh allows remote logon and management. For details, refer to other related documents.

Run sudo apt-get install ssh openssh-server to install openssh-server.

If you have installed ssh, proceed to the next step.

4. Create an ssh password-less logon to the Local Machine

First, convert it to a hadoop user, run the following command: su-hadoopusr, and then enter the password.

Ssh key generation methods include rsa and dsa. By default, rsa is used.

1. Create an ssh-key. We use the rsa method. Enter the command: ssh-keygen-t rsa-P ""

Enter file in which to save the key (/home/hadoopusr/. ssh/id_rsa): ", and press Enter. The following information is displayed:

1. Created directory '/home/hadoopusr/. ssh '.
2. Your identification has been saved in/home/hadoopusr/. ssh/id_rsa.
3. Your public key has been saved in/home/hadoopusr/. ssh/id_rsa.pub.
4. The key fingerprint is:
5. D4: 29: 00: 6e: 20: f0: d9: c6: a2: 9b: cd: 22: 60: 44: af: eb hadoopusr @ shan-pc
6. The key's randomart image is:
7. + -- [RSA 2048] ---- +
8. | +... |
9. |. O. *... |
10. |. +. * O |
11. |... +... |
12. | Oo S |
13. | O =. |
14. | =. O |
15. | O. |
16. | E |
17. + ----------------- +

Created directory '/home/hadoopusr/.ssh'.Your identification has been saved in /home/hadoopusr/.ssh/id_rsa.Your public key has been saved in /home/hadoopusr/.ssh/id_rsa.pub.The key fingerprint is:d4:29:00:6e:20:f0:d9:c6:a2:9b:cd:22:60:44:af:eb hadoopusr@shan-pcThe key's randomart image is:+--[ RSA 2048]----+|+.. ...          ||.o.*   . . .     || .+.*   o o      ||...+   . .       ||oo      S        ||o=.              ||=.o              ||o.               || E               |+-----------------+

 

(Note: After you press enter ~ Two files are generated under/. ssh/: id_rsa and id_rsa.pub. These two files are paired .)

2. Enter ~ In the/. ssh/directory, append id_rsa.pub to the authorized_keys authorization file. Run the following command:

Cd ~ /. Ssh
Cat id_rsa.pub> authorized_keys

You can log on to the local machine without a password.

3. log on to localhost. Enter the command: ssh localhost

(Note: After you remotely log on to another machine through ssh, you control the remote machine. You must run the exit command to re-control the local host .)

4. Run the exit command. Enter the command: exit

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

1 2 Next Page