Build a Hadoop-0.19.0 environment on javastuserver8.10

Building a Hadoop-0.19.0-based distributed Runtime Environment (VMwareworkstation) on javastuserver8.10 has long been a favorite of distributed technology, and has been trying to build a distributed programming environment, I have seen many articles about Hadoop recently, and Google has also developed a search engine using Hadoop. I want to try it myself, I found several articles on the Internet to discuss how to build the Hadoop environment.

Build a Hadoop-0.19.0-based distributed operating environment on Ubuntu Server 8.10

(VMware workstation Edition)

For a long time, I have always liked distributed technology and have been trying to build a distributed programming environment. Recently I have seen many articles discussing Hadoop, I also saw that Google has also developed a search engine using Hadoop, so I tried to make some fun. I found several articles on the Internet and discussed how to build the Hadoop environment, however, I didn't quite understand what I was talking about. I followed these articles Step by step. I thought it would be very smooth, but I encountered a lot of difficulties and difficulties. I checked them online for a long time, I tried many methods and took a lot of detours to solve the problem. Fortunately, all the problems have been solved. Last night, the environment was finally set up and run again. The Wordcount example of Hadoop was successfully passed, today, I am going to explain my "build a Journey" over the past few days. It is an annotation for my work.

Ubuntu is selected because the operating system is simple, easy to configure, and occupies a small amount of hard disk space. On T61, 2 GB of memory supports two virtual machines for Ubuntu servers, and the speed is very fast, virtual machine environment I chose VMware-workstation-6.5.1-126130, the official website Download. I started to install VMware on my XP Professional and installed Ubuntu Server 8.10 on it. Everything went fine. It was set by default, including hard disk (8 GB) and memory (512 MB) allocation, etc. If the configuration is low, you need to renew the header. Two points are worth mentioning,

1. Install the SSH Server during Ubuntu installation, so that you do not have to install the SSH software in the future because Hadoop communicates with each machine through SSH. Do not install the JDK that comes with Ubuntu, that is, Virual Machine Host (BSD OpenJDK). It is useless. We need sun jdk.

2. network configuration. I have read many articles about network configuration on the VM on the Internet, but I have not read many articles because I have never touched VMware. Let's just talk about my experience. I set up an environment at home and an environment in my organization. Broadband PPPoE is used for accessing the Internet at home, and the Unit is the LAN of the static IP address through proxy. Because I don't want to use NAT to access the Internet, I use the Bridged method and Replicated physical network connection state. This method makes virtual machines independent from the Internet and easy to access, you can see and find it.

URL Configuration

Home: I use DHCP to dynamically allocate the website address, the configuration file/etc/network/interfaces. I have not changed it, that is:

Iface eth0 inet dhcp

Auto eth0

Unit:/etc/network/interfaces, IP address, subnet mask, and gateway are modified.

Iface eth0 inet static

Address 10.8.83.200

Netmask 255.255.255.0

Gateway 10.8.83.254

Auto eth0

The command is as follows: sudo vi/etc/network/interfaces. As for the vi Operation Command, I won't talk about it here. Check it online.

DNS Configuration:

I don't care about the configuration at home, and it is also automatically obtained. The Unit configuration changes the/etc/resolv. conf file. Add the following code to the last line of the file: nameserver 10.8.83.11.

Note the following two important configurations:

1. You need to configure/etc/hostname:

Command: sudo/etc/hostname

Change localhost to your favorite name, for example, UBUNTU 1. In this way, the host name of your installed system will be unique.

2. You need to configure/etc/hosts.

Command: sudo/etc/hosts

Open this file and the following text will be displayed in the first line of the file:

127.0.0.1 localhost

Do not read the following text. Add the previous line as follows:

127.0.0.1 ubuntu1.localdomain ubuntu1

Save the file. If you are a static IP address, replace 127.0.0.1 with the IP address you specified. If you are a dynamic dhcp ip address, you need to find the IP address allocated to the virtual machine dynamically and replace 127.0.0.1 with 127.0.0.1.

This completes the network configuration and restarts the network to make it take effect: sudo/etc/init. d/networking restart

Because the Organization accesses the Internet through the proxy, the organization also needs to configure the proxy. I have configured the proxy during installation, but the agent cannot be connected once accessed. I guess the configuration is wrong, internet Lookup, many people say that if you set a global proxy, you can add configuration in profile. In fact, my requirement is very simple, that is, I can use apt update, so I chose to modify ~ /. Bashrc, add the following content at the end of the. bashrc file (replace yourproxyaddress and proxyport according to your actual situation ):

Export http: // [user: password] @ yourproxyaddress: proxyport.

The command used is as follows: sudo vi ~ /. Bashrc

In this way, you can ping www.sohu.com at home or in the Organization. No problem.