Ubuntu12.04 Hadoop standalone Construction

It is mysterious to hear from others about Hadoop all the time, but it is only known after reading the introduction that Hadoop is distributed in an all-round way. Now distributed, big data is quite popular, and people have to join in.

Let's talk about my environment:

Www.bkjia.com @ bkjia-virtual :~ /Downloads $ uname-
Linux bkjia-virtual 3.11.0-17-generic #31 ~ Precise1-Ubuntu SMP Tue Feb 4 21:29:23 UTC 2014 i686 i686 i386 GNU/Linux

Preparations: Hadoop is an Apache product. You know, this is of course related to Java, so you have to have a Java compiler, whether you are OpenJDK or javasejdk, you need the whole one.

Refer:

Now, go to the topic.

1, we have to go to the Hadoop official website to download, I choose the latest hadoop-2.3.0 version, other versions remember to check, after all, the test environment, of course, with the latest good.

Decompress the package and copy it to the specified location.

Www.bkjia.com @ bkjia-virtual :~ /Downloads $ tar-xvf hadoop-2.3.0.tar.gz

Www.bkjia.com @ bkjia-virtual :~ /Downloads $ sudo cp-r hadoop-2.3.0/usr/local/hadoop/

2. Why? I want to talk about the version problem, because the location of this configuration file is very different from the previous version.

Of course, the version configuration file address is

Www.bkjia.com @ bkjia-virtual:/usr/local/hadoop/etc/hadoop $ ls-al
Total 128
Drwxr-xr-x 2 bkjia 4096 Feb 27.
Drwxr-xr-x 3 bkjia 4096 Feb 27 ..
-Rw-r -- 1 bkjia 3589 Feb 27 capacity-scheduler.xml
-Rw-r -- 1 bkjia 1335 Feb 27 configuration. xsl
-Rw-r -- 1 bkjia 318 Feb 27 container-executor.cfg
-Rw-r -- 1 bkjia 860 Feb 27 core-site.xml
-Rw-r -- 1 bkjia 3589 Feb 27 hadoop-env.cmd
-Rw-r -- 1 bkjia 3402 Feb 27 hadoop-env.sh
-Rw-r -- 1 bkjia 1774 Feb 27 hadoop-metrics2.properties
-Rw-r -- 1 bkjia 2490 Feb 27 hadoop-metrics.properties
-Rw-r -- 1 bkjia 9257 Feb 27 hadoop-policy.xml
-Rw-r -- 1 bkjia 984 Feb 27 hdfs-site.xml
-Rw-r -- 1 bkjia 1449 Feb 27 httpfs-env.sh
-Rw-r -- 1 bkjia 1657 Feb 27 httpfs-log4j.properties
-Rw-r -- 1 bkjia 21 Feb 27 httpfs-signature.secret
-Rw-r -- 1 bkjia 620 Feb 27 httpfs-site.xml
-Rw-r -- 1 bkjia 11169 Feb 27 log4j. properties
-Rw-r -- 1 bkjia 918 Feb 27 mapred-env.cmd
-Rw-r -- 1 bkjia 1383 Feb 27 mapred-env.sh
-Rw-r -- 1 bkjia 4113 Feb 27 mapred-queues.xml.template
-Rw-r -- 1 bkjia 758 Feb 27 mapred-site.xml.template
-Rw-r -- 1 bkjia 10 Feb 27 :09 slaves
-Rw-r -- 1 bkjia 2316 Feb 27 ssl-client.xml.example
-Rw-r -- 1 bkjia 2268 Feb 27 ssl-server.xml.example
-Rw-r -- 1 bkjia 2178 Feb 27 yarn-env.cmd
-Rw-r -- 1 bkjia 4084 Feb 27 yarn-env.sh
-Rw-r -- 1 bkjia 772 Feb 27 yarn-site.xml

The configuration files we want to modify include

The hadoop-env.sh found JAVA_HOME and modified it to this.

# The java implementation to use.

2 export JAVA_HOME =/usr/lib/jvm/jdk1.7.0 _ 45

Core-site.xml

Www.bkjia.com @ bkjia-virtual:/usr/local/hadoop/etc/hadoop $ cat core-site.xml
<? Xml version = "1.0" encoding = "UTF-8"?>
<? Xml-stylesheet type = "text/xsl" href = "configuration. xsl"?>
<! --
Licensed under the Apache License, Version 2.0 (the "License ");
You may not use this file before t in compliance with the License.
You may obtain a copy of the License

Http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
Distributed under the License is distributed on an "as is" BASIS,
Without warranties or conditions of any kind, either express or implied.
See the License for the specific language governing permissions and
Limitations under the License. See accompanying LICENSE file.
-->

<! -- Put site-specific property overrides in this file. -->

<Configuration>
<Property>
<Name> fs. defaultFS </name>
<Value> hdfs: // 127.0.0.1: 9000 </value>
</Property>
</Configuration>

Hdfs-site.xml

Www.bkjia.com @ bkjia-virtual:/usr/local/hadoop/etc/hadoop $ cat hdfs-site.xml
<? Xml version = "1.0" encoding = "UTF-8"?>
<? Xml-stylesheet type = "text/xsl" href = "configuration. xsl"?>
<! --
Licensed under the Apache License, Version 2.0 (the "License ");
You may not use this file before t in compliance with the License.
You may obtain a copy of the License

Http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
Distributed under the License is distributed on an "as is" BASIS,
Without warranties or conditions of any kind, either express or implied.
See the License for the specific language governing permissions and
Limitations under the License. See accompanying LICENSE file.
-->

<! -- Put site-specific property overrides in this file. -->

<Configuration>
<Property>
<Name> dfs. namenode. name. dir </name>
<Value> file:/usr/local/hadoop/dfs/name </value>
</Property>
<Property>
<Name> dfs. datanode. data. dir </name>
<Value> file:/usr/local/hadoop/dfs/data </value>
</Property>
</Configuration>

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

1 2 Next Page