Fedora20 installation hadoop-2.5.1, hadoop-2.5.1

Fedora20 installation hadoop-2.5.1, hadoop-2.5.1

First of all, I would like to thank the author lxdhdgss. His blog article directly helped me install hadoop. Below is his revised version for jdk1.8 installed on fedora20.

Go to the hadoop official website to copy the link address (hadoop2.5.1 address http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-2.5.1/hadoop-2.5.1.tar.gz), in the terminal wget url (download link address), will be directly downloaded to the current directory. Decompress tar xvf apache-hadoop-2.5.1.tar.gzto decompress the tar.gz file. You will see a folder named hadoop-2.5.1, copy it to your usual installation of software, such as I am/opt/lib64/below, the command is mv hadoop-2.5.1/opt/lib64 /.

 

Now we need to configure Hadoop.

 

Run vim/etc/profile to add

 

Xml Code

1. # Set the jdk installation path
   Export JAVA_HOME =/opt/lib64/jdk1.8.0 _ 20
2. # Specify CLASSPATH and PATH
   Export CLASSPATH = $ CLASSPATH: $ JAVA_HOME/lib: $ JAVA_HOME/jre/lib
   Export PATH = $ JAVA_HOME/bin: $ JAVA_HOME/jre/bin: $ PATH: $ HOME/bin
3. # Set hadoop Environment Variables
   Export HADOOP_HOME =/opt/lib64/hadoop-2.5.1
   Export PATH = $ HADOOP_HOME/bin: $ PATH
   Export CLASSPATH = $ HADOOP_HOME/lib: $ CLASSPATH

 

Save (ESC,: wq)

Oh, don't forget to run the source/etc/profile command on the terminal to make the modified profile take effect immediately.

 

Then go to the etc/hadoop/(not the system's etc, but under hadoop) under hadoop, ls, and see a lot of configuration files.

First vi hadoop-env.sh

 

Java code

1. Export JAVA_HOME =/opt/lib64/jdk1.8.0 _ 20 save and exit

 

 

Vim yarn-env.sh.

 

Java code

1. Export JAVA_HOME =/opt/lib64/jdk1.8.0 _ 20 save and exit

 

 

Vim core-site.xml.

 

Xml Code

1. <Configuration>
2. <Property>
3. <Name> fs. default. name </name>
4. <Value> hdfs :/// localhost: 9000 </value>
5. </Property>
6. <Property>
7. <Name> hadoop. tmp. dir </name>
8. <Value>/home/hadoop/xxx/tmp </value>
9. </Property>
10. </Configuration>

 

Vim hdfs-site.xml.

 

Xml Code

1. <Configuration>
2. <Property>
3. <Name> dfs. namenode. name. dir </name>
4. <Value> file:/home/hadoop/dfs/name </value>
5. </Property>
6. <Property>
7. <Name> dfs. namenode. data. dir </name>
8. <Value> file:/home/hadoop/dfs/data </value>
9. </Property>
10. <Property>
11. <Name> dfs. replication </name>
12. <Value> 1 </value>
13. </Property>
14. </Configuration>

------------------------- Supplement --------------- ssh Login-free settings

There is also: because the original article does not set how to verify the ssh Login, this section will be supplemented.

Since openssh is installed by default on fedora, you do not need to install ssh again. Directly configure the following:

Configure SSH password-free login. The following is the bash command, which is directly entered on the terminal (preferably the root identity, how to root? You only need to enter su on the terminal, and then the root password will be able to log on to the root account)

You can. This is all done.

-----------------------------------------------------------------------------

Okay, now you can start it.

Format it first. This is very important.

 

Bash command

1. Hadoop namenode-format

This statement basically determines the structure of your hadoop system. In our hadoop system, it is generally a namenode + multiple datanode.

 

Namenode is equivalent to the customer room table, and datanode is equivalent to a specific room.

 

Start hadoop.

To the root directory of hadoop,

Bash command

1. Sbin/start-dfs.sh

Then start yarn, because version 2.5.0 has long been running in a better architecture like yarn.

Terminal display Starting namenodes on [localhost]
Root @ localhost's password:
Localhost: starting namenode, logging to/opt/lib64/hadoop-2.5.1/logs/hadoop-root-namenode-localhost.localdomain.out
Root @ localhost's password:
Localhost: starting datanode, logging to/opt/lib64/hadoop-2.5.1/logs/hadoop-root-datanode-localhost.localdomain.out
Starting secondary namenodes [0.0.0.0]
Root@0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to/opt/lib64/hadoop-2.5.1/logs/hadoop-root-secondarynamenode-localhost.localdomain.out

 

 

Terminal display

1. Sbin/start-yarn.sh

Check the enabled process.

 

 

Bash command terminal display [root @ localhost hadoop-2.5.1] # jps
30305 DataNode
26243 org. eclipse. equinox. launcher_1.3.0.v20140415-2008.jar
26645 org. eclipse. equinox. launcher_1.3.0.v20120522-1813.jar
30583 SecondaryNameNode
8216 ResourceManager
30092 NameNode
Jps 30831
8511 NodeManager

Hadoop has been started successfully.

 

Open http: // localhost: 50070 and you can see the hadoop management interface. The last item in the navigation bar can also view the files in hadoop.

Let's try to upload a file to/home, and create a new file, vim test.txt, and write something at will. Save.

Then upload the file.

 

Bash command

1. Hadoop fs-put test.txt/test. first (it indicates that the local text is placed under the dfs root directory, and the name "test. first" can be randomly named)

Refresh the browser background to see if there is test. first (Open http: // localhost: 50070, select utilties and Click browse the file system ).

 

You can also create a directory for hadoop first.

 

Bash command

1. Hadoop fs-mkdir/xxx

The Upload File above can also be changed to the following:

 

 

Bash command

1. Hadoop fs-put test.txt/xxx/test.txt

For other operations, try it yourself.

Special thanks to lxdhdgss, whose blog address is: http://www.iteye.com/topic/1135085

 

How to install fedora20

If you have no installation experience, we recommend using a virtual machine test. I have installed or learned about it. You can use the installation instructions on the official page. You can download the officially supported fedorahosted.org/liveusb-creator/, or use www.pendrivelinux.com/yumi-multiboot-usb-creator/ to burn a USB flash disk for installation by using a USB flash disk. For the installation process, follow the instructions. Fedora is relatively simple. Be sure not to format the useful Disk Area.

Installation of Fedora 20

First, tell lz that you should not consider the flash drive officially made by fedora. It is a waste!
As for Apsara stack, I have never succeeded.
I used to create a fedora program in a linux system ).

It is a little troublesome, but it is very effective.

First, lz creates a USB flash drive for the ubuntu system with a soft dish. (This Is Because ubuntu can be used to successfully create a USB flash drive, and the method is very detailed. It is very useful in Baidu Library)
Then download a fedora installation image. Remember the download location.

Boot with a USB flash drive to enter the live system of ubuntu.
Copy the downloaded fedora image to the/home path of ubuntu (the path here can be different, just for convenience of demonstration ).
Then insert a USB flash drive, which is used to create a fedora boot disk. Generally, the first USB flash drive, that is, the ubuntu boot disk, will be/dev/sdb, and the disk you insert will be/dev/sdc. If you are not sure, enter sudo fdisk-l on the terminal to check whether there are two partitions in the ubuntu USB flash drive. One partition is the USB flash drive you have inserted.

Open the terminal and enter the location of the feodra image. The demo is/home.
Cd/home
Create a boot disk.
Sudo dd if = fedora image full name of =/dev/sdc (this is another USB flash drive you inserted)
After the burning is completed, a string of text will be displayed, prompting you how much data you burn into the USB flash drive. If the size is exactly the size of the fedora image, congratulations,
If it fails, dd if =... is used ...... This command can be used again.

After the installation is complete, restart the computer and start to install fedora.