Apache Hadoop Getting Started Tutorial Chapter III

Source: Internet
Author: User
Tags free ssh hdfs dfs

Password-free SSH settings

Now verify that you can log in to localhost with ssh without entering your password:

$ ssh localhost
1
If you do not enter a password, you cannot log in to localhost with SSH and execute the following command:

$ ssh-keygen-t Rsa-p "-F ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys
1
2
3
Perform

The following shows a job that runs a MapReduce locally, and here are the steps to run it.

(1) format a new Distributed File system:

$ Bin/hdfs Namenode-format
1
(2) Start the NameNode daemon and the DataNode daemon:

$ sbin/start-dfs.sh
1
The log of the Hadoop daemon is written to the $HADOOP _log_dir directory ($HADOOP _home/logs by default)

(3) Browse NameNode's network interface, their address defaults to:

namenode-http://localhost:50070/
1
(4) Create the HDFS directory to perform the MapReduce job:

$ Bin/hdfs Dfs-mkdir/user
$ Bin/hdfs dfs-mkdir/user/<username>
1
2
(5) Copy the input file to the Distributed File system:

$ Bin/hdfs dfs-put Etc/hadoop input
1
(6) Run the sample program provided by the release version:

$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input Output ' dfs[a-z. +
1
(7) View output file

Copy the output file from the Distributed file system to the local file system view:

$ bin/hdfs dfs-get Output output
$ cat output/*****
1
2
Alternatively, view the output file on the Distributed File system:

$ Bin/hdfs Dfs-cat output/*
1
(8) After completing all the actions, stop the daemon:

$ sbin/stop-dfs.sh
* * You need to learn to continue reading the next chapter. **

Many people know that I have big data training materials, all naïve thought I have a full set of big data development, Hadoop, spark and other video learning materials. I want to say that you are right, I do have big data development, Hadoop, Spark's full set of video materials.
If you are interested in big data development You can add a group to receive free learning materials: 763835121

Apache Hadoop Getting Started Tutorial Chapter III

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.