Hadoop Basic Operations Command

Hadoop Basic Operations Command

In this article, we default to the view that the Hadoop environment is already configured for direct use by operational personnel.

Suppose the installation directory for Hadoop is hadoop_home to/home/admin/hadoop.

Start and close
Start Hadoop
1. Enter the Hadoop_home directory.

2. Perform SH bin/start-all.sh

Turn off Hadoop
1. Enter the Hadoop_home directory.

2. Perform SH bin/stop-all.sh

File actions
Hadoop uses HDFs, which is similar to the disk system we use. Wildcard characters are supported, such as *.

View File List
View the files in the/user/admin/aaron directory in HDFs.

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs-ls/user/admin/aaron

In this way, we found the files in the/user/admin/aaron directory in HDFs.

We can also list all the files in the/user/admin/aaron directory in HDFs (including the files under subdirectories).

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs-lsr/user/admin/aaron

Create a file directory
View the/user/admin/aaron directory in HDFs and create a new directory called Newdir.

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs-mkdir/user/admin/aaron/newdir

deleting files
Delete a file named Needdelete in the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs-rm/user/admin/aaron/needdelete

Delete the/user/admin/aaron directory in HDFs and all files in the directory

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs-rmr/user/admin/aaron

Uploading files
Upload a native/home/admin/newfile file to the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs–put/home/admin/newfile/user/admin/aaron/

Download files
Download newfile files in/user/admin/aaron directory in HDFs to native/home/admin/newfile

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs–get/user/admin/aaron/newfile/home/admin/newfile

View Files
We can directly view the file directly in the HDFs, the function and the class is cat similar

View NewFile files in the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop fs–cat/home/admin/newfile

MapReduce Job operation
Submit MapReduce Job
In principle, all mapreduce jobs in Hadoop are a jar package.

Run a/home/admin/hadoop/job.jar mapreduce job

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop jar/home/admin/hadoop/job.jar [jobmainclass] [Jobargs]

Kill a running job
Suppose job_id is: job_201005310937_0053

1. Enter the Hadoop_home directory.

2. Perform sh bin/hadoop job-kill job_201005310937_0053

More Hadoop commands
These Hadoop operations commands are the most commonly used. If you want to learn more, you can get instructions for the command in the following way.

1. Enter the Hadoop_home directory.

2. Perform SH bin/hadoop

We can see more instructions for the command:

Usage:hadoop [--config Confdir] COMMAND

Where COMMAND is one of:

Namenode-format Format the DFS filesystem

Secondarynamenode Run the DFS secondary Namenode

Namenode Run the DFS namenode

Datanode Run a DFS datanode

Dfsadmin Run a DFS admin client

Fsck Run a DFS filesystem checking utility

FS Run a generic filesystem user Client

Balancer run a cluster balancing utility

Jobtracker run the MapReduce job Tracker node

Pipes run a pipes job

Tasktracker Run a MapReduce task Tracker node

Job Manipulate MapReduce Jobs

Queue get information regarding Jobqueues

Version Print the version

Jar <jar> Run a jar file

Distcp <srcurl> <desturl> Copy file or directories recursively

Archive-archivename NAME <src>* <dest> Create a Hadoop archive

Daemonlog nonattached The log level for each daemon

Or

CLASSNAME run the class named CLASSNAME

Most commands Print Help when invoked W parameters.

This article from Csdn Blog, reproduced please indicate the source: http://blog.csdn.net/gpcuster/archive/2010/06/04/5648702.aspx