Hadoop Basic Operations Command (reprint)

Hadoop Basic Operations Command

Reprint: http://www.cnblogs.com/gpcuster/archive/2010/06/04/1751538.html

In this article, we assume that the Hadoop environment is already configured by the OPS staff to work directly.

Assume that the installation directory for Hadoop is Hadoop_home/home/admin/hadoop.

Start and close start Hadoop

1. Enter the Hadoop_home directory.
2. Execute SH bin/start-all.sh
3. Enter the Hadoop_home directory.
4. Execute SH bin/stop-all.sh

Close Hadoop file operations

Hadoop uses HDFs, which is similar in functionality to the disk systems we use. and wildcard characters such as * are supported.

View a list of files

View the files in the/user/admin/aaron directory in HDFs.

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs-ls/user/admin/aaron

In this way, we find the files in the/user/admin/aaron directory in HDFs.

We can also list all the files in the/user/admin/aaron directory in HDFs (including the files under subdirectories).

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs-lsr/user/admin/aaron

Create a file directory

View the/user/admin/aaron directory in HDFs and create a new directory called Newdir.

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs-mkdir/user/admin/aaron/newdir

deleting files

Delete a file named Needdelete in the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs-rm/user/admin/aaron/needdelete

Delete the/user/admin/aaron directory in HDFs and all files in that directory

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs-rmr/user/admin/aaron

Uploading files

Upload a native/home/admin/newfile file into the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs–put/home/admin/newfile/user/admin/aaron/

Download file

Download the NewFile file in the/user/admin/aaron directory in HDFs to the native/home/admin/newfile

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs–get/user/admin/aaron/newfile/home/admin/newfile

View Files

We can view the files directly in HDFs, and the functions are similar to those of cat

View NewFile files in the/user/admin/aaron directory in HDFs

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop fs–cat/home/admin/newfile

The MapReduce job operation submits a mapreduce job

In principle, all of the mapreduce jobs in Hadoop are a jar package.

Run a/home/admin/hadoop/job.jar mapreduce job

1. Enter the Hadoop_home directory.
2. Execution sh bin/hadoop jar/home/admin/hadoop/job.jar [jobmainclass] [Jobargs]

Kill a running job

Suppose job_id is: job_201005310937_0053

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop job-kill job_201005310937_0053

More commands for Hadoop

The operations commands described above for Hadoop are our most common. If you want to learn more, you can get instructions for the command in the following ways.

1. Enter the Hadoop_home directory.
2. Execute SH bin/hadoop

We can see instructions for more commands:

Usage:hadoop [--config Confdir] COMMAND

Where COMMAND is one of:

Namenode-format Format the DFS filesystem

Secondarynamenode Run the DFS secondary Namenode

Namenode Run the DFS namenode

Datanode Run a DFS datanode

Dfsadmin Run a DFS admin client

Fsck Run a DFS filesystem checking utility

FS Run a generic filesystem user Client

Balancer run a cluster balancing utility

Jobtracker run the MapReduce job Tracker node

Pipes run a pipes job

Tasktracker Run a MapReduce task Tracker node

Job Manipulate MapReduce Jobs

Queue get information regarding Jobqueues

Version Print the version

Jar <jar> Run a jar file

Distcp <srcurl> <desturl> Copy file or directories recursively

Archive-archivename NAME <src>* <dest> Create a Hadoop archive

Daemonlog Get/set The log level for each daemon

Or

CLASSNAME run the class named CLASSNAME

Most commands print help when invoked w/o parameters.

Hadoop Basic Operations Command (reprint)