Introduction to some common commands in hadoop

Assume that the Hadoop installation directory HADOOP_HOME is/home/admin/hadoop.
Start and close
Start Hadoop
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/start-all.sh

Disable Hadoop
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/stop-all.sh
File Operations
Hadoop uses HDFS to implement functions similar to the disk system we use. Wildcard characters are also supported, such *.

View the file list
View the files in the/user/admin/aaron directory of hdfs.
1. Go to the HADOOP_HOME directory.
2. Run sh bin/hadoop fs-ls/user/admin/aaron.
In this way, we find the files in the/user/admin/aaron directory in hdfs.
We can also list all the files in the/user/admin/aaron directory in hdfs (including files in subdirectories ).
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop fs-lsr/user/admin/aaron

Create a file directory
Check the/user/admin/aaron directory in hdfs and create a new directory named newDir.
1. Go to the HADOOP_HOME directory.
2. Run sh bin/hadoop fs-mkdir/user/admin/aaron/newDir.

Delete an object
Delete the needDelete file under the/user/admin/aaron directory in hdfs
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop fs-rm/user/admin/aaron/needDelete
Delete the/user/admin/aaron directory in hdfs and all files in the directory
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop fs-rmr/user/admin/aaron

Upload files
Upload a local/home/admin/newFile file to the/user/admin/aaron directory of hdfs.
1. Go to the HADOOP_HOME directory.
2. Run sh bin/hadoop fs-put/home/admin/newFile/user/admin/aaron/

Download files
Download the newFile file in the/user/admin/aaron directory of hdfs to the local machine/home/admin/newFile.
1. Go to the HADOOP_HOME directory.
2. Run sh bin/hadoop fs-get/user/admin/aaron/newFile/home/admin/newFile.

View files
We can directly view the file in hdfs. The function is similar to cat.
View the newFile file in the/user/admin/aaron directory of hdfs
1. Go to the HADOOP_HOME directory.
2. Run sh bin/hadoop fs-cat/home/admin/newFile.

MapReduce Job operations
Submit a MapReduce Job
In principle, all Hadoop MapReduce jobs are a jar package.
Run a/home/admin/hadoop/job. jar MapReduce Job
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop jar/home/admin/hadoop/job. jar [jobMainClass] [jobArgs]

Killing a running Job
Assume that Job_Id is job_20100531_37_0053.
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop job-kill job_20100531_37_0053

More Hadoop commands
The Hadoop operation commands described above are the most common ones. If you want to learn more, you can obtain the command instructions as follows.
1. Go to the HADOOP_HOME directory.
2. Execute sh bin/hadoop
We can see the description of more commands:

Copy codeThe Code is as follows: Usage: hadoop [-- config confdir] COMMAND
Where COMMAND is one:
Namenode-format the DFS filesystem
Secondarynamenode run the DFS secondary namenode
Namenode run the DFS namenode
Datanode run a DFS datanode
Dfsadmin run a DFS admin client
Fsck run a DFS filesystem checking utility
Fs run a generic filesystem user client
Balancer run a cluster balancing utility
Jobtracker run the MapReduce job Tracker node
Pipes run a Pipes job
Tasktracker run a MapReduce task Tracker node
Job manipulate MapReduce jobs
Queue get information regarding JobQueues
Version print the version
Jar <jar> run a jar file
Distcp <srcurl> <desturl> copy file or directories recursively
Archive-archiveName NAME <src> * <dest> create a hadoop archive
Daemonlog get/set the log level for each daemon
Or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.