Original address: http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_shell.html
FS Shell
Cat
Chgrp
chmod
Chown
Copyfromlocal
Copytolocal
Cp
Du
Dus
Expunge
Get
Getmerge
Ls
Lsr
Mkdir
Movefromlocal
Mv
Put
Rm
RMr
Setrep
Stat
Tail
Test
Text
Touchz
FS ShellThe call file system (FS) shell command should use the form Bin/
The installation of this article only covers Hadoop-common, Hadoop-hdfs, Hadoop-mapreduce, and Hadoop-yarn, and does not include hbase, Hive, and pig.http://blog.csdn.net/aquester/article/details/246210051. planning 1.1. list of machines
NameNode
Secondarynamenode
Datanodes
172
Apache-->hadoop's official Website document Command learning:http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_shell.html
FS Shell
The call file system (FS) shell command should use the bin/hadoop fs scheme://authority/path. For the HDFs file system, Scheme is HDFs, to the local file system, scheme is file. The scheme and authority parameters are optional, and if not specified, the default scheme specified in the configuration is used. An HDFs
Last week, the team led the research to Kerberos, to be used in our large cluster, and the research task was assigned to me. This week's words were probably done with a test cluster. So far the research is still relatively rough, many online data are CDH clusters, and our cluster is not used CDH, so in the process of integrating Kerberos there are some differences.
The test environment is a cluster of 5 machines, and the Hadoop version is 2.7.2. The 5
Read file:is the process by which HDFs reads files:Here is a detailed explanation:1. When the client begins to read a file, the client first obtains the Datanode information for the first few blocks of the file from Namenode. (steps)2. Start calling read (), the Read () method, first to read the first time from the Namenode to obtain a few blocks, when the read is completed, then go to Namenode take a block of datanode information. (Step 3,4,5)3. Call the Close method to complete the read. (Step
[Hadoop] 5. cloudera manager (3) and hadoopcloudera installed on HadoopInstall
Http://blog.sina.com.cn/s/blog_75262f0b0101aeuo.html
Before that, install all the files in the cm package
This is because CM depends on postgresql and requires postgresql to be installed on the local machine. If it is installed online, it is automatically installed in Yum mode. Because it is offline, postgresql cannot be installed automatically.
Check whether postgresql
1. By default, the Yarn log only displays info and above level information, and it is necessary to display the necessary debug information when the system is developed two times.
2. Configure yarn to print debug information to the log file, just modify its startup script sbin/yarn-daemon.sh, and change the info to debug (this step only).
Export Yarn_root_logger=${yarn_root_logger:-debug,rfa}
3. For HDFs, the modification method is similar, only need to modify the sbin/
Original articles, reproduced please mark from http://blog.csdn.net/lsttoy/article/details/53406840.First, go to Apache to see the official support version
You can see that hadoop2.4.x later versions basically support hbase1.2.4.The installation starts next.
The first step is to download the latest version from the Apache Foundation
Https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/1.2.4/hbase-1.2.4-bin.tar.gz
If you can not go to csdn and other major sites to download.
Step two , unzip to the
Configuration
The following properties should is in the core-site.xml of all the nodes in the cluster.
Hadoop.http.filter.initializers:add to the Org.apache.hadoop.security.AuthenticationFilterInitializer Initializer class.
Hadoop.http.authentication.type:Defines authentication used for the HTTP web-consoles. The Supported values Are:simple | Kerberos | #AUTHENTICATION_HANDLER_CLASSNAME #. The Dfeault value is simple.
Hadoop.http.authentication.token.validity:Indicates how long (in s
Install hadoop in Centos and connect to Eclipse
Hadoop has been planned for a long time and has not been put on the agenda until recently. It took some time to build hadoop under centos. The "setbacks" that we experienced before and after can definitely be written into a tear-down history of thousands of characters. There have been both online tutorials, as well
I downloaded the latest version of the Ubuntu64 (14.04) Desktop version of the system, in the installation of hadoop2.6.0, because the official Hadoop is on the 32-bit machine compiled, it is necessary to download the Hadoop source code to compile.Preparation: HADOOP-2.6.0-SRCJdk1.7.0_75 (because the latest version of the JDK is the 1.8.0_31 version, and I am usi
Reference Document http://blog.csdn.net/licongcong_0224/article/details/12972889 Reference document http://www.powerxing.com/install-hadoop/ Reference Document http://www.powerxing.com/install-hadoop-cluster/
Hadoop cluster installation configuration tutorial
Critical: Note that all host names need to be set for specification. You cannot use underscores to ma
Hadoop pseudo-distributed mode configuration and installation
Hadoop pseudo-distributed mode configuration and installation
The basic installation of hadoop has been introduced in the previous hadoop standalone mode. This section describes the basic simulation and deployment of had
Install and configure Hadoop in Linux
Before installing Hadoop on Linux, you need to install two programs:
JDK 1.6 or later;
We recommend that you install OpenSSH for SSH (Secure Shell protocol ).
The following describes the reasons for installing these two programs:
Hadoop is developed using Java. JDK is required for compiling
0 Basic Learning Hadoop to get started work Line guidance Questions Guide: What are the basics of 1.hadoop programming? What problems do 2.hadoop programming need to be aware of? 3. How to create a MapReduce program and how it contains several parts? 4. How can I connect to eclipse remotely and what problems might you encounter? 5. How to compile
Recently, when you want to exclude the logic of user thumb ups, you need to combine nginx access. only part of log logs and Mysql records can be used for joint query. Previous nginx logs are stored in hadoop, while mysql Data is not imported into hadoop, to do this, you have to import some tables in Mysql into HDFS. Although the name of Sqoop was too early
Recently, when you want to exclude the logic of use
Today in Bluemix easy to build a Hadoop cluster, Candide is the Hadoop command to forget to find out, today's supplement restudying
FS Shell
Calling the file system (FS) shell command should use the form of Bin/hadoop FS cat
How to use: Hadoop fs-cat uri [uri ...]
The path specifies the contents of the file to be e
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.