FileSystem for file operations in Hadoop

File Path Problems:

The path of the local file (linux) must start with file: //, and then add the actual file path. Example: file: // home/myHadoop/test

The file path in the cluster starts. Example:/temp/test

Command line operation. For example, to delete a hadoop cluster, operate on any node:

1. delete files in the Cluster

Hdfs dfs-rmr/temp/test hadoop 2.2.0

Hadoop fs-rmr/temp/test earlier version

2. commands for deleting Local Machine nodes

Hdfs dfs-rmr file: // home/myhadoop/test

Hadoop fs-rmr/temp/test

View hadoop Filesystem shell command: http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/FileSystemShell.html

Java operation file org. apache. hadoop. fs. FileSystem class

Fileystem is an abstract class that can be used to obtain a local file system or a cluster file system.

Configuration conf = new Configuration ();

Fileystem fs = FileSystem. get (URI. create (path), conf );

Here, path is the file path mentioned above. If it is a cluster file path, the cluster file system is obtained. If it is a local file path, the local file system is obtained.

Example: implement the shell command of copyFromLocal

Package com. alm. main;

Import java. io. IOException;
Import java.net. URI;

Import org. apache. hadoop. conf. Configuration;
Import org. apache. hadoop. fs. FileSystem;
Import org. apache. hadoop. fs. Path;

Public class FileSystemTest {
Public static void main (String args []) throws IOException {
FileSystem fs;
Configuration conf = new Configuration ();
Fs = FileSystem. get (URI. create ("hdfs: // 192.168.1.85: 9000"), conf );

Fs. copyFromLocalFile (false, true, new Path ("file: // home/myhadoop2/Desktop/testcp1"), new Path ("hdfs: // 192.168.1.85: 9000/testforcp/testcp1 "));


}
}

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)