Java Operation HDFs Instance

Environment: Window7+eclipse+vmware virtual machines + built-in Hadoop environment (master, SLAVE01, SLAVE02)

Content: Primarily in a Windows environment, take advantage of how Eclipse operates HDFs, such as uploading files, deleting files, creating folders, viewing node information, and more.

For the Eclipse development environment, please refer to: http://www.cnblogs.com/bookwed/p/4816521.html

1, new Maven project, (mainly because of the introduction of some jar package, unless it is particularly clear to introduce which jar packages can not be built MAVEN project)

The details of creating a Web project are not described below, and the key Pom dependency information is posted here, primarily in Hadoop's basic and HDFS packages

    <dependency>        <groupId>org.apache.hadoop</groupId>        <artifactid>hadoop-hdfs</ artifactid>        <version>2.6.  2</version>    </dependency>    <dependency>        <groupId>org.apache.hadoop< /groupid>        <artifactId>hadoop-common</artifactId>        <version>2.6.  2</version>    </dependency>      <dependency>        <groupid>commons-logging </groupId>        <artifactId>commons-logging</artifactId>        <version>1.2 </version>    </dependency>    

2, create a common Java class, write code, paste part of the code, as follows:

1  Public classOperahdfs {2      Public Static voidMain (String args[])throwsIOException {3         //test to create a new file4         //byte[] contents = "Hello World hi \n--created by eclipse\n". GetBytes ();5         //createFile ("/eclipse/first.txt", contents); //or CreateFile ("HDFs://192.168.137.56:9000/eclipse/first.txt ", contents); 6         7         //Test upload local file8         //uploadfile ("D:\\c.txt", "/eclipse/");9         Ten         //Test Renaming One         //rename ("/eclipse/c.txt", "/eclipse/cc.txt"); A          -         //Test Delete Files -         //Delete ("/eclipse/cc.txt");//Using relative Paths the         //Delete ("/eclipse2"); //Delete directory -          -         //Test new Catalog -         //mkdir ("/eclipse2/"); +          -         //Test Read File +         //readFile ("/eclipse/first.txt"); A          at         //test whether the file exists -         //fileisexists ("/eclipse/first.txt"); -          - Getnodemsghdfs (); -          -     } in  -     //1. Create a new file (directly generate the first.txt under the specified path, i.e.:/eclipse/first.txt) to      Public Static voidCreateFile (String DST,byte[] contents)throwsIOException { +Configuration conf =NewConfiguration (); -System.out.println ("-----------:" +conf); theConf.set ("Fs.defaultfs", "hdfs://192.168.137.56:9000");//Master *FileSystem fs =filesystem.get (conf); $Path Dstpath =NewPath (DST);//Target PathPanax Notoginseng         //open an output stream -Fsdataoutputstream OutputStream =fs.create (dstpath); the Outputstream.write (contents); + outputstream.close (); A fs.close (); theSystem.out.println ("File creation succeeded! "); +     } -  $     //2. Uploading Local Files $      Public Static voidUploadFile (string src, string dst)throwsIOException { -Configuration conf =NewConfiguration (); -Conf.set ("Fs.defaultfs", "hdfs://192.168.137.56:9000");//Master theFileSystem fs =filesystem.get (conf); -Path Srcpath =NewPath (SRC);//Source PathWuyiPath Dstpath =NewPath (DST);//Target Path the         //call the file System file copy function, the preceding parameter refers to whether to delete the original file, true to delete, the default is False -Fs.copyfromlocalfile (false, Srcpath, Dstpath); Wu  -         //Print File path AboutSystem.out.println ("Upload to" + Conf.get ("Fs.default.name"))); $         //lists all files under the specified path -System.out.println ("------------list files------------" + \ n "); -filestatus[] Filestatus =Fs.liststatus (dstpath); -          for(Filestatus file:filestatus) { ASystem.out.println (File.getpath () + "--" +file.getgroup () + "--" +file.getblocksize () + "--" +file.getlen () + "--"); +         } the fs.close (); -}
}

3, the complete code, please refer to: Http://pan.baidu.com/s/1eRsXp6M password: 9tg9, there are some examples of compressed files.

　　

　　

Java Operation HDFs Instance