HDFs file content Append (Append) version hdfs2.x support

The HDFs design does not support appending content to the file, so the design has its background (if you want to learn more about the append of HDFs , refer to the file appends in HDFs: http://blog.cloudera.com/blog/2009/07/file-appends-in-hdfs/), but starting with HDFs2.x support to file Additional content can be found in https://issues.apache.org/jira/browse/HADOOP-8230. Can look again http://www.quora.com/HDFs/is-HDFs- an-append-only-file-system-then-how-do-people-modify-the-files-stored-on-HDFS. As HADOOP-8230, it is only necessary to modify the following attribute in HDFs-site.xml to True.

<property>    <name>dfs.support.append</name>    <value>true</value></ Property>

How to append content to the HDFS file at the command line I haven't found the appropriate method yet. However, we can implement the file content append through the API provided by Hadoop. Here I have written a simple test program:

Package Com.wyp;import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.fs.filesystem;import Org.apache.hadoop.fs.path;import org.apache.hadoop.io.ioutils;import java.io.*;import java.net.URI;/** * blog:http  ://www.iteblog.com/* date:14-1-2 * Time: PM 6:09 */public class Appendcontent {public static void main (string[] args) {StringHDFs_path = "HDFs://mycluster/home/wyp/wyp.txt ";//file path Configuration conf = new configuration ();        Conf.setboolean ("Dfs.support.append", true);        String Inpath = "/home/wyp/append.txt";        FileSystem fs = null; try {fs = Filesystem.get (Uri.create (HDFs_path), Conf); ToAppendThe file stream, Inpath for file InputStream in = new Bufferedinputstream (new FileInputStream (Inpath)); OutputStream out = Fs.append (New Path (HDFs_path));        Ioutils.copybytes (in, out, 4096, true);        } catch (IOException e) {e.printstacktrace (); }    }}

Package the above code into a jar (here I named HDFs. jar) file and upload it to the machine, for example, I upload to my home directory, before the program runs, let's look at the contents of the Wyp.txt file in HDFs .

[[email protected] ~]$/home/q/hadoop-2.2.0/bin/hadoop FS                                              -cat/home/wyp/wyp.txt123456[[email protected] ~]$

OK, let's take a look at the contents of the/home/wyp/append.txt file:

[Email protected] ~]$ vim append.txtwyp  append test

After reading the two files that are involved in the code, we run the HDFs. jar

[[email protected] ~]$/home/q/hadoop-2.2.0/bin/hadoop jar                                        hdfs. Jar Com.wyp.AppendContent

After running, look at the wyp.txt content

[[email protected] ~]$/home/q/hadoop-2.2.0/bin/hadoop FS                                          -cat/home/wyp/wyp.txt123456wyp  Append test

OK, the Wyp.txt file has been append the contents of the Append.txt file. this blog post except special statement, all are original!

Respect the original, reproduced please specify: Reproduced from the past memory (http://www.iteblog.com/)
This link address: "hdfs file contents append (Append) "(http://www.iteblog.com/archives/881)
E-mail:[email protected]