The Append method of filesystem in Hadoop

Today in the append operation times of filesystem with Hadoop 1.1.2 The following exception is available:

1  Org.apache.hadoop.ipc.RemoteException:java.io.IOException:Append is not supported. Please see the dfs.support.append configuration parameter.

Google a bit, found that the Hadoop 1.x version does not support filesystem append operation, the official Hadoop 1.1.2 Release Notes are as follows:

• HADOOP-8230. Major improvement reported by ELI2 and fixed by Eli
  Enable Sync by default and disable append
  

  Append is not supported in Hadoop 1.x. Upgrade to 2.x if you need append. If you are enabled dfs.support.append for HBase, you ' re OK, as durable sync (why HBase required dfs.support.append) are now ENA Bled by default. If you really need the previous functionality, to turn on the Append functionality set the flag "Dfs.support.broken.append "To true.

The above explanation clearly mentions that if you need to use the append operation, you need to upgrade to the Hadoop 2.x version. And you need to include the following configuration in the Conf hdfs.site.xml file:

 <  property  >  2  <  name  >  dfs.support.append</ name      >  3  <  value  >  true</ value  >  4   Property  >   

The Hadoop API also provides settings to support the content append, the code is as follows:

1 New Configuration (); 2 true);

However, appending content to an existing file is an appropriate operation for the following reasons:

"Raghava Mail reply:" In Short, appendsinchHDFS is extremely experimental anddangerous. Most would advise toLeave this disabled. Your best option for "Append"Like behavior are toRewrite the file with Newcontent being added at theEnd. Append support was briefly introduced and  ThenRemoved asa number ofIssues came up. I believe theOpen(parent) JIRA issue tracking this is:http://issues.apache.org/jira/browse/hdfs-265

If you want to delve into its reasons, you can refer to the website: http://issues.apache.org/jira/browse/HDFS-265