Hadoop2.2.0 HDFS currently does not support concurrent append

After multiple clients initiate concurrent append operations on the same file, only the client that was last initiated obtains the append lock. Other clients report the following error:

At org. apache. Hadoop. hdfs. server. namenode. FSNamesystem. appendFile (FSNamesystem. java: 2347)
At org. apache. hadoop. hdfs. server. namenode. NameNodeRpcServer. append (NameNodeRpcServer. java: 508)
At org. apache. hadoop. hdfs. protocolPB. ClientNamenodeProtocolServerSideTranslatorPB. append (ClientNamenodeProtocolServerSideTranslatorPB. java: 320)
At org. apache. hadoop. hdfs. protocol. proto. ClientNamenodeProtocolProtos $ ClientNamenodeProtocol $2. callBlockingMethod (ClientNamenodeProtocolProtos. java: 59572)
At org. apache. hadoop. ipc. ProtobufRpcEngine $ Server $ ProtoBufRpcInvoker. call (ProtobufRpcEngine. java: 585)
At org. apache. hadoop. ipc. RPC $ Server. call (RPC. java: 928)
At org. apache. hadoop. ipc. Server $ Handler $ 1.run( Server. java: 2048)
At org. apache. hadoop. ipc. Server $ Handler $ 1.run( Server. java: 2044)
At java. security. AccessController. doPrivileged (Native Method)
At javax. security. auth. Subject. doAs (Subject. java: 415)
At org. apache. hadoop. security. UserGroupInformation. doAs (UserGroupInformation. java: 1491)
At org. apache. hadoop. ipc. Server $ Handler. Execute (Server. java: 2042)
At org. apache. hadoop. ipc. Client. call (Client. java: 1347)
At org. apache. hadoop. ipc. Client. call (Client. java: 1300)
At org. apache. hadoop. ipc. ProtobufRpcEngine $ Invoker. invoke (ProtobufRpcEngine. java: 206)
At com. sun. proxy. $ Proxy9.append (Unknown Source)
At sun. reflect. GeneratedMethodAccessor10.invoke (Unknown Source)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43)
At java. lang. reflect. Method. invoke (Method. java: 606)
At org. apache. hadoop. io. retry. RetryInvocationHandler. invokeMethod (RetryInvocationHandler. java: 186)
At org. apache. hadoop. io. retry. RetryInvocationHandler. invoke (RetryInvocationHandler. java: 102)
At com. sun. proxy. $ Proxy9.append (Unknown Source)
At org. apache. hadoop. hdfs. protocolPB. ClientNamenodeProtocolTranslatorPB. append (ClientNamenodeProtocolTranslatorPB. java: 245)
At org. apache. hadoop. hdfs. DFSClient. callAppend (DFSClient. java: 1480)
At org. apache. hadoop. hdfs. DFSClient. append (DFSClient. java: 1520)
At org. apache. hadoop. hdfs. DFSClient. append (DFSClient. java: 1508)
At org. apache. hadoop. hdfs. DistributedFileSystem $4. doCall (DistributedFileSystem. java: 310)
At org. apache. hadoop. hdfs. DistributedFileSystem $4. doCall (DistributedFileSystem. java: 306)
At org. apache. hadoop. fs. FileSystemLinkResolver. resolve (FileSystemLinkResolver. java: 81)
At org. apache. hadoop. hdfs. DistributedFileSystem. append (DistributedFileSystem. java: 306)
At org. apache. hadoop. fs. filesystems. append (FileSystem. java: 1160)
At com. goojje. cart. etl. CartCommon. write (CartCommon. java: 84)
At com. goojje. cart. CartClientMain. pushData (CartClientMain. java: 63)
At com. goojje. cart. CartClientMain. main (CartClientMain. java: 29)
Org. apache. hadoop. ipc. RemoteException (java. io. IOException): java. lang. NullPointerException
At com. goojje. cart. index. CartIndexer. buildIndex (CartIndexer. java: 49)
At com. goojje. cart. ipc. server. CartProtocolImpl. saveIndex (CartProtocolImpl. java: 49)
At sun. reflect. GeneratedMethodAccessor4.invoke (Unknown Source)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43)
At java. lang. reflect. Method. invoke (Method. java: 606)
At org. apache. hadoop. ipc. WritableRpcEngine $ Server $ WritableRpcInvoker. call (WritableRpcEngine. java: 481)
At org. apache. hadoop. ipc. RPC $ Server. call (RPC. java: 928)
At org. apache. hadoop. ipc. Server $ Handler $ 1.run( Server. java: 2048)
At org. apache. hadoop. ipc. Server $ Handler $ 1.run( Server. java: 2044)
At java. security. AccessController. doPrivileged (Native Method)
At javax. security. auth. Subject. doAs (Subject. java: 415)
At org. apache. hadoop. security. UserGroupInformation. doAs (UserGroupInformation. java: 1491)
At org. apache. hadoop. ipc. Server $ Handler. Execute (Server. java: 2042)
At org. apache. hadoop. ipc. Client. call (Client. java: 1347)
At org. apache. hadoop. ipc. Client. call (Client. java: 1300)
At org. apache. hadoop. ipc. WritableRpcEngine $ Invoker. invoke (WritableRpcEngine. java: 231)
At com. sun. proxy. $ Proxy10.saveIndex (Unknown Source)
At com. goojje. cart. CartClientMain. pushData (CartClientMain. java: 64)
At com. goojje. cart. CartClientMain. main (CartClientMain. java: 29)

The analysis shows that Hadoop currently does not support concurrent write operations. The following is a theoretical explanation of this problem:

Copy local files to HDFS

Download files from HDFS to local

Upload local files to HDFS

Common commands for HDFS basic files

Introduction to HDFS and MapReduce nodes in Hadoop

Hadoop practice Chinese version + English version + Source Code [PDF]

Hadoop: The Definitive Guide (PDF]