2014-08-25 15:35:05,691 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:DatanodeRegistration ( 10.130.136.136:50010, storageid=ds-1533727399-10.130.136.136-50010-1388038551296, infoPort=50075, ipcPort=50020) :D Ataxceiver
java.net.sockettimeoutexception:480000 Millis Timeout while waiting for channel to is ready for write. ch:java.nio.channels.socketchannel[connected local=/10.130.136.136:50010 remote=/10.130.136.136:34264]
At Org.apache.hadoop.net.SocketIOWithTimeout.waitForIO (socketiowithtimeout.java:246)
At Org.apache.hadoop.net.SocketOutputStream.waitForWritable (socketoutputstream.java:159)
At Org.apache.hadoop.net.SocketOutputStream.transferToFully (socketoutputstream.java:198)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks (blocksender.java:392)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock (blocksender.java:490)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock (dataxceiver.java:202)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.run (dataxceiver.java:104)
At Java.lang.Thread.run (thread.java:724)
2014-08-25 15:35:06,464 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace:src:/ 10.130.136.136:37121, dest:/10.130.136.136:50010, bytes:67108864, Op:hdfs_write, Cliid:dfsclient_hb_rs_xx, 60020,1388115177740_1837727868_26, offset:0, srvid:ds-1533727399-10.130.136.136-50010-1388038551296, Blockid:blk_ -3628597342762703578_40720686, duration:6339411379
2014-08-25 15:35:06,464 INFO Org.apache.hadoop.hdfs.server.datanode.DataNode:PacketResponder 2 for Block blk_- 3628597342762703578_40720686 terminating
2014-08-25 15:35:06,465 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:Receiving block blk_- 7509787569548089877_40720689 src:/10.130.136.136:37142 dest:/10.130.136.136:50010
2014-08-25 15:35:06,724 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace:src:/ 10.130.136.136:50010, dest:/10.130.136.136:33647, bytes:5921280, Op:hdfs_read, Cliid:dfsclient_hb_rs_xx, 60020,1388115177740_1837727868_26, offset:388096, srvid:ds-1533727399-10.130.136.136-50010-1388038551296, Blockid : blk_2616588945174162483_33797955, duration:547889496646
2014-08-25 15:35:06,725 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:DatanodeRegistration ( 10.130.136.136:50010, storageid=ds-1533727399-10.130.136.136-50010-1388038551296, infoPort=50075, ipcPort=50020): Got exception while serving blk_2616588945174162483_33797955 to/10.130.136.136:
java.net.sockettimeoutexception:480000 Millis Timeout while waiting for channel to is ready for write. ch:java.nio.channels.socketchannel[connected local=/10.130.136.136:50010 remote=/10.130.136.136:33647]
At Org.apache.hadoop.net.SocketIOWithTimeout.waitForIO (socketiowithtimeout.java:246)
At Org.apache.hadoop.net.SocketOutputStream.waitForWritable (socketoutputstream.java:159)
At Org.apache.hadoop.net.SocketOutputStream.transferToFully (socketoutputstream.java:198)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks (blocksender.java:392)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock (blocksender.java:490)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock (dataxceiver.java:202)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.run (dataxceiver.java:104)
At Java.lang.Thread.run (thread.java:724)
2014-08-25 15:35:06,725 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:DatanodeRegistration ( 10.130.136.136:50010, storageid=ds-1533727399-10.130.136.136-50010-1388038551296, infoPort=50075, ipcPort=50020) :D Ataxceiver
java.net.sockettimeoutexception:480000 Millis Timeout while waiting for channel to is ready for write. ch:java.nio.channels.socketchannel[connected local=/10.130.136.136:50010 remote=/10.130.136.136:33647]
At Org.apache.hadoop.net.SocketIOWithTimeout.waitForIO (socketiowithtimeout.java:246)
At Org.apache.hadoop.net.SocketOutputStream.waitForWritable (socketoutputstream.java:159)
At Org.apache.hadoop.net.SocketOutputStream.transferToFully (socketoutputstream.java:198)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks (blocksender.java:392)
At Org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock (blocksender.java:490)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock (dataxceiver.java:202)
At Org.apache.hadoop.hdfs.server.datanode.DataXceiver.run (dataxceiver.java:104)
At Java.lang.Thread.run (thread.java:724)
Workaround:<property>
<name>dfs.socket.timeout</name>
<value>900000</value>
</property>
<property>
<name>dfs.datanode.handler.count</name>
<value>20</value>
</property>
<property>
<name>dfs.namenode.handler.count</name>
<value>30</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>10800000</value>
<description>set to minutes, default 8 * $ * 1000,just for 480000 millis timeout while waiting for channel to IS Ready for W
Rite</description>
</property>
480000 Millis Timeout while waiting for channel to is ready for write exception handling
