Hadoop deployment Error

The standalone deployment of hadoop is simple and error-prone, but it has little value and significance for the production environment, but can be quickly used for development.

There are many and strange reasons for hadoop deployment errors.

For example, if the user name is different, the client and server communication may cause an authentication failure error. The user names of the client and server nodes should be consistent, and ssh login without authentication should be established for each node.

1. the following error occurs:

13/07/09 13:57:07 INFO ipc.Client: Retrying connect to server: master/192.168.2.200:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

Or

java.net.ConnectException: Call to master/192.168.2.200:9000 failed on connection exception: java.net.ConnectException: Connection refused        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1136)        at org.apache.hadoop.ipc.Client.call(Client.java:1112)        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:601)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)        at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:379)        at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:378)        at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:400)        at org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)        at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)        at java.lang.Thread.run(Thread.java:722)Caused by: java.net.ConnectException: Connection refused        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:719)        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:453)        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:579)        at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:202)        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1243)        at org.apache.hadoop.ipc.Client.call(Client.java:1087)        ... 14 more

This is because the client cannot connect to the server. It may be because the server has not started or the firewall has been started.

2. the following error occurs:

13/07/09 13:57:36 ERROR hdfs.DFSClient: Failed to close file /tmp/web304069331.logorg.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web304069331.log could only be replicated to 0 nodes, instead of 1        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:606)        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:415)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)        at org.apache.hadoop.ipc.Client.call(Client.java:1107)        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:601)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

There are many reasons. The namenode node and the datanode node cannot communicate normally. According to the query start log, the datanode node cannot parse the machine name. Therefore, modify the/etc/hostname and/etc/hosts files.

3. An error occurs:

13/07/09 13:59:01 INFO hdfs. DFSClient: Exception in createBlockOutputStream 192.168.2.201: 50010 java.net. socketTimeoutException: 63000 millis timeout while waiting for channel to be ready for connect. ch: java. nio. channels. socketChannel [connection-pending remote =/192.168.2.201: 50010] 13/07/09 13:59:01 INFO hdfs. DFSClient: Abandoning blk _-6965665250189110825_167913/07/09 13:59:01 INFO hdfs. DFSClient: Excluding datanode 192.168.2.201: 5001013/07/09 13:59:01 WARN hdfs. DFSClient: DataStreamer Exception: org. apache. hadoop. ipc. remoteException: java. io. IOException: File/tmp/webbench 901718.log cocould only be replicated to 0 nodes, instead of 1 at org. apache. hadoop. hdfs. server. namenode. FSNamesystem. getAdditionalBlock (FSNamesystem. java: 1920) at org. apache. hadoop. hdfs. server. namenode. nameNode. addBlock (NameNode. java: 783) at sun. reflect. generatedMethodAccessor6.invoke (Unknown Source) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 606) at org. apache. hadoop. ipc. RPC $ Server. call (RPC. java: 587) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1432) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1428) at java. security. accessController. doPrivileged (Native Method) at javax. security. auth. subject. doAs (Subject. java: 415) at org. apache. hadoop. security. userGroupInformation. doAs (UserGroupInformation. java: 1190) at org. apache. hadoop. ipc. server $ Handler. run (Server. java: 1426) at org. apache. hadoop. ipc. client. call (Client. java: 1107) at org. apache. hadoop. ipc. RPC $ Invoker. invoke (RPC. java: 229) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at sun. reflect. nativeMethodAccessorImpl. invoke0 (Native Method) at sun. reflect. nativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 57) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 601) at org. apache. hadoop. io. retry. retryInvocationHandler. invokeMethod (RetryInvocationHandler. java: 85) at org. apache. hadoop. io. retry. retryInvocationHandler. invoke (RetryInvocationHandler. java: 62) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. locateFollowingBlock (DFSClient. java: 3720) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. nextBlockOutputStream (DFSClient. java: 3580) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. access $2600 (DFSClient. java: 2783) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream $ DataStreamer. run (DFSClient. java: 3023) 13/07/09 13:59:01 WARN hdfs. DFSClient: Error Recovery for blk _-6965665250189110825_1679 bad datanode [0] nodes = null13/07/09 13:59:01 WARN hdfs. DFSClient: cocould not get block locations. source file "/tmp/website901718.log"-Aborting...org. apache. hadoop. ipc. remoteException: java. io. IOException: File/tmp/webbench 901718.log cocould only be replicated to 0 nodes, instead of 1 at org. apache. hadoop. hdfs. server. namenode. FSNamesystem. getAdditionalBlock (FSNamesystem. java: 1920) at org. apache. hadoop. hdfs. server. namenode. nameNode. addBlock (NameNode. java: 783) at sun. reflect. generatedMethodAccessor6.invoke (Unknown Source) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 606) at org. apache. hadoop. ipc. RPC $ Server. call (RPC. java: 587) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1432) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1428) at java. security. accessController. doPrivileged (Native Method) at javax. security. auth. subject. doAs (Subject. java: 415) at org. apache. hadoop. security. userGroupInformation. doAs (UserGroupInformation. java: 1190) at org. apache. hadoop. ipc. server $ Handler. run (Server. java: 1426) at org. apache. hadoop. ipc. client. call (Client. java: 1107) at org. apache. hadoop. ipc. RPC $ Invoker. invoke (RPC. java: 229) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at sun. reflect. nativeMethodAccessorImpl. invoke0 (Native Method) at sun. reflect. nativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 57) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 601) at org. apache. hadoop. io. retry. retryInvocationHandler. invokeMethod (RetryInvocationHandler. java: 85) at org. apache. hadoop. io. retry. retryInvocationHandler. invoke (RetryInvocationHandler. java: 62) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. locateFollowingBlock (DFSClient. java: 3720) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. nextBlockOutputStream (DFSClient. java: 3580) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. access $2600 (DFSClient. java: 2783) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream $ DataStreamer. run (DFSClient. java: 3023) ^ C July 09, 2:00:46 org. apache. coyote. abstractProtocol pauseINFO: Pausing ProtocolHandler ["http-bio-8080"] July 09, 2:00:46 org. apache. coyote. abstractProtocol pauseINFO: Pausing ProtocolHandler ["ajp-bio-8009"] July 09, 2:00:46 org. apache. catalina. core. standardService stopInternalINFO: Stopping service Catalina13/07/09 14:00:46 ERROR hdfs. DFSClient: Failed to close file/tmp/webbench 901718.logorg. apache. hadoop. ipc. remoteException: java. io. IOException: File/tmp/webbench 901718.log cocould only be replicated to 0 nodes, instead of 1 at org. apache. hadoop. hdfs. server. namenode. FSNamesystem. getAdditionalBlock (FSNamesystem. java: 1920) at org. apache. hadoop. hdfs. server. namenode. nameNode. addBlock (NameNode. java: 783) at sun. reflect. generatedMethodAccessor6.invoke (Unknown Source) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 606) at org. apache. hadoop. ipc. RPC $ Server. call (RPC. java: 587) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1432) at org. apache. hadoop. ipc. server $ Handler $ 1.run( Server. java: 1428) at java. security. accessController. doPrivileged (Native Method) at javax. security. auth. subject. doAs (Subject. java: 415) at org. apache. hadoop. security. userGroupInformation. doAs (UserGroupInformation. java: 1190) at org. apache. hadoop. ipc. server $ Handler. run (Server. java: 1426) at org. apache. hadoop. ipc. client. call (Client. java: 1107) at org. apache. hadoop. ipc. RPC $ Invoker. invoke (RPC. java: 229) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at sun. reflect. nativeMethodAccessorImpl. invoke0 (Native Method) at sun. reflect. nativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 57) at sun. reflect. delegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 43) at java. lang. reflect. method. invoke (Method. java: 601) at org. apache. hadoop. io. retry. retryInvocationHandler. invokeMethod (RetryInvocationHandler. java: 85) at org. apache. hadoop. io. retry. retryInvocationHandler. invoke (RetryInvocationHandler. java: 62) at com. sun. proxy. $ Proxy7.addBlock (Unknown Source) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. locateFollowingBlock (DFSClient. java: 3720) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. nextBlockOutputStream (DFSClient. java: 3580) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream. access $2600 (DFSClient. java: 2783) at org. apache. hadoop. hdfs. DFSClient $ DFSOutputStream $ DataStreamer. run (DFSClient. java: 3023) July 09 2:00:46 org. apache. catalina. loader. webappClassLoader clearReferencesThreads

This is because the communication between the client and the data node fails. The client program should be able to communicate with all nodes to ensure normal data transmission.

4. hadoop namenode-format:

Format aborted in /home/cnsworder/hadoop/name

Delete the folder and format it again.