解決:啟動hadoop-2.6.3集群遇到的datanode啟動不了 - 推酷 http://www.tuicool.com/articles/2y6B3uA
hadoop日志報錯信息如下:
2016-01-22 12:09:22,467 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /data/hadoop/tmp/dfs/data/in_use.lock acquired by nodename 13286@slave01
2016-01-22 12:09:22,469 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /data/hadoop/tmp/dfs/data: namenode clusterID = CID-ee797f67-7aa7-4ba5-8a13-ae174b6d0cf1; datanode clusterID = CID-9bd341cd-b887-49cb-b519-19772a3818c9
2016-01-22 12:09:22,470 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to master/192.168.1.120:9000. Exiting.
java.io.IOException: All specified directories are failed to load.
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1338)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1304)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:314)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:226)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:867)
at java.lang.Thread.run(Thread.java:745)
2016-01-22 12:09:22,471 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to master/192.168.1.120:9000
2016-01-22 12:09:22,473 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
2016-01-22 12:09:24,474 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
2016-01-22 12:09:24,476 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
2016-01-22 12:09:24,477 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
從日志上看是因為 datanode的clusterID 和 namenode的clusterID 不匹配饱亮。
解決方法:
根據日志中的路徑/data/hadoop/tmp/dfs蠢挡,master可以看到name目錄解藻,salve可以看到data目錄轻局,
將name/current下的VERSION中的clusterID復制到data/current下的VERSION中怜校,覆蓋掉原來的clusterID撤逢,目的是讓兩個保持一致窒朋。
然后重啟,就可以看到slave上的DataNode進程已經起來腋粥。
出現該問題的原因:在第一次格式化dfs后晦雨,啟動并使用了hadoop,后來又重新執(zhí)行了格式化命令(hdfs namenode -format)隘冲,這時namenode的clusterID會重新生成闹瞧,而datanode的clusterID 保持不變。