Hadoop程序運行中的Error(1)-Error: org.apache.hadoop.hdfs.BlockMissingException

15/03/18 09:59:21 INFO mapreduce.Job: Task Id : attempt_1426641074924_0002_m_000000_2, Status : FAILED

Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-35642051-192.168.199.91-1419581604721:blk_1073743091_2267 file=/filein/file_128M.txt

at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:882)

at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:563)

at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:793)

at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:840)

at java.io.DataInputStream.readFully(DataInputStream.java:195)

at java.io.DataInputStream.readFully(DataInputStream.java:169)

at com.mr.AESEn.DataRecordReader.nextKeyValue(DataRecordReader.java:94)

at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:533)

at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)

at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

 

以上是MapReduce程序在運行過程當中產生的Error-Error: org.apache.hadoop.hdfs.BlockMissingException。

在網上查了下，可能的緣由有如下兩種：一是datanode節點有break的，另外一種是datanode之間的通訊有問題。秉着如上的緣由，我開始排錯：

 

[hadoop@cMaster hadoop-2.5.2]$ ssh cSlave00

Last login: Tue Mar 17 08:38:10 2015 from missie-pc.lan

[hadoop@cSlave00 ~]$ jps

3952 Jps

2910 NodeManager

 

[hadoop@cMaster hadoop-2.5.2]$ ssh cSlave01

Last login: Tue Mar 17 08:38:13 2015 from missie-pc.lan

[hadoop@cSlave01 ~]$ jps

3051 NodeManager

2714 DataNode

4562 Jps

 

[hadoop@cMaster hadoop-2.5.2]$ ssh cSlave02

Last login: Tue Mar 17 08:38:15 2015 from missie-pc.lan

[hadoop@cSlave02 ~]$ jps

4154 Jps

2921 NodeManager

 

能夠發現，cSlave00與cSlave02節點的DataNode都crash掉了。

 

因而：

1.關閉yarn與dfs

[hadoop@cMaster hadoop-2.5.2]$ sbin/stop-yarn.sh 

[hadoop@cMaster hadoop-2.5.2]$ sbin/stop-dfs.sh

 

2.從新啓動dfs與yarn

[hadoop@cMaster hadoop-2.5.2]$ sbin/start-dfs.sh

[hadoop@cMaster hadoop-2.5.2]$ sbin/start-yarn.sh

 

15/03/18 10:04:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Starting namenodes on [cMaster]

cMaster: starting namenode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-namenode-cMaster.out

cSlave00: starting datanode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-datanode-cSlave00.out

cSlave02: starting datanode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-datanode-cSlave02.out

cSlave01: starting datanode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-datanode-cSlave01.out

Starting secondary namenodes [0.0.0.0]

0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-secondarynamenode-cMaster.out

15/03/18 10:04:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

 

 

starting yarn daemons

starting resourcemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-resourcemanager-cMaster.out

cSlave01: starting nodemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-nodemanager-cSlave01.out

cSlave02: starting nodemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-nodemanager-cSlave02.out

cSlave00: starting nodemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-nodemanager-cSlave00.out

 

 

如此，便能解決以前出現的Error: org.apache.hadoop.hdfs.BlockMissingException問題了。

 

參考文章：http://blog.csdn.net/yonghutwo/article/details/39937185