由于最近集群xcievers错误频频发生,已经到了影响集群正常运营的地步,于是决定修改集群所有的datanode节点的配置,并重启datanode,欲添加的配置项如下:
<property> <name>dfs.datanode.max.xcievers</name> <value>256</value> </property>修改完配置文件后,在重启datanode时发生了麻烦,datanode启动日志报出了版本不一致错误,详细日志如下:
2012-07-10 16:39:14,991 INFO org.apache.Hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.203.0 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May 4 07:57:50 PDT 2011 ************************************************************/ 2012-07-10 16:39:15,244 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-07-10 16:39:15,268 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-07-10 16:39:15,270 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-07-10 16:39:15,270 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2012-07-10 16:39:15,412 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-07-10 16:39:15,557 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Incompatible build versions: namenode BV = ; datanode BV = 1099333 2012-07-10 16:39:15,662 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Java.io.IOException: Incompatible build versions: namenode BV = ; datanode BV = 1099333 at org.apache.hadoop.hdfs.server.datanode.DataNode.handshake(DataNode.java:528) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:331) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573) 2012-07-10 16:39:15,663 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:datanode在获取namenode版本时出错,无法获取到namenode的版本。
分别尝试了一下解决办法:
1)重新安装hadoop,失败
2)对比hadoop datanode文件存储目录中的storage文件,对比其中的namespace id和storage id,没有发现异常,失败
3)修改hdfs-site.xml配置,将数据存储目录修改为新建文件夹,启动datanode,失败
4)查看namenode的启动日志,发现namenode最后一次启动时,版本升级为:0.20.203.1-snapshot,⊙﹏⊙b汗