HBase中服务器时间不同步造成的regionserver启动失败

今天配置hbase的时候发现有几台机器的regionserver启不来。报错如下:

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)

at java.lang.reflect.Constructor.newInstance(Constructor.java:513)

at org.apache.Hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)

at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)

at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1506)

at org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1470)

at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:563)

at java.lang.Thread.run(Thread.java:662)

Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server s3,60020,1304927875246 has been rejected; Reported time is too far out of sync with master.  Time difference of 41450ms > max allowed of 30000ms

at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:181)

at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:129)

at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:613)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)

at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)

at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)

at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)

at $Proxy5.regionServerStartup(Unknown Source)

at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1502)

错误里指出节点机的时间和master的时间差距大于30000ms,就是30秒时无法启动服务。之前hadoop没有遇到过这个问题,所以hbase存在系统时间同步的问题,并且误差要再30s以内。

在网上查了一下,要配置服务器时间同步需要用到ntp,最好能连接外网的时间服务器。内网里面配起来比较麻烦点,所以就手动改了时间,再启动就可以了。

还有另外一个方案,就是修改hbase最大时间误差。

在hbase-site.xml添加配置 hbase.master.maxclockskew 180000 Time difference of regionserver from master

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/psdgy.html