高可用Hadoop平台-Oozie工作流之Hadoop调度(2)

<!-- OOZIE --> <property> <name>hadoop.proxyuser.hadoop.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.hadoop.groups</name> <value>*</value> </property>

  然后,在去修改Oozie的oozie-site.xml文件,在该文件中添加如下内容如下:

oozie-site.xml

<property> <name>oozie.services</name> <value> org.apache.oozie.service.SchedulerService, org.apache.oozie.service.InstrumentationService, org.apache.oozie.service.MemoryLocksService, org.apache.oozie.service.UUIDService, org.apache.oozie.service.ELService, org.apache.oozie.service.AuthorizationService, org.apache.oozie.service.UserGroupInformationService, org.apache.oozie.service.HadoopAccessorService, org.apache.oozie.service.JobsConcurrencyService, org.apache.oozie.service.URIHandlerService, org.apache.oozie.service.DagXLogInfoService, org.apache.oozie.service.SchemaService, org.apache.oozie.service.LiteWorkflowAppService, org.apache.oozie.service.JPAService, org.apache.oozie.service.StoreService, org.apache.oozie.service.SLAStoreService, org.apache.oozie.service.DBLiteWorkflowStoreService, org.apache.oozie.service.CallbackService, org.apache.oozie.service.ActionService, org.apache.oozie.service.ShareLibService, org.apache.oozie.service.CallableQueueService, org.apache.oozie.service.ActionCheckerService, org.apache.oozie.service.RecoveryService, org.apache.oozie.service.PurgeService, org.apache.oozie.service.CoordinatorEngineService, org.apache.oozie.service.BundleEngineService, org.apache.oozie.service.DagEngineService, org.apache.oozie.service.CoordMaterializeTriggerService, org.apache.oozie.service.StatusTransitService, org.apache.oozie.service.PauseTransitService, org.apache.oozie.service.GroupsService, org.apache.oozie.service.ProxyUserService, org.apache.oozie.service.XLogStreamingService, org.apache.oozie.service.JvmPauseMonitorService </value> </property> <property> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> <value>*=/home/hadoop/hadoop-2.6.0/etc/hadoop</value> </property>

  在修改完相关文件后,下面我们去创建Oozie的sharelib,其命令如下所示:

oozie-setup.sh sharelib create -fs hdfs://cluster1

  然后使用shareliblist命令查看相关内容,命令如下所示:

oozie admin -shareliblist -oozie http://nna:11000/oozie

  若成功创建,会生成如下图所示内容:

高可用Hadoop平台-Oozie工作流之Hadoop调度

  若未出现相应内容,请检查相关信息是否配置正确即可。

启动Oozie服务

oozied.sh start

  注:在启动时,这里建议打开oozie的启动日志,动态观察相关日志信息,也许会出现一些异常信息,比如:

Caused by: java.lang.NoClassDefFoundError: org/htrace/Trace at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:214) Caused by: java.lang.NoClassDefFoundError: com/google/protobuf/ServiceException at org.apache.hadoop.ipc.ProtobufRpcEngine.<clinit>(ProtobufRpcEngine.jav

  这些异常信息大多是由于我们在前面打包生成war包时,由于缺少相关的依赖JAR包导致的,这里我们将缺少的JAR从Hadoop的share目录下找到对应的JAR拷贝到Oozie运行war包容器下即可,如:

$OOZIE_HOME/oozie-server/webapps/oozie/WEB-INF/lib

  这里,异常都有相应的提示,大家耐心的按照提示解决异常即可,在解决相关异常后,我们就可以去创建相关工作流。

  Oozie给我们提供了相关示例让我去参考配置,下面我们将examples/apps下的文件上传到HDFS当中去,这里我上传在HDFS的/oozie目录下。接下来,我给大家去演示一个定时任务。首先,我们进去到apps目录下的cron目录,这是一个定时任务的示例,其中包含:coordinator.xml、job.properties和workflow.xml三个文件,这里我们对其进行配置。

coordinator.xml

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/2ea69aade9e10fb56541ccb65cbd6730.html