Spark0.9分布式运行MLlib的线性回归算法(3)

14/02/17 21:54:33 INFO LinearRegressionWithSGD: Final model weights 0.6226986501625317,0.26562471165823115,-0.13304380020663167,0.21671917665388107,0.3037175607477254,-0.2007533914066441,0.013953499241049204,0.20270603251011174

14/02/17 21:54:33 INFO LinearRegressionWithSGD: Final model intercept 2.46997566387807

14/02/17 21:54:33 INFO SparkContext: Starting job: reduce at SimpleApp.scala:24

14/02/17 21:54:33 INFO DAGScheduler: Got job 22 (reduce at SimpleApp.scala:24) with 2 output partitions (allowLocal=false)

14/02/17 21:54:33 INFO DAGScheduler: Final stage: Stage 22 (reduce at SimpleApp.scala:24)

14/02/17 21:54:33 INFO DAGScheduler: Parents of final stage: List()

14/02/17 21:54:33 INFO DAGScheduler: Missing parents: List()

14/02/17 21:54:33 INFO DAGScheduler: Submitting Stage 22 (MappedRDD[45] at map at SimpleApp.scala:24), which has no missing parents

14/02/17 21:54:33 INFO DAGScheduler: Submitting 2 missing tasks from Stage 22 (MappedRDD[45] at map at SimpleApp.scala:24)

14/02/17 21:54:33 INFO TaskSchedulerImpl: Adding task set 22.0 with 2 tasks

14/02/17 21:54:33 INFO TaskSetManager: Starting task 22.0:0 as TID 42 on executor 2: slaver01 (NODE_LOCAL)

14/02/17 21:54:33 INFO TaskSetManager: Serialized task 22.0:0 as 2122 bytes in 1 ms

14/02/17 21:54:33 INFO TaskSetManager: Starting task 22.0:1 as TID 43 on executor 1: slaver02 (NODE_LOCAL)

14/02/17 21:54:33 INFO TaskSetManager: Serialized task 22.0:1 as 2122 bytes in 0 ms

14/02/17 21:54:34 INFO TaskSetManager: Finished TID 42 in 58 ms on slaver01 (progress: 0/2)

14/02/17 21:54:34 INFO DAGScheduler: Completed ResultTask(22, 0)

14/02/17 21:54:34 INFO TaskSetManager: Finished TID 43 in 60 ms on slaver02 (progress: 1/2)

14/02/17 21:54:34 INFO TaskSchedulerImpl: Remove TaskSet 22.0 from pool

14/02/17 21:54:34 INFO DAGScheduler: Completed ResultTask(22, 1)

14/02/17 21:54:34 INFO DAGScheduler: Stage 22 (reduce at SimpleApp.scala:24) finished in 0.057 s

14/02/17 21:54:34 INFO SparkContext: Job finished: reduce at SimpleApp.scala:24, took 0.081040964 s

14/02/17 21:54:34 INFO SparkContext: Starting job: count at SimpleApp.scala:24

14/02/17 21:54:34 INFO DAGScheduler: Got job 23 (count at SimpleApp.scala:24) with 2 output partitions (allowLocal=false)

14/02/17 21:54:34 INFO DAGScheduler: Final stage: Stage 23 (count at SimpleApp.scala:24)

14/02/17 21:54:34 INFO DAGScheduler: Parents of final stage: List()

14/02/17 21:54:34 INFO DAGScheduler: Missing parents: List()

14/02/17 21:54:34 INFO DAGScheduler: Submitting Stage 23 (MappedRDD[44] at map at SimpleApp.scala:20), which has no missing parents

14/02/17 21:54:34 INFO DAGScheduler: Submitting 2 missing tasks from Stage 23 (MappedRDD[44] at map at SimpleApp.scala:20)

14/02/17 21:54:34 INFO TaskSchedulerImpl: Adding task set 23.0 with 2 tasks

14/02/17 21:54:34 INFO TaskSetManager: Starting task 23.0:0 as TID 44 on executor 2: slaver01 (NODE_LOCAL)

14/02/17 21:54:34 INFO TaskSetManager: Serialized task 23.0:0 as 2011 bytes in 0 ms

14/02/17 21:54:34 INFO TaskSetManager: Starting task 23.0:1 as TID 45 on executor 1: slaver02 (NODE_LOCAL)

14/02/17 21:54:34 INFO TaskSetManager: Serialized task 23.0:1 as 2011 bytes in 0 ms

14/02/17 21:54:34 INFO TaskSetManager: Finished TID 45 in 44 ms on slaver02 (progress: 0/2)

14/02/17 21:54:34 INFO DAGScheduler: Completed ResultTask(23, 1)

14/02/17 21:54:34 INFO TaskSetManager: Finished TID 44 in 51 ms on slaver01 (progress: 1/2)

14/02/17 21:54:34 INFO TaskSchedulerImpl: Remove TaskSet 23.0 from pool

14/02/17 21:54:34 INFO DAGScheduler: Completed ResultTask(23, 0)

14/02/17 21:54:34 INFO DAGScheduler: Stage 23 (count at SimpleApp.scala:24) finished in 0.025 s

14/02/17 21:54:34 INFO SparkContext: Job finished: count at SimpleApp.scala:24, took 0.0633455 s

training Mean Squared Error = 0.4424462080486391

14/02/17 21:54:34 INFO ConnectionManager: Selector thread was interrupted!

[success] Total time: 41 s, completed Feb 17, 2014 9:54:34 PM

[root@master scala]#

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/57ac5067cb2a1031d353778c5e1861d1.html