Cinder组件装在Controller节点，节点发生故障重启后，发现挂载了volumn的实例无法启动，错误日志如下：
2013-06-28 09:48:46 14338 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/Python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in wrapped
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in wrapped
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 176, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp pass
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 162, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 197, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info())
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 191, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 970, in start_instance
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.power_on_instance(context, instance)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in wrapped
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in wrapped
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 176, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp pass
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 162, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 197, in decorated_function
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info())
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp KeyError: 'instance'
2013-06-28 09:48:46 14338 TRACE nova.openstack.common.rpc.amqp
造成的原因分析：应该是实例VM在启动时，无法挂载虚拟磁盘volumn，查看cinder状态，发现果然不能创建及挂载volumn，错误日志如下：