3、部署CloudFoundry遇到的问题 3.1、上传Cloudfoundry release包时碰到的问题
执行上传命令：
bosh upload release /var/vcap/releases/cf-release/CF_Release_VF-131.1-dev.tgz
报错如下：
E, [2013-05-24T06:38:59.646076 #24082] [task:1] ERROR -- : Could not create object, 400/
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/blobstore_client-0.5.0/lib/blobstore_client/simple_blobstore_client.rb:30:in `create_file'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/blobstore_client-0.5.0/lib/blobstore_client/base.rb:30:in `create'
/var/vcap/packages/director/bosh/director/lib/director/blob_util.rb:9:in `block in create_blob'
/var/vcap/packages/director/bosh/director/lib/director/blob_util.rb:9:in `open'
/var/vcap/packages/director/bosh/director/lib/director/blob_util.rb:9:in `create_blob'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:373:in `create_package'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:291:in `block (2 levels) in create_packages'
/var/vcap/packages/director/bosh/director/lib/director/event_log.rb:58:in `track'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:289:in `block in create_packages'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:286:in `each'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:286:in `create_packages'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:272:in `process_packages'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:131:in `process_release'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:48:in `block in perform'
/var/vcap/packages/director/bosh/director/lib/director/lock_helper.rb:47:in `block in with_release_lock'
/var/vcap/packages/director/bosh/director/lib/director/lock.rb:58:in `lock'
/var/vcap/packages/director/bosh/director/lib/director/lock_helper.rb:47:in `with_release_lock'
/var/vcap/packages/director/bosh/director/lib/director/jobs/update_release.rb:48:in `perform'
/var/vcap/packages/director/bosh/director/lib/director/job_runner.rb:98:in `perform_job'
/var/vcap/packages/director/bosh/director/lib/director/job_runner.rb:29:in `block in run'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/bosh_common-0.5.4/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/bosh/director/lib/director/job_runner.rb:29:in `run'
/var/vcap/packages/director/bosh/director/lib/director/jobs/base_job.rb:8:in `perform'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/resque-1.15.0/lib/resque/job.rb:127:in `perform'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:163:in `perform'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:130:in `block in work'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `loop'
/var/vcap/packages/director/bosh/director/vendor/bundle/ruby/1.9.1/gems/resque-1.15.0/lib/resque/worker.rb:116:in `work'
/var/vcap/packages/director/bosh/director/bin/worker:77:in `<main>'
不能在bosh管理机中的blobstore虚拟机上创建文件，分析原因，可能是磁盘空间问题，SSH到blodstore的虚拟机上，查看目录/var/vcap/store的挂载点大小：
root@1154e252-382e-4cf7-bb2d-09adbc97a954:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 1.3G 956M 273M 78% /
none 241M 168K 241M 1% /dev
none 247M 0 247M 0% /dev/shm
none 247M 52K 247M 1% /var/run
none 247M 0 247M 0% /var/lock
none 247M 0 247M 0% /lib/init/rw
/dev/vdb2 20G 246M 18G 2% /var/vcap/data
/dev/loop0 124M 5.6M 118M 5% /tmp
挂载点无/var/vcap/store的格外挂载，根目录挂载大小为/dev/vda1，大小只有1.3G，上图中是删除了之前上传内容后的大小，没删除之前，/dev/vda1已经100%使用
解决办法：
1、创建大小20G（可根据实际情况设定）的/dev/vda2磁盘，挂载到/var/vcap/store，挂载前将其中的文件拷贝出来，挂载上之后再拷贝进去，结果：
root@1154e252-382e-4cf7-bb2d-09adbc97a954:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 1.3G 956M 273M 78% /
none 241M 168K 241M 1% /dev
none 247M 0 247M 0% /dev/shm
none 247M 52K 247M 1% /var/run
none 247M 0 247M 0% /var/lock
none 247M 0 247M 0% /lib/init/rw
/dev/vdb2 20G 246M 18G 2% /var/vcap/data
/dev/loop0 124M 5.6M 118M 5% /tmp
/dev/vda2 19G 1.8G 16G 11% /var/vcap/store
2、执行release删除命令
root@bosh-cli:/var/vcap/deployments# bosh releases
+---------------+-----------+
| Name | Versions |
+---------------+-----------+
| CF_Release_VF | 131.1-dev |
+---------------+-----------+
Releases total: 1
root@bosh-cli:/var/vcap/deployments# bosh delete release CF_Release_VF
3、再次执行release上传命令。
3.2、上传cf-service release包时发生错误错误信息如下：
root@bosh-cli:~/src/cloudfoundry/cf-services-release# bosh upload release
Upload release `CF-VF-Service-Release-0.1-dev.yml' to `bosh' (type 'yes' to continue): yes
Copying packages
----------------
ruby (0.1-dev) SKIP
libyaml (0.1-dev) SKIP
mysql (0.1-dev) FOUND LOCAL
ruby_next (0.1-dev) SKIP
postgresql_node (0.1-dev) FOUND LOCAL
mysqlclient (0.1-dev) SKIP
syslog_aggregator (0.1-dev) FOUND LOCAL
mysql_gateway (0.1-dev) FOUND LOCAL
postgresql92 (0.1-dev) FOUND LOCAL
mysql55 (0.1-dev) FOUND LOCAL
postgresql_gateway (0.1-dev) FOUND LOCAL
postgresql91 (0.1-dev) FOUND LOCAL
mysql_node (0.1-dev) FOUND LOCAL
postgresql (0.1-dev) FOUND LOCAL
sqlite (0.1-dev) SKIP
common (0.1-dev) SKIP
....
Release info
------------
Name: CF-VF-Service-Release
Version: 0.1-dev
Packages
- ruby (0.1-dev)
- libyaml (0.1-dev)
- mysql (0.1-dev)
- ruby_next (0.1-dev)
- postgresql_node (0.1-dev)
- mysqlclient (0.1-dev)
- syslog_aggregator (0.1-dev)
- mysql_gateway (0.1-dev)
- postgresql92 (0.1-dev)
- mysql55 (0.1-dev)
- postgresql_gateway (0.1-dev)
- postgresql91 (0.1-dev)
- mysql_node (0.1-dev)
- postgresql (0.1-dev)
- sqlite (0.1-dev)
- common (0.1-dev)
Jobs
- mysql_node_external (0.1-dev)
- postgresql_node (0.1-dev)
- mysql_gateway (0.1-dev)
- postgresql_gateway (0.1-dev)
- mysql_node (0.1-dev)
- rds_mysql_gateway (0.1-dev)
....
Director task 20
Extracting release
extracting release (00:00:08)
Done 1/1 00:00:08
Verifying manifest
verifying manifest (00:00:00)
Done 1/1 00:00:00
Resolving package dependencies
resolving package dependencies (00:00:00)
Done 1/1 00:00:00
Creating new packages
ruby/0.1-dev: Could not fetch object, 404/ (00:00:01)
Error 1/16 00:00:01
Error 100: Could not fetch object, 404/
Task 20 error
E, [2013-07-22T02:45:32.218762 #16460] [task:20] ERROR -- : Could not fetch object, 404/
/var/vcap/packages/director/gem_home/gems/blobstore_client-1.5.0.pre.3/lib/blobstore_client/dav_blobstore_client.rb:48:in `get_file'
/var/vcap/packages/director/gem_home/gems/blobstore_client-1.5.0.pre.3/lib/blobstore_client/base.rb:50:in `get'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/blob_util.rb:16:in `block (2 levels) in copy_blob'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/blob_util.rb:15:in `open'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/blob_util.rb:15:in `block in copy_blob'
/var/vcap/packages/ruby/lib/ruby/1.9.1/tmpdir.rb:83:in `mktmpdir'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/blob_util.rb:13:in `copy_blob'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:352:in `create_package'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:283:in `block (2 levels) in create_packages'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/event_log.rb:58:in `track'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:281:in `block in create_packages'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:278:in `each'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:278:in `create_packages'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:264:in `process_packages'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:134:in `process_release'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:47:in `block in perform'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/lock_helper.rb:47:in `block in with_release_lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/lock.rb:58:in `lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/lock_helper.rb:47:in `with_release_lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/update_release.rb:47:in `perform'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_runner.rb:98:in `perform_job'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_runner.rb:29:in `block in run'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_runner.rb:29:in `run'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/jobs/base_job.rb:8:in `perform'
/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/job.rb:125:in `perform'
/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:186:in `perform'
/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:149:in `block in work'
/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:128:in `loop'
/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:128:in `work'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/bin/worker:74:in `<top (required)>'
/var/vcap/packages/director/bin/worker:23:in `load'
/var/vcap/packages/director/bin/worker:23:in `<main>'
初步分析，Ruby的package之所以SKIP，是因为bosh检测到系统中之前以上传到同版本的package，所以跳过该包的上传，但是为什么后边加载这个package的时候找不到了呢？苦苦思索，还是无法确定问题发生原因；再分析package上传后是存储在虚拟机blobstore之中的，但是上传信息是存于别的位置，检查blobstore状态，running状态，正常，突然想到，之前因为网络故障，重启过blobstore虚拟机，检查发现var/vcap/store挂载的/dev/vda消失，发现问题，解决如下：
1、重新挂载/dev/vda2到/var/vcap/store下
2、配置blobstore虚拟机开机挂载/dev/vda2
再次执行upload命令，一切正常。
3.3 部署CloudFoundry services时发生的错误部署Services时，因为需要创建大量的虚拟机、创建大量的Volumes，所以可能会因为OpenStack环境本身的Quota限制引起一些错误，下边这个错误是因为Cinder本身初始配额为10.当创建的volume超过10的时候，就会发生错误：
Bosh中的报错信息：
E, [2013-08-22T08:45:38.099667 #8647] [task:323] ERROR -- : OpenStack API Request Entity Too Large error. Check task debug log for details.
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.3/lib/cloud/openstack/helpers.rb:20:in `cloud_error'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.3/lib/cloud/openstack/helpers.rb:39:in `rescue in with_openstack'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.3/lib/cloud/openstack/helpers.rb:25:in `with_openstack'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.3/lib/cloud/openstack/cloud.rb:361:in `block in create_disk'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.3/lib/cloud/openstack/cloud.rb:342:in `create_disk'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/instance_updater.rb:377:in `block in update_persistent_disk'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/database/query.rb:338:in `_transaction'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/database/query.rb:300:in `block in transaction'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/database/connecting.rb:236:in `block in synchronize'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/connection_pool/threaded.rb:104:in `hold'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/database/connecting.rb:236:in `synchronize'
/var/vcap/packages/director/gem_home/gems/sequel-3.43.0/lib/sequel/database/query.rb:293:in `transaction'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/instance_updater.rb:376:in `update_persistent_disk'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/instance_updater.rb:73:in `block in update'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/instance_updater.rb:39:in `step'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/instance_updater.rb:73:in `update'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_updater.rb:63:in `block (5 levels) in update'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_updater.rb:60:in `block (4 levels) in update'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/event_log.rb:58:in `track'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.3/lib/director/job_updater.rb:59:in `block (3 levels) in update'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_pool.rb:83:in `call'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_pool.rb:83:in `block (2 levels) in create_thread'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_pool.rb:67:in `loop'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.3/lib/common/thread_pool.rb:67:in `block in create_thread'
OpenStack中的报错信息：
2013-08-22 16:44:57 ERROR nova.api.openstack [req-75ffcfe3-34ea-4e8d-ab5f-685a890d4378 57be829ed997455f9600a4f46f7dbbef 8cf196acd0494fb0bc8d04e47ff77893] Caught error: VolumeLimitExceeded: Maximum number of volumes allowed (10) exceeded (HTTP 413) (Request-ID: req-30e6e7d6-313d-46b1-9522-b3b20dd3e2ab)
2013-08-22 16:44:57 24345 TRACE nova.api.openstack Traceback (most recent call last):
2013-08-22 16:44:57 24345 TRACE nova.api.openstack File "/usr/lib/Python2.7/dist-packages/nova/api/openstack/__init__.py", line 78, in __call__
2013-08-22 16:44:57 24345 TRACE nova.api.openstack return req.get_response(self.application)
......
2013-08-22 16:44:57 24345 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 109, in request
2013-08-22 16:44:57 24345 TRACE nova.api.openstack raise exceptions.from_response(resp, body)
2013-08-22 16:44:57 24345 TRACE nova.api.openstack OverLimit: VolumeLimitExceeded: Maximum number of volumes allowed (10) exceeded (HTTP 413) (Request-ID: req-30e6e7d6-313d-46b1-9522-b3b20dd3e2ab)
2013-08-22 16:44:57 24345 TRACE nova.api.openstack
解决办法：修改Cinder的quota配额，并重启cinder服务：
cinder quota-update <tenant_id> --volumes 20
cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i restart; done
问题解决