Fuel for OpenStack

Could not create volume after restart environment ceph ha

Bug #1363994 reported by Egor Kotko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Medium
Dennis Dmitriev
Fuel for OpenStack 6.0
5.1.x
Fix Released
Medium
Dennis Dmitriev
Fuel for OpenStack 5.1.1-updates

Bug Description

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "490"
  build_id: "2014-08-31_00-01-17"
  astute_sha: "bc60b7d027ab244039f48c505ac52ab8eb0a990c"
  fuellib_sha: "2cfa83119ae90b13a5bac6a844bdadfaf5aeb13f"
  ostf_sha: "4dcd99cc4bfa19f52d4b87ed321eb84ff03844da"
  nailgun_sha: "d25ed02948a8be773e2bd87cfe583ef7be866bb2"
  fuelmain_sha: "109812be3425408dd7be192b5debf109cb1edd4c"

System test: "ceph_ha_restart" was fail on OSTF (Create volume and attach it to instance (fuel_health.tests.smoke.test_create_volume.VolumesTest.test_volume_create). Test status: failure, message: Volume was not created.)
Log (cinder-api.log on node-1) contains:
http://paste.openstack.org/show/104156/

http://jenkins-product.srt.mirantis.net:8080/view/0_master_swarm/job/master_fuelmain.system_test.centos.thread_3/153/testReport/%28root%29/ceph_ha_restart/ceph_ha_restart/

Revision history for this message
Egor Kotko (ykotko) wrote :
Changed in fuel:
importance: Undecided → Medium
Vladimir Kuklin (vkuklin)
Changed in fuel:
status: New → Confirmed
Revision history for this message
Andrew Woodward (xarses) wrote :

traceback shows rabbit connection errors, removing ceph tag

tags: added: rabbit
Aleksey Kasatkin (alekseyk-ru)
Changed in fuel:
assignee: nobody → Fuel Library Team (fuel-library)
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

In the test "ceph_ha_restart" volumes are not created because of cinder-scheduler and cinder-volume services are not started. Cinder services depend on rabbitmq functionality and become available only after 'rabbitmq' cluster start.

It could be checked with "cinder service-list" command on any controller of the cluster.

no longer affects: fuel/5.1.x
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/123014

Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Dennis Dmitriev (ddmitriev)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (stable/5.1)

Fix proposed to branch: stable/5.1
Review: https://review.openstack.org/123015

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/123014
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=27908dd829fcaf7d3395f3da3c5a42263a629532
Submitter: Jenkins
Branch: master

commit 27908dd829fcaf7d3395f3da3c5a42263a629532
Author: Dennis Dmitriev <email address hidden>
Date: Sun Sep 21 22:15:17 2014 +0300

    Added check for Cinder services

    After cluster nodes restart, volumes creation are failed
    for several minutes because cinder services are down
    due to rabbitmq is not ready yet. We need wait until
    cinder-scheduler and cinder-volumes are up on controllers
    before doing OSTF tests.

    Change-Id: Ie9c35368e0f5e399ee2fe69c7132a4232fe40690
    Closes-Bug: #1363994

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (stable/5.1)

Reviewed: https://review.openstack.org/123015
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=396210066bdfbaa21697e516a8c63ced6493bc5d
Submitter: Jenkins
Branch: stable/5.1

commit 396210066bdfbaa21697e516a8c63ced6493bc5d
Author: Dennis Dmitriev <email address hidden>
Date: Sun Sep 21 22:15:17 2014 +0300

    Added check for Cinder services

    After cluster nodes restart, volumes creation are failed
    for several minutes because cinder services are down
    due to rabbitmq is not ready yet. We need wait until
    cinder-scheduler and cinder-volumes are up on controllers
    before doing OSTF tests.

    Change-Id: Ie9c35368e0f5e399ee2fe69c7132a4232fe40690
    Closes-Bug: #1363994

Dennis Dmitriev (ddmitriev)
Changed in fuel:
status: Fix Committed → In Progress
Revision history for this message
Dmitry Borodaenko (angdraug) wrote :

Why is this back to In Progress?

Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

It was mistake.

Changed in fuel:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.