pause/resume failing (workload status races)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Landscape Server |
Fix Released
|
High
|
Andreas Hasenack | ||
16.06 |
Fix Released
|
High
|
Andreas Hasenack | ||
ceilometer-agent (Juju Charms Collection) |
Fix Released
|
High
|
Ryan Beisner | ||
cinder (Juju Charms Collection) |
Fix Released
|
High
|
David Ames | ||
glance (Juju Charms Collection) |
Fix Released
|
High
|
David Ames | ||
keystone (Juju Charms Collection) |
Fix Released
|
High
|
David Ames |
Bug Description
Cinder, keystone, glance, swift-proxy are failing pause/resume amulet full tests @ master as of May 10-13+, whereas they were all passing prior.
To date, I've only observed this failure on Trusty targets (ie. upstart). I've not pinned this to any one cause, though it could be an init/image revision, or an existing race that is simply exacerbated by slower performance during higher load.
## Timing observation in one repro, cinder@master
20:11:38 Test issues pause action
20:11:49 Test declares failure (based on juju action fetch returning non-zero).
20:12:22 Services on units get SIGTERM
## Juju status
Shows the charm has declared itself not-paused successfully, which is what the Amulet test goes by:
http://
=== original bug description ===
cinder@master amulet test often fails the pause action test
Observed and reproduced on trusty-liberty, trusty-mitaka test combos.
## Unit status
cinder/0 blocked idle 1.25.5 1 10.5.1.105 Services should be paused but these services running: cinder-api, cinder-volume, cinder-scheduler, Paused. Use 'resume' action to resume normal service.
## Amulet failure (trusty-mitaka, cinder@master 2016 May 12)
2016-05-12 18:11:56,004 test_910_
Running command: juju action do --format=json cinder/0 pause
Traceback (most recent call last):
File "/tmp/bundletes
deployment.
File "/tmp/bundletes
getattr(self, test)()
File "/tmp/bundletes
assert self._wait_
AssertionError: Pause action failed.
PASS: 1 ERROR: 1 Total: 2 (869.865811 sec)
## Juju status
http://
Related branches
- Alex Kavanagh (community): Approve
- charmers: Pending requested
-
Diff: 51 lines (+10/-1)2 files modifiedcharmhelpers/core/host.py (+4/-0)
tests/core/test_host.py (+6/-1)
- Ryan Beisner (community): Approve
- Edward Hope-Morley: Approve
-
Diff: 364 lines (+78/-94)2 files modifiedcharmhelpers/core/host.py (+16/-23)
tests/core/test_host.py (+62/-71)
description: | updated |
Changed in cinder (Juju Charms Collection): | |
status: | New → In Progress |
Changed in glance (Juju Charms Collection): | |
status: | New → In Progress |
Changed in keystone (Juju Charms Collection): | |
status: | New → In Progress |
Changed in cinder (Juju Charms Collection): | |
importance: | Undecided → High |
Changed in glance (Juju Charms Collection): | |
importance: | Undecided → High |
Changed in keystone (Juju Charms Collection): | |
importance: | Undecided → High |
Changed in cinder (Juju Charms Collection): | |
assignee: | nobody → David Ames (thedac) |
Changed in glance (Juju Charms Collection): | |
assignee: | nobody → David Ames (thedac) |
Changed in keystone (Juju Charms Collection): | |
assignee: | nobody → David Ames (thedac) |
Changed in cinder (Juju Charms Collection): | |
milestone: | none → 16.07 |
Changed in glance (Juju Charms Collection): | |
milestone: | none → 16.07 |
Changed in keystone (Juju Charms Collection): | |
milestone: | none → 16.07 |
summary: |
- pause/resume failing + pause/resume failing (workload status races) |
Changed in ceilometer-agent (Juju Charms Collection): | |
importance: | Undecided → High |
status: | New → In Progress |
milestone: | none → 16.07 |
assignee: | nobody → Ryan Beisner (1chb1n) |
tags: | added: kanban-cross-team landscape maintenance-mode |
tags: | removed: kanban-cross-team |
Changed in landscape: | |
assignee: | nobody → Andreas Hasenack (ahasenack) |
importance: | Undecided → High |
status: | New → Fix Committed |
milestone: | none → 16.06 |
Changed in cinder (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in glance (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in keystone (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in ceilometer-agent (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in landscape: | |
status: | Fix Committed → Fix Released |
Logs, artifacts available at: /openstack- ci-reports. ubuntu. com/artifacts/ test_charm_ pipeline_ amulet_ full/openstack/ charm-cinder/ 314773/ 5/2016- 05-12_14- 12-06/index. html
https:/