The Tempest scenario test TestStampPattern was unskipped on 2/13:
https://review.openstack.org/#/c/431800/
Since then the ceph jobs have been failing, e.g.:
http://logs.openstack.org/25/433825/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-trusty/4a58a2e/console.html#_2017-02-14_23_40_42_737153
2017-02-14 23:40:42.736898 | tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern[compute,id-10fd234a-515c-41e5-b092-8323060598c5,image,network,volume]
2017-02-14 23:40:42.736948 | ---------------------------------------------------------------------------------------------------------------------------------------------
2017-02-14 23:40:42.736959 |
2017-02-14 23:40:42.736975 | Captured traceback:
2017-02-14 23:40:42.736992 | ~~~~~~~~~~~~~~~~~~~
2017-02-14 23:40:42.737013 | Traceback (most recent call last):
2017-02-14 23:40:42.737037 | File "tempest/test.py", line 103, in wrapper
2017-02-14 23:40:42.737061 | return f(self, *func_args, **func_kwargs)
2017-02-14 23:40:42.737094 | File "tempest/scenario/test_stamp_pattern.py", line 112, in test_stamp_pattern
2017-02-14 23:40:42.737115 | keypair['private_key'])
2017-02-14 23:40:42.737153 | File "tempest/scenario/test_stamp_pattern.py", line 89, in _wait_for_volume_available_on_the_system
2017-02-14 23:40:42.737176 | raise lib_exc.TimeoutException
2017-02-14 23:40:42.737203 | tempest.lib.exceptions.TimeoutException: Request timed out
2017-02-14 23:40:42.737218 | Details: None
They fail while waiting for the attached volume to show up on the guest, and it never does.
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22_wait_for_volume_available_on_the_system%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20build_name%3A*ceph*&from=7d
There are only 11 hits in the ceph jobs in the last 24 hours but that's still pretty high. The ceph xenial job which runs on newton, ocata and master is non-voting so people probably aren't noticing, but the ceph trusty job runs on stable/mitaka and is voting and is failing.
We can probably pull some debug code out of this older patch I had:
https:/
Also, in that change I'm trying to be smarter about the device name used for the volume attachment rather than rely on the tempest.conf since the libvirt driver in nova completely ignores the device name passed in when attaching the volume.