[stack@devstack-f21 ~]$ [admin] cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| 1d7ce8ac-b440-44aa-80a7-932a938186a9 | in-use | cv1 | 1 | glusterfs | false | f886d5e8-257e-47b0-8b51-155d8a927af5 |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
[stack@devstack-f21 ~]$ [admin] cinder snapshot-list
+--------------------------------------+--------------------------------------+-----------+--------------+------+
| ID | Volume ID | Status | Display Name | Size |
+--------------------------------------+--------------------------------------+-----------+--------------+------+
| 22387337-8583-4d68-96a3-fa851056d0cd | 1d7ce8ac-b440-44aa-80a7-932a938186a9 | available | cv1-snap2 | 1 |
| ca7646b9-ca33-48e2-b27f-ea55010f7354 | 1d7ce8ac-b440-44aa-80a7-932a938186a9 | available | cv1-snap1 | 1 |
+--------------------------------------+--------------------------------------+-----------+--------------+------+
[stack@devstack-f21 ~]$ [admin] nova list
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+
| f886d5e8-257e-47b0-8b51-155d8a927af5 | vm1 | ACTIVE | - | Running | private=10.0.0.3, fdaf:7a10:7b10:0:f816:3eff:fe4b:69ae |
+--------------------------------------+------+--------+------------+-------------+--------------------------------------------------------+
[stack@devstack-f21 ~]$ [admin] nova stop vm1
Request to stop server vm1 has been accepted.
[stack@devstack-f21 ~]$ [admin] nova list
+--------------------------------------+------+---------+------------+-------------+--------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+---------+------------+-------------+--------------------------------------------------------+
| f886d5e8-257e-47b0-8b51-155d8a927af5 | vm1 | SHUTOFF | - | Shutdown | private=10.0.0.3, fdaf:7a10:7b10:0:f816:3eff:fe4b:69ae |
+--------------------------------------+------+---------+------------+-------------+--------------------------------------------------------+
Here are the steps used to re-create the fail:
[stack@devstack-f21 ~]$ [admin] cinder list ------- ------- ------- ------- ----+-- ------+ ------- ------- +------ +------ ------- +------ ----+-- ------- ------- ------- ------- ------- -+ ------- ------- ------- ------- ----+-- ------+ ------- ------- +------ +------ ------- +------ ----+-- ------- ------- ------- ------- ------- -+ b440-44aa- 80a7-932a938186 a9 | in-use | cv1 | 1 | glusterfs | false | f886d5e8- 257e-47b0- 8b51-155d8a927a f5 | ------- ------- ------- ------- ----+-- ------+ ------- ------- +------ +------ ------- +------ ----+-- ------- ------- ------- ------- ------- -+
+------
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+------
| 1d7ce8ac-
+------
[stack@devstack-f21 ~]$ [admin] cinder snapshot-list ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------+ ------- ------- +------ + ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------+ ------- ------- +------ + 8583-4d68- 96a3-fa851056d0 cd | 1d7ce8ac- b440-44aa- 80a7-932a938186 a9 | available | cv1-snap2 | 1 | ca33-48e2- b27f-ea55010f73 54 | 1d7ce8ac- b440-44aa- 80a7-932a938186 a9 | available | cv1-snap1 | 1 | ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------+ ------- ------- +------ +
+------
| ID | Volume ID | Status | Display Name | Size |
+------
| 22387337-
| ca7646b9-
+------
[stack@devstack-f21 ~]$ [admin] nova list ------- ------- ------- ------- ----+-- ----+-- ------+ ------- -----+- ------- -----+- ------- ------- ------- ------- ------- ------- ------- ------+ ------- ------- ------- ------- ----+-- ----+-- ------+ ------- -----+- ------- -----+- ------- ------- ------- ------- ------- ------- ------- ------+ 257e-47b0- 8b51-155d8a927a f5 | vm1 | ACTIVE | - | Running | private=10.0.0.3, fdaf:7a10: 7b10:0: f816:3eff: fe4b:69ae | ------- ------- ------- ------- ----+-- ----+-- ------+ ------- -----+- ------- -----+- ------- ------- ------- ------- ------- ------- ------- ------+
+------
| ID | Name | Status | Task State | Power State | Networks |
+------
| f886d5e8-
+------
[stack@devstack-f21 ~]$ [admin] nova stop vm1
Request to stop server vm1 has been accepted.
[stack@devstack-f21 ~]$ [admin] nova list ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------+ ------- ------+ ------- ------- ------- ------- ------- ------- ------- ------- + ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------+ ------- ------+ ------- ------- ------- ------- ------- ------- ------- ------- + 257e-47b0- 8b51-155d8a927a f5 | vm1 | SHUTOFF | - | Shutdown | private=10.0.0.3, fdaf:7a10: 7b10:0: f816:3eff: fe4b:69ae | ------- ------- ------- ------- ----+-- ----+-- ------- +------ ------+ ------- ------+ ------- ------- ------- ------- ------- ------- ------- ------- +
+------
| ID | Name | Status | Task State | Power State | Networks |
+------
| f886d5e8-
+------
[stack@devstack-f21 ~]$ [admin] cinder snapshot-delete cv1-snap1
[stack@devstack-f21 ~]$ [admin] cinder snapshot-list ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -----+- -----+ ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -----+- -----+ 8583-4d68- 96a3-fa851056d0 cd | 1d7ce8ac- b440-44aa- 80a7-932a938186 a9 | available | cv1-snap2 | 1 | ca33-48e2- b27f-ea55010f73 54 | 1d7ce8ac- b440-44aa- 80a7-932a938186 a9 | error_deleting | cv1-snap1 | 1 | ------- ------- ------- ------- ----+-- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -----+- -----+
+------
| ID | Volume ID | Status | Display Name | Size |
+------
| 22387337-
| ca7646b9-
+------
Exception from nova (n-cpu) screen:
2015-07-06 09:33:00.479 ERROR oslo_messaging. rpc.dispatcher [req-695dd8c5- 2722-4cf2- ab0a-583b0dacd3 88 nova service] Exception during message handling: Requested operation is not valid: domain is not running rpc.dispatcher Traceback (most recent call last): rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 142, in _dispatch_and_reply rpc.dispatcher executor_callback)) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 186, in _dispatch rpc.dispatcher executor_callback) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 130, in _do_dispatch rpc.dispatcher result = func(ctxt, **new_args) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/server. py", line 142, in inner rpc.dispatcher return func(*args, **kwargs) rpc.dispatcher File "/opt/stack/ nova/nova/ exception. py", line 89, in wrapped rpc.dispatcher payload) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 119, in __exit__ rpc.dispatcher six.reraise( self.type_ , self.value, self.tb) rpc.dispatcher File "/opt/stack/ nova/nova/ exception. py", line 72, in wrapped rpc.dispatcher return f(self, context, *args, **kw) rpc.dispatcher File "/opt/stack/ nova/nova/ compute/ manager. py", line 2954, in volume_ snapshot_ delete rpc.dispatcher snapshot_id, delete_info) rpc.dispatcher File "/opt/stack/ nova/nova/ virt/libvirt/ driver. py", line 2024, in volume_ snapshot_ delete rpc.dispatcher base_file = delete_ info['base_ file'] rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 119, in __exit__ rpc.dispatcher six.reraise( self.type_ , self.value, self.tb) rpc.dispatcher File "/opt/stack/ nova/nova/ virt/libvirt/ driver. py", line 2017, in volume_ snapshot_ delete rpc.dispatcher "snapshots.") % ver rpc.dispatcher File "/opt/stack/ nova/nova/ virt/libvirt/ driver. py", line 2003, in _volume_ snapshot_ delete rpc.dispatcher # paths are maintained relative by qemu. rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ eventlet/ tpool.py" , line 183, in doit rpc.dispatcher result = proxy_call( self._autowrap, f, *args, **kwargs) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ eventlet/ tpool.py" , line 141, in proxy_call rpc.dispatcher rv = execute(f, *args, **kwargs) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ eventlet/ tpool.py" , line 122, in execute rpc.dispatcher six.reraise(c, e, tb) rpc.dispatcher File "/usr/lib/ python2. 7/site- packages/ eventlet/ tpool.py" , line 80, in tworker rpc.dispatcher rv = meth(*args, **kwargs) rpc.dispatcher File "/usr/lib64/ python2. 7/site- packages/ libvirt. py", line 642, in blockCommit rpc.dispatcher if ret == -1: raise libvirtError ('virDomainBloc kCommit( ) failed', dom=self) rpc.dispatcher libvirtError: Requested operation is not valid: domain is not running rpc.dispatcher
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.
2015-07-06 09:33:00.479 TRACE oslo_messaging.