OpenStack Manila Charm

test_manila_share functional test always fails on creating a share

Bug #1935022 reported by Aurelien Lourot on 2021-07-08

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Manila Charm	In Progress	Undecided	Chris MacNaughton
	OpenStack Manila-Ganesha Charm	New	Undecided	Unassigned

Bug Description

On all architectures (seen at least on bionic-stein and focal-ussuri), visible in this gate:
https://review.opendev.org/c/openstack/charm-manila/+/778890
https://openstack-ci-reports.ubuntu.com/artifacts/00d/778890/9/check/ganesha-focal-ussuri/00d412c/job-output.txt

2021-07-06 08:46:45 [INFO] test_manila_share (zaza.openstack.charm_tests.manila_ganesha.tests.ManilaGaneshaTests)
2021-07-06 08:46:45 [INFO] Test that a Manila share can be accessed on two instances.
2021-07-06 08:46:45 [INFO] ...
2021-07-06 08:46:48 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:46:50 [INFO] Using keystone API V3 (or later) for overcloud auth
scp: /usr/local/share/ca-certificates/keystone_juju_ca_cert.crt: No such file or directory
2021-07-06 08:46:52 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:46:57 [INFO] Launching instance zaza-manilatests-ins-1
2021-07-06 08:46:59 [INFO] Checking instance is active
2021-07-06 08:46:59 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:47:01 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:47:03 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:47:08 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:47:16 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:47:32 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:48:05 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:49:05 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:50:06 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in BUILD state, waiting for ACTIVE
2021-07-06 08:51:07 [INFO] resource: resource c466ff13-00ee-491e-93c5-3e786d401f8a in ACTIVE state, waiting for ACTIVE
2021-07-06 08:51:07 [INFO] Checking cloud init is complete
2021-07-06 08:51:47 [INFO] Assigning floating ip.
2021-07-06 08:51:47 [INFO] Creating floatingip
2021-07-06 08:51:50 [INFO] Assigned floating IP 172.16.244.200 to zaza-manilatests-ins-1
2021-07-06 08:51:55 [INFO] Testing ssh access.
2021-07-06 08:51:55 [INFO] Attempting to ssh to zaza-manilatests-ins-1(172.16.244.200)
2021-07-06 08:51:55 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2021-07-06 08:51:55 [INFO] Authentication (publickey) failed.
2021-07-06 08:51:55 [INFO] Authentication (publickey) failed.
2021-07-06 08:51:57 [INFO] Attempting to ssh to zaza-manilatests-ins-1(172.16.244.200)
2021-07-06 08:51:57 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2021-07-06 08:51:57 [INFO] Authentication (publickey) failed.
2021-07-06 08:51:57 [INFO] Authentication (publickey) failed.
2021-07-06 08:51:59 [INFO] Attempting to ssh to zaza-manilatests-ins-1(172.16.244.200)
2021-07-06 08:51:59 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2021-07-06 08:52:00 [INFO] Authentication (publickey) failed.
2021-07-06 08:52:00 [INFO] Authentication (publickey) failed.
2021-07-06 08:52:03 [INFO] Removing already existing instance (c466ff13-00ee-491e-93c5-3e786d401f8a) with requested name (zaza-manilatests-ins-1)
2021-07-06 08:52:10 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:52:12 [INFO] Using keystone API V3 (or later) for overcloud auth
scp: /usr/local/share/ca-certificates/keystone_juju_ca_cert.crt: No such file or directory
2021-07-06 08:52:14 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:52:18 [INFO] Launching instance zaza-manilatests-ins-1
2021-07-06 08:52:19 [INFO] Checking instance is active
2021-07-06 08:52:19 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:52:21 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:52:23 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:52:27 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:52:36 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:52:52 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:53:25 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:54:25 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in BUILD state, waiting for ACTIVE
2021-07-06 08:55:26 [INFO] resource: resource 96c23fcc-e08a-4d43-922a-dde858735f3d in ACTIVE state, waiting for ACTIVE
2021-07-06 08:55:26 [INFO] Checking cloud init is complete
2021-07-06 08:56:35 [INFO] Assigning floating ip.
2021-07-06 08:56:35 [WARNING] A floating IP already exists but ports do not match Potentially creating more than one.
2021-07-06 08:56:35 [INFO] Creating floatingip
2021-07-06 08:56:40 [INFO] Assigned floating IP 172.16.244.209 to zaza-manilatests-ins-1
2021-07-06 08:56:45 [INFO] Testing ssh access.
2021-07-06 08:56:45 [INFO] Attempting to ssh to zaza-manilatests-ins-1(172.16.244.209)
2021-07-06 08:56:45 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2021-07-06 08:56:45 [INFO] Authentication (publickey) successful!
2021-07-06 08:56:45 [INFO] Running uname -n on zaza-manilatests-ins-1
2021-07-06 08:56:47 [INFO] SSH to zaza-manilatests-ins-1(172.16.244.209) succesfull
2021-07-06 08:56:52 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:56:55 [INFO] Using keystone API V3 (or later) for overcloud auth
scp: /usr/local/share/ca-certificates/keystone_juju_ca_cert.crt: No such file or directory
2021-07-06 08:56:57 [INFO] looking at application: {'name': 'keystone', 'type': {'pkg': 'keystone', 'origin_setting': 'openstack-origin'}}
2021-07-06 08:57:02 [INFO] Launching instance zaza-manilatests-ins-2
2021-07-06 08:57:03 [INFO] Checking instance is active
2021-07-06 08:57:04 [INFO] resource: resource f66a6b97-39cf-4274-9a67-a8cca981651b in BUILD state, waiting for ACTIVE
2021-07-06 08:57:05 [INFO] resource: resource f66a6b97-39cf-4274-9a67-a8cca981651b in BUILD state, waiting for ACTIVE
2021-07-06 08:57:08 [INFO] resource: resource f66a6b97-39cf-4274-9a67-a8cca981651b in BUILD state, waiting for ACTIVE
2021-07-06 08:57:12 [INFO] resource: resource f66a6b97-39cf-4274-9a67-a8cca981651b in BUILD state, waiting for ACTIVE
2021-07-06 08:57:21 [INFO] resource: resource f66a6b97-39cf-4274-9a67-a8cca981651b in ACTIVE state, waiting for ACTIVE
2021-07-06 08:57:21 [INFO] Checking cloud init is complete
2021-07-06 09:01:37 [INFO] Assigning floating ip.
2021-07-06 09:01:38 [WARNING] A floating IP already exists but ports do not match Potentially creating more than one.
2021-07-06 09:01:38 [INFO] Creating floatingip
2021-07-06 09:01:40 [INFO] Assigned floating IP 172.16.244.201 to zaza-manilatests-ins-2
2021-07-06 09:01:45 [INFO] Testing ssh access.
2021-07-06 09:01:45 [INFO] Attempting to ssh to zaza-manilatests-ins-2(172.16.244.201)
2021-07-06 09:01:45 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2021-07-06 09:01:46 [INFO] Authentication (publickey) successful!
2021-07-06 09:01:46 [INFO] Running uname -n on zaza-manilatests-ins-2
2021-07-06 09:01:48 [INFO] SSH to zaza-manilatests-ins-2(172.16.244.201) succesfull
2021-07-06 09:01:59 [INFO] Waiting for a share to become available: resource ece44a4a-59d0-4840-9256-792e17e080cb in available state, waiting for available
2021-07-06 09:02:14 [INFO] ERROR
2021-07-06 09:02:14 [INFO] Cleaning up test Nova servers
2021-07-06 09:02:31 [INFO] Cleaning up test FiPs reservations
2021-07-06 09:02:33 [INFO] Cleaning up test shares
2021-07-06 09:02:36 [INFO] Cleaning up test shares servers (if found)
2021-07-06 09:02:37 [INFO] ======================================================================
2021-07-06 09:02:37 [INFO] ERROR: test_manila_share (zaza.openstack.charm_tests.manila_ganesha.tests.ManilaGaneshaTests)
2021-07-06 09:02:37 [INFO] Test that a Manila share can be accessed on two instances.
2021-07-06 09:02:37 [INFO] ----------------------------------------------------------------------
2021-07-06 09:02:37 [INFO] Traceback (most recent call last):
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/zaza/openstack/charm_tests/manila/tests.py", line 338, in test_manila_share
2021-07-06 09:02:37 [INFO] share.allow(access_type='ip', access=fip_2, access_level='rw')
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/v2/shares.py", line 82, in allow
2021-07-06 09:02:37 [INFO] self, access_type, access, access_level, metadata)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/api_versions.py", line 399, in substitution
2021-07-06 09:02:37 [INFO] return method.func(obj, *args, **kwargs)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/v2/shares.py", line 555, in allow
2021-07-06 09:02:37 [INFO] share, access_type, access, access_level, "os-allow_access")
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/v2/shares.py", line 548, in _do_allow
2021-07-06 09:02:37 [INFO] access_params)[1]["access"]
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/v2/shares.py", line 672, in _action
2021-07-06 09:02:37 [INFO] return self.api.client.post(url, body=body)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/common/httpclient.py", line 177, in post
2021-07-06 09:02:37 [INFO] return self._cs_request(url, 'POST', **kwargs)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/common/httpclient.py", line 136, in _cs_request
2021-07-06 09:02:37 [INFO] **kwargs)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/common/httpclient.py", line 150, in _cs_request_with_retries
2021-07-06 09:02:37 [INFO] resp, body = self.request(url, method, **kwargs)
2021-07-06 09:02:37 [INFO] File "/home/ubuntu/src/review.opendev.org/openstack/charm-manila/build/builds/manila/.tox/func-target/lib/python3.6/site-packages/manilaclient/common/httpclient.py", line 128, in request
2021-07-06 09:02:37 [INFO] raise exceptions.from_response(resp, method, url)
2021-07-06 09:02:37 [INFO] manilaclient.common.apiclient.exceptions.BadRequest: Access rules cannot be added while the share or any of its replicas or migration copies has its access_rules_status set to error. Deny any rules in error state and try again. (HTTP 400) (Request-ID: req-b96e0fb5-d430-4be9-af5e-fd3db01fd479)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-08-16: Fix proposed to charm-manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-manila/+/804756

Changed in charm-manila:
status:	New → In Progress

Chris MacNaughton (chris.macnaughton) on 2021-08-16

Changed in charm-manila:
assignee:	nobody → Chris MacNaughton (chris.macnaughton)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-09-12: Change abandoned on charm-manila (master)

Change abandoned by "James Page <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-manila/+/804756
Reason: This review is > 12 weeks without comment, and failed testing the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.