[1.06-70] SI launch is failing as svc-mon is looking to authenticate user in a stale/non-existent tenant

Bug #1340346 reported by Ganesha HV
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Fix Committed
High
Rudra Rugge
R1.1
Fix Committed
High
Rudra Rugge
R2.20
New
Medium
Ganesha HV

Bug Description

1]. Created a project 'TestECMPSvcChain0' and added a user 'TestECMPSvcChain0' as a admin and also 'admin' as a Member.
2]. Created a service-instance with service-scaling and launched 3 instances in the same.
3]. The service-instance is not launched successfully with the following error :

Traceback (most recent call last):
  File "/opt/contrail/api-venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 1015, in launch_arc
    monitor.process_poll_result(result)
  File "/opt/contrail/api-venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 857, in process_poll_result
    func(idents)
  File "/opt/contrail/api-venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 797, in _addmsg_service_instance_service_template
    self._create_svc_instance_vm(st_obj, si_obj)
  File "/opt/contrail/api-venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 495, in _create_svc_instance_vm
    nics, proj_obj.name, flavor)
  File "/opt/contrail/api-venv/lib/python2.7/site-packages/svc_monitor/svc_monitor.py", line 905, in _create_svc_vm
    flavor = n_client.flavors.find(name=flavor_name)
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 184, in find
    matches = self.findall(**kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 222, in findall
    listing = self.list(**list_kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/flavors.py", line 103, in list
    return self._list("/flavors%s%s" % (detail, query_string), "flavors")
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 61, in _list
    _resp, body = self.api.client.get(url)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 229, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 202, in _cs_request
    self.authenticate()
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 329, in authenticate
    auth_url = self._v2_auth(auth_url)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 411, in _v2_auth
    return self._authenticate(url, body)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 423, in _authenticate
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 195, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 189, in request
    raise exceptions.from_response(resp, body, url, method)
Unauthorized: User 5bb33e7bb42d49f9a68978ed505b6ddd is unauthorized for tenant 84e8958626634a97948fcfa22bdc4401 (HTTP 401)

4]. It is seen that though the user exists, the tenant it is referring to is non-existent :

(api-venv)root@nodec4:/var/log/contrail# keystone user-list
WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored).
+----------------------------------+-------------------+---------+---------------------+
| id | name | enabled | email |
+----------------------------------+-------------------+---------+---------------------+
| f33ae47ca65b4fe18820b539a3413526 | TestECMPSvcChain0 | True | |
| 5bb33e7bb42d49f9a68978ed505b6ddd | admin | True | <email address hidden> |
| 19b66aa3d2514dd78fdf401bae3642ce | cinder | True | <email address hidden> |
| c95afaf99d2f44b1996cb95f334c7024 | demo | True | <email address hidden> |
| 2e2b26c3122245ef86bf3f2115dd260f | glance | True | <email address hidden> |
| 99f152ece64e451c8ddb363331af65c6 | neutron | True | <email address hidden> |
| 5654ddf0744148998c2d9fafece00775 | nova | True | <email address hidden> |
+----------------------------------+-------------------+---------+---------------------+
(api-venv)root@nodec4:/var/log/contrail# keystone tenant-list
WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored).
+----------------------------------+--------------------+---------+
| id | name | enabled |
+----------------------------------+--------------------+---------+
| 979cb99dc0d74fdc81ded28765317dac | TestECMPSvcChain0 | True |
| e66fb526c06c4c9a98b110d9ac206936 | admin | True |
| dc4ce1a074b14f3e831b43ab863cb83a | demo | True |
| 3488872267bf4beab977519f97c64be4 | invisible_to_admin | True |
| 9cab150dec974174a194ad88e562043d | service | True |
+----------------------------------+--------------------+---------+

I have kept the logs at
/home/bhushana/Documents/technical/bugs/<bug-id> on 10.204.216.50

Login with the following credentials:

USN : bhushana
PWD : bhu@123

Changed in juniperopenstack:
milestone: r1.06-fcs → r1.10-fcs
Changed in juniperopenstack:
milestone: r1.10-fcs → none
tags: added: config
tags: added: releasenote
information type: Proprietary → Public
Changed in juniperopenstack:
importance: Critical → High
Rudra Rugge (rrugge)
Changed in juniperopenstack:
status: New → Fix Committed
Revision history for this message
Ganesha HV (ganeshahv) wrote :

Steps to automate:
=================

- Create a Tenant in Keystone.
- Create a VN in this tenant in Contrail
- Create a Service Instance in this tenant, with a least one leg in a network from this Tenant.
- Delete the SI
- Delete the VN
- Delete the Tenant.

- recreate the tenant with the same name
- recreate the VN with the same name
- recreate the same Service Instance

=> the Service Instance remains in "Spawing" state.

tags: added: automation
tags: removed: releasenote
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.