Comment 6 for bug 1945662

Looking at mysql I do not see the project reference going stale it seems to delete immediately after project is deleted.

mysql cluster has three units all show old project before deletion

mysql> select * from project where name='project_PROD-LSEC-bugtest';

| id | name | extra | description | enabled | domain_id | parent_id | is_domain |
+----------------------------------+---------------------------+-------+-------------+---------+----------------------------------+----------------------------------+-----------+
| c4dfcd7702ad46d9a99c1be182e94157 | project_PROD-LSEC-bugtest | {} | | 1 | 4fa535a86d474ef98f65881501ddc1da | 4fa535a86d474ef98f65881501ddc1da | 0 |

openstack project delete c4dfcd7702ad46d9a99c1be182e94157

All three units are cleared

mysql> select * from project where name='project_PROD-LSEC-bugtest';
Empty set (0.00 sec)

However we get a failure with reference to the deleted project.

Using parameters {'auth_url': 'https://keystone.xxx:5000/v3', 'project_name': 'project_LSEC-bugtest', 'project_domain_id': '4fa535a86d474ef98f65881501ddc1da', 'username': 'project_member_LSEC-bugtest', 'user_domain_id': '4fa535a86d474ef98f65881501ddc1da', 'password': '***'}
Get auth_ref
REQ: curl -g -i --cacert "/home/ubuntu/xx/temp.cert" -X GET https://keystone.xxx:5000/v3 -H "Accept: application/json" -H "User-Agent: openstacksdk/0.55.0 keystoneauth1/4.3.1 python-requests/2.25.1 CPython/3.6.9"
Starting new HTTPS connection (1): keystone.xxx:5000
https://keystone.xxx:5000 "GET /v3 HTTP/1.1" 200 271
RESP: [200] Connection: Keep-Alive Content-Length: 271 Content-Type: application/json Date: Thu, 30 Sep 2021 18:43:52 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.41 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-341dec89-10e3-4fd3-b9dd-b4bc495334d0
RESP BODY: {"version": {"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "https://keystone.xxx:5000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}}
GET call to https://keystone.xxx:5000/v3 used request id req-341dec89-10e3-4fd3-b9dd-b4bc495334d0
Making authentication request to https://keystone.xxx:5000/v3/auth/tokens
https://keystone.xxx:5000 "POST /v3/auth/tokens HTTP/1.1" 404 113
Request returned failure status: 404
Could not find project: c4dfcd7702ad46d9a99c1be182e94157. (HTTP 404) (Request-ID: req-ef3134d3-36d5-4e41-bb8a-497e85e45334)
clean_up CreateFloatingIP: Could not find project: c4dfcd7702ad46d9a99c1be182e94157. (HTTP 404) (Request-ID: req-ef3134d3-36d5-4e41-bb8a-497e85e45334)
END return value: 1

My guess is that it is token related