OpenStack Snap

Bootstrap doesn't complete due to juju/traefik errors

Bug #2037631 reported by Matt Verran
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Snap
New
Undecided
Unassigned

Bug Description

During bootstrap, it hangs on 28/29 services, querying the logs for traefik-0 shows many of these errors preventing juju configuring.

2023-09-28T08:55:46.880Z [container-agent] 2023-09-28 08:55:46 INFO juju-log ingress:60: HTTP Request: GET https://10.152.183.1/api/v1/namespaces/openstack/services/traefik "HTTP/1.1 200 OK"
2023-09-28T08:55:46.950Z [container-agent] 2023-09-28 08:55:46 ERROR juju-log ingress:60: invalid databag contents: expecting json. {'host': 'cinder-0.cinder-endpoints.openstack.svc.cluster.local', 'model': 'openstack', 'name': 'cinder', 'port': '8776'}
2023-09-28T08:55:46.980Z [container-agent] 2023-09-28 08:55:46 WARNING juju-log ingress:60: <ops.model.Relation ingress:66> is using a deprecated ingress v1 protocol to talk to Traefik. Please inform the maintainers of 'cinder' that they should bump to v2.
2023-09-28T08:55:46.991Z [container-agent] 2023-09-28 08:55:46 WARNING juju-log ingress:60: provider: <charms.traefik_k8s.v1.ingress.IngressPerAppProvider object at 0x7f1c56632df0>
2023-09-28T08:55:47.057Z [container-agent] 2023-09-28 08:55:47 WARNING juju-log ingress:60: providing ingress over ingress v1: handling it as ingress per leader (legacy)

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

snap list openstack microceph microk8s juju
Name Version Rev Tracking Publisher Notes
juju 3.2.3 24162 3.2/stable canonical✓ -
microceph 0+git.c07ce73 585 quincy/stable canonical✓ -
microk8s v1.26.8 5883 1.26-strict/stable canonical✓ -
openstack 2023.1 274 2023.1/stable canonical✓ -

Revision history for this message
Matt Verran (mv-2112) wrote :

Think i raised this in the wrong place. As it stands this seems to be preventing me deploying a working setup.

affects: snap.openstack → snap-openstack
Revision history for this message
Matt Verran (mv-2112) wrote :

It appears you already have the start of a fix - for example:-

https://opendev.org/openstack/charm-placement-k8s/commit/d5ff0ed6a141647e566410118ef51b61ed2231ac

How do we pick up newer charms when we see these kinds of issues?

Revision history for this message
James Page (james-page) wrote :

Hi Matt - you're seeing two different issues here

As you correctly identified we do have some work in the pipeline to use the new interface definition for Traefik which fixes an issue with full distribution of load to the backend units of a service.

There has also been a regression in the performance of the hooks in the traefik charm which means it takes a long time to complete processing of all hook events.

The community around this charm are aware of the issue and some improvements have landed into their development branch.

Revision history for this message
James Page (james-page) wrote :

That said it should still complete bootstrap - just takes a while (which is not great).

Revision history for this message
Matt Verran (mv-2112) wrote :

Ok, so probably the correct definition of this bug would be that the Traefik charm regression means installation times out here before completing. I'm going to investigate if simply re-running bootstrap again is a workaround/mitigation for this until the fixes land.

Revision history for this message
Matt Verran (mv-2112) wrote :

Can confirm re-running works as mitigation in the meantime.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.