Add enabled_provider_drivers to juju configuration

Bug #2029857 reported by Alan Baghumian
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Octavia Charm
In Progress
Wishlist
Neil Campbell

Bug Description

Hello There!

The Octavia OVN provider driver is currently immature and not 100% reliable and occasionally causes Octavia API outage with 500 errors:

[Tue Aug 01 00:05:41.506270 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] octavia_lib.api.drivers.exceptions.DriverAgentNotFound: Unable to open the driver agent socket: /var/run/octavia/status.sock
[Tue Aug 01 00:05:41.506300 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460]
[Tue Aug 01 00:05:41.506326 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] During handling of the above exception, another exception occurred:
[Tue Aug 01 00:05:41.506329 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460]
[Tue Aug 01 00:05:41.506353 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] Traceback (most recent call last):
[Tue Aug 01 00:05:41.506403 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] File "/usr/bin/octavia-wsgi", line 52, in <module>
[Tue Aug 01 00:05:41.506408 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] application = setup_app()
[Tue Aug 01 00:05:41.506416 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] File "/usr/lib/python3/dist-packages/octavia/api/app.py", line 56, in setup_app
[Tue Aug 01 00:05:41.506420 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] _init_drivers()
[Tue Aug 01 00:05:41.506444 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] File "/usr/lib/python3/dist-packages/octavia/api/app.py", line 46, in _init_drivers
[Tue Aug 01 00:05:41.506448 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] driver_factory.get_driver(provider)$ juju run -a octavia 'sudo sed -i s/"enabled_provider_drivers = .*"/"enabled_provider_drivers = amphora:The Octavia Amphora driver"/g /etc/octavia/octavia.conf; sudo systemctl restart octavia-*; sudo systemctl status octavia-*'
[Tue Aug 01 00:05:41.506456 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] File "/usr/lib/python3/dist-packages/octavia/api/drivers/driver_factory.py", line 49, in get_driver
[Tue Aug 01 00:05:41.506460 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] raise exceptions.ProviderNotFound(prov=provider)
[Tue Aug 01 00:05:41.506474 2023] [wsgi:error] [pid 2852319:tid 139662793074432] [remote 127.0.0.1:54460] octavia.common.exceptions.ProviderNotFound: Provider 'ovn' was not found.

It would be great if we have this juju configuration option in order to properly disable this provider driver until it is 100% solid.

Currently we can do this using a juju run, but it will not survive a reboot unless the Octavia config file is made read-only.

Thanks much! Please let me know if there are any questions.

(1) https://storyboard.openstack.org/#!/story/2008710

Revision history for this message
Alan Baghumian (alanbach) wrote :

In case anyone needs it:

$ juju run -a octavia 'sudo sed -i s/"enabled_provider_drivers = .*"/"enabled_provider_drivers = amphora:The Octavia Amphora driver"/g /etc/octavia/octavia.conf; sudo systemctl restart octavia-*; sudo systemctl status octavia-*'

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Yes, I think this would be a useful addition.

Essentially, add a 'enabled_drivers' config option, or equivalent, to enable the operator to proactively enable drivers. Unfortunately, there will be an overlap with 'enable_amphora', which whilst we can't remove, we should deprecate.

The default for 'enabled_drivers' could initially be "ovn" to be compatible with the current 'enable_amphora' option's default. Then, if enabled_drivers is set to anything other that "ovn", the 'enable_amphora' would be ignored, and just the enabled_drivers used.

Changed in charm-octavia:
importance: Undecided → Wishlist
status: New → Triaged
tags: added: good-first-bug
Changed in charm-octavia:
assignee: nobody → Neil Campbell (neil-campbell)
Revision history for this message
Felipe Reyes (freyes) wrote : Re: [Bug 2029857] Re: Add enabled_provider_drivers to juju configuration

On Thu, 2023-08-03 at 22:05 +0000, Alan Baghumian wrote:
> In case anyone needs it:
>
> $ juju run -a octavia 'sudo sed -i s/"enabled_provider_drivers =
> .*"/"enabled_provider_drivers = amphora:The Octavia Amphora driver"/g
> /etc/octavia/octavia.conf; sudo systemctl restart octavia-*; sudo
> systemctl status octavia-*'
>
how is this different from setting enable-amphora to True?

https://charmhub.io/octavia/configure#enable-amphora

https://opendev.org/openstack/charm-octavia/src/branch/master/src/templates/xena/octavia.conf#L9

Revision history for this message
Alan Baghumian (alanbach) wrote :

Hi Felipe,

It's basically the opposite! Hypothetically something along the lines of this:

{% if options.enable_amphora -%}
enabled_provider_drivers = amphora:The Octavia Amphora driver,ovn:Octavia OVN driver
{% else if options.disable_ovn -%}
enabled_provider_drivers = amphora:The Octavia Amphora driver
{% else -%}
enabled_provider_drivers = ovn:Octavia OVN driver
{% endif -%}

Revision history for this message
Felipe Reyes (freyes) wrote :

I'm confused now. IIUC you want to have your environment running with the amphora driver instead of ovn, because it's not working as expected, so to get the amphora driver you just need to run "juju config octavia enable-amphora=True", doesn't this give you the results you want?

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

> I'm confused now. IIUC you want to have your environment running with the amphora driver instead of ovn, because it's not working as expected, so to get the amphora driver you just need to run "juju config octavia enable-amphora=True", doesn't this give you the results you want?

No, sadly not; the ovn driver is enabled *as well*, and if it fails then the octavia service fails as well. This request is to allow selecting which drivers, and only which drivers, the operator wants.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-octavia (master)
Changed in charm-octavia:
status: Triaged → In Progress
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

Im as confused as Felipe. If you set enable_amphora=False, disable_ovn=True then you should get
"enabled_provider_drivers = amphora:The Octavia Amphora driver", per Alan's comments on #4. The user experience is terrible but, I think you still can achieve what you want.

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

> Im as confused as Felipe. If you set enable_amphora=False, disable_ovn=True then you should get
"enabled_provider_drivers = amphora:The Octavia Amphora driver", per Alan's comments on #4. The user experience is terrible but, I think you still can achieve what you want.

@sombrafam where is the 'disable_ovn' option? it doesn't appear in the built charm. Also, there's no enable_amphora option, it's "enable-amphora".

The current master branch template for octavia does this from xena (which is the latest template version):

[api_settings]
{% if options.enable_amphora -%}
enabled_provider_drivers = amphora:The Octavia Amphora driver,ovn:Octavia OVN driver
{% else -%}
enabled_provider_drivers = ovn:Octavia OVN driver
{% endif -%}

i.e. there is no way to switch OFF the ovn driver at the moment. This bug is about switching off the ovn driver completely as if it fails the octavia API exe fails.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.