openstack-loadbalancer hook failed: "loadbalancer-relation-changed"

Bug #1991558 reported by Bas de Bruijne
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Loadbalancer Charm
New
Undecided
Unassigned

Bug Description

In testrun https://solutions.qa.canonical.com/testruns/testRun/72b29a95-7cdd-47f6-a028-073989abd4e7 which is openstack yoga on jammy, openstack-loadbalancer fails with the following logs:

```
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1595, in _run
    result = run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-openstack-loadbalancer-1/network-get', 'public', '--format=json')' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 153, in <module>
    main(OpenstackLoadbalancerCharm)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/main.py", line 431, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/main.py", line 142, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/interface_openstack_loadbalancer/loadbalancer.py", line 213, in _on_relation_changed
    self.on.lb_requested.emit()
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 150, in _process_lb_requests
    self._send_loadbalancer_response()
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 115, in _send_loadbalancer_response
    for binding, vips in self._get_space_vip_mapping().items():
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 104, in _get_space_vip_mapping
    for binding_name, subnets in self._get_binding_subnet_map().items():
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 94, in _get_binding_subnet_map
    network = self.model.get_binding(binding_name).network
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 556, in network
    self._network = Network(self._backend.network_get(self.name, self._relation_id))
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1851, in network_get
    return self._run(*cmd, return_output=True, use_json=True)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1597, in _run
    raise ModelError(e.stderr)
ops.model.ModelError: b'ERROR no network config found for binding "public"\n'
```

This could be related to LP: #1991552 which we are seeing a lot on OpenStack on jammy.

Crashdumps and configs can be found here:
https://oil-jenkins.canonical.com/artifacts/72b29a95-7cdd-47f6-a028-073989abd4e7/index.html

tags: added: cdo-qa foundations-engine
Revision history for this message
Bas de Bruijne (basdbruijne) wrote :

I'm also seeing the same symptom but with a different traceback in https://solutions.qa.canonical.com/v2/testruns/161701b7-1956-4bdd-9a38-8930d98a7c76

```
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 153, in <module>
    main(OpenstackLoadbalancerCharm)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/main.py", line 431, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/main.py", line 142, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/interface_openstack_loadbalancer/loadbalancer.py", line 213, in _on_relation_changed
    self.on.lb_requested.emit()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 149, in _process_lb_requests
    self._configure_haproxy()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 146, in _configure_haproxy
    _render_configs()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 778, in wrapped_f
    return restart_on_change_helper(
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 864, in restart_on_change_helper
    _post_restart_on_change_helper(checksums,
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 934, in _post_restart_on_change_helper
    restart_functions[service_name](service_name)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 33, in reload_service
    subprocess.check_call(['systemctl', 'reload', service_name])
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['systemctl', 'reload', 'haproxy']' returned non-zero exit status 1.
```

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.