== Env
focal/ussuri + ovn
nova-compute-dpdk 21.2.1 active 4 nova-compute charmstore stable 334 ubuntu Unit is ready
ovn-chassis-dpdk 20.03.2 error 4 ovn-chassis charmstore stable 17 ubuntu hook failed: "ovsdb-relation-departed"
nova-compute/ovn-chassis overlay: https://pastebin.canonical.com/p/2kBXYmdzqt/
== Problem
Unit falls into the error state with the following traceback in the unit logs:
unit-ovn-chassis-dpdk-0: 12:36:58 DEBUG unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: Changing permissions on existing content: 33184 -> 416
unit-ovn-chassis-dpdk-0: 12:36:58 INFO unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: CompletedProcess(args=('ovs-vsctl', '--no-wait', 'set-ssl', '/etc/ovn/key_host', '/etc/ovn/cert_host', '/etc/ovn/ovn-chassis.crt'), returncode=0, stdout='')
unit-ovn-chassis-dpdk-0: 12:36:58 ERROR unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/reactive/ovn_chassis_charm_handlers.py", line 114, in configure_ovs
charm_instance.configure_ovs(
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 142, in configure_ovs
super().configure_ovs(sb_conn, mlockall_changed)
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 747, in configure_ovs
self.run(*cmd)
File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 546, in run
cp = subprocess.run(
File "/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('ovs-vsctl', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-type=geneve', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-ip=10.35.83.3', '--', 'set', 'open-vswitch', '.', 'external-ids:system-id=u0400s1entcomp02.fqdn', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-remote=', '--', 'set', 'open-vswitch', '.', 'external_ids:ovn-match-northd-version=true')' returned non-zero exit status 1.
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed Traceback (most recent call last):
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/hooks/ovsdb-relation-departed", line 22, in <module>
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed main()
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed bus.dispatch(restricted=restricted_mode)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed _invoke(other_handlers)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed handler.invoke()
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed self._action(*args)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/reactive/ovn_chassis_charm_handlers.py", line 114, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed charm_instance.configure_ovs(
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 142, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed super().configure_ovs(sb_conn, mlockall_changed)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 747, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed self.run(*cmd)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 546, in run
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed cp = subprocess.run(
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/usr/lib/python3.8/subprocess.py", line 516, in run
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed raise CalledProcessError(retcode, process.args,
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed subprocess.CalledProcessError: Command '('ovs-vsctl', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-type=geneve', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-ip=10.35.83.3', '--', 'set', 'open-vswitch', '.', 'external-ids:system-id=u0400s1entcomp02.fqdn', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-remote=', '--', 'set', 'open-vswitch', '.', 'external_ids:ovn-match-northd-version=true')' returned non-zero exit status 1.
unit-ovn-chassis-dpdk-0: 12:36:58 ERROR juju.worker.uniter.operation hook "ovsdb-relation-departed" (via explicit, bespoke hook script) failed: exit status 1
unit-ovn-chassis-dpdk-0: 12:36:58 DEBUG juju.machinelock machine lock released for ovn-chassis-dpdk/0 uniter (run relation-departed (205; unit: ovn-central/2, departee: ovn-chassis-dpdk/0) hook)
Trying to run the above command manually:
# ovs-vsctl -- set open-vswitch . external-ids:ovn-encap-type=geneve -- set open-vswitch . external-ids:ovn-encap-ip=10.35.83.3 -- set open-vswitch . external-ids:system-id=u0400s1entcomp02.fqdn -- set open-vswitch . external-ids:ovn-remote= -- set open-vswitch . external_ids:ovn-match-northd-version=true
ovs-vsctl: external-ids:ovn-remote=: argument does not end in "=" followed by a value.
https:/ /github. com/openstack- charmers/ charm-layer- ovn/pull/ 63