| 2021-12-14 10:31:00 |
Frode Nordahl |
description |
The ovn-central charm is doing DNS lookups during the update-status hook and this crashed the charm. Although it's a reactive charm, it probably shouldn't be doing this much work, and should certainly handle a DNS lookup failure better than just crashing:
2021-12-13 16:42:36 ERROR juju-log Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-ovn-central-0/charm/reactive/layer_openstack.py", line 134, in default_request_certificates
for cn, req in instance.get_certificate_requests().items():
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms_openstack/charm/classes.py", line 290, in get_certificate_requests
return cert_utils.get_certificate_request(
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/openstack/cert_utils.py", line 142, in get_certificate_request
req.add_hostname_cn()
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/openstack/cert_utils.py", line 93, in add_hostname_cn
'cn': get_hostname(ip),
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/network/ip.py", line 523, in get_hostname
result = ns_query(rev)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/network/ip.py", line 479, in ns_query
answers = dns.resolver.query(address, rtype)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/dns/resolver.py", line 1100, in query
return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/dns/resolver.py", line 898, in query
raise NoNameservers(request=request, errors=errors)
dns.resolver.NoNameservers: All nameservers failed to answer the query 158.0.16.172.in-addr.arpa. IN PTR: Server 127.0.0.53 UDP port 53 answered SERVFAIL
This happened on ServerStack which does have a known propensity for DNS lookup failures. |
As exemplified by this trace from the ovn-central charm the layer-openstack code is doing DNS lookups during the update-status hook and this crashed the charm. Although it's a reactive charm, it probably shouldn't be doing this much work, and should certainly handle a DNS lookup failure better than just crashing:
2021-12-13 16:42:36 ERROR juju-log Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-ovn-central-0/charm/reactive/layer_openstack.py", line 134, in default_request_certificates
for cn, req in instance.get_certificate_requests().items():
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charms_openstack/charm/classes.py", line 290, in get_certificate_requests
return cert_utils.get_certificate_request(
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/openstack/cert_utils.py", line 142, in get_certificate_request
req.add_hostname_cn()
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/openstack/cert_utils.py", line 93, in add_hostname_cn
'cn': get_hostname(ip),
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/network/ip.py", line 523, in get_hostname
result = ns_query(rev)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/charmhelpers/contrib/network/ip.py", line 479, in ns_query
answers = dns.resolver.query(address, rtype)
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/dns/resolver.py", line 1100, in query
return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.8/site-packages/dns/resolver.py", line 898, in query
raise NoNameservers(request=request, errors=errors)
dns.resolver.NoNameservers: All nameservers failed to answer the query 158.0.16.172.in-addr.arpa. IN PTR: Server 127.0.0.53 UDP port 53 answered SERVFAIL
This happened on ServerStack which does have a known propensity for DNS lookup failures. |
|
