unit test failure with dnspython 2.3.0

Bug #2002950 reported by Thomas Goirand
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Designate
In Progress
High
Unassigned

Bug Description

Hi,

The package maintainer of dnspython in Debian would like that we release Bookworm with dnspython 2.30.0. However, we noticed that Designate 15.0.0 fails its unit tests with that version of dnspython. Below is log of unit test failures.

Please help us fixing these issues and upgrade dnspython.

======================================================================
FAIL: designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_create
designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_create
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/mock.py", line 1379, in patched
    return func(*newargs, **newkeywargs)
  File "/root/designate/designate-15.0.0/designate/tests/unit/agent/test_handler.py", line 112, in test_receive_create
    request = dns.message.from_wire(binascii.a2b_hex(payload))
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1289, in from_wire
    m = reader.read()
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1173, in read
    factory = _message_factory_from_opcode(dns.opcode.from_flags(flags))
  File "/usr/lib/python3/dist-packages/dns/opcode.py", line 70, in from_flags
    return Opcode((flags & 0x7800) >> 11)
  File "/usr/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 14 is not a valid Opcode

======================================================================
FAIL: designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_create_bad_notifier
designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_create_bad_notifier
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/root/designate/designate-15.0.0/designate/tests/unit/agent/test_handler.py", line 133, in test_receive_create_bad_notifier
    request = dns.message.from_wire(binascii.a2b_hex(payload))
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1289, in from_wire
    m = reader.read()
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1173, in read
    factory = _message_factory_from_opcode(dns.opcode.from_flags(flags))
  File "/usr/lib/python3/dist-packages/dns/opcode.py", line 70, in from_flags
    return Opcode((flags & 0x7800) >> 11)
  File "/usr/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 14 is not a valid Opcode

======================================================================
FAIL: designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_delete
designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_delete
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/mock.py", line 1379, in patched
    return func(*newargs, **newkeywargs)
  File "/root/designate/designate-15.0.0/designate/tests/unit/agent/test_handler.py", line 154, in test_receive_delete
    request = dns.message.from_wire(binascii.a2b_hex(payload))
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1289, in from_wire
    m = reader.read()
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1173, in read
    factory = _message_factory_from_opcode(dns.opcode.from_flags(flags))
  File "/usr/lib/python3/dist-packages/dns/opcode.py", line 70, in from_flags
    return Opcode((flags & 0x7800) >> 11)
  File "/usr/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 14 is not a valid Opcode

======================================================================
FAIL: designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_delete_bad_notifier
designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_receive_delete_bad_notifier
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/root/designate/designate-15.0.0/designate/tests/unit/agent/test_handler.py", line 173, in test_receive_delete_bad_notifier
    request = dns.message.from_wire(binascii.a2b_hex(payload))
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1289, in from_wire
    m = reader.read()
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1173, in read
    factory = _message_factory_from_opcode(dns.opcode.from_flags(flags))
  File "/usr/lib/python3/dist-packages/dns/opcode.py", line 70, in from_flags
    return Opcode((flags & 0x7800) >> 11)
  File "/usr/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 14 is not a valid Opcode

======================================================================
FAIL: designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_transfer_source
designate.tests.unit.agent.test_handler.AgentRequestHandlerTest.test_transfer_source
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/mock.py", line 1379, in patched
    return func(*newargs, **newkeywargs)
  File "/root/designate/designate-15.0.0/designate/tests/unit/agent/test_handler.py", line 195, in test_transfer_source
    request = dns.message.from_wire(binascii.a2b_hex(payload))
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1289, in from_wire
    m = reader.read()
  File "/usr/lib/python3/dist-packages/dns/message.py", line 1173, in read
    factory = _message_factory_from_opcode(dns.opcode.from_flags(flags))
  File "/usr/lib/python3/dist-packages/dns/opcode.py", line 70, in from_flags
    return Opcode((flags & 0x7800) >> 11)
  File "/usr/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 14 is not a valid Opcode

summary: - unit test failure with dnspython 2.30.0
+ unit test failure with dnspython 2.3.0
Revision history for this message
Michael Johnson (johnsom) wrote :

I did a deep dive on this last week and commented in the IRC channel with what I found:

The designate "agent" code is using an opcode (14) that is "unassigned" per the RFCs[1] and IANA[2] in the agent communication protocol:

https://github.com/openstack/designate/blob/master/designate/backend/private_codes.py#L26

This is a bad practice as unassigned opcodes could be assigned at any time and likely will conflict with the usage in Designate.

With dnspython 2.3.0rc1, dnspython is now validating the opcodes[3] and unassigned opcodes are not considered valid.

We have identified four possible paths forward:
1. Try to convince dnspython to include "unassigned" opcodes
2. Monkey patch dnspython to include the opcode 14 designate is using (this resolves the issue in testing)
3. Rewrite the agent communication protocol to use the UPDATE protocol properly and switch to opcode 5
4. Deprecate and remove agents

I think option 1 just infects another project with a bad practice, so I am not in favor of this path.
Option 3 will be significant work and I don't think we currently have developer resources available to work on this until at least the Bobcat development cycle, if not later (given other priority issues).

Based on community feedback in the IRC channel, I think I will check the agent code for "last patch contributions" and propose the deprecation of the agent framework on the discuss mailing list. In the mean time, I will propose a patch that monkey patches dnspython to add opcode 14 to buy us time and to complete the deprecation cycle required by stable policy.

[1] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2
[2] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-5
[3] https://github.com/rthalley/dnspython/blob/master/dns/opcode.py#L24

Changed in designate:
status: New → Confirmed
importance: Undecided → High
Changed in designate:
status: Confirmed → In Progress
Revision history for this message
Michael Johnson (johnsom) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to designate (master)

Reviewed: https://review.opendev.org/c/openstack/designate/+/870678
Committed: https://opendev.org/openstack/designate/commit/41bf7dd319966e3fa0182b88b0869ab13b52092e
Submitter: "Zuul (22348)"
Branch: master

commit 41bf7dd319966e3fa0182b88b0869ab13b52092e
Author: Michael Johnson <email address hidden>
Date: Mon Jan 16 22:40:24 2023 +0000

    Workaround issue with dnspython >= 2.3.0

    The dnspython module >= 2.3.0 now validates the opcodes used when building DNS messages. This breaks Designate because designate is using an "unassigned"[1] opcode 14 for command/control messages inside the backend agents framework.

    This patch adds a workaround to override the dnspython opcode enum to include
    opcode 14. This will give us time to either remove the agent framework via deprecation or to change the agent framework protocol to not rely on unassigned opcode values.

    [1] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-5

    Partial-Bug: #2002950

    Change-Id: If10443b2e361aa1b467fb64124ad3c82540bcddd

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.