Designate

RecordSetNotFound error on create

Bug #1304152 reported by Sam Morrison
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Designate
Fix Released
Medium
Darshan Sanghani
Designate 2014.2 "juno"

Bug Description

When creating a record we get the below error in the logs, the record creation all works fine, just looks like an unhandled exception

==> /var/log/designate/designate-central.log <==
2014-04-08 13:36:11.493 30080 ERROR designate.openstack.common.rpc.amqp [req-1ce0193f-5b50-4c23-8a14-f04089bc6b2b 89 2f6f7e75fc0f453d9c127b490b02e9e3 - - -] Exception during message handling
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp Traceback (most recent call last):
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/openstack/common/rpc/amqp.py", line 462, in _process_data
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp **args)
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/central/service.py", line 724, in find_recordset
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp return self.storage_api.find_recordset(context, criterion)
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/storage/api.py", line 546, in find_recordset
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp return self.storage.find_recordset(context, criterion)
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/storage/impl_sqlalchemy/__init__.py", line 518, in find_recordset
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp recordset = self._find_recordsets(context, criterion, one=True)
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp File "/opt/designate/designate/storage/impl_sqlalchemy/__init__.py", line 484, in _find_recordsets
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp raise exceptions.RecordSetNotFound()
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp RecordSetNotFound
2014-04-08 13:36:11.493 30080 TRACE designate.openstack.common.rpc.amqp

Revision history for this message
Darshan Sanghani (darshan104) wrote :

This is actually a handled exception which has to pass through the Queue to be handled.

In v1/records.py method _find_or_create_recordset(...): This calls the Central's find_recordset(), in a case where it cannot find the recordset an exception is raised, which is returned to the caller. Which is then handled by calling a create_recordset(). Expected behavior.

Revision history for this message
Sam Morrison (sorrison) wrote :

Hi Darshan,

Please see this from the operators perspective, they see errors in the logs and think something is wrong.

If it's dumping a stack trace it is clearly not a handled exception.

Revision history for this message
Darshan Sanghani (darshan104) wrote :

Fair enough. I would have thought the same. The only way I think this can be achieved is by making Central's find_recordset() aware if it is being called as a routine in the API's find_or_create_recordset() or not. Let me see if I can patch it and submit for review.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to designate (master)

Fix proposed to branch: master
Review: https://review.openstack.org/107166

Changed in designate:
assignee: nobody → Darshan Sanghani (darshan104)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/107195

Changed in designate:
assignee: Darshan Sanghani (darshan104) → Kiall Mac Innes (kiall)
Kiall Mac Innes (kiall)
Changed in designate:
assignee: Kiall Mac Innes (kiall) → Darshan Sanghani (darshan104)
milestone: none → juno-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to designate (master)

Reviewed: https://review.openstack.org/107195
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=2c9460505d07d0e46765a552b637e5a3296b667b
Submitter: Jenkins
Branch: master

commit 2c9460505d07d0e46765a552b637e5a3296b667b
Author: Kiall Mac Innes <email address hidden>
Date: Tue Jul 15 23:26:09 2014 +0100

    Avoid logging "Expected" exceptions returned over RPC

    When an expected exception, such as a NotFound or Duplicate is passed over
    RPC, we identify it via the `expected` attribute and raise oslo.messagings
    ExpectedException. The internals of ExpectedException effectively save the
    current exception details (type, trace, etc) inside the ExpectedException,
    when received on the client side, the ExpectedException is detected, and
    the saved details are used to bubble the original exception back up. On
    both sides of the RPC, logging is avoided when ExpectedException is used.

    Change-Id: I143882181e3346c4ebe1f0ad491c75a4db549fe4
    Closes-Bug: #1304152

Changed in designate:
status: In Progress → Fix Committed
Kiall Mac Innes (kiall)
Changed in designate:
importance: Undecided → Medium
Revision history for this message
Darshan Sanghani (darshan104) wrote :

Cannot reproduce this anymore for any exceptions that are set to expected. We should be good.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on designate (master)

Change abandoned by Darshan Sanghani (<email address hidden>) on branch: master
Review: https://review.openstack.org/107166
Reason: Was fixed by another commit which changed the way RPC server is setup, by overloading the dispatcher to patch the expected exceptions.

Russell Bryant (russellb)
Changed in designate:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in designate:
milestone: juno-2 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.