intermittent AristaProvisionedVlansStorageTestCase failure

Salvatore and I have been looking into this issue. I think we've got an interim fix.

Check if this works for you: http://paste.openstack.org/show/45738/

If yes, push it on gerrit.

Mark,
I know the fix and can push a patch later today. I was getting ready to
open a bug. You beat me to it. Please let me know how to proceed.
Thanks
Sukhdev
 On Sep 4, 2013 9:41 AM, "Mark McClain" <email address hidden> wrote:

> Salvatore and I have been looking into this issue. I think we've got an
> interim fix.
>
> ** Changed in: neutron
> Status: New => Triaged
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1220784
>
> Title:
> intermittent AristaProvisionedVlansStorageTestCase failure
>
> Status in OpenStack Neutron (virtual network service):
> Triaged
>
> Bug description:
> Unit test are sometimes failing with:
>
> 2013-09-04 14:58:53.719 | FAIL:
> neutron.tests.unit.ml2.drivers.test_arista_mechanism_driver.AristaProvisionedVlansStorageTestCase.test_remembers_multiple_networks
> 2013-09-04 14:58:53.719 | tags: worker-2
> 2013-09-04 14:58:53.720 |
> ----------------------------------------------------------------------
> 2013-09-04 14:58:53.720 | Empty attachments:
> 2013-09-04 14:58:53.720 | pythonlogging:''
> 2013-09-04 14:58:53.720 | stderr
> 2013-09-04 14:58:53.720 | stdout
> 2013-09-04 14:58:53.720 |
> 2013-09-04 14:58:53.720 | Traceback (most recent call last):
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/neutron/tests/unit/ml2/drivers/test_arista_mechanism_driver.py",
> line 118, in test_remembers_multiple_networks
> 2013-09-04 14:58:53.721 | (expected_num_nets, num_nets_provisioned))
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
> line 322, in assertEqual
> 2013-09-04 14:58:53.721 | self.assertThat(observed, matcher, message)
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
> line 417, in assertThat
> 2013-09-04 14:58:53.722 | raise MismatchError(matchee, matcher,
> mismatch, verbose)
> 2013-09-04 14:58:53.722 | MismatchError: 100 != 101: There should be 100
> nets, not 101
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/neutron/+bug/1220784/+subscriptions
>

I am going to make tests more defensive and clear everything explicitly
after every test. Just ti maje it bullet proof.
Sukhdev
On Sep 4, 2013 10:25 AM, "Salvatore Orlando" <email address hidden>
wrote:

> Check if this works for you: http://paste.openstack.org/show/45738/
>
> If yes, push it on gerrit.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1220784
>
> Title:
> intermittent AristaProvisionedVlansStorageTestCase failure
>
> Status in OpenStack Neutron (virtual network service):
> Triaged
>
> Bug description:
> Unit test are sometimes failing with:
>
> 2013-09-04 14:58:53.719 | FAIL:
> neutron.tests.unit.ml2.drivers.test_arista_mechanism_driver.AristaProvisionedVlansStorageTestCase.test_remembers_multiple_networks
> 2013-09-04 14:58:53.719 | tags: worker-2
> 2013-09-04 14:58:53.720 |
> ----------------------------------------------------------------------
> 2013-09-04 14:58:53.720 | Empty attachments:
> 2013-09-04 14:58:53.720 | pythonlogging:''
> 2013-09-04 14:58:53.720 | stderr
> 2013-09-04 14:58:53.720 | stdout
> 2013-09-04 14:58:53.720 |
> 2013-09-04 14:58:53.720 | Traceback (most recent call last):
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/neutron/tests/unit/ml2/drivers/test_arista_mechanism_driver.py",
> line 118, in test_remembers_multiple_networks
> 2013-09-04 14:58:53.721 | (expected_num_nets, num_nets_provisioned))
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
> line 322, in assertEqual
> 2013-09-04 14:58:53.721 | self.assertThat(observed, matcher, message)
> 2013-09-04 14:58:53.721 | File
> "/home/jenkins/workspace/gate-neutron-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
> line 417, in assertThat
> 2013-09-04 14:58:53.722 | raise MismatchError(matchee, matcher,
> mismatch, verbose)
> 2013-09-04 14:58:53.722 | MismatchError: 100 != 101: There should be 100
> nets, not 101
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/neutron/+bug/1220784/+subscriptions
>

That would be awesome.
I would not recommend using sqlalchemy magic to clear data from tables, as it might have unexpected side effects on tables with foreign keys.

You can use a context manager to ensure everything a routine is created is then destroyed or do that manually for every test.

Another way would be to run queries in a way that you don't risk loading stuff you don't want to load.

clearing the db is very defensive too anyway. You completely destroy the db at any run, so there's no risk of leftovers.

I have submitted the patch for this.

https://review.openstack.org/45089

Please give it a review.
Thanks
-Sukhdev

Fix proposed to branch: master
Review: https://review.openstack.org/45091

Reviewed: https://review.openstack.org/45091
Committed: http://github.com/openstack/neutron/commit/9b76de161d690faee341a87e8066d40329b4f5b0
Submitter: Jenkins
Branch: master

commit 9b76de161d690faee341a87e8066d40329b4f5b0
Author: Mark McClain <email address hidden>
Date: Wed Sep 4 14:36:11 2013 -0400

    ensure that Arista test destroys the database

    Out of order test execution has revealed that the database was not
    properly cleaned up after each test run. This patch adds a call to
    clear the database after each test.

    Fixes bug: 1220784

    Change-Id: I8a706887f89f21ede88177e441a3b562fd8f74dc