neutron

With DVR enabled instances sometimes fail to get metadata

  1. Series kilo
  2. Bug #1521524
Bug #1521524 reported by Oleg Bondarev
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Oleg Bondarev
Kilo
New
Undecided
Unassigned

Bug Description

Rally scenario which creates VMs with floating IPs at a high rate sometimes fails with SSHTimeout when trying to connect to the VM by floating IP. At the same time pings to the VM are fine.

It appeared that VMs may sometimes fail to get public key from metadata. That happens because metadata proxy process was started after VM boot.

Further analysis showed that l3 agent on compute node was not notified about new VM port at the time this port was created.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/251718

OpenStack Infra (hudson-openstack)
Changed in neutron:
assignee: Oleg Bondarev (obondarev) → Swaminathan Vasudevan (swaminathan-vasudevan)
OpenStack Infra (hudson-openstack)
Changed in neutron:
assignee: Swaminathan Vasudevan (swaminathan-vasudevan) → Oleg Bondarev (obondarev)
OpenStack Infra (hudson-openstack)
Changed in neutron:
assignee: Oleg Bondarev (obondarev) → Swaminathan Vasudevan (swaminathan-vasudevan)
Revision history for this message
Swaminathan Vasudevan (swaminathan-vasudevan) wrote :

This patch should address this scenario, if the above statement is true.

https://review.openstack.org/#/c/253685/

Revision history for this message
Swaminathan Vasudevan (swaminathan-vasudevan) wrote :

Both the patches, one shown above and the one shown below should address this issue.

https://review.openstack.org/#/c/251718/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/251718
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=87ce73141dddb840ffc523f253fdf87aef11d1c7
Submitter: Jenkins
Branch: master

commit 87ce73141dddb840ffc523f253fdf87aef11d1c7
Author: Oleg Bondarev <email address hidden>
Date: Tue Dec 1 11:47:05 2015 +0300

    Notify about port create/update unconditionally

    The notification about port create/update should be done no matter
    if host (or any other field) was changed or not - it should be up to
    handler to decide how to handle it.

    This also fixes the bug when there was actually no notification to l3
    dvr agent on compute node on new port creation:
    _get_host_port_if_changed() always returned None - this is a regression
    from commit 2ee08c3464c53abaf9bc5493132ad7958611e3b8

    Closes-Bug: #1521524
    Change-Id: I5bb416d2aaab632526f06a5620c0a4ea96340d21

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/255261

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/liberty)

Reviewed: https://review.openstack.org/255261
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7fc31ffd28506fda98b5b8c2c78465e79d5c884a
Submitter: Jenkins
Branch: stable/liberty

commit 7fc31ffd28506fda98b5b8c2c78465e79d5c884a
Author: Oleg Bondarev <email address hidden>
Date: Tue Dec 1 11:47:05 2015 +0300

    Notify about port create/update unconditionally

    The notification about port create/update should be done no matter
    if host (or any other field) was changed or not - it should be up to
    handler to decide how to handle it.

    This also fixes the bug when there was actually no notification to l3
    dvr agent on compute node on new port creation:
    _get_host_port_if_changed() always returned None - this is a regression
    from commit 2ee08c3464c53abaf9bc5493132ad7958611e3b8

    Closes-Bug: #1521524
    Change-Id: I5bb416d2aaab632526f06a5620c0a4ea96340d21
    (cherry picked from commit 87ce73141dddb840ffc523f253fdf87aef11d1c7)

tags: added: in-stable-liberty
Oleg Bondarev (obondarev)
Changed in neutron:
assignee: Swaminathan Vasudevan (swaminathan-vasudevan) → Oleg Bondarev (obondarev)
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/neutron 8.0.0.0b2

This issue was fixed in the openstack/neutron 8.0.0.0b2 development milestone.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 7.0.2

This issue was fixed in the openstack/neutron 7.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/273274

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/kilo)

Reviewed: https://review.openstack.org/273274
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4581c48599353ccaa4dc6b5ffcd0b9158872cbba
Submitter: Jenkins
Branch: stable/kilo

commit 4581c48599353ccaa4dc6b5ffcd0b9158872cbba
Author: Oleg Bondarev <email address hidden>
Date: Tue Dec 1 11:47:05 2015 +0300

    Notify about port create/update unconditionally

    The notification about port create/update should be done no matter
    if host (or any other field) was changed or not - it should be up to
    handler to decide how to handle it.

    This also fixes the bug when there was actually no notification to l3
    dvr agent on compute node on new port creation:
    _get_host_port_if_changed() always returned None - this is a regression
    from commit 2ee08c3464c53abaf9bc5493132ad7958611e3b8

    Conflicts:
     neutron/plugins/ml2/plugin.py
     neutron/tests/unit/plugins/ml2/test_plugin.py

    Closes-Bug: #1521524
    Change-Id: I5bb416d2aaab632526f06a5620c0a4ea96340d21
    (cherry picked from commit 87ce73141dddb840ffc523f253fdf87aef11d1c7)

tags: added: in-stable-kilo
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 2015.1.4

This issue was fixed in the openstack/neutron 2015.1.4 release.

Ihar Hrachyshka (ihar-hrachyshka)
tags: removed: liberty-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

This issue was fixed in the openstack/neutron 2015.1.4 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.