upgrade of openvswitch packages resets alternative binaries to auto

Bug #1836713 reported by James Page
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
Undecided
Unassigned
Queens
Fix Released
Undecided
James Page
Stein
Fix Released
Undecided
James Page
Train
Fix Released
Undecided
Unassigned
openvswitch (Ubuntu)
Fix Released
High
James Page
Bionic
Fix Released
High
James Page
Disco
Won't Fix
High
Unassigned
Eoan
Fix Released
High
James Page

Bug Description

[Impact]
Package upgrades on installations using the dpdk binary will be automatically switched back to the non-dpdk binary on upgrade.

This will break all configured networking within openvswitch.

[Test Case]
sudo apt install openvswitch-switch-dpdk
sudo update-alternatives --set ovs-vswitchd /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk
sudo update-alternatives --query ovs-vswitchd
Name: ovs-vswitchd
Link: /usr/sbin/ovs-vswitchd
Status: manual
Best: /usr/lib/openvswitch-switch/ovs-vswitchd
Value: /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk

Alternative: /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk
Priority: 50

Alternative: /usr/lib/openvswitch-switch/ovs-vswitchd
Priority: 100

sudo apt install --reinstall openvswitch-switch-dpdk
sudo update-alternatives --query ovs-vswitchd
Name: ovs-vswitchd
Link: /usr/sbin/ovs-vswitchd
Status: auto
Best: /usr/lib/openvswitch-switch/ovs-vswitchd
Value: /usr/lib/openvswitch-switch/ovs-vswitchd

Alternative: /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk
Priority: 50

Alternative: /usr/lib/openvswitch-switch/ovs-vswitchd
Priority: 100

[Regression Potential]
Low - the fix was been in Ubuntu since Eoan and the maintainer script usage of update-alternatives was broken since the -dpdk binary was introducted.

The main challenge is actually upgrading a -dpdk installation without disabling the -dpdk binary with the existing prerm script.

To avoid this:

sudo sed -i "/update-alternatives/d" /var/lib/dpkg/info/openvswitch-switch-dpdk.prerm

before completing the package upgrade thus ensuring the -dpdk version of the binary never gets removed.

[Original Bug Report]
Upgrading and existing openvswitch installation which has been manually configured to use the DPDK alternative binary using:

  sudo update-alternatives --set ovs-vswitchd /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk

results in the ovs-vswitchd being reset back to 'auto':

Setting up openvswitch-switch (2.11.0-0ubuntu2~cloud0) ...
update-alternatives: using /usr/lib/openvswitch-switch/ovs-vswitchd to provide /usr/sbin/ovs-vswitchd (ovs-vswitchd) in auto mode

The prerm maintainer scripts always remove the alternatives, which purges any manual setting done of the binaries.

Revision history for this message
James Page (james-page) wrote :

This is a somewhat tricky issue as the alternative removal is done in the prerm script of the currently installed package version; so we can fix it for future updates, but the act of fixing it will cause any -dpdk based installations to revert back to the non-dpdk version of the binary.

Changed in openvswitch (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in openvswitch (Ubuntu Eoan):
assignee: nobody → James Page (james-page)
Changed in openvswitch (Ubuntu Disco):
importance: Undecided → High
Changed in openvswitch (Ubuntu Bionic):
importance: Undecided → High
Changed in openvswitch (Ubuntu Disco):
status: New → Triaged
Changed in openvswitch (Ubuntu Bionic):
status: New → Triaged
Changed in openvswitch (Ubuntu Eoan):
status: Triaged → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openvswitch - 2.12.0~git20190903.1cdf291dc-0ubuntu1

---------------
openvswitch (2.12.0~git20190903.1cdf291dc-0ubuntu1) eoan; urgency=medium

  * New upstream snapshot from 2.12 branch.
  * d/p/0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch: Drop,
    included upstream.
  * d/control: Bumped Standards-Version to 4.4.0.
  * d/control,rules: Drop Python 2 support.
  * d/rules: Disable testing of DPDK build on arm64 as builders don't
    have the required crc32 CPU feature.
  * d/control: Version BD for libdpdk-dev to ensure buid without overlinking.

 -- James Page <email address hidden> Fri, 06 Sep 2019 09:46:28 +0100

Changed in openvswitch (Ubuntu Eoan):
status: In Progress → Fix Released
Steve Langasek (vorlon)
Changed in openvswitch (Ubuntu Disco):
status: Triaged → Won't Fix
Revision history for this message
James Troup (elmo) wrote :

This broke several customer Bionic DPDK based clouds today.

James Page (james-page)
description: updated
Changed in cloud-archive:
status: New → Fix Released
Changed in openvswitch (Ubuntu Bionic):
assignee: nobody → James Page (james-page)
status: Triaged → In Progress
description: updated
James Page (james-page)
description: updated
Revision history for this message
James Page (james-page) wrote :

Upgrading an install that exhibits this bug is a major trip-hazard, as the act of upgrading the package to the new version will call the broken prerm to execute, resulting in the -dpdk binary being disabled again.

To avoid this:

sudo sed -i "/update-alternatives/d" /var/lib/dpkg/info/openvswitch-switch-dpdk.prerm

before completing the package upgrade thus ensuring the -dpdk version of the binary never gets removed.

James Page (james-page)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello James, or anyone else affected,

Accepted openvswitch into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openvswitch/2.9.8-0ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in openvswitch (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello James, or anyone else affected,

Accepted openvswitch into stein-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:stein-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-stein-needed to verification-stein-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-stein-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-stein-needed
Revision history for this message
James Page (james-page) wrote :

Hello James, or anyone else affected,

Accepted openvswitch into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-queens-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Verified successfully

tags: added: verification-done verification-done-bionic verification-queens-done verification-stein-done
removed: verification-needed verification-needed-bionic verification-queens-needed verification-stein-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Note: I think the testcase has a typo for the SRU, where the first and second 'update-alternatives --query' should both have:

Value: /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openvswitch - 2.9.8-0ubuntu0.18.04.2

---------------
openvswitch (2.9.8-0ubuntu0.18.04.2) bionic-security; urgency=medium

  * SECURITY UPDATE: packet parsing vulnerability
    - debian/patches/CVE-2020-35498.patch: support extra padding length in
      lib/conntrack.c, lib/dp-packet.h, lib/flow.c, tests/classifier.at.
    - CVE-2020-35498

 -- Marc Deslauriers <email address hidden> Thu, 28 Jan 2021 14:49:10 -0500

Changed in openvswitch (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote : Update Released

The verification of the Stable Release Update for openvswitch has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package openvswitch - 2.11.5-0ubuntu0.19.04.1~cloud0
---------------

 openvswitch (2.11.5-0ubuntu0.19.04.1~cloud0) bionic; urgency=medium
 .
   * d/openvswitch-switch{-dpdk}.prerm,postinst: Tidy with case statements,
     only remove alternatives when package is being removed (LP: #1836713).
   * New upstream point release (LP: #1912225) including security fixes for:
   * SECURITY UPDATE: buffer overflow decoding malformed packets in lldp
     - CVE-2015-8011
   * SECURITY UPDATE: Externally triggered memory leak in lldp
     - CVE-2020-27827

Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for openvswitch has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package openvswitch - 2.9.8-0ubuntu0.18.04.1~cloud0
---------------

 openvswitch (2.9.8-0ubuntu0.18.04.1~cloud0) xenial-queens; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 openvswitch (2.9.8-0ubuntu0.18.04.1) bionic; urgency=medium
 .
   * d/openvswitch-switch{-dpdk}.prerm,postinst: Tidy case statements,
     only remove alternatives when package is being removed (LP: #1836713).
   * New upstream stable release (LP: #1912201):
     - debian/patches/CVE-2015-8011.patch,CVE-2020-27827.patch: Dropped,
       included in release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.