Ubuntu
cloud-init package

cloud-init should delete hook file in /etc/NetworkManager/dispatcher.d during upgrade

Bug #2027861 reported by Reinhard Munz on 2023-07-15

This bug affects 2 people

	Status	Importance	Assigned to
cloud-init (Ubuntu)	Fix Released	Undecided	Unassigned
Focal	Fix Released	Undecided	Unassigned
Jammy	Fix Released	Undecided	Unassigned
Lunar	Fix Released	Undecided	Unassigned

Bug Description

During upgrade of cloud-init there are dpkg warnings that '/etc/NetworkManager' may not be deleted. The delete fails due to a hook script file of cloud-init inside a sub-folder, to be exact at '/etc/NetworkManager/dispatcher.d/hook-network-manager'.

Package upgrade should handle deletion of the hook script and thus not fail to delete the directory.

The following is the output from my command line of the upgrade and the following investigation as to which files prevented deletion of the directory and caused the warning.

rm@g0-srv-vie:~$ sudo apt -o APT::Get::Always-Include-Phased-Updates=true upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
Get more security updates through Ubuntu Pro with 'esm-apps' enabled:
exo-utils libexo-2-0 libexo-common
Learn more about Ubuntu Pro at https://ubuntu.com/pro
#
# An OpenSSL vulnerability has recently been fixed with USN-6188-1 & 6119-1:
# CVE-2023-2650: possible DoS translating ASN.1 object identifiers.
# Ensure you have updated the package to its latest version.
#
The following packages will be upgraded:
cloud-init
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 533 kB of archives.
After this operation, 26.6 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://at.archive.ubuntu.com/ubuntu jammy-updates/main amd64 cloud-init all 23.2.1-0ubuntu0~22.04.1 [533 kB]
Fetched 533 kB in 0s (3,138 kB/s)
Preconfiguring packages ...
(Reading database ... 203794 files and directories currently installed.)
Preparing to unpack .../cloud-init_23.2.1-0ubuntu0~22.04.1_all.deb ...
Unpacking cloud-init (23.2.1-0ubuntu0~22.04.1) over (23.1.2-0ubuntu0~22.04.1) ...
dpkg: warning: unable to delete old directory '/etc/NetworkManager/dispatcher.d': Directory not empty
dpkg: warning: unable to delete old directory '/etc/NetworkManager': Directory not empty
Setting up cloud-init (23.2.1-0ubuntu0~22.04.1) ...
Installing new version of config file /etc/cloud/cloud.cfg ...
Installing new version of config file /etc/cloud/templates/chrony.conf.alpine.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.centos.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.cos.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.debian.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.fedora.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.freebsd.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.opensuse-leap.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.opensuse-microos.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.opensuse-tumbleweed.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.opensuse.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.photon.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.rhel.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.sle-micro.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.sle_hpc.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.sles.tmpl ...
Installing new version of config file /etc/cloud/templates/chrony.conf.ubuntu.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.alpine.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.debian.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.fedora.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.freebsd.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.opensuse.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.photon.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.rhel.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.sles.tmpl ...
Installing new version of config file /etc/cloud/templates/ntp.conf.ubuntu.tmpl ...
Installing new version of config file /etc/cloud/templates/ntpd.conf.openbsd.tmpl ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for rsyslog (8.2112.0-2ubuntu2.2) ...
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
rm@g0-srv-vie:~$ sudo ls -lah /etc/NetworkManager
total 20K
drwxr-xr-x 3 root root 4.0K Aug 9 2022 .
drwxr-xr-x 146 root root 12K Jul 15 19:24 ..
drwxr-xr-x 2 root root 4.0K Apr 28 06:44 dispatcher.d
rm@g0-srv-vie:~$ sudo ls -lah /etc/NetworkManager/dispatcher.d
total 12K
drwxr-xr-x 2 root root 4.0K Apr 28 06:44 .
drwxr-xr-x 3 root root 4.0K Aug 9 2022 ..
-rwxr-xr-x 1 root root 719 May 18 2022 hook-network-manager
rm@g0-srv-vie:~$ sudo cat /etc/NetworkManager/dispatcher.d/hook-network-manager
#!/bin/sh
# This file is part of cloud-init. See LICENSE file for license information.

# This script hooks into NetworkManager(8) via its scripts
# arguments are 'interface-name' and 'action'
#
is_azure() {
    local dmi_path="/sys/class/dmi/id/board_vendor" vendor=""
    if [ -e "$dmi_path" ] && read vendor < "$dmi_path"; then
        [ "$vendor" = "Microsoft Corporation" ] && return 0
    fi
    return 1
}

is_enabled() {
    # only execute hooks if cloud-init is enabled and on azure
    [ -e /run/cloud-init/enabled ] || return 1
    is_azure
}

if is_enabled; then
    case "$1:$2" in
        *:up) exec cloud-init dhclient-hook up "$1";;
        *:down) exec cloud-init dhclient-hook down "$1";;
    esac
fi

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: cloud-init 23.2.1-0ubuntu0~22.04.1
ProcVersionSignature: Ubuntu 5.15.0-76.83-generic 5.15.99
Uname: Linux 5.15.0-76-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: pass
Date: Sat Jul 15 19:32:09 2023
InstallationDate: Installed on 2022-11-24 (233 days ago)
InstallationMedia: Ubuntu-Server 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809)
PackageArchitecture: all
ProcEnviron:
LANG=en_US.UTF-8
TERM=xterm-256color
PATH=(custom, no user)
SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.NetworkManager.dispatcher.d.hook-network-manager: [deleted]

See original description

Tags:

Revision history for this message

Reinhard Munz (rmgradient0) wrote on 2023-07-15:

AutoInstallUserData.txt Edit (3.6 KiB, text/plain; charset="utf-8")
CurtinConfigCurtHooks.txt Edit (4.5 KiB, text/plain; charset="utf-8")
CurtinConfigExtract.txt Edit (4.5 KiB, text/plain; charset="utf-8")
CurtinConfigInitial.txt Edit (339 bytes, text/plain; charset="utf-8")
CurtinConfigPartitioning.txt Edit (4.5 KiB, text/plain; charset="utf-8")
CurtinLog.txt Edit (141.8 KiB, text/plain; charset="utf-8")
Dependencies.txt Edit (7.0 KiB, text/plain; charset="utf-8")
InstallerCloudCfg.txt Edit (542 bytes, text/plain; charset="utf-8")
ProbeData.txt Edit (49.1 KiB, text/plain; charset="utf-8")
ProcCpuinfoMinimal.txt Edit (1.7 KiB, text/plain; charset="utf-8")
SubiquityClientDebug.txt Edit (269.6 KiB, text/plain; charset="utf-8")
SubiquityServerDebug.txt Edit (218.7 KiB, text/plain; charset="utf-8")
cloud-init-log-warnings.txt Edit (3.7 KiB, text/plain; charset="utf-8")
cloud-init-output.log.txt.txt Edit (137.6 KiB, text/plain; charset="utf-8")
logs.tgz.gz Edit (321.1 KiB, application/x-gzip)
lshw.txt.txt Edit (47.5 KiB, text/plain; charset="utf-8")
user_data.txt.txt Edit (358 bytes, text/plain; charset="utf-8")

Revision history for this message

Launchpad Janitor (janitor) wrote on 2023-07-18:

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in cloud-init (Ubuntu):
status:	New → Confirmed

Revision history for this message

James Falcon (falcojr) wrote on 2023-07-31:

See https://github.com/canonical/cloud-init/pull/4264

Changed in cloud-init (Ubuntu):
status:	Confirmed → Fix Committed

Andreas Hasenack (ahasenack) on 2023-07-31

description:

updated

Revision history for this message

Launchpad Janitor (janitor) wrote on 2023-08-01:

This bug was fixed in the package cloud-init - 23.3~4ga31370af-0ubuntu1

---------------
cloud-init (23.3~4ga31370af-0ubuntu1) mantic; urgency=medium

  * Upstream snapshot based on upstream/main at a31370af.
    - Bugs fixed in this snapshot: (LP: #1923363, #2028562, #2028784)
  * d/cloud-init.maintscript: Remove the unused hook-network-manager
    conffile. (LP: #2027861)
  * d/control: Add python3-passlib as needed for testing

-- James Falcon <email address hidden> Mon, 31 Jul 2023 11:38:42 -0500

Changed in cloud-init (Ubuntu):
status:	Fix Committed → Fix Released

Revision history for this message

Tired Sysadmin (wearyofallthiscrap) wrote on 2023-08-08:

So perhaps I'm missing something, but that 23.3 package doesn't seem to be showing up on Jammy Jellyfish; an upgrade today got the same 23.2.1-0ubuntu0~22.04.1 against which this bug was originally filed.

Do the fixes normally take this long to propagate? (The breaking change was important enough to go to an LTS release, so presumably the fixing change qualifies also.)

Revision history for this message

James Falcon (falcojr) wrote on 2023-08-14:

23.3 is currently still in development. This fix has only made it to the development release of Ubuntu so far (Mantic). 23.3 should be released upstream soon (likely sometime this week), and then released to the currently supported releases of Ubuntu over the week or two after that.

Revision history for this message

Tired Sysadmin (wearyofallthiscrap) wrote on 2023-09-14:

Status a month later:

Reading the pull link in comment #3 and the conversation linked to from that in turn: https://github.com/canonical/cloud-init/pull/4298 It looks like this fix got reverted because of personnel policy (user committing the fix wasn't allowed to make packaging changes at the time, or similar). So even though 23.3 finally got released two weeks ago, this fix isn't listed in the release notes for https://github.com/canonical/cloud-init/releases/tag/23.3 and doesn't seem to have made it in.

So, other sysadmins: for now you'll still need to clean up the directories manually to avoid the otherwise harmless warnings from dpkg. This fix will be committed eventually once Canonical's paperwork catches up.

Revision history for this message

James Falcon (falcojr) wrote on 2023-09-14:

Tired Sysadmin, what you say isn't true. That PR you reference was an attempt for me to get an upload to happen a day or two earlier, but I realized that it wouldn't work.

The change isn't in upstream because it doesn't affect upstream. It is an Ubuntu-packaging specific issue, so it only exists within Ubuntu packaging. It will be included as part of the Ubuntu packaging when 23.3 is released to our supported series. 23.3 hasn't made it to supported series as that requires a separate process. See https://wiki.ubuntu.com/StableReleaseUpdates and https://wiki.ubuntu.com/CloudinitUpdates for details. The 23.3 release with the packaging change you're looking for has been uploaded as is currently waiting on review to be accepted into the proposed pocket. Once it is accepted, it will be available for testing. Once testing and requisite soak time has completed, it will be released to -updates accordingly.

Revision history for this message

Tired Sysadmin (wearyofallthiscrap) wrote on 2023-09-14:

I misunderstood what you meant in comment #6. It sounded to me like it was going into upstream. My bad.

I've seen the StableReleaseUpdates page, but the simple fact that little bugs like this one and 1803601, to pick examples at random, get introduced into LTS releases has my managers questioning the effectiveness of that policy in actual practice. :-) Obviously they're not major breakage, but when the log analyzer is flagging error messages from a "safe" upgrade to a major package, the chiefs start asking why the Ubuntu maintainers aren't seeing those errors during testing. I keep explaining how minor the warnings are, but bugs in LTS certainly don't make my job day easier, heheh, and how long it takes to return to normal-LTS-smoothness is frustrating.

Revision history for this message

Steve Langasek (vorlon) wrote on 2023-09-15: Please test proposed package

#10

Hello Reinhard, or anyone else affected,

Accepted cloud-init into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.3.1-0ubuntu1~23.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-init (Ubuntu Lunar):
status:	New → Fix Committed
tags:	added: verification-needed verification-needed-lunar

Revision history for this message

Steve Langasek (vorlon) wrote on 2023-09-15:

#11

Hello Reinhard, or anyone else affected,

Accepted cloud-init into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.3.1-0ubuntu1~22.04.1 in a few hours, and then in the -proposed repository.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-init (Ubuntu Jammy):
status:	New → Fix Committed
tags:	added: verification-needed-jammy
Changed in cloud-init (Ubuntu Focal):
status:	New → Fix Committed
tags:	added: verification-needed-focal

Revision history for this message

Steve Langasek (vorlon) wrote on 2023-09-15:

#12

Hello Reinhard, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.3.1-0ubuntu1~20.04.1 in a few hours, and then in the -proposed repository.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message

James Falcon (falcojr) wrote on 2023-10-04:

#13

focal.txt Edit (19.3 KiB, text/plain)

Revision history for this message

James Falcon (falcojr) wrote on 2023-10-04:

#14

jammy.txt Edit (17.1 KiB, text/plain)

Revision history for this message

James Falcon (falcojr) wrote on 2023-10-04:

#15

lunar.txt Edit (14.1 KiB, text/plain)

Revision history for this message

James Falcon (falcojr) wrote on 2023-10-04:

#16

Verifications for focal, jammy, and lunar have been uploaded.

Warning is still present on upgrade, but file is removed by the end of the process.

tags:	added: verification-done-focal verification-done-jammy verification-done-lunar removed: verification-needed-focal verification-needed-jammy verification-needed-lunar
tags:	added: verification-done removed: verification-needed

Revision history for this message

Launchpad Janitor (janitor) wrote on 2023-10-05:

#17

This bug was fixed in the package cloud-init - 23.3.1-0ubuntu1~23.04.1

---------------
cloud-init (23.3.1-0ubuntu1~23.04.1) lunar; urgency=medium

  * Upstream snapshot based on upstream/main at ee9078a7.
  * d/cloud-init.maintscript: Remove the unused hook-network-manager
    conffile. (LP: #2027861)
  * d/patches/retain-old-groups.patch:
    - Retain original groups in cloud.cfg.tmpl
  * d/control: Add python3-passlib as needed for testing
  * d/cloud-init.templates: enable Akamai by default.
    Add Akamai to the default templates to allow datasource discovery.
  * d/po/templates.pot: refresh with debconf-updatepo
  * Upstream snapshot based on 23.3.1. (LP: #2033310).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.3.1/ChangeLog

-- James Falcon <email address hidden> Wed, 06 Sep 2023 11:28:36 -0500

Changed in cloud-init (Ubuntu Lunar):
status:	Fix Committed → Fix Released

Revision history for this message

Andreas Hasenack (ahasenack) wrote on 2023-10-05: Update Released

#18

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message

Launchpad Janitor (janitor) wrote on 2023-10-05:

#19

This bug was fixed in the package cloud-init - 23.3.1-0ubuntu1~22.04.1

---------------
cloud-init (23.3.1-0ubuntu1~22.04.1) jammy; urgency=medium

  * d/cloud-init.maintscript: Remove the unused hook-network-manager
    conffile. (LP: #2027861)
  * d/patches/retain-old-groups.patch:
    - Retain original groups in cloud.cfg.tmpl
  * Upstream snapshot based on upstream/main at 5509b790.
  * d/control: Add python3-passlib as needed for testing
  * d/cloud-init.templates: enable Akamai by default.
    Add Akamai to the default templates to allow datasource discovery.
  * d/po/templates.pot: refresh with debconf-updatepo
  * Upstream snapshot based on 23.3.1. (LP: #2033310).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.3.1/ChangeLog

-- James Falcon <email address hidden> Wed, 06 Sep 2023 12:09:54 -0500

Changed in cloud-init (Ubuntu Jammy):
status:	Fix Committed → Fix Released

Revision history for this message

Launchpad Janitor (janitor) wrote on 2023-10-05:

#20

This bug was fixed in the package cloud-init - 23.3.1-0ubuntu1~20.04.1

---------------
cloud-init (23.3.1-0ubuntu1~20.04.1) focal; urgency=medium

-- James Falcon <email address hidden> Wed, 06 Sep 2023 12:11:17 -0500

Changed in cloud-init (Ubuntu Focal):
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntucloud-init package

cloud-init should delete hook file in /etc/NetworkManager/dispatcher.d during upgrade

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntu
cloud-init package