Upgrade to Focal now removes chrony

If I add the following pinning chrony is kept:

# cat /etc/apt/preferences.d/chrony-pin
Package: systemd-timesyncd
Pin: release n=focal*
Pin-Priority: -1

Ntp packages is affected the same way.

Systemd's dependencies (systemd-timesyncd | time-daemon) allows keeping chrony, ntp, etc., but APT chooses systemd-timesyncd. Maybe we need a quirk or an adjustment of a quirk in u-r-u.

Failed to reproduce in a chroot with `apt full-upgrade`.
Failed to reproduce in a lxd container with `apt full-upgrade`.
Can reproduce in a lxd container with do-release-upgrade -d

```
Calculating the changes
  MarkInstall systemd-timesyncd:amd64 < none -> 245.4-4ubuntu1 @un uN Ib > FU=1
   Removing: chrony
    MarkDelete chrony:amd64 < 3.2-4ubuntu4.2 -> 3.5-6ubuntu3 @ii umU Ib > FU=0

Do you want to start the upgrade?
```

I think there must be a bug in how do-release-upgrade calculates the upgrade, and it doesn't prefer/resolve existing time-daemon virtual package, and instead picks the first real one.

@xnox - yes I also only saw it happening in do-release-upgrade.
I was even going for a VM instead of a container knowing that some of the time services will auto-disable themselves when running in a container.

Marking the apt task as invalid as per xnox's analysis

The reason systemd-timesyncd is forcibly installed on upgrade is that it is in the minimal task:

$ apt-cache show systemd-timesyncd | grep Task
Task: minimal, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-minimal-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-desktop-default-languages, ubuntu-core

I see that

Installed tasks are: {'standard', 'server', 'openssh-server', 'minimal', 'cloud-image'}

and then we try to force mark each package in those to be installed/upgraded. However standard & minimal, are already handled via BaseMetaPkgs, and should not be "force installed".

When I ignore "standard" and "minimal" tasks from "installedTasks" I get better results:

Installed tasks are: {'openssh-server', 'cloud-image', 'server'}

53 new packages are going to be installed. 419 packages are going to
be upgraded.

And chrony remains installed.

IMO the real root cause, is tasks including packages transitively (why is that?):
https://people.canonical.com/~ubuntu-archive/germinate-output/ubuntu.focal/minimal

Starting with groovy I would prefer fixing that and landing a minimal change to fix focal upgrades.

This bug was fixed in the package ubuntu-release-upgrader - 1:20.10.1

---------------
ubuntu-release-upgrader (1:20.10.1) groovy; urgency=medium

  [ Dimitri John Ledkov ]
  * DistUpgradeCache: Ignore BaseMetaPkgs' & obsolete tasks during server upgrade
    (LP: #1872902)

  [ Brian Murray ]
  * Update for 20.04 -> 20.10.

 -- Brian Murray <email address hidden> Tue, 05 May 2020 14:37:21 -0700

Hello Christian, or anyone else affected,

Accepted ubuntu-release-upgrader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:20.04.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted ubuntu-release-upgrader (1:20.04.19) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10 (armhf)
update-notifier/3.192.30 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-release-upgrader

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

I tested upgrading an lxc container with chrony installed by downloading the tarball from http://archive.ubuntu.com/ubuntu/dists/focal-proposed/main/dist-upgrader-all/current/ as the meta-release files have not been updated for Focal. During the upgrade process chrony was kept and systemd-timesyncd was not installed. Additionally, I ran 'tasksel install standard' and 'tasksel install minimal' to ensure that no packages in either task were removed and none were. (Although doing that installed systemd-timesyncd.)

root@welcomed-cow:/tmp/u-r-u# head /var/log/dist-upgrade/main.log
2020-05-06 17:51:12,543 INFO Using config files '['./DistUpgrade.cfg']'
2020-05-06 17:51:12,543 INFO uname information: 'Linux welcomed-cow 5.4.0-26-generic #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020 x86_64'
2020-05-06 17:51:12,750 INFO apt version: '1.9.4'
2020-05-06 17:51:12,750 INFO python version: '3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008]'
2020-05-06 17:51:12,753 INFO release-upgrader version '20.04.19' started

I also tested upgrading an 19.10 Ubuntu desktop install with chrony installed to 20.04 LTS and chrony remained installed.

bdmurray@clean-eoan-amd64:/tmp/u-r-u-proposed$ apt-cache policy chrony
chrony:
  Installed: 3.5-6ubuntu6
  Candidate: 3.5-6ubuntu6
  Version table:
 *** 3.5-6ubuntu6 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
bdmurray@clean-eoan-amd64:/tmp/u-r-u-proposed$ apt-cache policy systemd-timesyncd
systemd-timesyncd:
  Installed: (none)
  Candidate: 245.4-4ubuntu3
  Version table:
     245.4-4ubuntu3 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
bdmurray@clean-eoan-amd64:/tmp/u-r-u-proposed$ head /var/log/dist-upgrade/main.log
2020-05-06 11:12:55,719 INFO Using config files '['./DistUpgrade.cfg']'
2020-05-06 11:12:55,719 INFO uname information: 'Linux clean-eoan-amd64 5.3.0-51-generic #44-Ubuntu SMP Wed Apr 22 21:09:44 UTC 2020 x86_64'
2020-05-06 11:12:56,050 INFO apt version: '1.9.4'
2020-05-06 11:12:56,051 INFO python version: '3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008]'
2020-05-06 11:12:56,053 INFO release-upgrader version '20.04.19' started
2020-05-06 11:12:56,056 INFO locale: 'en_US' 'UTF-8'
2020-05-06 11:12:56,118 DEBUG Using 'DistUpgradeViewText' view
2020-05-06 11:12:56,151 DEBUG enable dpkg --force-overwrite
2020-05-06 11:12:56,182 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'

Testing in an Ubuntu 18.04 LTS lxc container.

root@bionic:/tmp/u-r-u-proposed# apt-cache policy chrony
chrony:
  Installed: 3.5-6ubuntu6
  Candidate: 3.5-6ubuntu6
  Version table:
 *** 3.5-6ubuntu6 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
root@bionic:/tmp/u-r-u-proposed# apt-cache policy systemd-timesyncd
systemd-timesyncd:
  Installed: (none)
  Candidate: 245.4-4ubuntu3
  Version table:
     245.4-4ubuntu3 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
root@bionic:/tmp/u-r-u-proposed# head /var/log/dist-upgrade/main.log
2020-05-06 18:53:45,805 INFO Using config files '['./DistUpgrade.cfg.bionic']'
2020-05-06 18:53:45,805 INFO uname information: 'Linux bionic 5.4.0-26-generic #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020 x86_64'
2020-05-06 18:53:46,054 INFO apt version: '1.6.12'
2020-05-06 18:53:46,054 INFO python version: '3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]'
2020-05-06 18:53:46,058 INFO release-upgrader version '20.04.19' started
2020-05-06 18:53:46,063 WARNING can't import view 'DistUpgradeViewGtk3' (Namespace Vte not available)
2020-05-06 18:53:46,104 WARNING can't import view 'DistUpgradeViewKDE' (No module named 'PyQt4')

Additionally, I ran 'tasksel install standard' and 'tasksel install minimal' to ensure that no packages in either task were removed and none were. (Although doing that installed systemd-timesyncd.)

Testing on an Ubuntu 18.04 LTS system with ubuntu-desktop and chrony installed.

bdmurray@clean-bionic-amd64:/tmp/u-r-u-proposed$ apt-cache policy chrony
chrony:
  Installed: 3.5-6ubuntu6
  Candidate: 3.5-6ubuntu6
  Version table:
 *** 3.5-6ubuntu6 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
bdmurray@clean-bionic-amd64:/tmp/u-r-u-proposed$ apt-cache policy systemd-timesyncd
systemd-timesyncd:
  Installed: (none)
  Candidate: 245.4-4ubuntu3
  Version table:
     245.4-4ubuntu3 500
        500 http://192.168.10.7/ubuntu focal/main amd64 Packages
bdmurray@clean-bionic-amd64:/tmp/u-r-u-proposed$ head /var/log/dist-upgrade/main.log
2020-05-06 12:26:14,373 INFO Using config files '['./DistUpgrade.cfg.bionic']'
2020-05-06 12:26:14,373 INFO uname information: 'Linux clean-bionic-amd64 4.15.0-99-generic #100-Ubuntu SMP Wed Apr 22 20:32:56 UTC 2020 x86_64'
2020-05-06 12:26:14,733 INFO apt version: '1.6.12'
2020-05-06 12:26:14,733 INFO python version: '3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]'
2020-05-06 12:26:14,737 INFO release-upgrader version '20.04.19' started
2020-05-06 12:26:14,740 INFO locale: 'en_US' 'UTF-8'
2020-05-06 12:26:14,805 DEBUG Using 'DistUpgradeViewText' view
2020-05-06 12:26:14,843 DEBUG enable dpkg --force-overwrite
2020-05-06 12:26:14,870 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'

I had the same results with tasksel as I did with the lxc container upgrade.

I retested the armhf autopkgtest failures and update-manager and update-notifier have now both passed.

This bug was fixed in the package ubuntu-release-upgrader - 1:20.04.19

---------------
ubuntu-release-upgrader (1:20.04.19) focal; urgency=medium

  [ Dimitri John Ledkov ]
  * DistUpgradeCache: Ignore BaseMetaPkgs' & obsolete tasks during server upgrade
    (LP: #1872902)

  [ Brian Murray ]
  * data/mirrors.cfg: update as some mirrors have been removed as they do not
    mirror Ubuntu 20.04 LTS.
  * data/demoted.bionic.cfg: updated for packages that were removed nearer
    the release date.

 -- Brian Murray <email address hidden> Tue, 05 May 2020 15:26:02 -0700

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.