mailman3 package missing in jammy, however mailman3 package from Debian 11 works with the other mailman3 packages from Jammy

Bug #1999197 reported by AlainKnaff
64
This bug affects 10 people
Affects Status Importance Assigned to Milestone
mailman-hyperkitty (Ubuntu)
Invalid
Undecided
Unassigned
Jammy
In Progress
Undecided
Unassigned
mailman3 (Ubuntu)
Invalid
Undecided
Unassigned
Jammy
Fix Committed
Undecided
Unassigned

Bug Description

[ Impact ]

There is currently no version of Mailman in Ubuntu 22.04, meaning that people who depend on its functionality have limited options:

i) Stay on an older version of Ubuntu, but miss out on other update packages.
ii) Use a non-LTS Ubuntu release, but this results in a less stable experience and Mailman is principally a server package.
iii) Manually install the Debian package, but this requires manually updating it.

[ Test Plan ]

mailman3 should be available in Jammy with all DEP-8 tests passing.

[ Where problems could occur ]

Several packages are still present in Jammy, such as mailman3-web, despite mailman3 being removed. They will also have to be tested and potentially updated to be compatible with the version backported from Lunar.

[Original description]

1)
root@ubuntu22:~# lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04

2)
root@ubuntu22:~# apt-cache policy mailman3
mailman3:
  Installed: 3.2.2-1
  Candidate: 3.2.2-1
  Version table:
 *** 3.2.2-1 100
        100 /var/lib/dpkg/status

(shows version held from focal, see below)

3) I expect Jammy (22.04.1) to have a mailman3 versions to go along with the supplied mailman3-web package
4) What happened: mailman3 package missing in jammy, and mailman3-web supplied but incompatible with mailman3 version held over from focal.

When upgrading from Ubuntu 20.04.05 (focal) to 22.04.01 (jammy), mailman3 and mailman3-full packages are de-installed during upgrade. Strangely enough mailman3-web still makes it.

Of course, this results in a non-functional mailman3 installation (just a front-end without the backend).

Adding back the focal repositories and manually reinstalling mailman3 and mailman3-full 3.2.2-1 from focal into jammy gets mailman3 mostly up and running again (i.e. correctly distributes mails to lists).

Rest API password (i.e. admin_pass from /etc/mailman3/mailman.cfg needs to be copied to MAILMAN_REST_API_PASS in /etc/mailman3/mailman-web.py).

After doing this, front end *mostly* works.

However, lists cannot be viewed when logged in as admin, instead the following error is displayed:

Something went wrong
HTTP Error 404: {"title": "404 Not Found"}

Event downgrading mailman3-web from 0+20200530-2 to 0+20180916-10 and then rebooting does not help.

When this happens, the following is logged to /var/log/mailman3/web/mailman-web.log:

[pid: 749|app: 0|req: 9/9] 127.0.0.1 () {60 vars in 1359 bytes} [Fri Dec 9 00:21:22 2022] GET /mailman3/postorius/lists/ => generated 8958 bytes in 56 msecs (HTTP/1.1 200) 7 headers in 220 bytes (1 switches on core 0)
ERROR 2022-12-09 00:21:25,565 749 postorius Un-handled exception: HTTP Error 404: {"title": "404 Not Found"}
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/postorius/views/generic.py", line 74, in dispatch
    return super(MailingListView, self).dispatch(request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 301, in get
    primary_email = set_preferred(request.user, mm_user)
  File "/usr/lib/python3/dist-packages/postorius/utils.py", line 77, in set_preferred
    mm_user.preferred_address = primary_email.email
  File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py", line 57, in __setattr__
    super(User, self).__setattr__(name, value)
  File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py", line 131, in __setattr__
    return super(RESTObject, self).__setattr__(name, value)
  File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py", line 145, in preferred_address
    response, content = self._connection.call(url, {'email': email})
  File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 160, in call
    raise HTTPError(params.get('url'), response.status_code,
urllib.error.HTTPError: HTTP Error 404: {"title": "404 Not Found"}
WARNING 2022-12-09 00:21:25,569 749 django.request Not Found: /mailman3/postorius/lists/lilux-info.lilux.lu/
WARNING 2022-12-09 00:21:25,569 749 django.request Not Found: /mailman3/postorius/lists/lilux-info.lilux.lu/
[pid: 749|app: 0|req: 10/10] 127.0.0.1 () {60 vars in 1399 bytes} [Fri Dec 9 00:21:25 2022] GET /mailman3/postorius/lists/lilux-info.lilux.lu/ => generated 6374 bytes in 192 msecs (HTTP/1.1 404) 7 headers in 227 bytes (1 switches on core 1)

Revision history for this message
AlainKnaff (kubuntu-misc) wrote :

Just "holding" the mailman3 and mailman3-full packages (using apt-mark hold) before upgrading from focal to jammy prevents them from being uninstalled. So no need to reinstall them and synchronize the REST password. However, the problem with list administration still happens:

Something went wrong
HTTP Error 404: {"title": "404 Not Found"}

summary: - mailman3 package missing in jammy
+ mailman3 package missing in jammy, and keeping focal version causes HTTP
+ Error 404: {"title": "404 Not Found"}
Revision history for this message
AlainKnaff (kubuntu-misc) wrote : Re: mailman3 package missing in jammy, but mailman3-web supplied but incompatible with mailman3 held over from focal

Holding all packages containing mailman3 and also all packages containing django produces a (mostly) working mailman installation:

dpkg -l | awk '$2 ~ /mailman/ {print $2}' | xargs apt-mark hold
dpkg -l | awk '$2 ~ /django/ {print $2}' | xargs apt-mark hold

[ ... do upgrade ... ]

sed -i 's/from collections import/from collections.abc import/' /usr/lib/python3/dist-packages/mailmanclient/restbase/base.py
service mailman3-web restart

=> so this does seem to prove that the issue is that indeed the mailman3-web included with jammy is incomptible with previous mailman3. So supplying that elusive mailman3 package in jammy is indeed direly needed

description: updated
summary: - mailman3 package missing in jammy, and keeping focal version causes HTTP
- Error 404: {"title": "404 Not Found"}
+ mailman3 package missing in jammy, but mailman3-web supplied but
+ incompatible with mailman3 held over from focal
Revision history for this message
AlainKnaff (kubuntu-misc) wrote :

Package mailman3 from *Debian* 11 works with the other mailman3 packages from Ubuntu Jammy:

wget http://ftp.de.debian.org/debian/pool/main/m/mailman3/mailman3-full_3.3.3-1_all.deb
dpkg -i ~alain/mailman3_3.3.3-1_all.deb
apt install -f

If I had known it was that easy... :-)

summary: - mailman3 package missing in jammy, but mailman3-web supplied but
- incompatible with mailman3 held over from focal
+ mailman3 package missing in jammy, however mailman3 package from Debian
+ 11 works with the other mailman3 packages from Jammy
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mailman3 (Ubuntu):
status: New → Confirmed
Revision history for this message
Claus-Justus Heine (rotatingdrop) wrote :

Sad story. Seems to be this problem:

[Bug 1960547] Re: please remove mailman3, mailman-hyperkitty

And from mailman3 upstream it seems that Python 3.10 is only supported in the (not yet released) 3.3.8 version:

https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.html

Revision history for this message
Dave Moskovitz (dave-thinktank) wrote :

Yup, having this problem too.

Revision history for this message
Jeremy Gregson (jrggv) wrote :

A small correction to the advice about installing the Debian package. Yes, it works, but you need to install version 3.3.8 to fix a problem between Python 3.10 (installed with Ubuntu22) and mailman.
So, for me, the commands were:

wget http://ftp.de.debian.org/debian/pool/main/m/mailman3/mailman3_3.3.8-1_all.deb
dpkg -i ./mailman3_3.3.8-1_all.deb

...which throws an alert about missing dependencies, that you fix with...

apt install -f

information type: Public → Public Security
information type: Public Security → Public
Revision history for this message
Loeka Rogge (exur00) wrote :

[ Impact ]
There is currently no version of Mailman in Ubuntu 22.04, meaning that people who depend on its functionality have limited options:
Stay on an older version of Ubuntu, but miss out on other update packages
Use a non-LTS Ubuntu release, but this results in a less stable experience and Mailman is principally a server package.
Manually install the Debian package, but this requires manually updating it.
[ Test Plan ]
Examining the Jammy package list (https://packages.ubuntu.com/jammy/allpackages ) reveals that mailman3 is not included in Jammy.
[ Where problems could occur ]
Several packages are still present in Jammy, such as mailman3-web, despite mailman3 being removed. They will also have to be tested
and potentially updated to be compatible with the version backported from Mantic.

description: updated
Changed in mailman3 (Ubuntu):
status: Confirmed → Invalid
Changed in mailman3 (Ubuntu Jammy):
status: New → Triaged
Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

I backported the version from Lunar without changes. It built fine in Jammy and all DEP-8 tests are passing:

autopkgtest [20:45:42]: @@@@@@@@@@@@@@@@@@@@ summary
mailman3-nose2 PASS
mailman3-create-list PASS
mailman3-api PASS

It was uploaded and now it goes through the NEW queue, and we need a SRU team member/Archive Admin to process this package. vorlon might be the person you need.

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello AlainKnaff, or anyone else affected,

Accepted mailman3 into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mailman3/3.3.8-1ubuntu1~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mailman3 (Ubuntu Jammy):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (mailman3/3.3.8-1ubuntu1~22.04.1)

All autopkgtests for the newly accepted mailman3 (3.3.8-1ubuntu1~22.04.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

mailman3/3.3.8-1ubuntu1~22.04.1 (arm64, i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#mailman3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
AlainKnaff (kubuntu-misc) wrote :

Tested after installing the following versions from jammy-proposed:

 mailman3 3.3.8-1ubuntu1~22.04.1
 mailman3-full 3.3.8-1ubuntu1~22.04.1
 libapache2-mod-wsgi-py3 4.9.0-1ubuntu0.2

Following packages are still what we had before, as no newer ones were found in jammy-proposed:
 python3-mailmanclient 3.3.3-1 (from standard jammy)
 mailman3-web 0+20200530-2 (from standard jammy)
 python3-django-hyperkitty 1.3.5-1+really1.3.4-1 (from standard jammy)
 python3-mailman-hyperkitty 1.1.0-9 (from Debian, because neither in jammy nor in jammay-test)

With this config, tested by sending a mail to a test mailing list on our server:
 1. mail is distributed to subscribers
 2. mail is archived to hyperkitty, and browsable in web interface

So, the 3 new packages seem to work, but a python3-mailman-hyperkitty package is still missing.

In conclusion: the 3 new packages from jammy-proposed to work as intended, however python3-mailman-hyperkitty needed for archives and depended by mailman3-full, is still missing!

Revision history for this message
AlainKnaff (kubuntu-misc) wrote :

Oh, and btw, the instructions on how to add proposed are incomplete, here is a better link (albeit for an older version => change the distribution nickname):

https://askubuntu.com/questions/49609/how-do-i-add-the-proposed-repository

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

Thanks for testing! You are right, we are missing src:mailman-hyperkitty which provides python3-mailman-hyperkitty (mailman3-full dependency).

I am going to add a task for the backport of src:mailman-hyperkitty from Lunar to Jammy and upload it to NEW. I just tested and no change is needed to make it build.

Changed in mailman-hyperkitty (Ubuntu):
status: New → Invalid
Changed in mailman-hyperkitty (Ubuntu Jammy):
status: New → In Progress
Revision history for this message
Bert Van de Poel (bhack) wrote :

What would be a sensible point for us to attempt an upgrade from 20.04 to 22.04 to see if mailman2 to mailman3 works well now? I'm also not sure if there's a way to make do-release-upgrade use the proposed repo, so some hints on that front would also be welcome. We are excited and ready to make this work, so that many who've been keeping back their installations are finally able to upgrade to the current LTS.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.