Upgrade of erlang-base left rabbitmq-server dead

Bug #1749959 reported by Nils Toedtmann
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
erlang (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Last night, unattended-upgrades upgraded the erlang packages on our RabbitMQ server from 1:18.3-dfsg-1ubuntu3 to 1:18.3-dfsg-1ubuntu3.1. erlang-base's prerm script successfully found and stopped rabbitmq-server, and the postinst script tried to start it again:

# From /var/log/apt/term.log, same in /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
#...
Preparing to unpack .../erlang-base_1%3a18.3-dfsg-1ubuntu3.1_amd64.deb ...
Searching for services which depend on erlang and should be stopped...found: rabbitmq-server.
Stopping services which depend on erlang
  rabbitmq-server: stopping...done.
Services stopped successfully.
Killing epmd...it is not running.
Unpacking erlang-base (1:18.3-dfsg-1ubuntu3.1) over (1:18.3-dfsg-1ubuntu3) ...
Preparing to unpack .../erlang-nox_1%3a18.3-dfsg-1ubuntu3.1_all.deb ...
Unpacking erlang-nox (1:18.3-dfsg-1ubuntu3.1) over (1:18.3-dfsg-1ubuntu3) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up erlang-base (1:18.3-dfsg-1ubuntu3.1) ...
Searching for services which depend on erlang and should be started...found: rabbitmq-server.
Starting services which depend on erlang
  rabbitmq-server: starting...done.
Services started successfully.
#...

I looked into erlang-base.prerm and erlang-base.postinst and ran the relevant snippets manually, and it looks like they would have used "invoke-rc.d":

   invoke-rc.d rabbitmq-server stop > /dev/null 2>&1 # from erlang-base.prerm
   invoke-rc.d rabbitmq-server start > /dev/null 2>&1 # from erlang-base.postinst

But while the stop command succeeded, the start command did not, and there isn't any evidence in the logs that it was ever issued. This is the reason why I think this is a bug in the erlang package (or possibly invoke-rc.d/init-system-helpers?), and not in rabbitmq-server:

# journalctl
Feb 16 06:30:03 rabbit-b systemd[1]: Starting Daily apt upgrade and clean activities...
Feb 16 06:30:48 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:49 rabbit-b systemd[1]: Started ACPI event daemon.
Feb 16 06:30:49 rabbit-b systemd[1]: Stopping RabbitMQ Messaging Server...
Feb 16 06:30:50 rabbit-b rabbitmq[6596]: Stopping and halting node 'rabbit@rabbit-b' ...
Feb 16 06:30:51 rabbit-b systemd[1]: Stopped RabbitMQ Messaging Server.
Feb 16 06:30:52 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:52 rabbit-b systemd[1]: Started ACPI event daemon.
Feb 16 06:30:54 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:54 rabbit-b systemd[1]: Started ACPI event daemon.
# ... and the next log entry is from after unattended-upgrades had ended at 06:30:56

# /<email address hidden>
=INFO REPORT==== 16-Feb-2018::06:30:50 ===
Stopping RabbitMQ
#... lots of AMQP connection closures
=INFO REPORT==== 16-Feb-2018::06:30:51 ===
Halting Erlang VM
# ... and the next log entry is from 2h later when we manually started RabbitMQ again

This is Ubuntu 16.04.2 LTS with all security patches and the latest rabbitmq-server (3.5.7-1ubuntu0.16.04.2)

PS: Where are the sources for erlang-base.prerm and erlang-base.postinst? https://packages.debian.org/source/stretch/erlang points to http://anonscm.debian.org/viewvc/pkg-erlang/erlang/trunk/, but that link is dead

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in erlang (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.