Upgrade of erlang-base left rabbitmq-server dead
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
erlang (Ubuntu) |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
Last night, unattended-upgrades upgraded the erlang packages on our RabbitMQ server from 1:18.3-
# From /var/log/
#...
Preparing to unpack .../erlang-
Searching for services which depend on erlang and should be stopped...found: rabbitmq-server.
Stopping services which depend on erlang
rabbitmq-server: stopping...done.
Services stopped successfully.
Killing epmd...it is not running.
Unpacking erlang-base (1:18.3-
Preparing to unpack .../erlang-
Unpacking erlang-nox (1:18.3-
Processing triggers for man-db (2.7.5-1) ...
Setting up erlang-base (1:18.3-
Searching for services which depend on erlang and should be started...found: rabbitmq-server.
Starting services which depend on erlang
rabbitmq-server: starting...done.
Services started successfully.
#...
I looked into erlang-base.prerm and erlang-
invoke-rc.d rabbitmq-server stop > /dev/null 2>&1 # from erlang-base.prerm
invoke-rc.d rabbitmq-server start > /dev/null 2>&1 # from erlang-
But while the stop command succeeded, the start command did not, and there isn't any evidence in the logs that it was ever issued. This is the reason why I think this is a bug in the erlang package (or possibly invoke-
# journalctl
Feb 16 06:30:03 rabbit-b systemd[1]: Starting Daily apt upgrade and clean activities...
Feb 16 06:30:48 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:49 rabbit-b systemd[1]: Started ACPI event daemon.
Feb 16 06:30:49 rabbit-b systemd[1]: Stopping RabbitMQ Messaging Server...
Feb 16 06:30:50 rabbit-b rabbitmq[6596]: Stopping and halting node 'rabbit@rabbit-b' ...
Feb 16 06:30:51 rabbit-b systemd[1]: Stopped RabbitMQ Messaging Server.
Feb 16 06:30:52 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:52 rabbit-b systemd[1]: Started ACPI event daemon.
Feb 16 06:30:54 rabbit-b systemd[1]: Reloading.
Feb 16 06:30:54 rabbit-b systemd[1]: Started ACPI event daemon.
# ... and the next log entry is from after unattended-upgrades had ended at 06:30:56
# /<email address hidden>
=INFO REPORT==== 16-Feb-
Stopping RabbitMQ
#... lots of AMQP connection closures
=INFO REPORT==== 16-Feb-
Halting Erlang VM
# ... and the next log entry is from 2h later when we manually started RabbitMQ again
This is Ubuntu 16.04.2 LTS with all security patches and the latest rabbitmq-server (3.5.7-
PS: Where are the sources for erlang-base.prerm and erlang-
description: | updated |
Status changed to 'Confirmed' because the bug affects multiple users.