Upgrade from bionic-train to bionic-ussuri fails for cinder due to sys-kernel-config.mount not existing in unprivileged container

Bug #1930455 reported by Drew Freiberger
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Cinder Charm
Invalid
Undecided
Unassigned
python-rtslib-fb (Ubuntu)
New
Undecided
Unassigned

Bug Description

While upgrading cinder with openstack-origin=cloud:bionic-ussuri, after upgrading from bionic:distro, bionic-rocky, bionic-stein, bionic-train, I've run into an issue with the upgrade/install of the package python3-rtslib-fb. The configure script of the package tries to start the service rtslib-fb-targetctl, which has a "Requires=sys-kernel-config.mount" in it's service definition.

This mount is not present in the cinder container.

Looking at some related bug: https://bugs.launchpad.net/ubuntu/+source/python-rtslib-fb/+bug/1916715

It seems that the module configfs is needed for this mountpoint to exist.

My container's kernel is 4.15.0-76-generic.

cinder-1: 18:20:59 INFO unit.cinder/1.juju-log [cinder] Performing OpenStack upgrade to ussuri.
unit-cinder-1: 18:20:59 INFO unit.cinder/1.juju-log DEPRECATION WARNING: Function configure_installation_source is being removed on/around 2017-07 : use charmhelpers.fetch.add_source() instead.
unit-cinder-1: 18:21:00 INFO unit.cinder/1.juju-log Installing [] with options: ['--option=Dpkg::Options::=--force-confold']
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed Reading package lists...
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed Building dependency tree...
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed Reading state information...
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed The following packages were automatically installed and are no longer required:
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed python-pyudev python-rtslib-fb python3-ceilometerclient python3-contextlib2
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed Use 'apt autoremove' to remove them.
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed 1 not fully installed or removed.
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed After this operation, 0 B of additional disk space will be used.
unit-cinder-1: 18:21:00 DEBUG unit.cinder/1.config-changed Setting up python3-rtslib-fb (2.1.66-3.2~cloud0) ...
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed A dependency job for rtslib-fb-targetctl.service failed. See 'journalctl -xe' for details.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed invoke-rc.d: initscript rtslib-fb-targetctl, action "start" failed.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed ● rtslib-fb-targetctl.service - Restore LIO kernel target configuration
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Loaded: loaded (/lib/systemd/system/rtslib-fb-targetctl.service; disabled; vendor preset: enabled)
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Active: inactive (dead)
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:17 juju-bf8473-19-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:17 juju-bf8473-19-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:30 juju-bf8473-19-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:30 juju-bf8473-19-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:43 juju-bf8473-19-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:43 juju-bf8473-19-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:55 juju-bf8473-19-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:15:55 juju-bf8473-19-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:21:01 juju-bf8473-19-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Jun 01 18:21:01 juju-bf8473-19-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed dpkg: error processing package python3-rtslib-fb (--configure):
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed installed python3-rtslib-fb package post-installation script subprocess returned error exit status 1
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed Errors were encountered while processing:
unit-cinder-1: 18:21:01 DEBUG unit.cinder/1.config-changed python3-rtslib-fb
unit-cinder-1: 18:21:02 WARNING unit.cinder/1.config-changed E: Sub-process /usr/bin/dpkg returned an error code (1)

I tried modprobing configfs on the metal, then restarting the container, but I am still seeing failure to mount, permission denied.

ubuntu@juju-bf8473-18-lxd-2:/sys/kernel/config$ sudo systemctl start sys-kernel-config.mount
Job for sys-kernel-config.mount failed.
See "systemctl status sys-kernel-config.mount" and "journalctl -xe" for details.
ubuntu@juju-bf8473-18-lxd-2:/sys/kernel/config$ journalctl -xe
-- The result is RESULT.
Jun 01 18:31:45 juju-bf8473-18-lxd-2 systemd[1]: Dependency failed for Restore LIO kernel target configuration.
-- Subject: Unit rtslib-fb-targetctl.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit rtslib-fb-targetctl.service has failed.
--
-- The result is RESULT.
Jun 01 18:31:45 juju-bf8473-18-lxd-2 systemd[1]: rtslib-fb-targetctl.service: Job rtslib-fb-targetctl.service/start failed with result 'dependency'.
Jun 01 18:31:49 juju-bf8473-18-lxd-2 sudo[4421]: ubuntu : TTY=pts/0 ; PWD=/sys/kernel/config ; USER=root ; COMMAND=/bin/systemctl start sys-kernel-config.mount
Jun 01 18:31:49 juju-bf8473-18-lxd-2 sudo[4421]: pam_unix(sudo:session): session opened for user root by ubuntu(uid=0)
Jun 01 18:31:49 juju-bf8473-18-lxd-2 systemd[1]: sys-kernel-config.mount: Failed to reset devices.list: Operation not permitted
Jun 01 18:31:49 juju-bf8473-18-lxd-2 systemd[1]: Mounting Kernel Configuration File System...
-- Subject: Unit sys-kernel-config.mount has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit sys-kernel-config.mount has begun starting up.
Jun 01 18:31:49 juju-bf8473-18-lxd-2 mount[4426]: mount: /sys/kernel/config: permission denied.
Jun 01 18:31:49 juju-bf8473-18-lxd-2 systemd[1]: sys-kernel-config.mount: Mount process exited, code=exited status=32
Jun 01 18:31:49 juju-bf8473-18-lxd-2 systemd[1]: sys-kernel-config.mount: Failed with result 'exit-code'.
Jun 01 18:31:49 juju-bf8473-18-lxd-2 systemd[1]: Failed to mount Kernel Configuration File System.
-- Subject: Unit sys-kernel-config.mount has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit sys-kernel-config.mount has failed.
--
-- The result is RESULT.
Jun 01 18:31:49 juju-bf8473-18-lxd-2 sudo[4421]: pam_unix(sudo:session): session closed for user root

Revision history for this message
Drew Freiberger (afreiberger) wrote :

As my installation is not using iSCSI, I've removed the package on the cinder units and re-triggered the config-changed hook with juju resolved.

juju run -a cinder 'apt-get remove -y python3-rtslib-fb'
juju resolved cinder/0
juju resolved cinder/1
juju resolved cinder/2

tags: added: series-upgrade
tags: added: openstack-upgrade
removed: series-upgrade
Revision history for this message
Billy Olsen (billy-olsen) wrote :

We should fix this in the package and SRU it there rather than relying on a charm fix. I've raised a task against bug #1916715 to resolve it for focal, but this is where we should fix it.

Revision history for this message
Billy Olsen (billy-olsen) wrote :

I'm marking this as a duplicate of bug #1916715 per previous comment.

Changed in charm-cinder:
status: New → Invalid
Revision history for this message
Billy Olsen (billy-olsen) wrote :

Updated to be a duplicate of the correct bug - bug #1865037

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.