Activity log for bug #1855061

Date Who What changed Old value New value Message
2019-12-04 07:35:42 Frode Nordahl bug added bug
2019-12-04 07:35:54 Frode Nordahl bug task added charms.openstack
2019-12-04 07:46:35 Frode Nordahl description In charms.openstack we have this sequience of calls: source, key = os_utils.get_source_and_pgp_key( self.config[self.source_config_key]) fetch.add_source(source, key) fetch.apt_update(fatal=True) On a failed deployment I found this: 2019-12-03 15:07:10 INFO juju-log Reactive main running for hook install 2019-12-03 15:07:11 INFO juju-log Initializing Leadership Layer (is leader) 2019-12-03 15:07:11 DEBUG juju-log tracer: set flag leadership.is_leader 2019-12-03 15:07:11 DEBUG juju-log tracer> tracer: starting handler dispatch, 8 flags set tracer: set flag charms.openstack.do-default-charm.installed tracer: set flag charms.openstack.do-default-config.changed tracer: set flag charms.openstack.do-default-update-status tracer: set flag charms.openstack.do-default-upgrade-charm tracer: set flag config.changed tracer: set flag config.changed.source tracer: set flag config.set.source tracer: set flag leadership.is_leader 2019-12-03 15:07:11 DEBUG juju-log tracer: hooks phase, 0 handlers queued 2019-12-03 15:07:11 DEBUG juju-log tracer> tracer: main dispatch loop, 3 handlers queued tracer: ++ queue handler hooks/relations/tls-certificates/requires.py:109:broken:certificates tracer: ++ queue handler reactive/layer_openstack.py:26:default_config_changed tracer: ++ queue handler reactive/layer_openstack.py:9:default_install 2019-12-03 15:07:11 INFO juju-log Invoking reactive handler: reactive/layer_openstack.py:9:default_install 2019-12-03 15:07:12 INFO juju-log Installing ['ubuntu-cloud-keyring'] with options: ['--option=Dpkg::Options::=--force-confold'] 2019-12-03 15:07:12 DEBUG install Reading package lists... 2019-12-03 15:07:12 DEBUG install Building dependency tree... 2019-12-03 15:07:12 DEBUG install Reading state information... 2019-12-03 15:07:12 DEBUG install The following package was automatically installed and is no longer required: 2019-12-03 15:07:12 DEBUG install libfreetype6 2019-12-03 15:07:12 DEBUG install Use 'apt autoremove' to remove it. 2019-12-03 15:07:12 DEBUG install The following NEW packages will be installed: 2019-12-03 15:07:12 DEBUG install ubuntu-cloud-keyring 2019-12-03 15:07:12 DEBUG install 0 upgraded, 1 newly installed, 0 to remove and 42 not upgraded. 2019-12-03 15:07:12 DEBUG install Need to get 6008 B of archives. 2019-12-03 15:07:12 DEBUG install After this operation, 22.5 kB of additional disk space will be used. 2019-12-03 15:07:12 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ubuntu-cloud-keyring all 2018.09.18.1~18.04.0 [6008 B] 2019-12-03 15:07:13 DEBUG install Fetched 6008 B in 0s (14.7 kB/s) 2019-12-03 15:07:13 DEBUG install Selecting previously unselected package ubuntu-cloud-keyring. (Reading database ... 35333 files and directories currently installed.) 2019-12-03 15:07:13 DEBUG install Preparing to unpack .../ubuntu-cloud-keyring_2018.09.18.1~18.04.0_all.deb ... 2019-12-03 15:07:13 DEBUG install Unpacking ubuntu-cloud-keyring (2018.09.18.1~18.04.0) ... 2019-12-03 15:07:13 DEBUG install Setting up ubuntu-cloud-keyring (2018.09.18.1~18.04.0) ... 2019-12-03 15:07:15 DEBUG install Err:1 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train InRelease 2019-12-03 15:07:15 DEBUG install 503 Service Unavailable [IP: 10.219.3.124 3128] 2019-12-03 15:07:15 DEBUG install Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2019-12-03 15:07:15 DEBUG install Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2019-12-03 15:07:15 DEBUG install Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2019-12-03 15:07:15 DEBUG install Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2019-12-03 15:07:17 DEBUG install Reading package lists... 2019-12-03 15:07:17 DEBUG install W: Failed to fetch http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/bionic-updates/train/InRelease 503 Service Unavailable [IP: 10.219.3.124 3128] 2019-12-03 15:07:17 DEBUG install W: Some index files failed to download. They have been ignored, or old ones used instead. At this point execution should have stopped, but instead it continued with the in-distro packages, leading to a very confusing result. 2019-12-03 15:07:17 INFO juju-log Installing ['ovn-central'] with options: ['--option=Dpkg::Options::=--force-confold'] 2019-12-03 15:07:18 DEBUG install Reading package lists... 2019-12-03 15:07:18 DEBUG install Building dependency tree... 2019-12-03 15:07:18 DEBUG install Reading state information... 2019-12-03 15:07:18 DEBUG install The following package was automatically installed and is no longer required: 2019-12-03 15:07:18 DEBUG install libfreetype6 2019-12-03 15:07:18 DEBUG install Use 'apt autoremove' to remove it. 2019-12-03 15:07:18 DEBUG install The following additional packages will be installed: 2019-12-03 15:07:18 DEBUG install libpython-stdlib libpython2.7-minimal libpython2.7-stdlib openvswitch-common 2019-12-03 15:07:18 DEBUG install openvswitch-switch ovn-common python python-minimal python-six python2.7 2019-12-03 15:07:18 DEBUG install python2.7-minimal 2019-12-03 15:07:18 DEBUG install Suggested packages: 2019-12-03 15:07:18 DEBUG install openvswitch-doc python-doc python-tk python2.7-doc binfmt-support 2019-12-03 15:07:18 DEBUG install The following NEW packages will be installed: 2019-12-03 15:07:18 DEBUG install libpython-stdlib libpython2.7-minimal libpython2.7-stdlib openvswitch-common 2019-12-03 15:07:18 DEBUG install openvswitch-switch ovn-central ovn-common python python-minimal python-six 2019-12-03 15:07:18 DEBUG install python2.7 python2.7-minimal 2019-12-03 15:07:18 DEBUG install 0 upgraded, 12 newly installed, 0 to remove and 42 not upgraded. 2019-12-03 15:07:18 DEBUG install Need to get 7821 kB of archives. 2019-12-03 15:07:18 DEBUG install After this operation, 33.2 MB of additional disk space will be used. 2019-12-03 15:07:18 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-minimal amd64 2.7.15-4ubuntu4~18.04.2 [336 kB] 2019-12-03 15:07:18 DEBUG install Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7-minimal amd64 2.7.15-4ubuntu4~18.04.2 [1296 kB] 2019-12-03 15:07:19 DEBUG install Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-minimal amd64 2.7.15~rc1-1 [28.1 kB] 2019-12-03 15:07:19 DEBUG install Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-stdlib amd64 2.7.15-4ubuntu4~18.04.2 [1916 kB] 2019-12-03 15:07:19 DEBUG install Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7 amd64 2.7.15-4ubuntu4~18.04.2 [239 kB] 2019-12-03 15:07:19 DEBUG install Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython-stdlib amd64 2.7.15~rc1-1 [7620 B] 2019-12-03 15:07:19 DEBUG install Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python amd64 2.7.15~rc1-1 [140 kB] 2019-12-03 15:07:19 DEBUG install Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-six all 1.11.0-2 [11.3 kB] 2019-12-03 15:07:20 DEBUG install Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openvswitch-common amd64 2.9.2-0ubuntu0.18.04.3 [814 kB] 2019-12-03 15:07:20 DEBUG install Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openvswitch-switch amd64 2.9.2-0ubuntu0.18.04.3 [1506 kB] 2019-12-03 15:07:20 DEBUG install Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 ovn-common amd64 2.9.2-0ubuntu0.18.04.3 [821 kB] 2019-12-03 15:07:20 DEBUG install Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 ovn-central amd64 2.9.2-0ubuntu0.18.04.3 [706 kB] In charms.openstack we have this sequence of calls:         source, key = os_utils.get_source_and_pgp_key(             self.config[self.source_config_key])         fetch.add_source(source, key)         fetch.apt_update(fatal=True) On a failed deployment I found this: 2019-12-03 15:07:10 INFO juju-log Reactive main running for hook install 2019-12-03 15:07:11 INFO juju-log Initializing Leadership Layer (is leader) 2019-12-03 15:07:11 DEBUG juju-log tracer: set flag leadership.is_leader 2019-12-03 15:07:11 DEBUG juju-log tracer> tracer: starting handler dispatch, 8 flags set tracer: set flag charms.openstack.do-default-charm.installed tracer: set flag charms.openstack.do-default-config.changed tracer: set flag charms.openstack.do-default-update-status tracer: set flag charms.openstack.do-default-upgrade-charm tracer: set flag config.changed tracer: set flag config.changed.source tracer: set flag config.set.source tracer: set flag leadership.is_leader 2019-12-03 15:07:11 DEBUG juju-log tracer: hooks phase, 0 handlers queued 2019-12-03 15:07:11 DEBUG juju-log tracer> tracer: main dispatch loop, 3 handlers queued tracer: ++ queue handler hooks/relations/tls-certificates/requires.py:109:broken:certificates tracer: ++ queue handler reactive/layer_openstack.py:26:default_config_changed tracer: ++ queue handler reactive/layer_openstack.py:9:default_install 2019-12-03 15:07:11 INFO juju-log Invoking reactive handler: reactive/layer_openstack.py:9:default_install 2019-12-03 15:07:12 INFO juju-log Installing ['ubuntu-cloud-keyring'] with options: ['--option=Dpkg::Options::=--force-confold'] 2019-12-03 15:07:12 DEBUG install Reading package lists... 2019-12-03 15:07:12 DEBUG install Building dependency tree... 2019-12-03 15:07:12 DEBUG install Reading state information... 2019-12-03 15:07:12 DEBUG install The following package was automatically installed and is no longer required: 2019-12-03 15:07:12 DEBUG install libfreetype6 2019-12-03 15:07:12 DEBUG install Use 'apt autoremove' to remove it. 2019-12-03 15:07:12 DEBUG install The following NEW packages will be installed: 2019-12-03 15:07:12 DEBUG install ubuntu-cloud-keyring 2019-12-03 15:07:12 DEBUG install 0 upgraded, 1 newly installed, 0 to remove and 42 not upgraded. 2019-12-03 15:07:12 DEBUG install Need to get 6008 B of archives. 2019-12-03 15:07:12 DEBUG install After this operation, 22.5 kB of additional disk space will be used. 2019-12-03 15:07:12 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ubuntu-cloud-keyring all 2018.09.18.1~18.04.0 [6008 B] 2019-12-03 15:07:13 DEBUG install Fetched 6008 B in 0s (14.7 kB/s) 2019-12-03 15:07:13 DEBUG install Selecting previously unselected package ubuntu-cloud-keyring. (Reading database ... 35333 files and directories currently installed.) 2019-12-03 15:07:13 DEBUG install Preparing to unpack .../ubuntu-cloud-keyring_2018.09.18.1~18.04.0_all.deb ... 2019-12-03 15:07:13 DEBUG install Unpacking ubuntu-cloud-keyring (2018.09.18.1~18.04.0) ... 2019-12-03 15:07:13 DEBUG install Setting up ubuntu-cloud-keyring (2018.09.18.1~18.04.0) ... 2019-12-03 15:07:15 DEBUG install Err:1 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train InRelease 2019-12-03 15:07:15 DEBUG install 503 Service Unavailable [IP: 10.219.3.124 3128] 2019-12-03 15:07:15 DEBUG install Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2019-12-03 15:07:15 DEBUG install Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2019-12-03 15:07:15 DEBUG install Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2019-12-03 15:07:15 DEBUG install Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2019-12-03 15:07:17 DEBUG install Reading package lists... 2019-12-03 15:07:17 DEBUG install W: Failed to fetch http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/bionic-updates/train/InRelease 503 Service Unavailable [IP: 10.219.3.124 3128] 2019-12-03 15:07:17 DEBUG install W: Some index files failed to download. They have been ignored, or old ones used instead. At this point execution should have stopped, but instead it continued with the in-distro packages, leading to a very confusing result. 2019-12-03 15:07:17 INFO juju-log Installing ['ovn-central'] with options: ['--option=Dpkg::Options::=--force-confold'] 2019-12-03 15:07:18 DEBUG install Reading package lists... 2019-12-03 15:07:18 DEBUG install Building dependency tree... 2019-12-03 15:07:18 DEBUG install Reading state information... 2019-12-03 15:07:18 DEBUG install The following package was automatically installed and is no longer required: 2019-12-03 15:07:18 DEBUG install libfreetype6 2019-12-03 15:07:18 DEBUG install Use 'apt autoremove' to remove it. 2019-12-03 15:07:18 DEBUG install The following additional packages will be installed: 2019-12-03 15:07:18 DEBUG install libpython-stdlib libpython2.7-minimal libpython2.7-stdlib openvswitch-common 2019-12-03 15:07:18 DEBUG install openvswitch-switch ovn-common python python-minimal python-six python2.7 2019-12-03 15:07:18 DEBUG install python2.7-minimal 2019-12-03 15:07:18 DEBUG install Suggested packages: 2019-12-03 15:07:18 DEBUG install openvswitch-doc python-doc python-tk python2.7-doc binfmt-support 2019-12-03 15:07:18 DEBUG install The following NEW packages will be installed: 2019-12-03 15:07:18 DEBUG install libpython-stdlib libpython2.7-minimal libpython2.7-stdlib openvswitch-common 2019-12-03 15:07:18 DEBUG install openvswitch-switch ovn-central ovn-common python python-minimal python-six 2019-12-03 15:07:18 DEBUG install python2.7 python2.7-minimal 2019-12-03 15:07:18 DEBUG install 0 upgraded, 12 newly installed, 0 to remove and 42 not upgraded. 2019-12-03 15:07:18 DEBUG install Need to get 7821 kB of archives. 2019-12-03 15:07:18 DEBUG install After this operation, 33.2 MB of additional disk space will be used. 2019-12-03 15:07:18 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-minimal amd64 2.7.15-4ubuntu4~18.04.2 [336 kB] 2019-12-03 15:07:18 DEBUG install Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7-minimal amd64 2.7.15-4ubuntu4~18.04.2 [1296 kB] 2019-12-03 15:07:19 DEBUG install Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-minimal amd64 2.7.15~rc1-1 [28.1 kB] 2019-12-03 15:07:19 DEBUG install Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython2.7-stdlib amd64 2.7.15-4ubuntu4~18.04.2 [1916 kB] 2019-12-03 15:07:19 DEBUG install Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python2.7 amd64 2.7.15-4ubuntu4~18.04.2 [239 kB] 2019-12-03 15:07:19 DEBUG install Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpython-stdlib amd64 2.7.15~rc1-1 [7620 B] 2019-12-03 15:07:19 DEBUG install Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python amd64 2.7.15~rc1-1 [140 kB] 2019-12-03 15:07:19 DEBUG install Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-six all 1.11.0-2 [11.3 kB] 2019-12-03 15:07:20 DEBUG install Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openvswitch-common amd64 2.9.2-0ubuntu0.18.04.3 [814 kB] 2019-12-03 15:07:20 DEBUG install Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openvswitch-switch amd64 2.9.2-0ubuntu0.18.04.3 [1506 kB] 2019-12-03 15:07:20 DEBUG install Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 ovn-common amd64 2.9.2-0ubuntu0.18.04.3 [821 kB] 2019-12-03 15:07:20 DEBUG install Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 ovn-central amd64 2.9.2-0ubuntu0.18.04.3 [706 kB]