Kubernetes E2E Test Charm

import error packagefinder on focal

Bug #1870134 reported by Adam Stokes on 2020-04-01

This bug affects 2 people

	Status	Importance	Assigned to	Milestone
AWS IAM Subordinate Charm	Fix Released	High	Unassigned	AWS IAM Subordinate Charm 1.18+ck1
AWS Integrator Charm	Fix Released	High	Unassigned	AWS Integrator Charm 1.18+ck1
Azure Integrator Charm	Fix Released	High	Unassigned	Azure Integrator Charm 1.18+ck1
Calico Charm	Fix Released	High	Unassigned	Calico Charm 1.18+ck1
Canal Charm	Fix Released	High	Unassigned	Canal Charm 1.18+ck1
Containerd Subordinate Charm	Fix Released	High	Unassigned	Containerd Subordinate Charm 1.18+ck1
Docker Registry Charm	Fix Released	High	Unassigned	Docker Registry Charm 1.18+ck1
Docker Subordinate Charm	Fix Released	High	Unassigned	Docker Subordinate Charm 1.18+ck1
EasyRSA Charm	Fix Released	High	Unassigned	EasyRSA Charm 1.18+ck1
Etcd Charm	Fix Released	High	Unassigned	Etcd Charm 1.18+ck1
Flannel Charm	Fix Released	High	Cory Johns	Flannel Charm 1.18+ck1
Google Cloud Platform Integrator Charm	Fix Released	High	Unassigned	Google Cloud Platform Integrator Charm 1.18+ck1
Kata Containers Subordinate Charm	Fix Released	High	Unassigned	Kata Containers Subordinate Charm 1.18+ck1
Keepalived Charm	Fix Released	High	Unassigned	Keepalived Charm 1.20+ck1
Kubernetes API Load Balancer	Fix Released	High	Unassigned	Kubernetes API Load Balancer 1.18+ck1
Kubernetes Control Plane Charm	Fix Released	High	Unassigned	Kubernetes Control Plane Charm 1.18+ck1
Kubernetes E2E Test Charm	Fix Released	High	Unassigned	Kubernetes E2E Test Charm 1.18+ck1
Kubernetes Worker Charm	Fix Released	High	Unassigned	Kubernetes Worker Charm 1.18+ck1
Openstack Integrator Charm	Fix Released	High	Unassigned	Openstack Integrator Charm 1.18+ck1
Tigera Secure EE Charm	Fix Released	High	Unassigned	Tigera Secure EE Charm 1.18+ck1
vSphere Integrator Charm	Fix Released	High	Unassigned	vSphere Integrator Charm 1.18+ck1

Bug Description

Attempting to deploy flannel on focal with ck 1.18 results in flannel workload error:

unit-kubernetes-worker-1: 03:51:03 DEBUG unit.kubernetes-worker/1.juju-log kube-control:6: tracer: set flag config.default.allow-privileged
unit-kubernetes-master-0: 03:51:03 DEBUG unit.kubernetes-master/0.juju-log tracer: set flag config.set.image-registry
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm Traceback (most recent call last):
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/bin/pip", line 5, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal import main
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/__init__.py", line 40, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.cli.autocompletion import autocomplete
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.cli.main_parser import create_main_parser
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 12, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.commands import (
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 6, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.commands.completion import CompletionCommand
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/commands/completion.py", line 6, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.cli.base_command import Command
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 23, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm from pip._internal.index import PackageFinder
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm ImportError: cannot import name 'PackageFinder' from 'pip._internal.index' (/var/lib/juju/agents/unit-flannel-1/.venv/lib/python3.8/site-packages/pip/_internal/index/__init__.py)
unit-kubernetes-master-1: 03:51:03 DEBUG unit.kubernetes-master/1.juju-log certificates:7: tracer: set flag config.set.dashboard-auth
unit-kubernetes-worker-1: 03:51:03 DEBUG unit.kubernetes-worker/1.juju-log kube-control:6: tracer: set flag config.set.channel
unit-etcd-2: 03:51:03 DEBUG unit.etcd/2.juju-log tracer: set flag config.default.port
unit-kubernetes-worker-0: 03:51:03 INFO unit.kubernetes-worker/0.juju-log certificates:10: Invoking reactive handler: reactive/kubernetes_worker.py:1164:missing_kube_control
unit-kubernetes-master-0: 03:51:03 DEBUG unit.kubernetes-master/0.juju-log tracer: set flag config.default.image-registry
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm Traceback (most recent call last):
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/var/lib/juju/agents/unit-flannel-1/charm/hooks/upgrade-charm", line 8, in <module>
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm basic.bootstrap_charm_deps()
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "lib/charms/layer/basic.py", line 170, in bootstrap_charm_deps
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm check_call([pip, 'install', '-U', '--no-index', '-f', 'wheelhouse',
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm raise CalledProcessError(retcode, cmd)
unit-flannel-1: 03:51:03 DEBUG unit.flannel/1.upgrade-charm subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-flannel-1/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status 1.

Revision history for this message

Adam Stokes (adam-stokes) wrote on 2020-04-01:

juju-crashdump-8ac6c622-0b48-4c80-ae0f-cc112c434610.tar.xz Edit (4.4 MiB, application/x-tar)

Revision history for this message

Cory Johns (johnsca) wrote on 2020-04-01:

Per https://github.com/pypa/pip/issues/7620 it seems like there was an issue with the 20.0 release of pip. Focal seems to be on 20.0.2, but it seems that there's stilll an issue with it:

$ lxc launch ubuntu-daily:focal
Creating the instance
Instance name is: good-hawk
Starting good-hawk
$ lxc shell good-hawk
root@good-hawk:~# apt update
...
root@good-hawk:~# apt install python3-pip
...
root@good-hawk:~# pip3 --version
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

Revision history for this message

Cory Johns (johnsca) wrote on 2020-04-02:

It turns out that the pip, setuptools, and setuptools-scm bundling in layer:basic ends up downgrading these packages on Focal causing this issue, but only after the initial bootstrap. Flannel seems to run an upgrade-charm during the initial install for some reason, which is why we're seeing this on this charm and not others, but it will affect all reactive charms on Focal at the first upgrade.

I filed this against layer:basic at https://github.com/juju-solutions/layer-basic/issues/159 and have a PR in progress to address this at https://github.com/juju-solutions/layer-basic/pull/160

Changed in charm-flannel:
status:	New → In Progress
assignee:	nobody → Cory Johns (johnsca)

Revision history for this message

Cory Johns (johnsca) wrote on 2020-04-03:

The PR has been merged.

Changed in charm-flannel:
status:	In Progress → Fix Committed

Cory Johns (johnsca) on 2020-04-03

Changed in charm-flannel:
milestone:	none → 1.19

George Kraft (cynerva) on 2020-05-13

Changed in charm-kubernetes-worker:
status:	New → Fix Committed
milestone:	none → 1.19

Tim Van Steenburgh (tvansteenburgh) on 2020-05-20

Changed in charm-flannel:
milestone:	1.19 → 1.18+ck1
Changed in charm-kubernetes-worker:
milestone:	1.19 → 1.18+ck1

George Kraft (cynerva) on 2020-05-20

Changed in charm-aws-iam:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-aws-integrator:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-azure-integrator:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-calico:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-canal:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-containerd:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in layer-docker-registry:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-docker:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-easyrsa:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-etcd:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-flannel:
importance:	Undecided → High
Changed in charm-gcp-integrator:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-kata:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-keepalived:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-kubeapi-load-balancer:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-kubernetes-e2e:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-kubernetes-master:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-kubernetes-worker:
importance:	Undecided → High
Changed in charm-openstack-integrator:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-tigera-secure-ee:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed
Changed in charm-vsphere-integrator:
importance:	Undecided → High
milestone:	none → 1.18+ck1
status:	New → Fix Committed

Revision history for this message

George Kraft (cynerva) wrote on 2020-06-01:

Cherry-pick to stable:
https://github.com/charmed-kubernetes/layer-basic/commit/4761d6c966966441b8eb388af152b25ddcde6636
https://github.com/charmed-kubernetes/layer-basic/commit/a7ad4f1f6fddc6c767549c29a77f341c2dccacbf
https://github.com/charmed-kubernetes/layer-basic/commit/955e85312718dbc3818e471dc7c3ea983cb09ce8

Revision history for this message

George Kraft (cynerva) wrote on 2020-06-02:

Follow-up cherry-pick due to an issue encountered in testing:
https://github.com/charmed-kubernetes/layer-basic/commit/f761da8d4a2985b725f2ec79be703f3e63b0e29b

Tim Van Steenburgh (tvansteenburgh) on 2020-06-11

Changed in charm-aws-iam:
status:	Fix Committed → Fix Released
Changed in charm-aws-integrator:
status:	Fix Committed → Fix Released
Changed in charm-azure-integrator:
status:	Fix Committed → Fix Released
Changed in charm-calico:
status:	Fix Committed → Fix Released
Changed in charm-canal:
status:	Fix Committed → Fix Released
Changed in charm-containerd:
status:	Fix Committed → Fix Released
Changed in layer-docker-registry:
status:	Fix Committed → Fix Released
Changed in charm-docker:
status:	Fix Committed → Fix Released
Changed in charm-easyrsa:
status:	Fix Committed → Fix Released
Changed in charm-etcd:
status:	Fix Committed → Fix Released
Changed in charm-flannel:
status:	Fix Committed → Fix Released
Changed in charm-gcp-integrator:
status:	Fix Committed → Fix Released
Changed in charm-kata:
status:	Fix Committed → Fix Released
Changed in charm-keepalived:
status:	Fix Committed → Fix Released
Changed in charm-kubeapi-load-balancer:
status:	Fix Committed → Fix Released
Changed in charm-kubernetes-e2e:
status:	Fix Committed → Fix Released
Changed in charm-kubernetes-master:
status:	Fix Committed → Fix Released
Changed in charm-kubernetes-worker:
status:	Fix Committed → Fix Released
Changed in charm-openstack-integrator:
status:	Fix Committed → Fix Released
Changed in charm-tigera-secure-ee:
status:	Fix Committed → Fix Released
Changed in charm-vsphere-integrator:
status:	Fix Committed → Fix Released