Canonical Juju

juju scp broken in 2.9.24 release testing

Bug #1960844 reported by Bas de Bruijne
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Incomplete
High
Yang Kelvin Liu

Bug Description

As seen in testrun:

https://solutions.qa.canonical.com/testruns/testRun/ebab0f04-c2a2-4ae7-bf58-6e7b06e39042

Juju fails to scp files from the kubernetes-master unit.

------------------------------
2022-02-12-01:05:50 root ERROR [localhost] Command failed: juju scp -m foundation-openstack:kubernetes-openstack kubernetes-master/1:config /home/ubuntu/project/generated/kubernetes-openstack/kube.conf
2022-02-12-01:05:50 root ERROR [localhost] STDOUT follows:
b''
2022-02-12-01:05:50 root ERROR [localhost] STDERR follows:
ERROR cannot connect to any address: [172.16.0.110:22]
------------------------------

Crashdumps:
https://oil-jenkins.canonical.com/artifacts/ebab0f04-c2a2-4ae7-bf58-6e7b06e39042/index.html

The crashdumps are empty, even though `juju crashdump` uses scp instead of juju scp.

All occurrences of this bug:
https://solutions.qa.canonical.com/bugs/bugs/bug/1960844

See original description
Bas de Bruijne (basdbruijne)
description: updated
Revision history for this message
John A Meinel (jameinel) wrote :

Kubernetes containers don't run ssh for you to SCP the content from a container.
Juju fakes ssh support and scp support by mapping it to other kubernetes primitives (exec, IIRC)

So crashdump won't work for those cotnainers. I don't know why it is actually trying to connect to :22, since there isn't anything there.

Changed in juju:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Yang Kelvin Liu (kelvin.liu)
Revision history for this message
Yang Kelvin Liu (kelvin.liu) wrote :

The test was trying to juju scp to CK master node which is an OpenStack machine.

Revision history for this message
Yang Kelvin Liu (kelvin.liu) wrote :

I deployed CK bundle on AWS but I can't reproduce it.
And as we know, 2.9.24 had a few runs already, we didn't have this issue previously.
Additionally, the juju crashdumps are all empty as well. But we did have crashdumps last week.
So I think this should not be a Juju-related issue but a networking or proxy config issue.

juju ssh/scp is client-side only command if we don't use --proxy.
So it will be very helpful for debugging if the test runs commands with --debug in the future.

Revision history for this message
Bas de Bruijne (basdbruijne) wrote (last edit ):

Thanks for looking into this so quickly.
I checked with previous k8s on openstack testruns, normally crashdump does work on these containers. See for example: https://oil-jenkins.canonical.com/artifacts/3f96065d-e194-4afa-9de5-f0022f9b9211/index.html

I will see if I can reproduce the problem with --debug

Revision history for this message
Bas de Bruijne (basdbruijne) wrote :

I tried again with the same bundle, but juju scp seems to work fine now. Maybe because the snap updated to 2.9.25 and the issue is solved there already. I will reopen this bug if we run into this again.

Changed in juju:
status: Triaged → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.