| 2021-08-04 16:06:04 |
Alexander Balderson |
description |
Testing on baremetal
we ran into a case where test_cluster_service_endpoint failed when running /snap/bin/kubectl get po ... because of an Unauthorized, and we must be logged into the server; full error at the bottom of the report.
This is weird because the command is running from the k8s units themselves using juju run. We have a bug on baremetal about k8s master failing to connect to itself, LP: #1929234 it could be the same issue, but it manifested strangely, and everything is active-idle in juju status.
I'm going to submit a patch to the test to print the unit names that are checking the pod for some extra information; but something went wrong with the k8s unit.
Test run at: https://solutions.qa.canonical.com/testruns/testRun/81564c2d-702c-4f27-8a1a-a35c783a4915
Crashdump at: https://oil-jenkins.canonical.com/artifacts/81564c2d-702c-4f27-8a1a-a35c783a4915/generated/generated/kubernetes/juju-crashdump-kubernetes-2021-08-03-22.19.27.tar.gz
Test result xml at: https://oil-jenkins.canonical.com/artifacts/81564c2d-702c-4f27-8a1a-a35c783a4915/generated/generated/kubernetes/k8s-suite/test_clusterip_service_endpoint-junit.xml
______________________ test_clusterip_service_endpoint ________________________
Traceback (most recent call last):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 125, in test_clusterip_service_endpoint
await cleanup()
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 69, in cleanup
await retry_async_with_timeout(
File "/home/ubuntu/k8s-validation/jobs/integration/utils.py", line 198, in retry_async_with_timeout
if await func(*args):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 34, in is_pod_cleaned
pod = get_pod_yaml()
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 9, in get_pod_yaml
out = sh.kubectl.get("po", o="yaml", selector="app=hello-world")
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 1520, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 784, in __init__
self.wait()
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 841, in wait
self.handle_command_exit_code(exit_code)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 865, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_1:
RAN: /snap/bin/kubectl get po -o yaml --selector=app=hello-world
STDOUT:
apiVersion: v1
items: []
kind: List
metadata:
resourceVersion: ""
selfLink: ""
STDERR:
error: You must be logged in to the server (Unauthorized) |
Testing on baremetal
we ran into a case where test_cluster_service_endpoint failed when running /snap/bin/kubectl get po ... because of an Unauthorized, and we must be logged into the server; full error at the bottom of the report.
This is weird because the command is running from the k8s units themselves using juju run. We have a bug on baremetal about k8s master failing to connect to itself, LP: #1929234 it could be the same issue, but it manifested strangely, and everything is active-idle in juju status.
I'm going to submit a patch to the test to print the unit names that are checking the pod for some extra information; but something went wrong with the k8s unit.
Test run at: https://solutions.qa.canonical.com/testruns/testRun/81564c2d-702c-4f27-8a1a-a35c783a4915
Crashdump at: https://oil-jenkins.canonical.com/artifacts/81564c2d-702c-4f27-8a1a-a35c783a4915/generated/generated/kubernetes/juju-crashdump-kubernetes-2021-08-03-22.19.27.tar.gz
Test result xml at: https://oil-jenkins.canonical.com/artifacts/81564c2d-702c-4f27-8a1a-a35c783a4915/generated/generated/kubernetes/k8s-suite/test_clusterip_service_endpoint-junit.xml
all tests which hit this bug can be found at: https://solutions.qa.canonical.com/bugs/bugs/bug/1938926
______________________ test_clusterip_service_endpoint ________________________
Traceback (most recent call last):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 125, in test_clusterip_service_endpoint
await cleanup()
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 69, in cleanup
await retry_async_with_timeout(
File "/home/ubuntu/k8s-validation/jobs/integration/utils.py", line 198, in retry_async_with_timeout
if await func(*args):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 34, in is_pod_cleaned
pod = get_pod_yaml()
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 9, in get_pod_yaml
out = sh.kubectl.get("po", o="yaml", selector="app=hello-world")
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 1520, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 784, in __init__
self.wait()
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 841, in wait
self.handle_command_exit_code(exit_code)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 865, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_1:
RAN: /snap/bin/kubectl get po -o yaml --selector=app=hello-world
STDOUT:
apiVersion: v1
items: []
kind: List
metadata:
resourceVersion: ""
selfLink: ""
STDERR:
error: You must be logged in to the server (Unauthorized) |
|
