| 2022-06-16 15:18:21 |
Alexander Balderson |
bug |
|
|
added bug |
| 2022-06-16 15:18:21 |
Alexander Balderson |
attachment added |
|
juju-crashdump-kubernetes-aws-2022-06-15-04.12.04.tar.gz https://bugs.launchpad.net/bugs/1978973/+attachment/5597745/+files/juju-crashdump-kubernetes-aws-2022-06-15-04.12.04.tar.gz |
|
| 2022-06-16 15:19:46 |
Alexander Balderson |
description |
Deploying k8s 1.24 stable with keystone latest/stable one unit stays blocked waiting for auth-webhook token.
There are a bunch of errors in the cdk.master.auth-webhook.log and the juju log on the unit (kuberentes-control-plane_0) where it is trying to refresh secrets and then getting an unauthorized.
[2022-06-15 04:12:23 +0000] [102814] [INFO] Refreshing secrets
[2022-06-15 04:12:23 +0000] [102814] [WARNING] Unable to load secrets (1): error: You must be logged in to the server (Unauthorized)
We do see this pass in the lab though, about half the time. I'm wondering if there is a race where one k8s-CP sets up the tokens and then the other unit doesnt get its token in time before auth is required.
Testrun can be found at:
https://solutions.qa.canonical.com/testruns/testRun/5debbdb7-9e09-4fa8-8abe-5db4c373b969
crashdump at:
https://oil-jenkins.canonical.com/artifacts/5debbdb7-9e09-4fa8-8abe-5db4c373b969/generated/generated/kubernetes-aws/juju-crashdump-kubernetes-aws-2022-06-15-04.12.04.tar.gz
bundle at:
https://oil-jenkins.canonical.com/artifacts/5debbdb7-9e09-4fa8-8abe-5db4c373b969/generated/generated/lma-aws/bundle.yaml
crashdump is also attached |
Deploying k8s 1.24 stable with keystone latest/stable one unit stays blocked waiting for auth-webhook token.
There are a bunch of errors in the cdk.master.auth-webhook.log and the juju log on the unit (kuberentes-control-plane_0) where it is trying to refresh secrets and then getting an unauthorized.
[2022-06-15 04:12:23 +0000] [102814] [INFO] Refreshing secrets
[2022-06-15 04:12:23 +0000] [102814] [WARNING] Unable to load secrets (1): error: You must be logged in to the server (Unauthorized)
We do see this pass in the lab though, about half the time. I'm wondering if there is a race where one k8s-CP sets up the tokens and then the other unit doesnt get its token in time before auth is required.
Testrun can be found at:
https://solutions.qa.canonical.com/testruns/testRun/5debbdb7-9e09-4fa8-8abe-5db4c373b969
crashdump at:
https://oil-jenkins.canonical.com/artifacts/5debbdb7-9e09-4fa8-8abe-5db4c373b969/generated/generated/kubernetes-aws/juju-crashdump-kubernetes-aws-2022-06-15-04.12.04.tar.gz
bundle at:
https://oil-jenkins.canonical.com/artifacts/5debbdb7-9e09-4fa8-8abe-5db4c373b969/generated/generated/lma-aws/bundle.yaml
All occurrences of this bug can be found at:
https://solutions.qa.canonical.com/bugs/bugs/bug/1978973
crashdump is also attached |
|
| 2022-06-23 17:10:49 |
George Kraft |
charm-kubernetes-master: importance |
Undecided |
High |
|
| 2022-06-23 17:10:52 |
George Kraft |
charm-kubernetes-master: assignee |
|
George Kraft (cynerva) |
|
| 2022-06-23 17:10:55 |
George Kraft |
charm-kubernetes-master: status |
New |
In Progress |
|
| 2022-06-23 17:10:57 |
George Kraft |
charm-kubernetes-master: milestone |
|
1.24+ck1 |
|
| 2022-06-24 19:56:38 |
George Kraft |
charm-kubernetes-master: status |
In Progress |
Fix Committed |
|
| 2022-06-24 19:56:52 |
George Kraft |
tags |
cdo-qa foundations-engine |
backport-needed cdo-qa foundations-engine |
|
| 2022-08-01 19:38:04 |
Adam Dyess |
tags |
backport-needed cdo-qa foundations-engine |
cdo-qa foundations-engine |
|
| 2022-08-04 17:59:35 |
Adam Dyess |
charm-kubernetes-master: status |
Fix Committed |
Fix Released |
|
