Comment 3 for bug 1940527

Workaround by explicitly setting http 1.1 on the k-l-b unit (i'm guessing requests from kubectl-1.24 defaults to http/1.2, which doesn't honor the 'Upgrade' header):

juju run --unit kubeapi-load-balancer/0 -- sed -i -E "s/proxy_buffering\s+off;/proxy_buffering off;\nproxy_http_version 1.1;/g" /var/lib/juju/agents/unit-kubeapi-load-balancer-0/charm/templates/apilb.conf

Once the template is re-rendered (which happens every 5 minutes on update-status), kubectl exec should work again.