OpenStack Keystone Charm

Allow to disable wsgi socket rotation

Bug #2021550 reported by Hua Zhang on 2023-05-30

This bug affects 2 people

	Status	Importance	Assigned to
Charm Helpers	New	Undecided	Unassigned
OpenStack Ceilometer Charm	Fix Committed	Undecided	Unassigned
OpenStack Cinder Charm	Fix Committed	Wishlist	Unassigned
OpenStack Dashboard Charm	Fix Committed	Wishlist	Unassigned
OpenStack Glance Charm	Invalid	Wishlist	Unassigned
OpenStack Keystone Charm	Fix Committed	Wishlist	Unassigned
OpenStack Neutron API Charm	Invalid	Wishlist	Unassigned
OpenStack Nova Cloud Controller Charm	Fix Committed	Wishlist	Hua Zhang

Bug Description

The lp bug 1863232 introduces a new config WSGISocketRotation that allow to disable the wsgi socket rotation for apache's mod-wsgi, and this option is disabled in apache by default so the default behavior remains consistent with the previous versions. But for charm, there is no config to set whether WSGISocketRotation is turned on or off. Moreover, sometimes allowing customers to disable rotation can solve HTTP 503 issue mentioned in the lp bug 1863232

But this HTTP 503 issue is quite difficult to reproduce, I have made the following effort, but I still cannot reproduce them so far. I also asked the customer to run the following effort on his production env, it also failed to reproduce the problem.

#eg: on nova-cloud-controller/0, run:
cat <<EOF >http-request
GET / HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive

EOF
for i in {1..1000}; do echo $i; date; (cat http-request; sleep 1; cat http-request; sleep 9) | telnet 127.0.0.1 8754 2>&1 | while read line; do echo "$(date +'%T') == $line"; done >>output.txt; done

We did see HTTP 503 issue in the customer's production env, I cannot request the customer to directly modify WSGISocketRotation=Off in his production env to confirm this can fix the problem. However, I believe that since apache has provided a config to disable rotation, charm should also provide a config to allow customers to disable rotation. So, I propose adding the following config to all charms that include apache wsgi.

1, when disable-wsgi-rotaion=True, set 'WSGISocketRotation Off' in apache2

2, when disable-wsgi-rotaion=False, set 'WSGISocketRotation on' in apache2

The config disable-wsgi-rotaion will be set to False in charm side by default as well, this can avoid all potential regressions. Additionally, this config needs to be introduced to all charms that include apache wsgi, such as: horizon, keystone, glance, nova-cloud-controller, neutron-server, cinder etc

Tags:

Hua Zhang (zhhuabj) on 2023-05-30

tags:

added: sts

Alex Kavanagh (ajkavanagh) on 2023-05-30

summary:	- Allow to disable wsgi socket rotaion + Allow to disable wsgi socket rotation
Changed in charm-cinder:
importance:	Undecided → Wishlist
status:	New → Triaged
Changed in charm-openstack-dashboard:
importance:	Undecided → Wishlist
status:	New → Triaged
Changed in charm-glance:
importance:	Undecided → Wishlist
status:	New → Triaged
Changed in charm-keystone:
importance:	Undecided → Wishlist
status:	New → Triaged
Changed in charm-neutron-api:
importance:	Undecided → Wishlist
status:	New → Triaged
Changed in charm-nova-cloud-controller:
importance:	Undecided → Wishlist
status:	New → Triaged

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-06-12: Fix proposed to charm-nova-cloud-controller (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/885836

Changed in charm-nova-cloud-controller:
status:	Triaged → In Progress

Revision history for this message

Hua Zhang (zhhuabj) wrote on 2023-06-12:

charm-helpers PR is - https://github.com/juju/charm-helpers/pull/801

Changed in charm-nova-cloud-controller:
assignee:	nobody → Hua Zhang (zhhuabj)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-06-19: Fix proposed to charm-cinder (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-cinder/+/886356

Changed in charm-cinder:
status:	Triaged → In Progress

Hua Zhang (zhhuabj) on 2023-06-19

Changed in charm-glance:
status:	Triaged → Invalid
Changed in charm-neutron-api:
status:	Triaged → Invalid

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-06-19: Fix proposed to charm-openstack-dashboard (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/886373

Changed in charm-openstack-dashboard:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-06-19: Fix proposed to charm-glance (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-glance/+/886376

Changed in charm-glance:
status:	Invalid → In Progress
Changed in charm-keystone:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-06-19: Fix proposed to charm-keystone (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-keystone/+/886377

Revision history for this message

Hua Zhang (zhhuabj) wrote on 2023-07-06:

for ceilometer charm - https://review.opendev.org/c/openstack/charm-ceilometer/+/887793

OpenStack Infra (hudson-openstack) on 2023-09-04

Changed in charm-ceilometer:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-04: Change abandoned on charm-glance (master)

Change abandoned by "Zhang Hua <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-glance/+/886376
Reason: abandoning it because glance don't need this patch accourding to https://github.com/juju/charm-helpers/pull/801#issuecomment-1623502522

$ grep -r 'WSGIWorkerConfigContext' glance
glance/charmhelpers/contrib/openstack/context.py:class WSGIWorkerConfigContext(WorkerConfigContext):

Hua Zhang (zhhuabj) on 2023-09-04

Changed in charm-glance:
status:	In Progress → Invalid

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-05: Fix merged to charm-nova-cloud-controller (master)

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/885836
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/e0d3b6a36553f3627c86bdef420fd55e24e9a0b5
Submitter: "Zuul (22348)"
Branch: master

commit e0d3b6a36553f3627c86bdef420fd55e24e9a0b5
Author: zhhuabj <email address hidden>
Date: Mon Sep 4 18:15:31 2023 +0800

Support disabling apache wsgi socket rotation

    Bug LP 1863232 introduced a new Apache configuration option called
    WSGISocketRotation which allows users to disable wsgi socket
    rotation. This patch makes this configurable with a new
    wsgi-socket-rotation config option that defaults to the Apache
    default and can optionally be set to False.

Closes-Bug: #2021550
Change-Id: I85d48b646b3d0778aae3a7dd3942ad45b4af002d

Changed in charm-nova-cloud-controller:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-05: Fix merged to charm-keystone (master)

#10

Reviewed: https://review.opendev.org/c/openstack/charm-keystone/+/886377
Committed: https://opendev.org/openstack/charm-keystone/commit/b4ee292bb60c2558c6adc98e21545713a329fbf2
Submitter: "Zuul (22348)"
Branch: master

commit b4ee292bb60c2558c6adc98e21545713a329fbf2
Author: zhhuabj <email address hidden>
Date: Mon Sep 4 18:33:51 2023 +0800

Support disabling apache wsgi socket rotation

Closes-Bug: #2021550
Change-Id: Ia5852c3ebe84bd0355670f262cbe5e1cd433a08d

Changed in charm-keystone:
status:	In Progress → Fix Committed
Changed in charm-openstack-dashboard:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-05: Fix merged to charm-openstack-dashboard (master)

#11

Reviewed: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/886373
Committed: https://opendev.org/openstack/charm-openstack-dashboard/commit/5c34112030e8961d67b4718eb563cbe501eeed31
Submitter: "Zuul (22348)"
Branch: master

commit 5c34112030e8961d67b4718eb563cbe501eeed31
Author: zhhuabj <email address hidden>
Date: Mon Sep 4 18:30:45 2023 +0800

Support disabling apache wsgi socket rotation

Closes-Bug: #2021550
Change-Id: I671f4b7f655f12cc558fc64116e31f16560dd2e7

Changed in charm-cinder:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-05: Fix merged to charm-cinder (master)

#12

Reviewed: https://review.opendev.org/c/openstack/charm-cinder/+/886356
Committed: https://opendev.org/openstack/charm-cinder/commit/8b366fbf6b56373958b9579faf98bff898e62378
Submitter: "Zuul (22348)"
Branch: master

commit 8b366fbf6b56373958b9579faf98bff898e62378
Author: zhhuabj <email address hidden>
Date: Mon Sep 4 18:41:43 2023 +0800

Support disabling apache wsgi socket rotation

Closes-Bug: #2021550
Change-Id: Ieca45a470e7f827194780b2fee9f8c6049b94da3

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-cinder (stable/ussuri)

#13

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-cinder/+/897755

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-keystone (stable/ussuri)

#14

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-keystone/+/897756

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-nova-cloud-controller (stable/ussuri)

#15

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/897757

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-openstack-dashboard (stable/ussuri)

#16

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/897758

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-cinder (stable/yoga)

#17

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-cinder/+/897760

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-keystone (stable/yoga)

#18

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-keystone/+/897761

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-nova-cloud-controller (stable/yoga)

#19

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/897762

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-openstack-dashboard (stable/yoga)

#20

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/897763

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-cinder (stable/zed)

#21

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-cinder/+/897764

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-keystone (stable/zed)

#22

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-keystone/+/897765

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-nova-cloud-controller (stable/zed)

#23

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/897766

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-10: Fix proposed to charm-openstack-dashboard (stable/zed)

#24

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/897767

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-keystone (stable/2023.1)

#25

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-keystone/+/898539

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-nova-cloud-controller (stable/2023.1)

#26

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/898540

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-cinder (stable/2023.1)

#27

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-cinder/+/898541

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-openstack-dashboard (stable/2023.1)

#28

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-openstack-dashboard/+/898542

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-ceilometer (stable/2023.1)

#29

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-ceilometer/+/898545

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-ceilometer (stable/zed)

#30

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-ceilometer/+/898546

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-ceilometer (stable/yoga)

#31

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-ceilometer/+/898547

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-17: Fix proposed to charm-ceilometer (stable/ussuri)

#32

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-ceilometer/+/898548

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-18: Fix merged to charm-ceilometer (master)

#33

Reviewed: https://review.opendev.org/c/openstack/charm-ceilometer/+/887793
Committed: https://opendev.org/openstack/charm-ceilometer/commit/a4ffe0e28c997c8bd19f1bcbf9501a86b31701e3
Submitter: "Zuul (22348)"
Branch: master

commit a4ffe0e28c997c8bd19f1bcbf9501a86b31701e3
Author: zhhuabj <email address hidden>
Date: Tue Oct 17 18:13:35 2023 +0800

Support disabling apache wsgi socket rotation

Closes-Bug: #2021550
Change-Id: Ia50067e1f64a6003d5c2ca2ebc22903a1ab8d3ae

Changed in charm-ceilometer:
status:	In Progress → Fix Committed

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.