civetweb does not allow tuning of maximum socket connections

https://github.com/civetweb/civetweb/pull/776

This bug was fixed in the package ceph - 14.2.2-0ubuntu2

---------------
ceph (14.2.2-0ubuntu2) eoan; urgency=medium

  [ Eric Desrochers ]
  * Ensure that daemons are not automatically restarted during package
    upgrades (LP: #1840347):
    - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
      instead of "--no-restart-on-upgrade".
    - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.

  [ Jesse Williamson ]
  * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
    to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
    deployments (LP: #1838109).

  [ James Page ]
  * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
    ensure that required wal and db devices are present before
    activating OSD's (LP: #1828617).

  [ Steve Beattie ]
  * SECURITY UPDATE: RADOS gateway remote denial of service
    - d/p/CVE-2019-10222.patch: rgw: asio: check the remote endpoint
      before processing requests.
    - CVE-2019-10222

 -- James Page <email address hidden> Thu, 29 Aug 2019 13:54:25 +0100

Hello Jesse, or anyone else affected,

Accepted ceph into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/13.2.6-0ubuntu0.19.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Jesse, or anyone else affected,

Accepted ceph into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/12.2.12-0ubuntu0.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Jesse, or anyone else affected,

Accepted ceph into rocky-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:rocky-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-rocky-needed to verification-rocky-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-rocky-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Jesse, or anyone else affected,

Accepted ceph into stein-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:stein-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-stein-needed to verification-stein-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-stein-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

verification-done-bionic
verification-done-disco

Procedure:

* run: strings radosgw | grep max_connections
max_connections
max_connections value "%s" is invalid

run: sudo systemctl restart ceph-radosgw.target
...see no error in logs.
r
* edit /etc/ceph/ceph.conf
change:
    rgw frontends = civetweb port=70
to:
    rgw frontends = civetweb port=70 max_connections=1099329233823823823823832823832832
...see error in logs.

...restart with value in range (say, 1000), no error message.

* in the code there is no codepath setting this value without using the variable, internally: so seeing this behavior is evidence the feature is present.

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ceph - 13.2.6-0ubuntu0.19.04.4

---------------
ceph (13.2.6-0ubuntu0.19.04.4) disco; urgency=medium

  [ Eric Desrochers ]
  * Ensure that daemons are not automatically restarted during package
    upgrades (LP: #1840347):
    - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
      instead of "--no-restart-on-upgrade".
    - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.

  [ James Page ]
  * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
    ensure that required wal and db devices are present before
    activating OSD's (LP: #1828617).

  [ Jesse Williamson ]
  * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
    to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
    deployments (LP: #1838109).

 -- James Page <email address hidden> Fri, 30 Aug 2019 10:10:04 +0100

This bug was fixed in the package ceph - 12.2.12-0ubuntu0.18.04.3

---------------
ceph (12.2.12-0ubuntu0.18.04.3) bionic; urgency=medium

  [ James Page ]
  * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
    ensure that required wal and db devices are present before
    activating OSD's (LP: #1828617).

  [ Jesse Williamson ]
  * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
    to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
    deployments (LP: #1838109).

  [ James Page ]
  * d/p/rgw-gc-use-aio.patch: Cherry pick fix to switch to using AIO for
    garbage collection of objects in the Ceph RADOS Gateway
    (LP: #1838858).

  [ Eric Desrochers ]
  * Ensure that daemons are not automatically restarted during package
    upgrades (LP: #1840347):
    - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
      instead of "--no-restart-on-upgrade".
    - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.

 -- James Page <email address hidden> Fri, 30 Aug 2019 10:11:09 +0100

Hello Jesse, or anyone else affected,

Accepted ceph into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

This bug is still pending verification for UCA pockets.

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ceph - 13.2.6-0ubuntu0.19.04.4~cloud0
---------------

 ceph (13.2.6-0ubuntu0.19.04.4~cloud0) bionic-stein; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 ceph (13.2.6-0ubuntu0.19.04.4) disco; urgency=medium
 .
   [ Eric Desrochers ]
   * Ensure that daemons are not automatically restarted during package
     upgrades (LP: #1840347):
     - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
       instead of "--no-restart-on-upgrade".
     - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.
 .
   [ James Page ]
   * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
     ensure that required wal and db devices are present before
     activating OSD's (LP: #1828617).
 .
   [ Jesse Williamson ]
   * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
     to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
     deployments (LP: #1838109).

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ceph - 13.2.6-0ubuntu0.18.10.3~cloud0
---------------

 ceph (13.2.6-0ubuntu0.18.10.3~cloud0) bionic; urgency=medium
 .
   [ Eric Desrochers ]
   * Ensure that daemons are not automatically restarted during package
     upgrades (LP: #1840347):
     - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
       instead of "--no-restart-on-upgrade".
     - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.
 .
   [ James Page ]
   * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
     ensure that required wal and db devices are present before
     activating OSD's (LP: #1828617).
 .
   [ Jesse Williamson ]
   * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
     to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
     deployments (LP: #1838109).

Presumably we will need a way for the charm to apply this setting if needs be so adding charm-ceph-radosgw.

xenial-queens verified

test output:

root@juju-662fa0-xq-sru-test-7:~# sed -i -r 's/(rgw frontends = civetweb port=70).*/\1 max_connections=10000000000000000000000000000000/g' /etc/ceph/ceph.conf
root@juju-662fa0-xq-sru-test-7:~# systemctl restart ceph-radosgw@rgw.`hostname`
root@juju-662fa0-xq-sru-test-7:~# lsof -i :70
root@juju-662fa0-xq-sru-test-7:~# grep max_conn /var/log/ceph/ceph-client.rgw.juju-662fa0-xq-sru-test-7.log
2019-10-30 11:11:22.085139 7f8c89268000 0 civetweb: 0x55e1294108e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:23.446174 7f961780d000 0 civetweb: 0x55d4401128e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:25.497004 7f3569786000 0 civetweb: 0x555c414398e0: max_connections value "10000000000000000000000000000000" is invalid
root@juju-662fa0-xq-sru-test-7:~# sed -i -r 's/(rgw frontends = civetweb port=70).*/\1 max_connections=1000/g' /etc/ceph/ceph.conf
root@juju-662fa0-xq-sru-test-7:~# systemctl restart ceph-radosgw@rgw.`hostname`
root@juju-662fa0-xq-sru-test-7:~# lsof -i :70
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
radosgw 7663 ceph 37u IPv4 25171 0t0 TCP *:gopher (LISTEN)
root@juju-662fa0-xq-sru-test-7:~# grep max_conn /var/log/ceph/ceph-client.rgw.juju-662fa0-xq-sru-test-7.log
2019-10-30 11:11:22.085139 7f8c89268000 0 civetweb: 0x55e1294108e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:23.446174 7f961780d000 0 civetweb: 0x55d4401128e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:25.497004 7f3569786000 0 civetweb: 0x555c414398e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:26.860549 7fbbe5e42000 0 civetweb: 0x55b8810748e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:11:28.070096 7f093f962000 0 civetweb: 0x55c44a9ac8e0: max_connections value "10000000000000000000000000000000" is invalid
2019-10-30 11:12:15.341958 7f14baf35000 1 mgrc service_daemon_register rgw.juju-662fa0-xq-sru-test-7 metadata {arch=x86_64,ceph_version=ceph version 12.2.12 (1436006594665279fe734b4c15d7e08c13ebd777) luminous (stable),cpu=Intel Xeon E312xx (Sandy Bridge, IBRS update),distro=ubuntu,distro_description=Ubuntu 16.04.6 LTS,distro_version=16.04,frontend_config#0=civetweb port=70 max_connections=1000,frontend_type#0=civetweb,hostname=juju-662fa0-xq-sru-test-7,kernel_description=#195-Ubuntu SMP Tue Oct 1 09:35:25 UTC 2019,kernel_version=4.4.0-166-generic,mem_swap_kb=0,mem_total_kb=2047984,num_handles=1,os=Linux,pid=7663,zone_id=a3fe50ec-b375-4b10-ba6a-8e3bb8a15e89,zone_name=default,zonegroup_id=c35bd079-a42d-4540-acd1-bf77ba2ee5c8,zonegroup_name=default}

Since the others are already released ill mark them as verification-done as well.

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ceph - 12.2.12-0ubuntu0.18.04.3~cloud0
---------------

 ceph (12.2.12-0ubuntu0.18.04.3~cloud0) xenial-queens; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 ceph (12.2.12-0ubuntu0.18.04.3) bionic; urgency=medium
 .
   [ James Page ]
   * d/p/ceph-volume-wait-for-lvs.patch: Cherry pick inflight fix to
     ensure that required wal and db devices are present before
     activating OSD's (LP: #1828617).
 .
   [ Jesse Williamson ]
   * d/p/civetweb-755-1.8-somaxconn-configurable*.patch: Backport changes
     to civetweb to allow tuning of SOMAXCONN in Ceph RADOS Gateway
     deployments (LP: #1838109).
 .
   [ James Page ]
   * d/p/rgw-gc-use-aio.patch: Cherry pick fix to switch to using AIO for
     garbage collection of objects in the Ceph RADOS Gateway
     (LP: #1838858).
 .
   [ Eric Desrochers ]
   * Ensure that daemons are not automatically restarted during package
     upgrades (LP: #1840347):
     - d/rules: Use "--no-restart-after-upgrade" and "--no-stop-on-upgrade"
       instead of "--no-restart-on-upgrade".
     - d/rules: Drop exclusion for ceph-[osd,mon,mds] for restarts.