_gpgme_io_select breaks when more than 1024 file descriptors are open
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| gpgme1.0 (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
| Focal |
Fix Committed
|
High
|
Colin Watson | ||
| Jammy |
Fix Committed
|
High
|
Colin Watson | ||
Bug Description
[Impact]
Launchpad's buildd-manager is a complex event-driven daemon that manages several hundred builders on production. As such, it can often end up with more than 1024 file descriptors open. It also uses gpgme from time to time to fetch signing keys for PPAs when dispatching builds that involve PPAs.
On focal (and also on jammy), this breaks because `_gpgme_io_select` is implemented using select(2), whose manual page says: "WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applications should instead use poll(2) or epoll(7), which do not suffer this limitation." This is exactly the limitation we're running into, resulting in mysterious "Too many open files" errors despite resource limits being set to more than comfortable values.
Fortunately, this was fixed upstream in https:/
We've been running buildd-manager with a cherry-picked fix from upstream for a few weeks now, and I'd like to get that SRUed back into Ubuntu so that we don't accidentally lose this patch in the event of a security update or similar.
[Test Plan]
William Grant provided this minimal reproducer:
```
root@tmp:~# python3
Python 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gpg
>>> c = gpg.Context()
>>> list(c.keylist())
[]
>>> fs = []
>>> for i in range(1024):
... fs.append(
...
>>> list(c.keylist())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/
key = self.op_
File "/usr/lib/
raise excp
File "/usr/lib/
errorcheck(
File "/usr/lib/
raise GPGMEError(retval, extradata)
gpg.errors.
```
[Where problems could occur]
`_gpgme_io_select` is pretty central to gpgme, so if something is wrong the package will be pretty broken. On the other hand, that also means any glaring problems are likely to be caught by the package's test suite.
| Colin Watson (cjwatson) wrote | #1 |
| Changed in gpgme1.0 (Ubuntu): | |
| status: | New → Fix Released |
| Colin Watson (cjwatson) wrote | #2 |
| Changed in gpgme1.0 (Ubuntu Focal): | |
| assignee: | nobody → Colin Watson (cjwatson) |
| Changed in gpgme1.0 (Ubuntu Jammy): | |
| assignee: | nobody → Colin Watson (cjwatson) |
| Changed in gpgme1.0 (Ubuntu Focal): | |
| status: | New → In Progress |
| importance: | Undecided → High |
| Changed in gpgme1.0 (Ubuntu Jammy): | |
| status: | New → In Progress |
| importance: | Undecided → High |
| Steve Langasek (vorlon) wrote Please test proposed package | #3 |
| Changed in gpgme1.0 (Ubuntu Jammy): | |
| status: | In Progress → Fix Committed |
| tags: | added: verification-needed verification-needed-jammy |
| Changed in gpgme1.0 (Ubuntu Focal): | |
| status: | In Progress → Fix Committed |
| tags: | added: verification-needed-focal |
| Steve Langasek (vorlon) wrote | #4 |
| Ubuntu SRU Bot (ubuntu-sru-bot) wrote Autopkgtest regression report (gpgme1.0/1.16.0-1.2ubuntu4.2) | #5 |
| Ubuntu SRU Bot (ubuntu-sru-bot) wrote Autopkgtest regression report (gpgme1.0/1.13.1-7ubuntu2.2) | #6 |
Hello Colin, or anyone else affected,
Accepted gpgme1.0 into jammy-proposed. The package will build now and be available at https:/
Please help us by testing this new package. See https:/
If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-
Further information regarding the verification process can be found at https:/
N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.