qemu: CVE-2018-15746: seccomp: blacklist is not applied to all threads

Qemu 2.12 on Cosmic

for pid in $(pidof qemu-system-ppc64); do echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; done
PID 158126
Seccomp: 2
Seccomp: 0
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2

Hmm, why isn't this on by default in Bionic as I expect it ...
Anyway, as I thought the feature existed back then as well and users could have turned it on like
  "-sandbox on..."
I checked with that 2.11 in Bionic is also affected.

There is a useful one line reproducer:
$ qemu-system-x86_64 -sandbox on -nographic & pid=$!; sleep 2s; echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; kill -9 $pid

That will report something like
PID 23230
Seccomp: 2
Seccomp: 0

And the two lines should match.

Upstream commit [1]
This will bump the need for libseccomp to >=2.2 which has an update to the configure script [2].

Furthermore I think that the old (whitelist) approach might be able to trigger issues if extended to the other threads (which would be correct but might regress people).
Therefore again pre Bionic qemu needs double checks and consideration.

I'm not entirely sure on severity/priority and implications I'd leave X/T for the consideration of the security Team.

I think we can and should fix Cosmic for sure and I'd start on that now.
I have another Qemu SRU Prepped that I could bundle to unify uploads and testing.

I'll spawn tasks according to that assigning clearly who would work on what.

[1]: https://git.qemu.org/?p=qemu.git;a=commit;h=70dfabeaa79ba4d7a3b699abe1a047c8012db114
[2]: https://git.qemu.org/?p=qemu.git;a=commit;h=d0699bd37c48067cffbd80383172efc29da6d2f9

Subscribed James Page as FYI for cloud archive.
So far I only found qemu version related "be careful" things (like the assumption that the whitelist approach might regress when applied to all threads). But we might spot something more for Xenial which might be important for the UCA builds there - so subscribed for awareness of the discussions here.

Cosmic:
PPA: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3390
Diff: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_39a8dbb93caf4ec889f8a1b7f69885db/bileto-3390/2018-08-29_07:07:17/cosmic_qemu_content.diff

Upgraded a system to the new qemu:
The following packages will be upgraded:
  qemu-kvm qemu-system-ppc
2 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.
Need to get 8421 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://ppa.launchpad.net/ci-train-ppa-service/3390/ubuntu cosmic/main ppc64el qemu-kvm ppc64el 1:2.12+dfsg-3ubuntu5 [12.7 kB]
Get:2 http://ppa.launchpad.net/ci-train-ppa-service/3390/ubuntu cosmic/main ppc64el qemu-system-ppc ppc64el 1:2.12+dfsg-3ubuntu5 [8408 kB]
Fetched 8421 kB in 2s (3411 kB/s)
(Reading database ... 142908 files and directories currently installed.)
Preparing to unpack .../qemu-kvm_1%3a2.12+dfsg-3ubuntu5_ppc64el.deb ...
Unpacking qemu-kvm (1:2.12+dfsg-3ubuntu5) over (1:2.12+dfsg-3ubuntu4) ...
Preparing to unpack .../qemu-system-ppc_1%3a2.12+dfsg-3ubuntu5_ppc64el.deb ...
Unpacking qemu-system-ppc (1:2.12+dfsg-3ubuntu5) over (1:2.12+dfsg-3ubuntu4) ...
Setting up qemu-system-ppc (1:2.12+dfsg-3ubuntu5) ...
Setting up qemu-kvm (1:2.12+dfsg-3ubuntu5) ...
Processing triggers for man-db (2.8.4-2) ...

Manual check:
$ qemu-system-ppc64 -sandbox on -nographic & pid=$!; sleep 2s; echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; kill -9 $pid
[2] 31359

[2]+ Stopped qemu-system-ppc64 -sandbox on -nographic
PID 31359
Seccomp: 2
Seccomp: 2
Seccomp: 2

Libvirt started guest (with extended blakclist like -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny):
$ virsh list
 Id Name State
----------------------------------------------------
 2 b running

[1]+ Killed qemu-system-ppc64 -sandbox on -nographic

$ for pid in $(pidof qemu-system-ppc64); do echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; done
PID 31445
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2

Running regression tests on that PPA now which is slightly more interesting to set up than usual as we also need glibc from proposed to be installable ...

cosmic containers are sort of dead today due to bug 1789627 - so this takes a while

My opinion on how this could go:

- integrate changes for cosmic soon, deploy the blacklist variant as soon as practical. Beg and cajole people to test and report results. (Does this require a feature freeze exception bug?)

- handle bionic and earlier via SRU process -- this feels like a significant regression risk, and the consequences of it could be pretty severe for our users. Not all kernels will log seccomp denials either, making it extremely difficult to track down the root cause of potential regressions.

- we might not want to turn on even the blacklist variant by default in bionic and earlier due to the risk of regressions. We can always turn it on after cosmic has shipped and seen wider use.

Thanks

Hi Seth,
thanks for your thoughts!

Splitting my answers per Release:

== Cosmic ==

For cosmic it needs no FFe IMHO, for already having the Blacklist variant and using it for quite a while. We only extend it to the threads that were missing - in that scope it is only a bug fix.
- There the fix is ready and now also tested in various combinations

stage0-prep-cosmic-CVE-seccomp-run1-x86_64.status : Pass 4 Failed 0 Skip 0 + 0 - RC 0 in 12 minutes
stage1-migrate-cosmic-CVE-seccomp-run1-x86_64.status : Pass 276 Failed 0 Skip 0 + 0 - RC 0 in 62 minutes
stage2-cross-cosmic-CVE-seccomp-run1-x86_64.status : Pass 22 Failed 0 Skip 0 + 1 - RC 0 in 28 minutes
stage3-misc-cosmic-CVE-seccomp-run1-x86_64.status : Pass 103 Failed 0 Skip 0 + 0 - RC 0 in 29 minutes

stage0-prep-cosmic-CVE-seccomp-run1-s390x.status : Pass 3 Failed 0 Skip 0 + 0 - RC 0 in 44 minutes
stage1-migrate-cosmic-CVE-seccomp-run1-s390x.status : Pass 249 Failed 2 Skip 5 + 0 - RC 2 in 531 minutes
stage2-cross-cosmic-CVE-seccomp-run1-s390x.status : Pass 12 Failed 0 Skip 0 + 0 - RC 0 in 178 minutes
stage3-misc-cosmic-CVE-seccomp-run1-s390x.status : Pass 67 Failed 0 Skip 0 + 0 - RC 0 in 95 minutes

stage0-prep-cosmic-CVE-seccomp-run1-ppc64le.status : Pass 2 Failed 0 Skip 0 + 0 - RC 0 in 47 minutes
stage1-migrate-cosmic-CVE-seccomp-run1-ppc64le.status : Pass 276 Failed 0 Skip 0 + 0 - RC 0 in 101 minutes
stage2-cross-cosmic-CVE-seccomp-run1-ppc64le.status : Pass 4 Failed 0 Skip 0 + 0 - RC 0 in 8 minutes
stage3-misc-cosmic-CVE-seccomp-run1-ppc64le.status : Pass 48 Failed 0 Skip 1 + 0 - RC 0 in 20 minutes

The only two fails we see have existed before.
Given all that looks good and we were using it already I'll push that for Cosmic.

== Bionic ==
Bionic is different as I outlined and you also emphasized further.

First of all I'd NOT want to turn on blacklist filtering by default at all there.

But OTOH being not used by default means the only few that use it are those that want to rely on its function. So they would most likely want the fix to be in?

Bionic at least using the blacklist approach already makes this safer than in older relases.
So for Bionic I'd agree to the "prep something and cajole people that are using it already for testing of their cases".
I'll make a PPA ready for that.

The fact that not all kernels log seccomp denials is what makes me feel unsure. That would really be hard to debug.

If we want to go on further than this PPA and actually push something into Bionic depends on
a) positive test feedback
b) feedback at all that the feature is used
c) your security severity estimation if that is needed is high enough

If not a+b+c then I'd keep Bionic untouched.
Would you be able to "cajole the people" once I have a PPA to try?

== Xenial/Trusty ==
Still using the whitelist approach plus risk due to the obvious backport noise and older kernels behaving different makes this too much of a risk IMHO.
So I...

I have made a Bionic version with the fix available for testing in [1].
I'd ask people to test this. Since it changes behavior of a default disabled feature this call applies almost exclusively to people that have set up qemu with "-sandbox=..." or configured libvirt to do so with seccomp_sandbox = 1 in /etc/libvirt/qemu.conf.

@Seth - might I ask you to do a nicely worked call for testing since you so nicely coined it "cajole people to test and report results"?

Depending on the feedback to that we can make a Bionic decision then.

Note: there are plenty of qemu SRUs in flight (one in -proposed, one waiting) this PPA is ahead to all of them to be useful for a while - none of the other changes affect seccomp execution in any way.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3395

This bug was fixed in the package qemu - 1:2.12+dfsg-3ubuntu5

---------------
qemu (1:2.12+dfsg-3ubuntu5) cosmic; urgency=medium

  * d/p/ubuntu/lp-1789551-seccomp-set-the-seccomp-filter-to-all-threads.patch:
    ensure that the seccomp blacklist is applied to all threads (LP: #1789551)
    - CVE-2018-15746

 -- Christian Ehrhardt <email address hidden> Wed, 29 Aug 2018 08:50:36 +0200

Call for testing submitted to ubuntu-devel and ubuntu-server.
=> https://lists.ubuntu.com/archives/ubuntu-devel/2018-September/040483.html
=> https://lists.ubuntu.com/archives/ubuntu-server/2018-September/007740.html

IMHO We have three possible outcomes:
1. people actually use -sandbox in Bionic and verify -> SRU it
2. people actually use -sandbox in Bionic and found it broken -> good that we called for testing
3. people don't use it -> SRU it (not affecting a lot of people and fixing those who are unaware)

lets give this some time to generate feedback (or not)

$ for pid in $(pidof qemu-system-x86_64); do echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; done
PID 21066
Seccomp: 2
Seccomp: 0
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2

$ sudo apt-get update && sudo apt-get dist-upgrade -V
...
The following packages will be upgraded:
   qemu-block-extra (1:2.11+dfsg-1ubuntu7.4 => 1:2.11+dfsg-1ubuntu7.7~ppa1)
   qemu-kvm (1:2.11+dfsg-1ubuntu7.4 => 1:2.11+dfsg-1ubuntu7.7~ppa1)
   qemu-system-common (1:2.11+dfsg-1ubuntu7.4 => 1:2.11+dfsg-1ubuntu7.7~ppa1)
   qemu-system-x86 (1:2.11+dfsg-1ubuntu7.4 => 1:2.11+dfsg-1ubuntu7.7~ppa1)
   qemu-utils (1:2.11+dfsg-1ubuntu7.4 => 1:2.11+dfsg-1ubuntu7.7~ppa1)
5 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
...

$ for pid in $(pidof qemu-system-x86_64); do echo PID $pid; for task in /proc/$pid/task/*; do cat $task/status | grep Secc; done; done
PID 29666
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2
Seccomp: 2

From https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg04892.html

> Starting with libseccomp 2.2.0 and kernel >= 3.17, we can use
> seccomp_attr_set(ctx, > SCMP_FLTATR_CTL_TSYNC, 1) to update the policy
> on all threads.

== Xenial ==

Since Xenial fulfills those requirements, I'd think the QEMU patch would be backportable without too much effort. I haven't look at the code so take this with a grain of salt.

== Trusty ==

Trusty doesn't meet the requirements so addressing it there would probably be more involved and maybe not worth the time investment and the stability risks.

Thanks Simon for your checks, yeah I also set X/T to Won't Fix already.

Did you see my poll: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1789551/comments/10
Since you tested Bionic might I ask - are you an example of a real user of that feature on Bionic?

Yes, I've been using "-sandbox on" since at least Trusty and filled a few bugs about this feature in the past. I've replied to the "call for testing" email you sent but maybe my response is being delayed somehow.

I agree with you that Trusty should be "won't fix" but I would have hope that Xenial would be easier/less risky to fix ;)

On Wed, Sep 5, 2018 at 4:31 PM Simon Déziel <email address hidden>
wrote:

> Yes, I've been using "-sandbox on" since at least Trusty and filled a
> few bugs about this feature in the past. I've replied to the "call for
> testing" email you sent but maybe my response is being delayed somehow.
>

Thanks, I have seen the reply now.
Having feedback so fast is great and an indicator that we really want to
tackle Bionic.

I agree with you that Trusty should be "won't fix" but I would have hope
> that Xenial would be easier/less risky to fix ;)
>

No Xenial isn't easy either :-/
The big change was to switch from a whitelist (which often had issues with
uncommon setups) to a blacklist based approach.
And that change in terms of Ubuntu Releases was Xenial->Bionic.

So my current stance would be, give the ML some time if more people show
up, but considering the Bionic fix more likely to eventually take place.

So to summarize:
- our own testing on Bionic was good
- we had user feedback that it is actually used
- we had positive user test feedback
- further review of the suggested changes did not find issues (as long as the blacklist approach is used as in Bionic)

So I'll prep this for SRU to Bionic.

Also we had it a few more days in cosmic now with slowly testers/early adopters picking up and no one hit any issue (and remember in cosmic it is default on).

Added SRU Template and prepared the packaging of the changes to Bionic.

Thank you all for your participation in the discussions and checks around this.

Uploaded to Bionic-unapproved for SRU Team review

An upload of qemu to bionic-proposed has been rejected from the upload queue for the following reason: "The SRU itself is good but currently the .changes file comes with 2 bugs that se".

Rejection comment got cut: The SRU itself is good but currently the .changes file comes with 2 bugs that seem to be private (LP: #1780773, LP: #1790457). These need to be public for review and SRU tooling.

Hello , or anyone else affected,

Accepted qemu into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.11+dfsg-1ubuntu7.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Tested prior to update:
PID 4732
Seccomp: 2
Seccomp: 0

Update:
$ sudo apt install qemu-block-extra qemu-kvm qemu-system-common qemu-system-x86 qemu-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  grub-pc-bin
Use 'sudo apt autoremove' to remove it.
Suggested packages:
  samba vde2 ovmf
The following packages will be upgraded:
  qemu-block-extra qemu-kvm qemu-system-common qemu-system-x86 qemu-utils
5 upgraded, 0 newly installed, 0 to remove and 68 not upgraded.
Need to get 6782 kB of archives.
After this operation, 8192 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.6 [868 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.6 [661 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.6 [39.0 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 qemu-kvm amd64 1:2.11+dfsg-1ubuntu7.6 [13.2 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.6 [5201 kB]
Fetched 6782 kB in 1s (7023 kB/s)
(Reading database ... 86373 files and directories currently installed.)
Preparing to unpack .../qemu-utils_1%3a2.11+dfsg-1ubuntu7.6_amd64.deb ...
Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.6) over (1:2.11+dfsg-1ubuntu7.5) ...
Preparing to unpack .../qemu-system-common_1%3a2.11+dfsg-1ubuntu7.6_amd64.deb ...
Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.6) over (1:2.11+dfsg-1ubuntu7.5) ...
Preparing to unpack .../qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.6_amd64.deb ...
Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.6) over (1:2.11+dfsg-1ubuntu7.5) ...
Preparing to unpack .../qemu-kvm_1%3a2.11+dfsg-1ubuntu7.6_amd64.deb ...
Unpacking qemu-kvm (1:2.11+dfsg-1ubuntu7.6) over (1:2.11+dfsg-1ubuntu7.5) ...
Preparing to unpack .../qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.6_amd64.deb ...
Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.6) over (1:2.11+dfsg-1ubuntu7.5) ...
Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.6) ...
Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.6) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.6) ...
Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.6) ...
Setting up qemu-kvm (1:2.11+dfsg-1ubuntu7.6) ...

After the update:
PID 5482
Seccomp: 2
Seccomp: 2

Also worked with bigger configurations, setting verified.

This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu7.6

---------------
qemu (1:2.11+dfsg-1ubuntu7.6) bionic; urgency=medium

  [ Christian Ehrhardt ]
  * Add cpu model for z14 ZR1 (LP: #1780773)
  * d/p/ubuntu/lp-1789551-seccomp-set-the-seccomp-filter-to-all-threads.patch:
    ensure that the seccomp blacklist is applied to all threads (LP: #1789551)
    - CVE-2018-15746
  * improve s390x spectre mitigation with etoken facility (LP: #1790457)
    - debian/patches/ubuntu/lp-1790457-s390x-kvm-add-etoken-facility.patch
    - debian/patches/ubuntu/lp-1790457-partial-s390x-linux-headers-update.patch

  [ Phillip Susi ]
  * d/p/ubuntu/lp-1787267-fix-en_us-vnc-pipe.patch: Fix pipe, greater than and
    less than keys over vnc when using en_us kemaps (LP: #1787267).

 -- Christian Ehrhardt <email address hidden> Wed, 29 Aug 2018 11:46:37 +0200

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.