expr-simplify optimization slows click/snap policy compilation

Bug #1383858 reported by Jamie Strandboge
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
AppArmor
Triaged
Medium
Unassigned
apparmor (Ubuntu)
Fix Released
Medium
Jamie Strandboge
apparmor (Ubuntu RTM)
Fix Released
Critical
Jamie Strandboge
click-apparmor (Ubuntu)
Fix Released
Medium
Jamie Strandboge
click-apparmor (Ubuntu RTM)
Fix Released
Critical
Jamie Strandboge

Bug Description

AppArmor has several optimization options that can be used to help speed up policy compiles for certain types of policy. Currently, we are using expr tree simplification option by default, which has dramatic affects on policy compiles for the evince profile. However, with click profiles not using expr tree simplification (ie, adding the '-O no-expr-simplify' option) can improve click policy generation by 44% (375 vs 210 seconds).

On Krillin, the difference is even more substantial: 636 vs 233 seconds (63%).

Short term for rtm is to to use '-O no-expr-simplify' when compiling policy in /var/lib/apparmor/profiles but leave /etc/apparmor.d alone. We can do the same with click-apparmor. Note: the fix for bug #1385947 must be included with this fix.

The long term fix is to adjust expr tree simplification to be more efficient (at least as fast as without) and drop the '-O no-expr-simplify' option.

Justification: apparmor policy recompilation is not expected to happen as part of the normal user experience (see bug #1350598 for a lot of detail) and it is expected to only happen on upgrades from 14.10 to 15.04 or to fix very serious apparmor or apparmor policy bugs. None of these bugs are currently scheduled for OTA. However, *if* we ever need to fix one of these, policy will have to be recompiled.

Choices:
1. do nothing for RTM since policy recompiles are expected to be rare, but do apply this change to 15.04. Policy is expected to be recompiled on upgrades to 15.04 and upgrades would use the new option
2. apply this change in OTA. This is problematic because this change alone will trigger a policy recompilation that would not otherwise be needed. Optionally, this change could accompany a severe bug fix

Risk:
The change consists of a small modification to the apparmor upstart job and a change to the arguments click-apparmor gives to apparmor_parser. The risk assessment is considered low because of the size of the change and the simple test case will immediately indicate if either were applied incorrectly.

Test case:
1. run aa-status | wc -l and note the result
2. install the new apparmor and click-apparmor packages and verify there are no errors during installation
3. reboot
4. run aa-status | wc -l and compare to '1'
5. run 'sudo start apparmor' and make sure it finishes in a few seconds

If they are the same, it indicates the upstart job is properly loading the profiles generated by click apparmor.

While these changes may occur separately, landing them at the same time along with a regenerated custom tarball (for preinstalled policy) will reduce policy recompiles.

Changed in apparmor (Ubuntu):
status: New → Triaged
Changed in click-apparmor (Ubuntu):
status: New → Triaged
Changed in apparmor (Ubuntu):
importance: Undecided → High
Changed in click-apparmor (Ubuntu):
importance: Undecided → High
summary: - use 'no-expr-simplify' parser option for clicks
+ improve click apparmor policy times by using 'no-expr-simplify' parser
+ option for click policy
Changed in apparmor (Ubuntu):
importance: High → Critical
Changed in click-apparmor (Ubuntu):
importance: High → Critical
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in apparmor (Ubuntu):
assignee: nobody → Jamie Strandboge (jdstrand)
tags: added: rtm14
description: updated
description: updated
description: updated
summary: - improve click apparmor policy times by using 'no-expr-simplify' parser
- option for click policy
+ expr-simplify optimization slows click policy compilation
Changed in apparmor:
status: New → Triaged
importance: Undecided → Medium
description: updated
affects: apparmor (Ubuntu) → apparmor (Ubuntu RTM)
affects: click-apparmor (Ubuntu) → click-apparmor (Ubuntu RTM)
Changed in click-apparmor (Ubuntu RTM):
status: Triaged → In Progress
Changed in apparmor (Ubuntu RTM):
status: Triaged → In Progress
Changed in apparmor (Ubuntu):
milestone: none → ubuntu-15.01
Changed in click-apparmor (Ubuntu):
milestone: none → ubuntu-15.01
status: New → Triaged
Changed in apparmor (Ubuntu):
status: New → Triaged
Changed in click-apparmor (Ubuntu):
importance: Undecided → Medium
Changed in apparmor (Ubuntu):
importance: Undecided → Medium
Changed in click-apparmor (Ubuntu):
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in apparmor (Ubuntu):
assignee: nobody → Jamie Strandboge (jdstrand)
tags: added: application-confinement
Revision history for this message
Launchpad Janitor (janitor) wrote : Re: expr-simplify optimization slows click policy compilation

This bug was fixed in the package click-apparmor - 0.2.11.2

---------------
click-apparmor (0.2.11.2) utopic; urgency=medium

  * aa-clickhook: disable expr tree simplification when calling the parser
    since this considerably slows down click policy compiles (LP: #1383858)
 -- Jamie Strandboge <email address hidden> Wed, 22 Oct 2014 14:24:15 -0400

Changed in click-apparmor (Ubuntu):
status: Triaged → Fix Released
Changed in click-apparmor (Ubuntu):
milestone: ubuntu-15.01 → lucid-updates
milestone: lucid-updates → ubuntu-14.10
tags: added: aa-parser
description: updated
tags: added: touch-2014-10-30
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Here is the debdiff for apparmor, in case anyone needs it:

Changed in apparmor (Ubuntu):
status: Triaged → Fix Committed
tags: added: patch
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apparmor - 2.8.98-0ubuntu3

---------------
apparmor (2.8.98-0ubuntu3) vivid; urgency=medium

  * debian/lib/apparmor/functions: disable expr tree simplification for
    /var/lib/apparmor/profiles (LP: #1383858)
  * parser-dont-skip-read-cache-with-optimizations.patch: don't skip read
    cache when specifying '-O' (LP: #1385947)
 -- Jamie Strandboge <email address hidden> Tue, 28 Oct 2014 17:41:08 -0500

Changed in apparmor (Ubuntu):
status: Fix Committed → Fix Released
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apparmor - 2.8.96~2652-0ubuntu5.3

---------------
apparmor (2.8.96~2652-0ubuntu5.3) 14.09; urgency=medium

  * debian/lib/apparmor/functions: disable expr tree simplification for
    /var/lib/apparmor/profiles (LP: #1383858)
  * parser-dont-skip-read-cache-with-optimizations.patch: don't skip read
    cache when specifying '-O' (LP: #1385947)
  * 10-lp1371771.patch, 11-lp1371765.patch: refresh for the above
 -- Jamie Strandboge <email address hidden> Mon, 27 Oct 2014 07:13:42 -0500

Changed in apparmor (Ubuntu RTM):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package click-apparmor - 0.2.11.2

---------------
click-apparmor (0.2.11.2) utopic; urgency=medium

  * aa-clickhook: disable expr tree simplification when calling the parser
    since this considerably slows down click policy compiles (LP: #1383858)
 -- Jamie Strandboge <email address hidden> Wed, 22 Oct 2014 14:24:15 -0400

Changed in click-apparmor (Ubuntu RTM):
status: In Progress → Fix Released
summary: - expr-simplify optimization slows click policy compilation
+ expr-simplify optimization slows click/snap policy compilation
Revision history for this message
Ivan Baldo (ibaldo) wrote :

Hello.
Can you guys take a look at this suggestion please?
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1830502/comments/18
Thanks!!!

Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

This MR for apparmor_parser:

https://gitlab.com/apparmor/apparmor/-/merge_requests/711

helps quite a bit with accelerating the expression optimization, and actually makes it worth enabling them back.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.