Comment 3 for bug 1870201

We're still struggling with this bug in our spread tests. When running spread tests for *craft applications, we occasionally see the following failure:

+ snap install lxd
error: cannot perform the following tasks:
- Run configure hook of "lxd" snap if present (run hook "configure":
-----
cat: /proc/self/attr/current: Permission denied
/snap/lxd/23680/snap/hooks/configure: 5: exec: aa-exec: Permission denied
-----)

The failure is occurring in clean VMs with a stock Ubuntu image (18.04, 20.04, and 22.04). The failure rate for these steps is very low - less than 1% of the time. In our github workflow, we run ~350 spread tests, so the failure of the overall work flow is around 10-15% (although some weeks it has been closer to 50%).

The environment setup is brief. We apply the following steps to the image:

  apt-get install -y snapd
  snap install snapd
  snap wait system seed.loaded

  if [ "$SPREAD_SYSTEM" = "ubuntu-18.04-64" ] || [ "$SPREAD_SYSTEM" = "ubuntu-20.04-64" ]; then
      # Remove lxd and lxd-client deb packages as our implementation (pylxd) does not
      # nicely handle the snap and deb being installed at the same time.
      apt-get remove --purge --yes lxd lxd-client
  fi
  # Install and setup the lxd snap
  snap install lxd

------------------

Attempted solutions

I tried the 3 primary suggestions from this thread (https://forum.snapcraft.io/t/snapped-lxd-has-stopped-working-aa-exec-permission-denied/2356) with no luck:

1. Running snap interfaces (now snap connections)

This was suggested to check if the LXD snap is connected to lxd-support interface. This displays nothing, because the LXD snap is not installed yet.

2. Running snap install core20 and snap revert core20

This gives the error:
```
  + snap revert core20

  error: cannot revert "core20": no revision to revert to
```

3. Running snap install core20 and snap refresh core20

This still produces the same exec: aa-exec: Permission denied error.

I’ve captured the journal of a failure here (https://paste.ubuntu.com/p/XnyFRsvVfH/), with the failing section posted below:

```
oct142128-679423 audit[1763]: AVC apparmor="DENIED" operation="open" profile="snap.lxd.activate" name="/proc/1763/attr/current" pid=1763 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
oct142128-679423 audit[1764]: AVC apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
oct142128-679423 audit[1764]: AVC apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
oct142128-679423 kernel: kauditd_printk_skb: 4 callbacks suppressed
oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:46): apparmor="DENIED" operation="open" profile="snap.lxd.activate" name="/proc/1763/attr/current" pid=1763 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:47): apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
oct142128-679423 kernel: audit: type=1400 audit(1665782994.672:48): apparmor="DENIED" operation="exec" profile="snap.lxd.activate" name="/usr/bin/aa-exec" pid=1764 comm="daemon.activate" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
oct142128-679423 lxd.activate[1744]: The LXD snap was unable to run aa-exec, this usually indicates a LXD sideload.
oct142128-679423 lxd.activate[1744]: When sideloading, make sure to manually connect all interfaces.
oct142128-679423 systemd[1]: snap.lxd.activate.service: Deactivated successfully.
```