Support LXD multiple sub-uid mapping

Charm will also need some updates to support configuration of an expanded uid range for suid/guid.

Fix proposed to branch: master
Review: https://review.openstack.org/408072

Changes proposed to LXD charm to enable use of this feature (by extending the idmap ranges for the root user).

The changes as they stand in nova-lxd don't currently function - use of an unscoped key in extra specs for a flavor causes the ComputeCapabilitiesFilter to use them as keys to match to compute hosts - I think we need to switch to lxd: scoped values.

Reviewed: https://review.openstack.org/408164
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=216b7a5cd50bb0ef10edfeb6b551b75b083952ae
Submitter: Jenkins
Branch: stable/mitaka

commit 216b7a5cd50bb0ef10edfeb6b551b75b083952ae
Author: James Page <email address hidden>
Date: Wed Dec 7 15:03:24 2016 +0000

    Switch to using lxd: namespace for extra-specs

    Use of unscoped extra-specs confuses the ComputeCapabilitiesFilter,
    causing all LXD compute hosts to be excluded as targets for
    scheduling of instances.

    Switch supported extra-specs to the lxd: namespace to ensure that
    they are correctly ignored by other parts of Nova, but remain
    visible in the LXD compute driver:

      lxd_isolated -> lxd:isolated
      lxd_privileged_allowed -> lxd:privileged_allowed
      lxd_nested_allowed -> lxd:nested_allowed

    (also fixup branch configuration for stable/mitaka)

    Change-Id: I5ff696769c25639ff28eb029f27c8d22d5769adf
    Closes-Bug: 1648056

Reviewed: https://review.openstack.org/408168
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=1f04f663bc2674f7429f25f698abade4bc0bda2a
Submitter: Jenkins
Branch: stable/newton

commit 1f04f663bc2674f7429f25f698abade4bc0bda2a
Author: James Page <email address hidden>
Date: Wed Dec 7 15:11:23 2016 +0000

    Switch to using lxd: namespace for extra-specs

    Use of unscoped extra-specs confuses the ComputeCapabilitiesFilter,
    causing all LXD compute hosts to be excluded as targets for
    scheduling of instances.

    Switch supported extra-specs to the lxd: namespace to ensure that
    they are correctly ignored by other parts of Nova, but remain
    visible in the LXD compute driver:

      lxd_isolated -> lxd:isolated
      lxd_privileged_allowed -> lxd:privileged_allowed
      lxd_nested_allowed -> lxd:nested_allowed

    (also fixup branch configuration for stable/newton)

    Change-Id: I5ff696769c25639ff28eb029f27c8d22d5769adf
    Closes-Bug: 1648056
    (cherry picked from commit 3d8968140bc53ec2f0199816a9768309e644ea4f)

Reviewed: https://review.openstack.org/408072
Committed: https://git.openstack.org/cgit/openstack/charm-lxd/commit/?id=f0773c995220f400ca6b4d5eca04ad9ee919202a
Submitter: Jenkins
Branch: master

commit f0773c995220f400ca6b4d5eca04ad9ee919202a
Author: James Page <email address hidden>
Date: Wed Dec 7 12:58:23 2016 +0000

    Increase subuid/subgid range for root user

    To support use of distinct subuid/subgid ranges per LXD container,
    the default range for the root user must be increased to support
    > 1 running container in this configuration.

    Increase subuid/subgid range to support 5000 containers with distinct
    ranges. Restart LXD daemon if idmap configuration changes, to ensure
    that the full range of subid's are used.

    Change-Id: I8b87dad736abaffdbd7afac090429790d3b03c96
    Closes-Bug: 1648056

Fix proposed to branch: stable/16.10
Review: https://review.openstack.org/408243

Reviewed: https://review.openstack.org/408243
Committed: https://git.openstack.org/cgit/openstack/charm-lxd/commit/?id=eb44a8949532699e4e4100ca94029dc2b3cd0b34
Submitter: Jenkins
Branch: stable/16.10

commit eb44a8949532699e4e4100ca94029dc2b3cd0b34
Author: James Page <email address hidden>
Date: Wed Dec 7 12:58:23 2016 +0000

    Increase subuid/subgid range for root user

    To support use of distinct subuid/subgid ranges per LXD container,
    the default range for the root user must be increased to support
    > 1 running container in this configuration.

    Increase subuid/subgid range to support 5000 containers with distinct
    ranges. Restart LXD daemon if idmap configuration changes, to ensure
    that the full range of subid's are used.

    (also fix amulet tests for OpenStack Newton).

    Change-Id: I8b87dad736abaffdbd7afac090429790d3b03c96
    Closes-Bug: 1648056
    (cherry picked from commit f0773c995220f400ca6b4d5eca04ad9ee919202a)

Xenial/Mitaka packages consumable from:

  https://launchpad.net/~james-page/+archive/ubuntu/mitaka

until the SRU is accepted.

Xenial/Newton and Yakkety/Newton packages consumable from:

   https://launchpad.net/~james-page/+archive/ubuntu/newton

until the SRU is accepted.

This bug was fixed in the package nova-lxd - 15.0.0~dev44-0ubuntu1

---------------
nova-lxd (15.0.0~dev44-0ubuntu1) zesty; urgency=medium

  * New upstream release (LP: #1648056).

 -- James Page <email address hidden> Thu, 15 Dec 2016 11:30:53 +0000

Hello James, or anyone else affected,

Accepted nova-lxd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova-lxd/13.2.0-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Tested xenial proposed - using the lxd:isolated flavor property resulted in LXD containers on the same host having different subuid/subgid mappings.

The verification of the Stable Release Update for nova-lxd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package nova-lxd - 13.2.0-0ubuntu1

---------------
nova-lxd (13.2.0-0ubuntu1) xenial; urgency=medium

  * New upstream release for Openstack Mitaka (LP: #1649368, #1648056):
    - d/p/*: Dropped, no longer required as included upstream.

 -- Chuck Short <email address hidden> Mon, 12 Dec 2016 13:35:03 -0500

This bug was fixed in the package nova-lxd - 15.0.0~dev44-0ubuntu1~cloud0
---------------

 nova-lxd (15.0.0~dev44-0ubuntu1~cloud0) xenial-ocata; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 nova-lxd (15.0.0~dev44-0ubuntu1) zesty; urgency=medium
 .
   * New upstream release (LP: #1648056).

Hello James, or anyone else affected,

Accepted nova-lxd into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello James, or anyone else affected,

Accepted nova-lxd into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova-lxd/14.1.0-0ubuntu0.16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello James, or anyone else affected,

Accepted nova-lxd into newton-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:newton-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-newton-needed to verification-newton-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-newton-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Verified LXD isolation on yakkety:

$ ps -aef | grep init
165536 1934 1910 0 11:59 ? 00:00:00 /sbin/init
231072 3203 3185 0 11:59 ? 00:00:00 /sbin/init

container init and other processes running in distinct ranges.

Just to ensure complete transparency here; the LXD in yakkety does not support container isolation; the LXD team provide backports of newer stable LXD versions to all supported Ubuntu versions - so I

a) Tested with yakkety LXD

Driver correctly identified that the backend LXD did not support isolation and rejected the scheduling request.

b) Tested on yakkety with the LXD stable PPA (LXD 2.8)

Driver detected the feature and isolated LXD containers as detailed in #21

This bug was fixed in the package nova-lxd - 14.1.0-0ubuntu0.16.10.1

---------------
nova-lxd (14.1.0-0ubuntu0.16.10.1) yakkety; urgency=medium

  * New upstream version for Openstack Newton (LP: #1649304, #1648056).

 -- Chuck Short <email address hidden> Mon, 12 Dec 2016 10:06:01 -0500