Failed to launch nova-compute with zfs backend on Xenial

Bug #1815273 reported by Rikimaru Honjo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
nova-lxd
Fix Committed
Undecided
Unassigned

Bug Description

[Error detail]

I built an openstack environment with nova-lxd by devstack on Ubuntu 16.04 Xenial.
At that time, I add the following settings to my local.conf.

enable_plugin nova-lxd https://git.openstack.org/openstack/nova-lxd
LXD_BACKEND_DRIVER=zfs

As a result, nova-compute output the following error.

Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager [None req-cb7209c1-b444-4069-910b-baa1012bdf04 None None] Error updating resources for node lxd-1.: ProcessExecutionError: Unexpected error while running command.
Feb 09 05:03:55 lxd-1 nova-compute[27109]: Command: sudo nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H -p devstack
Feb 09 05:03:55 lxd-1 nova-compute[27109]: Exit code: 2
Feb 09 05:03:55 lxd-1 nova-compute[27109]: Stdout: u''
Feb 09 05:03:55 lxd-1 nova-compute[27109]: Stderr: u"invalid option 'p'\nusage:\n\tlist [-gHLPv] [-o property[,...]] [-T d|u] [pool] ... [interval [count]]\n\nthe following properties are supported:\n\n\tPROPERTY EDIT VALUES\n\n\tallocated
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager Traceback (most recent call last):
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 7745, in _update_available_resource_for_node
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager startup=startup)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova/nova/compute/resource_tracker.py", line 723, in update_available_resource
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager resources = self.driver.get_available_resource(nodename)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 1069, in get_available_resource
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager local_disk_info = _get_zpool_info(pool_name)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 228, in _get_zpool_info
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager total = _get_zfs_attribute('zpool', 'size')
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 212, in _get_zfs_attribute
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager run_as_root=True)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova/nova/utils.py", line 234, in execute
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager return RootwrapProcessHelper().execute(*cmd, **kwargs)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/opt/stack/nova/nova/utils.py", line 117, in execute
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager return processutils.execute(*cmd, **kwargs)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager cmd=sanitized_cmd)
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager ProcessExecutionError: Unexpected error while running command.
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager Command: sudo nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H -p devstack
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager Exit code: 2
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager Stdout: u''
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager Stderr: u"invalid option 'p'\nusage:\n\tlist [-gHLPv] [-o property[,...]] [-T d|u] [pool] ... [interval [count]]\n\nthe following properties are supported:\n\n\tPROPERTY EDIT
Feb 09 05:03:55 lxd-1 nova-compute[27109]: ERROR nova.compute.manager

[Cause of this error]

"-p" option of zpool command was added at zfs 0.7.0.
But, the latest version of zpool in Xenial is 0.6.5.6.
So this error was occurred.
Users who want to use zfs backend for nova-lxd should install zfs 0.7.0 or higher.

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Hi Rikimaru

The issue is of incompatible nova, nova-lxd and Host Ubuntu versions.

On devstack, you'll need to use a tagged version of nova-lxd (13.x) as the plugin. This will then be compatible with xenial, but might not be compatible with nova in devstack. Devstack is/has moved to bionic where this won't be a problem.

If you're not trying to develop nova + nova-lxd, it might be easier to just use the packages?

On Xenial, we validate that the nova and nova-lxd packages are compatible in the distro archives. The version osf nova-lxd that works on xenial can be found at https://launchpad.net/ubuntu/xenial/+source/nova-lxd

(i.e. nova-lxd 13.0.0.*)

So we support

distro = xenial-mitaka
xenial-pike
xenial-queens

However, please note that bionic is supported as bionic-rocky in distro as well.

Changed in nova-lxd:
status: New → Invalid
Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Marking as invalid as it's not something that will be fixed (i.e. nova-lxd mitaka, pike, queens works on xenial with the associated nova versions).

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova-lxd (master)

Reviewed: https://review.openstack.org/636001
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=6693f08db68626e27033a66e9751952be75ce7bc
Submitter: Zuul
Branch: master

commit 6693f08db68626e27033a66e9751952be75ce7bc
Author: Rikimaru Honjo <email address hidden>
Date: Sat Feb 9 08:13:46 2019 +0000

    Improve description about installing with devstack

    LXD_BACKEND_DRIVER=zfs should be specified in local.conf if LXD
    version is 3.0.
    In addition, LXD_BACKEND_DRIVER=zfs requires zfs 0.7.0 or higher.
    This patch adds those information to README and local.conf.sammple.

    Change-Id: I1692aefd2c4e8daba57629c5f99559ec9593fa5d
    Closes-Bug: #1815272
    Closes-Bug: #1815273

Changed in nova-lxd:
status: Invalid → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.