OpenStack Compute (nova)

_is_storage_shared_with() in libvirt/driver.py gives possibly false results if ssh keys not configured

Bug #1459782 reported by Chris Friesen
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Low
Unassigned

Bug Description

In virt.libvirt.driver.LibvirtDriver._is_storage_shared_with() we first check IP addresses and if they don't match then we'll try to use ssh to check whether the storage is actually shared or not.

If ssh keys are not set up between the compute nodes for the user running nova-compute then the call to utils.ssh_execute() will fail and we will return wrong information.

utils.ssh_execute() is also used in _cleanup_remote_migration() and migrate_disk_and_power_off() and would suffer from similar issues there.

Either we need to ensure that the requirement for pre-sharing the ssh keys is clearly documented, or we need to convert these to to use RPC calls like live migration.

melanie witt (melwitt)
tags: added: libvirt
Pawel Koniszewski (pawel-koniszewski)
tags: added: live-migrate
Revision history for this message
melanie witt (melwitt) wrote :

It sounds like this is part of the overall limitation of live migrate requiring ssh connectivity to work.

Revision history for this message
Chris Friesen (cbf123) wrote :

This is actually not part of live-migrate, but rather cold migrate. Live migration uses RPC calls to determine whether storage is shared.

And in any case, live migration doesn't require ssh connectivity to work since by default it is tunnelled via libvirt.

tags: removed: live-migrate
Revision history for this message
melanie witt (melwitt) wrote :

Sorry, I was thinking of resize/cold migration requiring ssh connectivity.

Revision history for this message
Chris Friesen (cbf123) wrote :

Do we document that resize/cold migration requires ssh connectivity? I don't remember reading about that anywhere.

Revision history for this message
melanie witt (melwitt) wrote :

I wasn't aware it was documented but a quick search yielded the Configuration Reference which has a brief explanation:

http://docs.openstack.org/kilo/config-reference/content/configuring-resize.html

Revision history for this message
Chris Friesen (cbf123) wrote :

Interesting, hadn't seen that. The way it's written it sounds like it's just for copying the disks over, but nova actually requires it for shared storage as well.

Markus Zoeller (markus_z) (mzoeller)
Changed in nova:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

Although it is documented right now, I think we should reuse the RPC call mechanic from the live-migration, right? Seems odd to me to have two different ways to determine if there is shared storage.

Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

The documentation issue is also reported in bug 1477296. I'd like to have this one as a bug for the currently two ways of how to determine if it is a shared storage.

Revision history for this message
Chris Friesen (cbf123) wrote :

Based on the code submission comments in commit 9290bdd it appears that it was expected that all the resize/migration code was going to be refactored in Havana to switch to using RPC like live migration. Looks like that never happened. That would fix the immediate issue raised in this bug.

There's a complication though. In virt.libvirt.utils.copy_image() we also rely on passwordless authentication to do either "rsync" or "scp" to copy the image file over when doing cold migration with local storage. So for the case of local storage we'd still need to set up passwordless ssh between compute nodes to handle cold migration.

Revision history for this message
Chris Friesen (cbf123) wrote :

As you mentioned in bug 1477296 the issue of needing ssh was raised on the mailing list at http://lists.openstack.org/pipermail/openstack-dev/2014-February/028137.html

Unfortunately that discussion didn't go very far, but the original post made it clear that dealing with the actual file copy was the main sticking point with getting rid of passwordless ssh completely.

Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

@<whoever-fixes-this-bug>:
The manuals describe the need for SSH for detecting if there is shared storage [1]. This should be updated as soon as this bug here is fixed.

[1] https://review.openstack.org/#/c/219294

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.