Implement stable MORef for VMware

Bug #1962771 reported by kiran pawar
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
kiran pawar

Bug Description

The "mo-ref" in VMware is not guaranteed to be stable. It can change after a recovery or simply un-registering and re-registering a vm. So making it stable is often necessary for the vcenter to be able to manage vms which were "lost" somehow.
Possible scenario is, when VMs reside on a ESXi host, which was down, and the ESXi host comes back, but has lost the vm. For the Stable VM refs, we need to restart the agent to clear out the cache. More problematic is the volume-ref as there is no fallback to the instance-uuid and we need to modify the db to reflect the new mo-ref.

Affected versions
vsphere (all), openstack (all).

Steps to reproduce:
For the VM:
1. Create an Instance
2. Stop the instance
3. Unregister the VM in vsphere
4. Register the VM in vsphere
5. Start the instance in openstack
6. An exception is raised

For the volume:
1. Create an instance with vmdk-volume attached
2. Unregister shadow-vm in vcenter
3. Register shadow-vm in vcenter
4. Detach the volume
5. An exception is raised

kiran pawar (kpdev)
Changed in nova:
assignee: nobody → kiran pawar (kiranpawar89)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Submitter: "Zuul (22348)"
Branch: master

commit 03fd208c562214476019ff2d7cc38f95c06e1348
Author: Fabian Wiesel <email address hidden>
Date: Sat Mar 5 09:29:11 2022 +0000

    VMware: Split out VMwareAPISession

    The VMwareAPISession object is not only used by the driver, but in
    practically all modules of vmwareapi. It reduces a bit the scope of
    the driver module itself.

    Partial-Bug: #1962771

    Change-Id: I4094b6031872bd3b5c871b9a82c7e01280a3352d

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Submitter: "Zuul (22348)"
Branch: master

commit 56055ede03fff633020e0f34fa4da5c8457a89c6
Author: Fabian Wiesel <email address hidden>
Date: Sat Mar 5 13:53:18 2022 +0000

    VMware: StableMoRefProxy for moref recovery

    The vmwareapi driver uses Managed-Object references throughout the code
    with the assumption that they are stable. It is however a database id,
    which may change during the runtime of the compute node. e.g. If an
    instance is unregistered and re-registerd in the vcenter, the moref will
    change. By wrapping a moref in a proxy object, with an additional method
    to resolve the openstack object to a moref, we can hide those changes
    from a caller.

    MoRef implementation with closure - should ease the transition to stable
    mo-refs One simply has to pass the search function as a closure to the
    MoRef instance, and the very same method will be called when an
    exception is raised for the stored reference.

    Stable Volume refs - The connection_info['data'] contains the
    managed-object reference (moref) as well as the uuid of the volume.
    When the moref become invalid for some reason, we can recover it by
    searching for the volume-uuid as the `config.instanceUuid` attribute
    of the shadow-vm.

    Stable VM Ref - By encapsulating all the parameters for searching for
    the vm-ref again, we can move the retry logic to the session object,
    where we can try to recover the vm-ref should it result in a
    ManagedObjectNotFound exception.

    Use refs as index for fakedb - It was previously using the object-id
    to lookup an object, meaning that you couldn't pass a newly created
    Managed-object-reference like you could over the vmware-api. Now the
    lookup happens over the ref-id string, and in turn some functions
    were refactored to take that into account.

    Partial-Bug: #1962771

    Change-Id: I2a3ddf95b7fe07630855b06e732f8764efb13e91

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.