Volume can't be detached if attachment delete api call fails with 504 gateway timeout
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Undecided
|
Takashi Kajinami | ||
Train |
In Progress
|
Undecided
|
Unassigned | ||
Ussuri |
In Progress
|
Undecided
|
Unassigned | ||
Victoria |
Fix Committed
|
Undecided
|
Unassigned | ||
Wallaby |
Fix Committed
|
Undecided
|
Unassigned | ||
Xena |
Fix Released
|
Undecided
|
Unassigned | ||
Yoga |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
Description
===========
When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout.
When this timeout occurs while detaching a volume, this results in un-detachable volume.
- nova-compute calls delete attachment api in cinder
- haproxy detects server timeout and returns 504
- cinder continues processing the API and removes the attachment
- nova-compute immediately aborts the volume detachment and leaves the bdm
- when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova
See for details https:/
Steps to reproduce
==================
* Stop cinder-volume
* Detach a volume from an instance
* Start cinder-volume
* Detach the volume again
Expected result
===============
* Volume can be detached after cinder-volume is recovered
Actual result
===============
* Volume can't be detached
Environment
===========
* The issue was initially found in stable/train
Logs & Configs
==============
* See https:/
Changed in nova: | |
assignee: | nobody → Takashi Kajinami (kajinamit) |
Fix proposed to branch: master /review. opendev. org/c/openstack /nova/+ /845543
Review: https:/