periodic checking is needed to stabilize DB state

Bug #1226250 reported by Samuel XJ
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Opinion
Low
Unassigned

Bug Description

We found that glance's DB state may remain inconsistent/confusing without proper periodic checking, in case faults occur.

Below are two examples:

1. When "glance image-create" is issued with "copy-from=http://..." option, the image state transits from "queued" to "saving," before the image is copied from http backend to the glance host. If the exeuction is interrupted at this time, then the image will stay in "saving" state.

While in such case, glance does return an error code to the external user who issued the command, we believe that keeping the image in the "saving" state may confuse other external users. It would be better to differentiate a failed image uploading with the case where an image is being uploaded. A periodic check may help in this case. Alternatively, one can trigger the state transition at the time when the image upload is believed to have failed.

2. If the execution related to a "glance image-delete" request is interrupted, then it is possible that the "status" in DB table "glance.images" has transited to "deleted" but the "deleted" field remains to be False. Since this case clearly indicates the occurance of an error, we think it may be better to periodically check the existence of such inconsistent states, transit "deleted" to True accordingly, and mark the occurance of error probably in a dedicated error DB table.

Revision history for this message
Feilong Wang (flwang) wrote :
Revision history for this message
Samuel XJ (samuel-xj) wrote : Re: [Bug 1226250] Re: periodic checking is needed to stabilize DB state

We think that the two belong to the same category. They are different in
details: cases described in Bug 1226233 require cross-checking states in
database and the image store (in our case, the local filesystem of the
glance host), while cases described in this post do not require such
efforts: the transition from "saving" to "killed" (or some other proper
state) can be triggered at the same time when an error is returned to the
(external) user; and the synchronization between "status=deleted" and
"deleted=True" involves checking in one DB table. Bug 1226233 may lead to
resource leakage and is harder (from our perspective) to be detected/fixed
by an external user/admin.

On Tue, Sep 17, 2013 at 12:50 AM, Fei Long Wang <email address hidden> wrote:

> Is this a dup of https://bugs.launchpad.net/glance/+bug/1226233 ?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1226250
>
> Title:
> periodic checking is needed to stabilize DB state
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/glance/+bug/1226250/+subscriptions
>

Ian Cordasco (icordasc)
Changed in glance:
importance: Undecided → Wishlist
Changed in glance:
importance: Wishlist → Low
Changed in glance:
assignee: nobody → Mohammed Ashraf (mohammed-asharaf)
Changed in glance:
assignee: Mohammed Ashraf (mohammed-asharaf) → nobody
Revision history for this message
Nikhil Komawar (nikhil-komawar) wrote :

I have moved it to Opinion. As per the discussion on the spec https://review.openstack.org/#/c/242682/ , we've come to an agreement that such clean up doesn't belong in Glance.

You are welcome to open a new lite-spec with link to this bug to get wider input.

Changed in glance:
status: New → Opinion
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.