[nailgun] Task 'deploy' remains in DB even if deployment was failed to start

Bug #1354401 reported by Aleksey Kasatkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Aleksey Kasatkin

Bug Description

  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "406"
  build_id: "2014-08-06_11-23-14"
  astute_sha: "99a790ad1b7526cbbd5bf8add0cb2b4e503fccd4"
  fuellib_sha: "513ec5cdcdef74c7419d5bae967b9edc7da8dbd7"
  ostf_sha: "be71965998364bf8e6415bd38b75c84b63aab867"
  nailgun_sha: "f64b06c788e2b92fcb8e678ea6d0c9b86e8d0ab7"
  fuelmain_sha: "124ea87f1ac1c06e27613fe3b31fd5fc6b39e82d"

1. Install 5.0-26
2. Upgrade to 5.1
3. Deploy simple/neutron/cinder (1 ctrl + 1 comp/cind)
4. Update env to 5.0.2 - it's failed
5. Rollback env - it's passed. OSTF tests passed.
6. Deploy button is active
7. Press 'Deploy'
8. Cannot stop env deployment.

The following screenshots shows the results of pressing 'Deploy' after rollback.

Tags: ui upgrade
Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :
Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :

2014-08-08 07:55:46.894 DEBUG [7f09f7f53740] (logger) Request PUT /api/clusters/3/changes from 10.20.0.1:38481 {}

2014-08-08 07:55:46.920 DEBUG [7f09f7f53740] (logger) Response code '400 Bad Request' for PUT /api/clusters/3/changes from 10.20.0.1:38481

Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :

2014-08-08 08:43:12 DEBUG [7f09f7f53740] (logger) Response code '400 Bad Request' for PUT /api/clusters/3/stop_deployment from 10.20.0.1:40025

Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :

[root@fuel tmp]# fuel env
id | status | name | mode | release_id | changes | pending_release_id
---|-------------|--------|-----------|------------|--------------------------------------|-------------------
3 | operational | 3e4rty | multinode | 2 | [{u'node_id': 6, u'name': u'disks'}] | None

Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :
Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :
Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :
Revision history for this message
Aleksey Kasatkin (alekseyk-ru) wrote :

Reproduced on :

  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "414"
  build_id: "2014-08-08_11-23-51"
  astute_sha: "b52910642d6de941444901b0f20e95ebbcb2b2e9"
  fuellib_sha: "d699fc178559e98cfd7d53b58478b46553ffe39e"
  ostf_sha: "e33390c275e225d648b36997460dc29b1a3c20ae"
  nailgun_sha: "5bc33457e5a1f108b071ed0ef2a771ea0b610b22"
  fuelmain_sha: "16c54168143061724e635a20a99545a756725b49"

Changed in fuel:
status: New → Confirmed
Revision history for this message
Kate Pimenova (kpimenova) wrote :

There is an backend bug:
The main problem is that backend starts task even when wrong changes are coming.

It is easy reproducible on fakeUI:
Deploy cluster - you'll see the cluster with disabled "Deploy changes" button. Then you can to try to put any changes to cluster using browser console, or just edit "Deploy changes" button using "inspect element" - remove the "disabled" class, then you can click on this button and start deployment with no changes on the cluster actually.
The result:
PUT http://0.0.0.0:8000/api/clusters/1/changes
400 Bad Request
Response: No changes to deploy
 - so there is an expected behavior.

But! For any case - backend starts an mysterious task.
It is easy visible in browser console:
[{"status": "running", "name": "deploy", "cluster": 1, "result": {}, "progress": 0, "message": null, "id": 27, "uuid": "a078732f-d376-4b7c-838b-41664a85a4b9"}]
UI also are viewing this task - so "progress bar" and "stop deployment" elements are showed for the cluster.

But - backend doesn't see it - So when we trying to stop this task: we'll se the " Nothing to stop - deployment is not running on environment '1'"

Changed in fuel:
assignee: Fuel UI Team (fuel-ui) → Fuel Python Team (fuel-python)
summary: - [ui] Cannot stop deployment if is started with error
+ [nailgun] Task 'deploy' remains in DB even if deployment was failed to
+ start
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Aleksey Kasatkin (alekseyk-ru)
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/114568

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/114568
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=bc9e377dbe010732bc2ba47161ed9d433998e07b
Submitter: Jenkins
Branch: master

commit bc9e377dbe010732bc2ba47161ed9d433998e07b
Author: Aleksey Kasatkin <email address hidden>
Date: Fri Aug 15 18:15:54 2014 +0300

    Delete task 'deploy' before raise exception on deployment start

    Closes-Bug: #1354401

    Change-Id: I5906cbfc094e0d0ca67158ba9fbec233eefb014e

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
pengfei (pengfei-qiu) wrote :

i'm still meeting this problem in fuel 6.1 when i add node to a present environment。 probably because i stopped the deployment several times.
Can anyone tell me how to fixed it temporarily? I don't want to redeploy the environment because it's in use.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.