Fuel stuck on stopping

Bug #1403796 reported by Jacolex
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Medium
Fuel Library (Deprecated)
5.1.x
Fix Released
Medium
Fuel Library (Deprecated)
6.0.x
Fix Released
Medium
Fuel Library (Deprecated)
6.1.x
Fix Released
Medium
Fuel Library (Deprecated)

Bug Description

Deployment of controller+Mongo had stucked on installing ubuntu (on the node bootstrap wasn't do anything). I canceled the process but it stucked on "stopping".

Tags: openstack
Revision history for this message
Jacolex (jacolex) wrote :

Fuel 5.1.1

fuel task list

id | status | name | cluster | progress | uuid
---|---------|-----------------|---------|----------|-------------------------------------
3 | running | deploy | 1 | 0 | 6074dd83-fb27-45f2-be32-cf0c898ae401
6 | running | provision | 1 | 0 | 23c552a5-1180-459f-8d0b-b240aaae7064
7 | running | deployment | 1 | 0 | 2cda5ef0-b60e-4c6f-a700-ca3c35eff381
8 | running | stop_deployment | 1 | 0 | 7c1a1971-9031-4ba2-ba9e-b45f560ba58f

tags: added: openstack
Revision history for this message
Tomasz 'Zen' Napierala (tzn) wrote :

Can you please attach diagnostic snapshot?

Changed in fuel:
status: New → Incomplete
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
milestone: none → 6.1
assignee: nobody → Fuel Library Team (fuel-library)
Revision history for this message
Jacolex (jacolex) wrote :

Here it is

Changed in fuel:
status: Incomplete → Confirmed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

The orchestrator logs looks ok:
2014-12-17T12:51:32 ./astute/astute.log:2014-12-17T12:51:32 info: [397] c040abf5-4074-4b01-b9e8-309d94f71679: Starting removing of nodes: ["1", "2", "3", "4", "5"]
2014-12-17T12:52:55 ./astute/astute.log:2014-12-17T12:52:55 info: [397] Starting OS provisioning for nodes: 1,2,3,4,5
2014-12-17T12:58:37 ./astute/astute.log:2014-12-17T12:58:37 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Starting deployment
2014-12-17T13:11:19 ./astute/astute.log:2014-12-17T13:11:19 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Spent 762.142930072 seconds on puppet run for following nodes(uids): 1
2014-12-17T13:11:19 ./astute/astute.log:2014-12-17T13:11:19 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Starting deployment
2014-12-17T13:36:40 ./astute/astute.log:2014-12-17T13:36:40 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Spent 1521.305410579 seconds on puppet run for following nodes(uids): 4,3
2014-12-17T13:36:40 ./astute/astute.log:2014-12-17T13:36:40 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Starting deployment
2014-12-17T13:38:36 ./astute/astute.log:2014-12-17T13:38:36 info: [397] 14b0f7b3-c532-4269-a47c-75927fc236bd: Spent 115.170055345 seconds on puppet run for following nodes(uids): 2,5
2014-12-17T13:38:36 ./astute/astute.log:2014-12-17T13:38:36 info: [397] Started updating no quorum policy for corosync cluster

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

I cannot find any log records regarding the task ids provided at https://bugs.launchpad.net/fuel/+bug/1403796/comments/1. It is strange.

Revision history for this message
Jacolex (jacolex) wrote :

After force stopping and restoring from backup i haven't this issue. I don't know how to reproduce it and what actually happend.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Ok, thank you for the information.
We need more reproducers for this issue, perhaps our QA team could help with that.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Ww still need more reproduces

Revision history for this message
Tatyana Dubyk (tdubyk) wrote :

I've reproduced this bug too. I uses 6.1 (#68 iso)
Error running RPC method stop_deploy_task: wrong number of arguments (0 for 1), trace:
["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:209:in `include?'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:209:in `stop_current_task'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:179:in `stop_deploy_task'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:144:in `dispatch_message'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:103:in `block in dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each_with_index'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:96:in `block in perform_service_job'"]

Revision history for this message
Tatyana Dubyk (tdubyk) wrote :

My configuration:
vcenter as hypervisor, storages - by default, OS by default
env:
1controller
1controller
1controller+cinder

Revision history for this message
Tatyana Dubyk (tdubyk) wrote :
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Thank you. I guess the fix for this issue should be backported despite on the medium priority, so I put it confirmed for affected releases

Revision history for this message
Tatyana Dubyk (tdubyk) wrote :

I've verified it on iso: 6.1 (buld 92) and now its works fine without any stucks.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.