Activity log for bug #1584044

Date Who What changed Old value New value Message
2016-05-20 13:04:20 Andrey Volochay bug added bug
2016-05-20 13:26:05 Dmitry Klenov tags area-python
2016-05-20 13:26:38 Dmitry Klenov nominated for series fuel/mitaka
2016-05-20 13:26:38 Dmitry Klenov bug task added fuel/mitaka
2016-05-20 13:26:38 Dmitry Klenov nominated for series fuel/7.0.x
2016-05-20 13:26:38 Dmitry Klenov bug task added fuel/7.0.x
2016-05-20 13:26:38 Dmitry Klenov nominated for series fuel/newton
2016-05-20 13:26:38 Dmitry Klenov bug task added fuel/newton
2016-05-20 13:26:38 Dmitry Klenov nominated for series fuel/8.0.x
2016-05-20 13:26:38 Dmitry Klenov bug task added fuel/8.0.x
2016-05-20 13:26:59 Dmitry Klenov fuel/newton: milestone 10.0
2016-05-20 13:27:02 Dmitry Klenov fuel/mitaka: milestone 9.0
2016-05-20 13:27:06 Dmitry Klenov fuel/8.0.x: milestone 8.0-updates
2016-05-20 13:27:09 Dmitry Klenov fuel/7.0.x: milestone 7.0-updates
2016-05-20 13:27:17 Dmitry Klenov fuel/newton: assignee Fuel Octane (fuel-octane-team)
2016-05-20 13:27:24 Dmitry Klenov fuel/mitaka: assignee Fuel Octane (fuel-octane-team)
2016-05-20 13:27:31 Dmitry Klenov fuel/8.0.x: assignee Fuel Octane (fuel-octane-team)
2016-05-20 13:27:38 Dmitry Klenov fuel/7.0.x: assignee Fuel Octane (fuel-octane-team)
2016-05-20 13:27:40 Dmitry Klenov fuel/newton: importance Undecided High
2016-05-20 13:27:42 Dmitry Klenov fuel/mitaka: importance Undecided High
2016-05-20 13:27:45 Dmitry Klenov fuel/8.0.x: importance Undecided High
2016-05-20 13:27:48 Dmitry Klenov fuel/7.0.x: importance Undecided High
2016-05-20 13:27:51 Dmitry Klenov fuel/7.0.x: status New Confirmed
2016-05-20 13:27:53 Dmitry Klenov fuel/8.0.x: status New Confirmed
2016-05-20 13:27:56 Dmitry Klenov fuel/mitaka: status New Confirmed
2016-05-20 13:27:59 Dmitry Klenov fuel/newton: status New Confirmed
2016-05-23 11:57:24 Alexey Stupnikov tags area-python area-python non-release
2016-06-08 10:19:09 Maksim Malchuk bug task deleted fuel/newton
2016-06-09 14:16:06 Andrey Volochay description Detailed bug description: During upgrade MOS6.0 to MOS8.0 via Octane(stable/8.0), I have faced with following bug. By doing second controller upgrade via Octane (octane upgrade-node <target_node_id> <seed_env_id>), Nailgun returns an error with code 500 2016-05-18 13:01:00.188 ERROR [7f39799c4880] (handlers) MOVE NODE FAILED Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/handlers.py", line 93, in POST upgrade.UpgradeHelper.assign_node_to_cluster(node, cluster) File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/upgrade.py", line 191, in assign_node_to_cluster node, netgroups_id_mapping) File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/objects/adapters.py", line 140, in set_bond_assignment_netgroups_ids node.node, mapping) File "/usr/lib/python2.7/site-packages/nailgun/network/manager.py", line 203, in set_bond_assignment_netgroups_ids netgroups_id_mapping[bond_assignment.network_id] KeyError: 7 Steps to reproduce: 1) Have MOS6.0 environment 2) Upgrade master node (fuel 6.0) to 6.1 and then to 7.0 3) Backup environment settings, following documented process 4) Prepare new master node (fuel 8.0) 5) Restore the backup from third step 5) Upgrade restored environment via Octane 6) Upgrade primary controller via Octane 7) Upgrade DB via Octane 8) Upgrade Control Plane via Octane 9) Upgrade second and third controllers Expected results: Provisioning of second and third controllers is started Actual result: Nailgun returns an error 500 which in turn breaks node upgrade. Reproducibility: Use network templates ans network bonding. Workaround: Comment out following code in nailgin/cluster_upgrade/upgrade.py, when first controller is moved and then restart Nailgun: orig_manager.set_bond_assignment_netgroups_ids( node, netgroups_id_mapping) Description of the environment: Versions of components: MOS8.0, Octane(stable/8.0) Network model: Environment uses network bonding Related projects installed: Octane Additional information: It seems like Nailgun falls at "set_bond_assignment_netgroups_ids" method because the method performs an action, which has been done before. Presumably it happens at assign_node_to_cluster method. 1) assign_node_to_cluster 2) node.update_cluster_assignment(seed_cluster) 3) update_cluster_assignment 4) set_network_template 5) apply_network_template #there is a condition IF, in case when network_template is not absent 6) assign_networks_by_template #there are mentioned actions Detailed bug description:  During upgrade MOS6.0 to MOS8.0 via Octane(stable/8.0), I have faced with following bug.  By doing second controller upgrade via Octane (octane upgrade-node <target_node_id> <seed_env_id>), Nailgun returns an error with code 500 2016-05-18 13:01:00.188 ERROR [7f39799c4880] (handlers) MOVE NODE FAILED Traceback (most recent call last):  File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/handlers.py", line 93, in POST    upgrade.UpgradeHelper.assign_node_to_cluster(node, cluster)  File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/upgrade.py", line 191, in assign_node_to_cluster    node, netgroups_id_mapping)  File "/usr/lib/python2.7/site-packages/nailgun/extensions/cluster_upgrade/objects/adapters.py", line 140, in set_bond_assignment_netgroups_ids    node.node, mapping)  File "/usr/lib/python2.7/site-packages/nailgun/network/manager.py", line 203, in set_bond_assignment_netgroups_ids    netgroups_id_mapping[bond_assignment.network_id] KeyError: 7 Steps to reproduce:  1) Have MOS6.0 environment  2) Upgrade master node (fuel 6.0) to 6.1 and then to 7.0  3) Backup environment settings, following documented process  4) Prepare new master node (fuel 8.0)  5) Restore the backup from third step  5) Upgrade restored environment via Octane  6) Upgrade primary controller via Octane  7) Upgrade DB via Octane  8) Upgrade Control Plane via Octane  9) Upgrade second and third controllers Expected results:  Provisioning of second and third controllers is started Actual result:  Nailgun returns an error 500 which in turn breaks node upgrade. Reproducibility:  Use network templates ans network bonding. Workaround:  Comment out following code in nailgin/cluster_upgrade/upgrade.py, when first controller is moved and then restart Nailgun:    orig_manager.set_bond_assignment_netgroups_ids(       node, netgroups_id_mapping) Description of the environment:  Versions of components: MOS8.0, Octane(stable/8.0)  Network model: Environment uses network bonding  Related projects installed: Octane Additional information:  It seems like Nailgun falls at "set_bond_assignment_netgroups_ids" method because the method performs an action, which has been done before.  Presumably it happens at assign_node_to_cluster method. 1) assign_node_to_cluster 2) node.update_cluster_assignment(seed_cluster) 3) update_cluster_assignment 4) set_network_template 5) apply_network_template #there is a condition IF, in case when network_template is not absent 6) assign_networks_by_template #there are mentioned actions
2016-06-09 14:16:44 Andrey Volochay attachment added network_template.yaml https://bugs.launchpad.net/fuel/+bug/1584044/+attachment/4680452/+files/network_template.yaml
2016-06-15 15:36:27 Nikita Zubkov fuel/8.0.x: assignee Fuel Octane (fuel-octane-team) Nikita Zubkov (zubchick)
2016-07-08 07:39:05 OpenStack Infra fuel: status Confirmed In Progress
2016-07-08 07:39:05 OpenStack Infra fuel: assignee Fuel Octane (fuel-octane-team) Anastasiya (atolochkova)
2016-07-08 09:23:43 Nikita Zubkov fuel/8.0.x: assignee Nikita Zubkov (zubchick)
2016-07-08 10:24:10 Oleksiy Molchanov fuel/8.0.x: assignee Fuel Octane (fuel-octane-team)
2016-07-28 15:16:29 OpenStack Infra fuel: assignee Anastasiya (atolochkova) Nikita Zubkov (zubchick)
2016-07-28 15:27:17 Nikita Zubkov fuel: assignee Nikita Zubkov (zubchick) Anastasiya (atolochkova)
2016-07-29 09:45:32 OpenStack Infra fuel: assignee Anastasiya (atolochkova) Nikita Zubkov (zubchick)
2016-08-15 13:27:58 Denis Meltsaykin fuel/8.0.x: milestone 8.0-updates 8.0-mu-3
2016-08-15 13:28:19 Denis Meltsaykin fuel/8.0.x: status Confirmed In Progress
2016-08-15 13:48:15 Denis Meltsaykin fuel/8.0.x: status In Progress Fix Committed
2016-08-24 21:06:10 OpenStack Infra fuel/mitaka: status Confirmed In Progress
2016-08-24 21:06:10 OpenStack Infra fuel/mitaka: assignee Fuel Octane (fuel-octane-team) Yuriy Taraday (yorik-sar)
2016-08-24 21:29:13 OpenStack Infra tags area-python non-release area-python in-stable-mitaka non-release
2016-08-26 15:51:04 Anton Chevychalov fuel/8.0.x: assignee Fuel Octane (fuel-octane-team) Anton Chevychalov (achevychalov)
2016-09-01 14:33:46 Nikita Zubkov fuel: status In Progress Fix Committed
2016-09-01 14:33:52 Nikita Zubkov fuel/mitaka: status In Progress Fix Committed
2016-09-15 10:04:16 Vadim Rovachev fuel/8.0.x: status Fix Committed Fix Released
2016-11-17 16:28:55 Ilya Kharin fuel/7.0.x: status Confirmed Won't Fix