[octane] nova db sync broken from Juno to Liberty

Bug #1573653 reported by Roman Sokolkov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Sergey Abramov
8.0.x
Won't Fix
High
Andrey Volkov
Mitaka
Fix Committed
High
Ilya Kharin
Newton
Fix Committed
High
Sergey Abramov

Bug Description

Affected versions: 8.0+

Nova bug: https://bugs.launchpad.net/bugs/1511466

nova-manage db sync fails if trying to migrate from Juno to Liberty

Workaround:

      "1) Download https://github.com/openstack/nova/archive/2015.1.3.tar.gz
      "2) Put to upgraded controller into /tmp/2015.1.3.tar.gz
      "3) Execute 'tar xf /tmp/2015.1.3.tar.gz -C /tmp'
      "4) Execute 'PYTHONPATH=/tmp/nova-2015.1.3/ nova-manage db sync'
      "5) Execute 'PYTHONPATH=/tmp/nova-2015.1.3/ nova-manage db migrate_flavor_data'
      "6) Execute 'rm -rf /tmp/nova-2015.1.3 /tmp/2015.1.3.tar.gz'

Dmitry Klenov (dklenov)
tags: added: area-python
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Roman, only N->(N+1) upgrades are supported - you can't skip Kilo: you should do Juno->Kilo, then Kilo->Liberty upgrade instead.

Based on the discussion in https://bugs.launchpad.net/nova/+bug/1511466 my understanding is that the reporter forgot to force data migration of remaining instances (db sync only migrates DB schema and does not touch DB rows for availability sake).

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

migrate_flavor_data was removed in https://review.openstack.org/#/c/196911/

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

This specific error is about Juno -> Liberty upgrade (6.0 -> 8.0), thus, Mitaka and Newton (9.0 and 10.0 respectively) are not affected.

Revision history for this message
Andrey Volkov (avolkov) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (master)

Fix proposed to branch: master
Review: https://review.openstack.org/326446

Changed in fuel:
assignee: Fuel Octane (fuel-octane-team) → Sergey Abramov (sabramov)
status: Invalid → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :

Nova doesn't support skipping releases, you have to upgrade from juno to kilo to liberty.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/nova (openstack-ci/fuel-8.0/liberty)

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Andrey Volkov <email address hidden>
Review: https://review.fuel-infra.org/21824

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (master)

Reviewed: https://review.openstack.org/326446
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=a2671ff513704f13c43b67b0d33491bbc1608389
Submitter: Jenkins
Branch: master

commit a2671ff513704f13c43b67b0d33491bbc1608389
Author: Sergey Abramov <email address hidden>
Date: Tue Jun 7 15:51:11 2016 +0300

    Fix migration in upgrade db from Juno to Liberty

    * patch nova (revert migrate_flavor_data delete)
    * soft delete instance_system_metadata where key = instance_type_id
    * split nova-manage db sync to 2 patch
    * call migrate_flavor_data between nova-manage db sync
    * call nova-manage db expand after sync

    required for upgrade db from 6.0 cluster to 8.0

    Change-Id: I7c3cd2812df7559af800febb34b558731d5bd9e9
    Closes-bug: 1573653

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/334948

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/8.0)

Reviewed: https://review.openstack.org/334948
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=e2b8de8cec2e58b5bdda8ba1a7ae52ebde610fa8
Submitter: Jenkins
Branch: stable/8.0

commit e2b8de8cec2e58b5bdda8ba1a7ae52ebde610fa8
Author: Sergey Abramov <email address hidden>
Date: Tue Jun 7 15:51:11 2016 +0300

    Fix migration in upgrade db from Juno to Liberty

    * patch nova (revert migrate_flavor_data delete)
    * soft delete instance_system_metadata where key = instance_type_id
    * split nova-manage db sync to 2 patch
    * call migrate_flavor_data between nova-manage db sync
    * call nova-manage db expand after sync

    required for upgrade db from 6.0 cluster to 8.0

    Change-Id: I7c3cd2812df7559af800febb34b558731d5bd9e9
    Closes-bug: 1573653
    (cherry picked from commit a2671ff513704f13c43b67b0d33491bbc1608389)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (master)

Fix proposed to branch: master
Review: https://review.openstack.org/351721

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-octane (master)

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/351721
Reason: This patch have to go in stable/mitaka instead of master.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/351728

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/mitaka)
Download full text (17.8 KiB)

Reviewed: https://review.openstack.org/351728
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=3101bf302ea0f2d501fd42a6ae6046c3587ac093
Submitter: Jenkins
Branch: stable/mitaka

commit a05d2ecf13e15c12f10e1517fae425e583e0d714
Author: Nikita Zubkov <email address hidden>
Date: Thu Aug 4 17:51:56 2016 +0300

    fix ssh connection auth

    According to spec [1] service user must use key authentication​ for ssh

    [1] https://github.com/openstack/fuel-specs/blame/ba4504/specs/9.0/fuel-nonroot-openstack-nodes.rst#L61-L64

    Change-Id: I461f659d4f0985283a43ec6a702bdd78af91a072

commit 7140398e501879a4e1f5605f3d1cc99195464850
Author: Sergey Abramov <email address hidden>
Date: Tue Aug 2 12:12:57 2016 +0300

    Setup upgrade levels for each fuel version

    Setup upgrade levels block in nova.conf for each fuel version,
    this is required for backward compatibility on RPC.

    Change-Id: I6ccbc34457d11616b015cdb1e46e733b0b49ce58

commit 9e2a419ac79ad591faee9afe16419bf4497eacad
Author: Vladimir Khlyunev <email address hidden>
Date: Mon Aug 1 21:31:56 2016 +0300

    Fix several typos

    Copypasting is evil, lets remove typos

    Change-Id: Icf4bec1a3de954911a469fa98380af2b6175da08

commit 013a7ded2275245a38ee58289dac2bc8b90fa37e
Author: Sergey Abramov <email address hidden>
Date: Wed Jul 20 16:05:10 2016 +0300

    Host evacuation using nova

    Evacuate instance from compute node using
    nova host-evacuate-live except host_evacuation.sh script

    Change-Id: I01a04cfa1ed2aafce7987e523e6b589dea21fbb0

commit 00a1d0dce1f7ae5db6706a59353403516f7913dc
Author: Sergey Abramov <email address hidden>
Date: Tue Jul 19 17:10:44 2016 +0300

    Change shutoff vm command

    * run nova commands using nova util function
    * host filter using get nova node handle method(
        hostname for fuel < 6.1 and fqdn for newer version)

    Change-Id: Ib5c373714e359982b13910a225787167db7b35e9

commit 49357c98313c5f551b13d0877581c5fb5e108db1
Author: Nikita Zubkov <email address hidden>
Date: Mon Jun 27 18:40:13 2016 +0300

    Add `stderr_log_level` parameter to popen

    Now posible to set logging level in popen for logging stderr

    fuel-bootstrap build comand stderr now logged with INFO level

    Change-Id: Ibaa2be54c68ce7a428e07052c917f54e99b95a5e

commit fb582b271dab4bdd10d27149a77c0e1f6a580b9e
Author: Sergey Abramov <email address hidden>
Date: Thu Jul 21 17:56:41 2016 +0300

    Add run_nova_cmd function

    This is required for running nova command on contoller without
    manipulating manipulating with command.

    It makes code easier and more readable

    Usage:

        from fuelclient.objects import environment as environment_obj

        from octane.util import env
        from octane.util import nova

        env_id = ...
        my_env = environment_obj.Environment(env_id)
        controller = env.get_one_controller(my_env)
        nova_cmd_just_run = [...]
        nova_cmd_with_output = [...]
        nova.run_nova_cmd(nova_cmd_just_run, controller, False)
        result = nova.run_nova_cmd(nova_cmd_w...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.