OpenStack Compute (nova)

db sync command should give user friendly message for invalid 'version' specified

Bug #1546441 reported by Dinesh Bhor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Low
Dinesh Bhor
Glance
In Progress
Low
Dinesh Bhor
OpenStack Compute (nova)
In Progress
Low
Dinesh Bhor
OpenStack Identity (keystone)
Opinion
Wishlist
Unassigned

Bug Description

db sync command should give user friendly message for invalid 'version' specified

The command:

$ nova-manage db sync 11111111111111111111111111111111111111111111111111

LOG:

2016-02-16 01:54:53.908 CRITICAL nova [-] OverflowError: range() result has too many items

2016-02-16 01:54:53.908 TRACE nova Traceback (most recent call last):
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/bin/nova-manage", line 10, in <module>
2016-02-16 01:54:53.908 TRACE nova sys.exit(main())
2016-02-16 01:54:53.908 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 1448, in main
2016-02-16 01:54:53.908 TRACE nova ret = fn(*fn_args, **fn_kwargs)
2016-02-16 01:54:53.908 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 932, in sync
2016-02-16 01:54:53.908 TRACE nova return migration.db_sync(version)
2016-02-16 01:54:53.908 TRACE nova File "/opt/stack/nova/nova/db/migration.py", line 26, in db_sync
2016-02-16 01:54:53.908 TRACE nova return IMPL.db_sync(version=version, database=database)
2016-02-16 01:54:53.908 TRACE nova File "/opt/stack/nova/nova/db/sqlalchemy/migration.py", line 57, in db_sync
2016-02-16 01:54:53.908 TRACE nova version)
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
2016-02-16 01:54:53.908 TRACE nova return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2016-02-16 01:54:53.908 TRACE nova File "<decorator-gen-15>", line 2, in _migrate
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
2016-02-16 01:54:53.908 TRACE nova return f(*a, **kw)
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 345, in _migrate
2016-02-16 01:54:53.908 TRACE nova changeset = schema.changeset(version)
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 82, in changeset
2016-02-16 01:54:53.908 TRACE nova changeset = self.repository.changeset(database, start_ver, version)
2016-02-16 01:54:53.908 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/repository.py", line 224, in changeset
2016-02-16 01:54:53.908 TRACE nova versions = range(int(start) + range_mod, int(end) + range_mod, step)
2016-02-16 01:54:53.908 TRACE nova OverflowError: range() result has too many items
2016-02-16 01:54:53.908 TRACE nova

The command:
$ nova-manage db sync 2147483

LOG:
CRITICAL nova [-] KeyError: <VerNum(315)>

2016-02-16 02:06:15.045 TRACE nova Traceback (most recent call last):
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/bin/nova-manage", line 10, in <module>
2016-02-16 02:06:15.045 TRACE nova sys.exit(main())
2016-02-16 02:06:15.045 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 1448, in main
2016-02-16 02:06:15.045 TRACE nova ret = fn(*fn_args, **fn_kwargs)
2016-02-16 02:06:15.045 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 932, in sync
2016-02-16 02:06:15.045 TRACE nova return migration.db_sync(version)
2016-02-16 02:06:15.045 TRACE nova File "/opt/stack/nova/nova/db/migration.py", line 26, in db_sync
2016-02-16 02:06:15.045 TRACE nova return IMPL.db_sync(version=version, database=database)
2016-02-16 02:06:15.045 TRACE nova File "/opt/stack/nova/nova/db/sqlalchemy/migration.py", line 57, in db_sync
2016-02-16 02:06:15.045 TRACE nova version)
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
2016-02-16 02:06:15.045 TRACE nova return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2016-02-16 02:06:15.045 TRACE nova File "<decorator-gen-15>", line 2, in _migrate
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
2016-02-16 02:06:15.045 TRACE nova return f(*a, **kw)
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 345, in _migrate
2016-02-16 02:06:15.045 TRACE nova changeset = schema.changeset(version)
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 82, in changeset
2016-02-16 02:06:15.045 TRACE nova changeset = self.repository.changeset(database, start_ver, version)
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/repository.py", line 225, in changeset
2016-02-16 02:06:15.045 TRACE nova changes = [self.version(v).script(database, op) for v in versions]
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/repository.py", line 189, in version
2016-02-16 02:06:15.045 TRACE nova return self.versions.version(*p, **k)
2016-02-16 02:06:15.045 TRACE nova File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/version.py", line 163, in version
2016-02-16 02:06:15.045 TRACE nova return self.versions[VerNum(vernum)]
2016-02-16 02:06:15.045 TRACE nova KeyError: <VerNum(315)>
2016-02-16 02:06:15.045 TRACE nova

The command:
$ nova-manage db sync something

LOG:
2016-02-17 00:47:20.287 CRITICAL nova [-] NovaException: version should be an integer

2016-02-17 00:47:20.287 TRACE nova Traceback (most recent call last):
2016-02-17 00:47:20.287 TRACE nova File "/usr/local/bin/nova-manage", line 10, in <module>
2016-02-17 00:47:20.287 TRACE nova sys.exit(main())
2016-02-17 00:47:20.287 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 1448, in main
2016-02-17 00:47:20.287 TRACE nova ret = fn(*fn_args, **fn_kwargs)
2016-02-17 00:47:20.287 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 932, in sync
2016-02-17 00:47:20.287 TRACE nova return migration.db_sync(version)
2016-02-17 00:47:20.287 TRACE nova File "/opt/stack/nova/nova/db/migration.py", line 26, in db_sync
2016-02-17 00:47:20.287 TRACE nova return IMPL.db_sync(version=version, database=database)
2016-02-17 00:47:20.287 TRACE nova File "/opt/stack/nova/nova/db/sqlalchemy/migration.py", line 51, in db_sync
2016-02-17 00:47:20.287 TRACE nova raise exception.NovaException(_("version should be an integer"))
2016-02-17 00:47:20.287 TRACE nova NovaException: version should be an integer
2016-02-17 00:47:20.287 TRACE nova

Same issues are present in glance, cinder and keystone also.

Tags: nova-manage
Dinesh Bhor (dinesh-bhor)
Changed in nova:
assignee: nobody → Dinesh Bhor (dinesh-bhor)
Changed in keystone:
assignee: nobody → Dinesh Bhor (dinesh-bhor)
Changed in glance:
assignee: nobody → Dinesh Bhor (dinesh-bhor)
Changed in cinder:
assignee: nobody → Dinesh Bhor (dinesh-bhor)
Revision history for this message
Sheel Rana (ranasheel2000) wrote :

I could see this issue in cinder

root@sheelrana-VirtualBox:/home/sheelrana/cross_project/openstack-specs# cinder-manage db sync 4324234234
2016-02-17 15:17:13.147 CRITICAL cinder [-] MemoryError

2016-02-17 15:17:13.147 TRACE cinder Traceback (most recent call last):
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/bin/cinder-manage", line 10, in <module>
2016-02-17 15:17:13.147 TRACE cinder sys.exit(main())
2016-02-17 15:17:13.147 TRACE cinder File "/opt/stack/cinder/cinder/cmd/manage.py", line 575, in main
2016-02-17 15:17:13.147 TRACE cinder fn(*fn_args)
2016-02-17 15:17:13.147 TRACE cinder File "/opt/stack/cinder/cinder/cmd/manage.py", line 214, in sync
2016-02-17 15:17:13.147 TRACE cinder return db_migration.db_sync(version)
2016-02-17 15:17:13.147 TRACE cinder File "/opt/stack/cinder/cinder/db/migration.py", line 61, in db_sync
2016-02-17 15:17:13.147 TRACE cinder init_version=init_version)
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/migration.py", line 79, in db_sync
2016-02-17 15:17:13.147 TRACE cinder migration = versioning_api.upgrade(engine, repository, version)
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
2016-02-17 15:17:13.147 TRACE cinder return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2016-02-17 15:17:13.147 TRACE cinder File "<decorator-gen-15>", line 2, in _migrate
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
2016-02-17 15:17:13.147 TRACE cinder return f(*a, **kw)
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 345, in _migrate
2016-02-17 15:17:13.147 TRACE cinder changeset = schema.changeset(version)
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 82, in changeset
2016-02-17 15:17:13.147 TRACE cinder changeset = self.repository.changeset(database, start_ver, version)
2016-02-17 15:17:13.147 TRACE cinder File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/repository.py", line 224, in changeset
2016-02-17 15:17:13.147 TRACE cinder versions = range(int(start) + range_mod, int(end) + range_mod, step)
2016-02-17 15:17:13.147 TRACE cinder MemoryError
2016-02-17 15:17:13.147 TRACE cinder

Changed in cinder:
status: New → Confirmed
importance: Undecided → Low
Sean Dague (sdague)
Changed in nova:
status: New → Confirmed
importance: Undecided → Low
tags: added: nova-manage
Steve Martinelli (stevemar)
Changed in keystone:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/284156

Changed in cinder:
status: Confirmed → In Progress
Morgan Fainberg (mdrnstm)
Changed in keystone:
importance: Low → Wishlist
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/289316

Changed in keystone:
assignee: Dinesh Bhor (dinesh-bhor) → Srushti Gadadare (srushti-gadadare)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Srushti Gadadare (<email address hidden>) on branch: master
Review: https://review.openstack.org/289316

Revision history for this message
Dinesh Bhor (dinesh-bhor) wrote :
Download full text (5.3 KiB)

Hi All,

I have found two more issues related to 'glance-manage db sync'.

Commands:
'glance-manage db sync' has two arguments 'version' and 'current_version'.
If you give 'current_version' greater than the actual current database version it
throws InvalidVersionError exception with traceback.

$ glance-manage db sync 45 56

LOG:
2016-03-11 10:21:37.520 CRITICAL glance [-] InvalidVersionError: 56
2016-03-11 10:21:37.520 TRACE glance Traceback (most recent call last):
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/bin/glance-manage", line 10, in <module>
2016-03-11 10:21:37.520 TRACE glance sys.exit(main())
2016-03-11 10:21:37.520 TRACE glance File "/opt/stack/glance/glance/cmd/manage.py", line 383, in main
2016-03-11 10:21:37.520 TRACE glance return CONF.command.action_fn(*func_args, **func_kwargs)
2016-03-11 10:21:37.520 TRACE glance File "/opt/stack/glance/glance/cmd/manage.py", line 146, in sync
2016-03-11 10:21:37.520 TRACE glance version=current_version)
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/migration.py", line 153, in db_version_control
2016-03-11 10:21:37.520 TRACE glance versioning_api.version_control(engine, repository, version)
2016-03-11 10:21:37.520 TRACE glance File "<decorator-gen-8>", line 2, in version_control
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
2016-03-11 10:21:37.520 TRACE glance return f(*a, **kw)
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/api.py", line 250, in version_control
2016-03-11 10:21:37.520 TRACE glance ControlledSchema.create(engine, repository, version)
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 140, in create
2016-03-11 10:21:37.520 TRACE glance version = cls._validate_version(repository, version)
2016-03-11 10:21:37.520 TRACE glance File "/usr/local/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 161, in _validate_version
2016-03-11 10:21:37.520 TRACE glance raise exceptions.InvalidVersionError(version)
2016-03-11 10:21:37.520 TRACE glance InvalidVersionError: 56
2016-03-11 10:21:37.520 TRACE glance

In case of 'current_version' greater than 0 and less than the current database version it throws DatabaseAlreadyControlledError.
$ glance-manage db sync 100 43

LOG:
2016-03-11 09:17:18.841 CRITICAL glance [-] DatabaseAlreadyControlledError
2016-03-11 09:17:18.841 TRACE glance Traceback (most recent call last):
2016-03-11 09:17:18.841 TRACE glance File "/usr/local/bin/glance-manage", line 10, in <module>
2016-03-11 09:17:18.841 TRACE glance sys.exit(main())
2016-03-11 09:17:18.841 TRACE glance File "/opt/stack/glance/glance/cmd/manage.py", line 353, in main
2016-03-11 09:17:18.841 TRACE glance return CONF.command.action_fn(*func_args, **func_kwargs)
2016-03-11 09:17:18.841 TRACE glance File "/opt/stack/glance/glance/cmd/manage.py", line 117, in sync
2016-03-11 09:17:18.841 TRACE glance version=current_version)
2016-03-11 ...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/296226

Changed in glance:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/296920

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/299171

Nikhil Komawar (nikhil-komawar)
Changed in glance:
importance: Undecided → Low
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (master)

Change abandoned by Sean McGinnis (<email address hidden>) on branch: master
Review: https://review.openstack.org/284156
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/296920
Reason: This patch has been sitting unchanged for more than 12 weeks. I am therefore going to abandon it to keep the nova review queue sane. Please feel free to restore the change if you're still working on it.

Steve Martinelli (stevemar)
Changed in keystone:
status: In Progress → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Steve Martinelli (<email address hidden>) on branch: master
Review: https://review.openstack.org/289316
Reason: no change in many months, i am abandoning this change from the keystone review queue

Revision history for this message
Steve Martinelli (stevemar) wrote :

Is this change still being pursued? Looks like the patches have not been looked at in many months. They all depended on an oslo.db change (https://review.openstack.org/#/c/296209/) which never merged.

Steve Martinelli (stevemar)
Changed in keystone:
status: Confirmed → Incomplete
assignee: Srushti Gadadare (srushti-gadadare) → nobody
Steve Martinelli (stevemar)
Changed in keystone:
status: Incomplete → Opinion
OpenStack Infra (hudson-openstack)
Changed in cinder:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on glance (master)

Change abandoned by Abhishek Kekane (<email address hidden>) on branch: master
Review: https://review.openstack.org/299171
Reason: No update for more than 4 months. Abandoning to clear the review queue. Feel free to reopen.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/cinder 11.0.0.0b3

This issue was fixed in the openstack/cinder 11.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on glance (master)

Change abandoned by Dinesh Bhor (<email address hidden>) on branch: master
Review: https://review.openstack.org/296226

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.