---[Proposed Solution (morganfainberg)]---
The solution to this bug it so eliminate all SQLite migrations in the unit tests and only test migrations with the functional test suite. This is something that will occur once David Stanek has the full functional test suite up and running.
This also implies that support for SQLite is suspect at best for all use-cases.
---[ Problem Description ]---
Some db migration tests succeed, while the same migrations, applied by db_sync fail. A good example of this is bug #1405673: migration 44 seems to be tested, but failes when applied in real life.
To illustrate the issue, http://paste.openstack.org/show/155081/ can be applied here https://github.com/openstack/keystone/blob/f5c06d619c762d5d39e8ec3bc752215ec02b2756/keystone/tests/test_sql_upgrade.py#L152-L163 . The tests will be very slow, but reliable (because the db will be in file, not in memory) and will fail. Here's the output:
(.venv)breton@bbobrov-pc ~/src/keystone (bug/1405673*) $ ./run_tests.sh SqlUpgradeTests
tools/with_venv.sh
keystone.tests.test_sql_upgrade.SqlUpgradeTests
test_blank_db_to_start OK 0.50
test_downgrade_endpoint_enabled_cols FAIL
test_downgrade_service_enabled_cols OK 20.78
test_add_actor_id_index OK 25.44
test_endpoint_region_downgrade_columns OK 27.31
test_downgrade_endpoint_enabled_data FAIL
test_id_mapping OK 21.70
test_assignment_table_migration OK 25.05
test_limited_trusts_upgrade OK 18.93
test_downgrade_service_enabled_data OK 24.67
test_downgrade_to_db_init_version FAIL
test_project_parent_id_downgrade OK 39.04
test_endpoint_region_migration OK 46.27
test_endpoint_region_upgrade_columns OK 36.07
test_region_url_downgrade OK 36.40
test_limited_trusts_downgrade OK 31.71
test_limited_trusts_downgrade_trusts_cleanup OK 33.84
test_region_url_upgrade OK 38.80
test_project_parent_id_upgrade OK 49.62
test_project_parent_id_cleanup OK 52.02
test_two_steps_forward_one_step_back FAIL
test_region_migration OK 34.42
test_remove_actor_id_index OK 48.16
test_start_version_db_init_version OK 0.95
test_upgrade_endpoint_enabled_cols OK 29.47
test_region_url_cleanup OK 42.90
test_token_user_id_and_trust_id_index_downgrade OK 96.98
test_upgrade_endpoint_enabled_data OK 101.70
test_token_user_id_and_trust_id_index_upgrade OK 125.88
test_upgrade_region_unique_description FAIL
test_upgrade_region_non_unique_description OK 91.90
test_upgrade_add_initial_tables OK 65.13
test_upgrade_service_enabled_data OK 97.24
test_upgrade_service_enabled_cols OK 80.83
[...]
Ran 34 tests in 557.405s
FAILED (failures=5)
I would like to see a move to eliminate SQLite from use in Keystone instead of trying to rely on it to "test" migrations. SQLite does not mirror any real production worthy system and tends to result in weird edge-cases.
Instead of "fixing" these migration tests we should move towards the functional test suite doing tests against MySQL and pgsql (in infra-gate) directly and drop SQLite support completely.
SQL migration tests need to be re-worked as well to mirror how nova handles them for pre-upgrade/