[SRU] Missing sqlalchemy-utils dep on ussuri

It seems sqlalchemy-utils may have been removed recently in error https://git.launchpad.net/ubuntu/+source/masakari/tree/debian/changelog?id=4d933765965f3d02cd68c696cc69cf53b7c6390d#n3

Thanks for reporting this Liam. I think I was able to recreate the same issue (focal ussuri):

root@f1:~# grep -A 1 "\[taskflow\]" /etc/masakari/masakari.conf
[taskflow]
connection = sqlite:////var/lib/masakari/masakari.sqlite
root@f1:~# su -s /bin/sh -c 'masakari-manage db sync' masakari
2020-06-10 12:38:48.164 9027 INFO migrate.versioning.api [-] 5 -> 6...
2020-06-10 12:38:48.166 9027 INFO masakari.engine.driver [-] Loading masakari notification driver 'taskflow_driver'
2020-06-10 12:38:48.333 9027 INFO keyring.backend [-] Loading KWallet
2020-06-10 12:38:48.342 9027 INFO keyring.backend [-] Loading SecretService
2020-06-10 12:38:48.345 9027 INFO keyring.backend [-] Loading Windows
2020-06-10 12:38:48.346 9027 INFO keyring.backend [-] Loading chainer
2020-06-10 12:38:48.346 9027 INFO keyring.backend [-] Loading macOS
Invalid input received: No module named 'sqlalchemy_utils'
root@f1:~# apt policy masakari-common
masakari-common:
  Installed: 9.0.0-0ubuntu0.20.04.2
  Candidate: 9.0.0-0ubuntu0.20.04.2
  Version table:
 *** 9.0.0-0ubuntu0.20.04.2 500

And error goes away after install python3-sqlalchemy-utils:

root@f1:~# sudo apt install python3-sqlalchemy-utils
root@f1:~# su -s /bin/sh -c 'masakari-manage db sync' masakari
2020-06-10 12:41:48.709 9204 INFO migrate.versioning.api [-] 5 -> 6...
2020-06-10 12:41:48.712 9204 INFO masakari.engine.driver [-] Loading masakari notification driver 'taskflow_driver'
2020-06-10 12:41:49.022 9204 INFO keyring.backend [-] Loading KWallet
2020-06-10 12:41:49.032 9204 INFO keyring.backend [-] Loading SecretService
2020-06-10 12:41:49.036 9204 INFO keyring.backend [-] Loading Windows
2020-06-10 12:41:49.037 9204 INFO keyring.backend [-] Loading chainer
2020-06-10 12:41:49.038 9204 INFO keyring.backend [-] Loading macOS
2020-06-10 12:41:49.845 9204 INFO migrate.versioning.api [-] done

It looks like sqlalchemy-utils should be a run-time dependency of taskflow:

https://opendev.org/openstack/taskflow/src/branch/master/taskflow/persistence/backends/sqlalchemy/tables.py#L22

I'm not sure why but all of the sqlalchemy dependencies for taskflow are listed in test-requirements.txt. So it looks like it could use a fix upstream as well.

Ok nevermind, they are listed in setup.cfg [extras] in the upstream source. Dropping upstream from this bug.

Removing masakari, never know how to handle that. Bug affects masakari but is not in masakari. Feel free to add back if you want.

Yep thats the traceback I'm seeing.

Charm shows:

2020-06-10 12:45:57 ERROR juju-log amqp:40: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-masakari-0/charm/reactive/masakari_handlers.py", line 50, in init_db
    charm_class.db_sync()
  File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms_openstack/charm/core.py", line 849, in db_sync
    subprocess.check_call(self.sync_cmd)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['masakari-manage', '--config-file', '/etc/masakari/masakari.conf', 'db', 'sync']' returned non-zero exit status 1.

2020-06-10 12:45:57 DEBUG amqp-relation-changed Traceback (most recent call last):
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/charm/hooks/amqp-relation-changed", line 22, in <module>
2020-06-10 12:45:57 DEBUG amqp-relation-changed main()
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/__init__.py", line 74, in main
2020-06-10 12:45:57 DEBUG amqp-relation-changed bus.dispatch(restricted=restricted_mode)
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
2020-06-10 12:45:57 DEBUG amqp-relation-changed _invoke(other_handlers)
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
2020-06-10 12:45:57 DEBUG amqp-relation-changed handler.invoke()
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
2020-06-10 12:45:57 DEBUG amqp-relation-changed self._action(*args)
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/charm/reactive/masakari_handlers.py", line 50, in init_db
2020-06-10 12:45:57 DEBUG amqp-relation-changed charm_class.db_sync()
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/var/lib/juju/agents/unit-masakari-0/.venv/lib/python3.6/site-packages/charms_openstack/charm/core.py", line 849, in db_sync
2020-06-10 12:45:57 DEBUG amqp-relation-changed subprocess.check_call(self.sync_cmd)
2020-06-10 12:45:57 DEBUG amqp-relation-changed File "/usr/lib/python3.6/subp...

We may need to revive the MIR bug for sqlalchemy-utils: pad.lv/1543641

The problem we have now is sqlalchemy-utils is in universe still and tasklfow is in main.

https://pad.lv/1543641

We'll try to pursue the MIR for groovy and get this added as a dependency. For prior releases we can add it as a 'Suggests' dependency as they don't need to be in main.

This is fixed in python-taskflow 4.2.0-0ubuntu2 for groovy.

Uploaded to focal unapproved queue.

I don't understand how adding a Suggests to the package in Focal will make any difference to the user impact. Please could you explain? What problem that the user is facing will adding a Suggests solve, and how?

I think that the core of the issue is that has a runtime depends on sqlalchemy-utils but sqlalchemy-utils isn't yet in main. A MIR has been re-opened for Groovy but sqlalchemy-utils (likely) won't get pushed into main for Focal.

Robie's question was how does adding python3-sqlachemy-utils to "Suggests" actually help users? They can still install python-taskflow (without --install-suggests) and not have python3-sqlalchemy-utils installed and still encounter the original crash.

An upload of python-taskflow to focal-proposed has been rejected from the upload queue for the following reason: "This has been rejected due to unanswered questions about the utility of providing this change as an update.".