Activity log for bug #1190986

Date Who What changed Old value New value Message
2013-06-14 12:15:50 Konrad Meier bug added bug
2013-06-17 01:52:06 Julian Edwards maas: status New Incomplete
2013-06-17 08:34:33 Konrad Meier attachment added maas-logfiles.tar.gz https://bugs.launchpad.net/maas/+bug/1190986/+attachment/3705410/+files/maas-logfiles.tar.gz
2013-06-18 01:18:32 Julian Edwards maas: status Incomplete Triaged
2013-06-18 01:18:34 Julian Edwards maas: importance Undecided Low
2013-10-03 21:43:14 David Britton tags theme-oil
2013-10-04 13:33:02 Gavin Panella maas: importance Low High
2013-10-07 15:54:19 Raphaël Badin maas: assignee Raphaël Badin (rvb)
2013-10-07 15:54:20 Raphaël Badin maas: status Triaged In Progress
2013-10-11 12:08:40 Raphaël Badin branch linked lp:~rvb/maas/nonce-bug-1190986-tasks
2013-10-11 12:08:59 Raphaël Badin branch linked lp:~rvb/maas/nonce-bug-1190986
2013-10-14 09:14:31 Raphaël Badin maas: status In Progress Fix Committed
2013-10-16 10:00:16 James Page bug task added maas (Ubuntu)
2013-10-16 10:00:23 James Page nominated for series Ubuntu Saucy
2013-10-16 10:00:23 James Page bug task added maas (Ubuntu Saucy)
2013-10-17 11:41:15 James Page maas (Ubuntu Saucy): status New Triaged
2013-10-17 11:41:19 James Page maas (Ubuntu Saucy): importance Undecided Medium
2014-01-16 00:17:12 Julian Edwards maas: milestone 14.04
2014-04-17 05:16:39 Julian Edwards maas: status Fix Committed Fix Released
2014-07-11 05:43:51 Launchpad Janitor branch linked lp:~lutostag/maas/1.5+nonce-generation-fix
2014-07-11 06:02:16 Launchpad Janitor branch linked lp:~lutostag/maas/trunk+nonce-generation-fix
2014-07-15 02:43:24 Julian Edwards nominated for series maas/1.5
2014-07-15 02:43:24 Julian Edwards bug task added maas/1.5
2014-07-15 02:43:24 Julian Edwards nominated for series maas/1.6
2014-07-15 02:43:24 Julian Edwards bug task added maas/1.6
2014-07-15 02:43:24 Julian Edwards nominated for series maas/trunk
2014-07-15 02:43:24 Julian Edwards bug task added maas/trunk
2014-07-15 02:43:39 Julian Edwards maas/trunk: assignee Raphaël Badin (rvb)
2014-07-15 02:43:41 Julian Edwards maas/trunk: status Fix Released In Progress
2014-07-15 02:43:48 Julian Edwards maas/trunk: milestone 14.04 1.6.1
2014-07-15 02:43:56 Julian Edwards maas/1.6: milestone 1.6.0
2014-07-15 02:44:00 Julian Edwards maas/1.5: milestone 1.5.4
2014-07-15 02:44:06 Julian Edwards maas/1.5: status New In Progress
2014-07-15 02:44:08 Julian Edwards maas/1.6: status New In Progress
2014-07-15 02:44:13 Julian Edwards maas/1.6: assignee Julian Edwards (julian-edwards)
2014-07-15 02:44:15 Julian Edwards maas/1.5: assignee Julian Edwards (julian-edwards)
2014-07-15 02:44:25 Julian Edwards maas/1.5: importance Undecided High
2014-07-15 02:44:28 Julian Edwards maas/1.6: importance Undecided High
2014-07-15 02:44:39 Julian Edwards maas/trunk: assignee Julian Edwards (julian-edwards)
2014-07-15 02:45:31 Launchpad Janitor branch linked lp:~julian-edwards/maas/nonce-fix-bug-1190986
2014-07-15 02:57:04 MAAS Lander maas/trunk: status In Progress Fix Committed
2014-07-15 03:07:12 Launchpad Janitor branch linked lp:~julian-edwards/maas/1.6-nonce-fix-bug-1190986
2014-07-15 03:23:43 Launchpad Janitor branch linked lp:~julian-edwards/maas/1.5
2014-07-15 03:31:52 MAAS Lander maas/1.5: status In Progress Fix Committed
2014-07-15 03:33:09 MAAS Lander maas/1.6: status In Progress Fix Committed
2014-07-30 16:08:22 Julian Edwards maas/1.5: status Fix Committed Fix Released
2014-07-30 16:08:28 Julian Edwards maas/trunk: milestone 1.6.1
2014-07-30 16:08:31 Julian Edwards maas/trunk: status Fix Committed Fix Released
2014-08-20 00:34:14 Julian Edwards maas/1.6: status Fix Committed Fix Released
2014-08-20 22:45:33 Andres Rodriguez bug task added juju-core
2014-08-20 22:45:49 Andres Rodriguez bug task added juju (Ubuntu)
2014-08-21 00:10:09 Ian Booth juju-core: milestone 1.21-alpha1
2014-08-21 00:10:28 Ian Booth juju-core: milestone 1.21-alpha1 1.20.6
2014-08-21 00:10:38 Ian Booth juju-core: importance Undecided High
2014-08-21 00:10:47 Ian Booth juju-core: status New Triaged
2014-08-22 00:51:04 Ian Booth juju-core: assignee Ian Booth (wallyworld)
2014-08-22 00:51:12 Ian Booth juju-core: status Triaged In Progress
2014-08-22 04:40:50 Ian Booth juju-core: status In Progress Fix Committed
2014-08-27 05:50:07 Launchpad Janitor juju (Ubuntu): status New Confirmed
2014-08-27 05:50:07 Launchpad Janitor juju (Ubuntu Saucy): status New Confirmed
2014-08-28 21:19:56 Aaron Bentley juju-core: status Fix Committed Fix Released
2014-08-29 19:53:25 Launchpad Janitor branch linked lp:ubuntu/trusty-proposed/maas
2014-08-29 20:05:27 Greg Lutostanski description The error below happens sometimes. (~5% of all requests fail) System: Ubuntu 12.04.2 MAAS-version: 1.2+bzr1373+dfsg-0ubuntu1~12.04.1 Juju-version: 0.7+bzr628+bzr631~precise1 $ juju status 2013-06-14 13:59:36,213 INFO Connecting to environment... 2013-06-14 13:59:36,295 ERROR Cannot connect to environment: Nonce already used: 55941407 Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/juju/providers/common/connect.py", line 43, in run client = yield self._internal_connect(share) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/lib/python2.7/dist-packages/juju/providers/common/connect.py", line 59, in _internal_connect candidates = yield self._provider.get_zookeeper_machines() File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/lib/python2.7/dist-packages/juju/providers/common/findzookeepers.py", line 32, in find_zookeepers machine = yield provider.get_machine(instance_id) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/lib/python2.7/dist-packages/juju/providers/maas/provider.py", line 125, in get_machines instances = yield self.maas_client.get_nodes(instance_ids) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 545, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/lib/python2.7/dist-packages/juju/providers/maas/maas.py", line 95, in _process_error raise ProviderError(error.response) ProviderError: Nonce already used: 55941407 2013-06-14 13:59:36,297 ERROR Nonce already used: 55941407 I think the problem is caused by the MAAS-database table "piston_nonce" In my system this table contains a hughe amount of tokens that are never deleted: maasdb=# SELECT count(id) FROM piston_nonce; count --------- 3619099 (1 row) SRU for secondary fix (branch lp:~julian-edwards/maas/nonce-fix-bug-1190986) [Impact] * Minimize possibility of hitting a nonce collision by using a longer nonce (uuid4) in both python-maas-client and commissioning scripts. [Testcase] * Do some api calls and make sure that the piston_nonce postgres table is using the larger uuid nonce for the calls [Regression Potential] * Minimal; replace random string of length 8 with uuidgen. The error below happens sometimes. (~5% of all requests fail) System: Ubuntu 12.04.2 MAAS-version: 1.2+bzr1373+dfsg-0ubuntu1~12.04.1 Juju-version: 0.7+bzr628+bzr631~precise1 $ juju status 2013-06-14 13:59:36,213 INFO Connecting to environment... 2013-06-14 13:59:36,295 ERROR Cannot connect to environment: Nonce already used: 55941407 Traceback (most recent call last):   File "/usr/lib/python2.7/dist-packages/juju/providers/common/connect.py", line 43, in run     client = yield self._internal_connect(share)   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks     result = result.throwExceptionIntoGenerator(g)   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator     return g.throw(self.type, self.value, self.tb)   File "/usr/lib/python2.7/dist-packages/juju/providers/common/connect.py", line 59, in _internal_connect     candidates = yield self._provider.get_zookeeper_machines()   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks     result = result.throwExceptionIntoGenerator(g)   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator     return g.throw(self.type, self.value, self.tb)   File "/usr/lib/python2.7/dist-packages/juju/providers/common/findzookeepers.py", line 32, in find_zookeepers     machine = yield provider.get_machine(instance_id)   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks     result = result.throwExceptionIntoGenerator(g)   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator     return g.throw(self.type, self.value, self.tb)   File "/usr/lib/python2.7/dist-packages/juju/providers/maas/provider.py", line 125, in get_machines     instances = yield self.maas_client.get_nodes(instance_ids)   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 545, in _runCallbacks     current.result = callback(current.result, *args, **kw)   File "/usr/lib/python2.7/dist-packages/juju/providers/maas/maas.py", line 95, in _process_error     raise ProviderError(error.response) ProviderError: Nonce already used: 55941407 2013-06-14 13:59:36,297 ERROR Nonce already used: 55941407 I think the problem is caused by the MAAS-database table "piston_nonce" In my system this table contains a hughe amount of tokens that are never deleted: maasdb=# SELECT count(id) FROM piston_nonce;   count ---------  3619099 (1 row)
2014-08-29 20:05:35 Greg Lutostanski summary ERROR Nonce already used [SRU] ERROR Nonce already used
2014-09-02 14:04:20 Chris J Arges nominated for series Ubuntu Trusty
2014-09-02 14:04:20 Chris J Arges bug task added juju (Ubuntu Trusty)
2014-09-02 14:04:20 Chris J Arges bug task added maas (Ubuntu Trusty)
2014-09-02 14:11:41 Greg Lutostanski maas (Ubuntu Trusty): importance Undecided High
2014-09-02 14:11:41 Greg Lutostanski maas (Ubuntu Trusty): status New Fix Committed
2014-09-02 14:11:41 Greg Lutostanski maas (Ubuntu Trusty): milestone trusty-updates
2014-09-02 14:16:32 Chris J Arges maas (Ubuntu): status Triaged Fix Released
2014-09-02 14:16:57 Chris J Arges maas (Ubuntu Saucy): status Triaged Won't Fix
2014-09-03 14:33:30 Chris J Arges bug added subscriber Ubuntu Stable Release Updates Team
2014-09-03 14:33:38 Chris J Arges bug added subscriber SRU Verification
2014-09-03 14:33:41 Chris J Arges tags theme-oil theme-oil verification-needed
2014-09-09 17:48:48 Launchpad Janitor juju (Ubuntu Trusty): status New Confirmed
2014-09-30 18:26:11 Andres Rodriguez tags theme-oil verification-needed theme-oil verification-done
2014-09-30 20:18:03 Launchpad Janitor maas (Ubuntu Trusty): status Fix Committed Fix Released
2014-12-05 07:07:40 Rolf Leggewie juju (Ubuntu Saucy): status Confirmed Won't Fix