MAAS

Image import fails

Bug #1743648 reported by Slawomir Gajowniczek
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
High
Igor Brovtsin
MAAS 3.4.0-beta1

Bug Description

2018-01-16 20:23:30 maasserver.bootresources: [critical] Importing boot resources failed.

Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 842, in worker
    return target()
  File "/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
    task()
  File "/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
    task()
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
    result = inContext.theWork()
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 875, in callInContext
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 232, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 169, in wrapper
    return func_in_reactor(*args, **kwargs).wait(LONGTIME)
  File "/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 231, in wait
    result.raiseException()
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 368, in raiseException
    raise self.value.with_traceback(self.tb)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/maasserver/bootresources.py", line 1262, in eb_import
    failure.trap(DatabaseLockNotHeld)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 342, in trap
    self.raiseException()
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 368, in raiseException
    raise self.value.with_traceback(self.tb)
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
    result = inContext.theWork()
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 875, in callInContext
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 232, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 710, in call_with_connection
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/__init__.py", line 178, in call_with_lock
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/bootresources.py", line 1333, in _import_resources_with_lock
    sources, product_mapping, notify=notify)
  File "/usr/lib/python3/dist-packages/maasserver/bootresources.py", line 1180, in download_all_boot_resources
    listener.register("sys_stop_import", stop_import)
  File "/usr/lib/python3/dist-packages/maasserver/listener.py", line 223, in register
    "System channel '%s' has already been registered." % channel)
maasserver.listener.PostgresListenerRegistrationError: System channel 'sys_stop_import' has already been registered.

Ubuntu 16.04 (armh) - rpi3

See original description

Related branches

~igor-brovtsin/maas:lp-1743648
MAAS Lander: Approve
Alberto Donato: Approve
Diff: 218 lines (+109/-31)
4 files modified
src/maasserver/bootresources.py (+25/-26)
src/maasserver/listener.py (+12/-0)
src/maasserver/tests/test_bootresources.py (+47/-5)
src/maasserver/tests/test_listener.py (+25/-0)
Slawomir Gajowniczek (imkebe)
description: updated
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Hi Slawomir,

Are you running maas on armhf? This seems like the systme doesn't have enough resources for MAAS to do what it needs to do.

Changed in maas:
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for MAAS because there has been no activity for 60 days.]

Changed in maas:
status: Incomplete → Expired
Revision history for this message
Andrey Grebennikov (agrebennikov) wrote :

resurrecting this bug as I've just hit it on 3.2.6
MAAS runs on a pretty powerful machine, x86.
Trying to add 22.10 with amd64 kernel.
Getting exactly same error.

Have reproducer ready.

Changed in maas:
status: Expired → New
status: New → Confirmed
Revision history for this message
Andrey Grebennikov (agrebennikov) wrote :
Revision history for this message
Andrey Grebennikov (agrebennikov) wrote (last edit ):
Download full text (7.5 KiB)

The very first failure of the boot resources import:

2022-11-04 21:58:04 sstreams: [info] com.ubuntu.maas:stable:centos-bases-download/com.ubuntu.maas.stable:centos-bases:8:amd64: to_add=['20211104_01'] to_remove=[]
2022-11-04 22:00:08 maasserver.bootresources: [critical] Importing boot resources failed.
        Traceback (most recent call last):
          File "/usr/lib/python3.8/threading.py", line 870, in run
            self._target(*self._args, **self._kwargs)
          File "/snap/maas/23947/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 821, in worker
            return target()
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
            task()
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
            task()
        --- <exception caught here> ---
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
            result = inContext.theWork()
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
            inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
          File "/snap/maas/23947/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 856, in callInContext
            return func(*args, **kwargs)
          File "/snap/maas/23947/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 202, in wrapper
            result = func(*args, **kwargs)
          File "/snap/maas/23947/lib/python3.8/site-packages/provisioningserver/utils/twisted.py", line 136, in wrapper
            return func_in_reactor(*args, **kwargs).wait(LONGTIME)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 231, in wait
            result.raiseException()
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/failure.py", line 467, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/snap/maas/23947/lib/python3.8/site-packages/maasserver/bootresources.py", line 1317, in eb_import
            failure.trap(DatabaseLockNotHeld)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/failure.py", line 439, in trap
            self.raiseException()
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/failure.py", line 467, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/snap/maas/23947/usr/lib/python3/dist-packages/twisted/python/t...

Read more...

Revision history for this message
Björn Tillenius (bjornt) wrote :

Seems to be a race condition. Not sure yet how to fix it, but we should have enough info to find it and fix it.

Changed in maas:
status: Confirmed → Triaged
importance: Undecided → High
milestone: none → 3.4.0
Adam Collard (adam-collard)
Changed in maas:
assignee: nobody → Igor Brovtsin (igor-brovtsin)
MAAS Lander (maas-lander)
Changed in maas:
status: Triaged → Fix Committed
Alberto Donato (ack)
Changed in maas:
milestone: 3.4.0 → 3.4.0-beta1
Alberto Donato (ack)
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.