[2.5] MAAS doesn't notice that rack controller is no longer connected to a vlan where it was previously running DHCP, causing it to fail to configure DHCP

Bug #1816451 reported by Andres Rodriguez
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Invalid
Critical
Blake Rouse

Bug Description

MAAS was unable to configure DHCP on a rack controller, which seemed to be the rack controller that had been previously configured to a VLAN, but was no longer present in such VLAN.

DHCP HA was configured for an untagged VLAN on subnet 192.168.1.0/24, with 'charybdis' and 'ypso' as primary/secondary, respectively. However, for whatever reason, while 'ypso' as a rack controller seems to be fully connected to MAAS, there are no interfaces associated to it.

While there are no interfaces associated to it, the rack controller seems to be part of the vlan, hence as a secondary in HA.

So, while I think the real issue could be that the rack is not correctly notifying its network links, the fact that a rack disappears from a vlan and then its install a secondary rack controller is indeed an issue.

2019-02-18 14:54:47 maasserver.rack_controller: [critical] Failed configuring DHCP on rack controller 'id:249'.
 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
     _inlineCallbacks(r, g, deferred)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1432, in _inlineCallbacks
     deferred.errback()
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 500, in errback
     self._startRunCallbacks(fail)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
     self._runCallbacks()
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python3/dist-packages/maasserver/rack_controller.py", line 256, in <lambda>
     d.addErrback(lambda f: f.trap(NoConnectionsAvailable))
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 359, in trap
     self.raiseException()
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 385, in raiseException
     raise self.value.with_traceback(self.tb)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 751, in configure_dhcp
     config = yield deferToDatabase(get_dhcp_configuration, rack_controller)
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
     result = inContext.theWork()
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 885, in callInContext
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 234, in wrapper
     result = func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 756, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 563, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python3.6/contextlib.py", line 52, in inner
     return func(*args, **kwds)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 684, in get_dhcp_configuration
     dhcp_snippets=dhcp_snippets, use_rack_proxy=use_rack_proxy)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 578, in get_dhcp_configure_for
     vlan, rack_controller)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 454, in make_failover_peer_config
     peer_address = get_ip_address_for_rack_controller(peer_rack, vlan)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 247, in get_ip_address_for_rack_controller
     return get_ip_address_for_interface(interface, vlan)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 228, in get_ip_address_for_interface
     for ip_address in interface.ip_addresses.all():
 builtins.AttributeError: 'NoneType' object has no attribute 'ip_addresses'

Changed in maas:
milestone: none → 2.5.2
importance: Undecided → Critical
status: New → Triaged
assignee: nobody → Blake Rouse (blake-rouse)
summary: [2.5] MAAS doesn't notice that rack controller is no longer connected to
- a network where it was previously running DHCP
+ a network where it was previously running DHCP, causing it to traceback
tags: added: performance track
Revision history for this message
Andres Rodriguez (andreserl) wrote :
summary: [2.5] MAAS doesn't notice that rack controller is no longer connected to
- a network where it was previously running DHCP, causing it to traceback
+ a vlan where it was previously running DHCP, causing it to fail to
+ configure DHCP
Revision history for this message
Andres Rodriguez (andreserl) wrote :
Revision history for this message
Andres Rodriguez (andreserl) wrote :
Changed in maas:
milestone: 2.5.2 → 2.5.3
Changed in maas:
milestone: 2.5.3 → 2.6.0
Revision history for this message
Adam Collard (adam-collard) wrote :

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

MAAS Team

Changed in maas:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.