networking-cisco

EMC-POC: Nexus plugin: enhancement request to have the replay code run in multi threads

Bug #1468843 reported by Danny Choi on 2015-06-25

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	networking-cisco	New	Undecided	Unassigned

Bug Description

Currently the replay code runs in single thread. As such, with multiple switches, it will replay config serially.

For example, both Nexus switches reboot. The first switch comes online will get the replay config served. Until all the config is replayed successfully, then the plugin will move onto the second switch.

If for some reason the replay for the first switch fails during the replay process, it will wait till the timeout expires (switch_heartbeat_time = 30) before it serves the second switch.

Tags:

Revision history for this message

Danny Choi (dannchoi) wrote on 2015-06-25:

Download full text (5.8 KiB)

Example

Replay fails at 13:18:24.572 at switch #1.
30 seconds later, at 13:18:54.984, it establishes connection with switch #2.

Example

Replay fails at 13:18:24.572 at switch #1. 
30 seconds later, at 13:18:54.984, it establishes connection with switch #2.

2015-06-25 13:18:24.571 38580 ERROR neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Nexus Driver cisco_nexus failed in replay _configure_port_binding
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus Traceback (most recent call last):
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/mech_cisco_nexus.py", line 526, in configure_switch_entries
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     port.vni)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/mech_cisco_nexus.py", line 434, in _configure_port_binding
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     vni)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/nexus_network_driver.py", line 334, in create_and_trunk_vlan
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     nexus_port)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/nexus_network_driver.py", line 297, in enable_vlan_on_trunk_int
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     intf_type, interface)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/nexus_network_driver.py", line 188, in get_interface_switch_trunk_allowed
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     response = self._get_config(nexus_host, confstr)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/nexus_network_driver.py", line 78, in _get_config
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     exc=e)
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus NexusConfigFailed: Failed to configure Nexus switch: 10.86.1.128 XML: 
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus     <cmd>show running-config interface ethernet 1/22</cmd>
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus . Reason: .
2015-06-25 13:18:24.571 38580 TRACE neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus 
2015-06-25 13:18:24.572 38580 ERROR neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Failed to configure port binding for switch 10.86.1.128, vlan 540 vni 0, port ethernet:1/22, reason Failed to configure Nexus switch: 10.86.1.128 XML: 
    <cmd>show running-config interface ethernet 1/22</cmd>
. Reason: .
2015-06-25 13:18:24.573 38580 WARNING neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Replay config failed for ip 10.86.1.128
2015-06-25 13:18:32.532 38580 DEBUG neutron.context [req-85848d49-6d53-46ec-bb54-9c783ec7605e None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:36.735 38580 DEBUG neutron.context [req-a91ce879-1688-4e9e-9fe1-226f88ba38bb None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:37.433 38580 DEBUG neutron.context [req-9bc9aa11-ca41-4e11-af3b-861c2cdd9c64 None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:37.925 38580 DEBUG neutron.context [req-233621a8-d550-4ca2-84f1-ca5fb3194d50 None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:38.191 38580 DEBUG neutron.context [req-8b4e21b0-48cb-4db9-b0e4-7c836a2d9e5e None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:47.296 38580 DEBUG neutron.context [req-7b24f7a7-4797-47a6-80f8-402c02b4fc2f None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:84
2015-06-25 13:18:54.574 38580 DEBUG neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] check_connections() switch 10.86.7.59 state 0 check_connections /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/mech_cisco_nexus.py:128
2015-06-25 13:18:54.983 38580 WARNING neutron.plugins.ml2.drivers.cisco.nexus.nexus_network_driver [-] GET call failed to return Nexus type
2015-06-25 13:18:54.983 38580 ERROR neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Received invalid Nexus type -1 for switch ip 10.86.7.59
2015-06-25 13:18:54.984 38580 INFO neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Re-established connection to switch ip 10.86.7.59
2015-06-25 13:18:54.984 38580 DEBUG neutron.plugins.ml2.drivers.cisco.nexus.mech_cisco_nexus [-] Replaying config for switch ip 10.86.7.59 replay_config /usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/cisco/nexus/mech_cisco_nexus.py:77

Danny Choi (dannchoi) on 2015-07-14

tags:

added: cisco ml2 nexus

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.