Nexus plugins fails to configure the network node connection when the network node is rebooted before launching the VM

Symptom:
   If the network node is rebooted,

       the following would be affected:
          1) creation of new VMs on tenant networks that existed prior to network
              node reboot would not get their VLAN config on the switch
     - Since the dhcp namespaces for these networks are already created
     after the reboot.

       the following would be unaffected:
          1) any existing VMs
              - since the tenant network's VLAN has already been configured on the switch
          2) any new VMs created on tenant networks already having VMs
              - since the tenant network's VLAN has already been configured on the switch
          3) any new tenant networks and their VMs created after the network node is back

Cause:
 Upon boots up, the dhcp namespace is created while the DB is still empty
   - VM triggered namespace creation is path where Nexus plugin configures the
      VLAN

******Workaround**********

To limit the number of VLANs configured at that trunk port where the neutron server is hanging off,
the other way around is to include this port as a list of ports associate with each compute node.

E.g. Network node is port 1, compute-1 is port 2, compute-2 is port 3
[ml2_mech_cisco_nexus:10.86.7.59]
compute-1=1/1,1/2
compute-2=1/1,1/3
ssh_port=22
username=admin
password=cisco123

This way the nexus plugin still manages the trunk configuration of the switch port connecting the network node and only allows the VLANs associated with tenant networks.

While verifying this workaround:

[ml2_mech_cisco_nexus:10.86.7.59]
compute-1=1/1,1/2
compute-2=1/1,1/3
ssh_port=22
username=admin
password=cisco123

I came across another issue.

When the DHCP namespace is automatically created after the network node reboot, there is a port binding failure for the corresponding DHCP neutron port:

[root@bxb-ds-46 ~]# ip netns
qdhcp-d1e57ac4-b775-4dd1-ab4d-c901ebb04c98
qrouter-3b3f07ff-f530-4cfb-8a4b-f85ea0d4441c
[root@bxb-ds-46 ~]# neutron port-list
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+
| 036acf84-a5c2-4789-a742-bc049d0d2dff | | fa:16:3e:72:c2:8e | {"subnet_id": "768ea98d-3b7e-43a5-b18a-d3a1a284c193", "ip_address": "10.86.7.239"} |
| 52a2db1a-1929-4c2c-8720-c08b0706b4cb | | fa:16:3e:d1:43:a6 | {"subnet_id": "e7ec9460-b1aa-4801-a128-345955a38467", "ip_address": "192.168.1.2"} |
| 7a4610d4-9a63-4eee-88ea-8086cb5702fc | | fa:16:3e:fb:98:0d | {"subnet_id": "e7ec9460-b1aa-4801-a128-345955a38467", "ip_address": "192.168.1.1"} |
+--------------------------------------+------+-------------------+——————————————————————————————————————————+
[root@bxb-ds-46 ~]# neutron port-show 52a2db1a-1929-4c2c-8720-c08b0706b4cb
+-----------------------+------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | bxb-ds-46 |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | binding_failed | <<<<<ERROR
| binding:vnic_type | normal |
| device_id | dhcpc63a1974-ca08-50bd-9dfd-34e0a562acaf-d1e57ac4-b775-4dd1-ab4d-c901ebb04c98 |
| device_owner | network:dhcp |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "e7ec9460-b1aa-4801-a128-345955a38467", "ip_address": "192.168.1.2"} |
| id | ...

Remove the cisco_nexus driver in ml2conf.ini file and saw the same issues after the Network node reboot.

1. DHCP namespace is created automatically.
2. DHCP port binding failure.

/etc/neutron/plugins/ml2/ml2_conf.ini:
mechanism_drivers =openvswitch /* remove cisco_nexus */

Retest on RHEL reference test bed without Cisco neutron:

[root@hh24-11 ~]# rpm -qi python-neutron
Name : python-neutron
Version : 2014.2.3
Release : 2.el7ost
Architecture: noarch
Install Date: Thu 16 Jul 2015 11:17:46 AM PDT
Group : Applications/System
Size : 11211657
License : ASL 2.0
Signature : RSA/SHA256, Sun 26 Apr 2015 08:03:39 AM PDT, Key ID 199e2f91fd431d51
Source RPM : openstack-neutron-2014.2.3-2.el7ost.src.rpm
Build Date : Mon 13 Apr 2015 06:23:08 AM PDT
Build Host : x86-021.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor : Red Hat, Inc.
URL : http://launchpad.net/neutron/
Summary : Neutron Python libraries

Test result:
1. DHCP namespace is created automatically after a reboot or neutron restart.
2. DHCP port binding is successful.

Before reboot:
==============
[root@hh24-11 ~]# neutron net-list
+--------------------------------------+---------+----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+---------+----------------------------------------------------+
| 0a7cafba-100d-4761-9799-96c32bd54cd5 | ext-net | 66a2a867-af61-4017-bd94-54f04d973073 2.2.2.0/24 |
| 1e0b9add-3df6-483c-840a-533b4323c2bc | private | c2e11c64-2269-486f-b1ce-15c358db129a 10.10.10.0/24 |
+--------------------------------------+---------+----------------------------------------------------+
[root@hh24-11 ~]# neutron subnet-list
+--------------------------------------+----------------+---------------+------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+----------------+---------------+------------------------------------------------+
| 66a2a867-af61-4017-bd94-54f04d973073 | | 2.2.2.0/24 | {"start": "2.2.2.2", "end": "2.2.2.200"} |
| c2e11c64-2269-486f-b1ce-15c358db129a | private-subnet | 10.10.10.0/24 | {"start": "10.10.10.2", "end": "10.10.10.254"} |
+--------------------------------------+----------------+---------------+------------------------------------------------+
[root@hh24-11 ~]# neutron port-list
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------------+
| 79acd252-7f56-461c-be2d-3b38ab0d878c | | fa:16:3e:5a:98:51 | {"subnet_id": "66a2a867-af61-4017-bd94-54f04d973073", "ip_address": "2.2.2.2"} |
| 98e09da6-104e-42e8-8802-39aaf02e3e3d | | fa:16:3e:24:2c:01 | {"subnet_id": "c2e11c64-2269-486f-b1ce-15c358db129a", "ip_address": "10.10.10.1"} |
+--------------------------------------+------...