Discovered Nodes Conflict with each other

From IRC, macs of nodes 00:26:6C:FE:57:4C and 00:26:6C:FE:28:C4

Bug #1, mac must be lowered before attempt to find it in the database

https://github.com/openstack/fuel-web/blob/efdd063170062d1bc881cd31c52bcee08bcdc9ee/nailgun/nailgun/objects/node.py#L219-L220

After it was not able to find by interface, it tries to use other interfaces, and it finds mellanox interface with mac 00:01, which exists in multiple nodes

"interfaces":[{"name":"enp3s0d1","state":"down","mac":"00:00:00:00:00:01","pxe":false,"driver":"mlx4_en","bus_info":"0000:03:00.0","max_speed":10000,"current_speed":null

After changes were suggested, same results.

`dockerctl shell nailgun`, and `find /usr/lib --name node.py`
add change mac=meta.get('mac') to mac=meta.get('mac', "").lower() and restart the container.

Adding screenshot of changes.

Hmm, actually it's lowered further https://github.com/openstack/fuel-web/blob/stable/8.0/nailgun/nailgun/objects/node.py#L85

But comment https://bugs.launchpad.net/fuel/+bug/1650646/comments/4 is still valid.

screenshot of changes number 2

snapshot number 3

I am able to reproduce this bug in a test environment by setting identical macs on two arbitrary interfaces on two nodes. We need to use a different method of node identification instead of MACs, for example system UUID + S/N from ohai output.

The UUID is difference on each Nodes in my environment.

The Mellanox Mac Address defaults to 00:00:00:00:00:01 when it is in Ethernet Mode. It would be nice for Mirantis Fuel can have the Mellanox switchable between IB and EN modes.

Fix proposed to branch: master
Review: https://review.openstack.org/414664

@Jainesh Nand
Please try the proposed fix and let us know if it resolves the issue. You will need to:
1. change /usr/lib/python2.7/site-packages/nailgun/objects/node.py as in the following review https://review.openstack.org/414664
2. restart nailgun (systemctl restart nailgun)
3. reboot bootstrapped nodes

@Jainesh Nand
The current version of the review (patch set 18) should resolve the issue, however the review is for master branch and might not apply directly on 8.0 or 9.0. If you you feel confident in Python please try to apply the changes similarly on your Fuel system and restart nailgun (if running 8.0 - do not restart the container if running 8.0 - this will revert any changes inside it; only restart nailgun service inside). Otherwise we need to wait for the change to be reviewed, and if approved we will need to make a backport.

Very invasive change and rare case. Moving out of 9.2 scope.

Change abandoned by Dmitry Sutyagin (<email address hidden>) on branch: master
Review: https://review.openstack.org/414664
Reason: Approach disapproved

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/421551

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/421552

New proposed fix for master: https://review.openstack.org/#/c/421539

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/newton
Review: https://review.opendev.org/421552
Reason: This repo is retired now, no further work will get merged.

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: master
Review: https://review.opendev.org/421539
Reason: This repo is retired now, no further work will get merged.

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/mitaka
Review: https://review.opendev.org/421551
Reason: This repo is retired now, no further work will get merged.