Node Discovery Information being overwritten
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
New
|
Undecided
|
Unassigned |
Bug Description
Issue:
Node Discovery Behavior
Synopsis:
1. “Name” assigned to the node on the discovery is not unique.
2. Node discovery information gets overwritten against the same “Name” and “Id”.
3. “Id” and “name” get updated with the information from the first node discovery after a node delete and stick to that thereafter.
Issue Re-production procedure:
1. Initial status on the fuel node listing.
[root@fuel ~]# fuel2 node list
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 1 | Untitled (1d:04) | discover | ubuntu | [] | 192.168.0.22 | 08:f1:ea:7e:fc:a0 | None | ProLiant DL360 Gen10 | True |
+----+-
2. Delete the discovered node.
[root@fuel ~]# fuel node --node-id 1 --delete-from-db --force
Nodes with ids [1] have been deleted from Fuel db.
3. Status on the fuel node listing.
• On the first listing, it is noticed that the node id and name get updated with information from the first discovery request after the above delete. Note, the “name” has the last 4 letters on the mac address from the discovered node.
• Note, the id is incremented after the delete on discovery of the first node.
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.22 | 08:f1:ea:7e:fc:a0 | None | ProLiant DL360 Gen10 | True |
+----+-
• On subsequent listings, it can be noticed from the discovery information is overwritten with the same “name” and “id”, we have 4 nodes here, all 4 discoveries cycle against the same “name” and “id”.
[root@fuel ~]# fuel2 node list
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.20 | 08:f1:ea:7e:1d:04 | None | ProLiant DL360 Gen10 | True |
+----+-
root@fuel ~]# fuel2 node list
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.21 | 08:f1:ea:7e:dc:98 | None | ProLiant DL360 Gen10 | True |
+----+-
root@fuel ~]# fuel2 node list
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.23 | 08:f1:ea:7e:ec:4d | None | ProLiant DL360 Gen10 | True |
+----+-
4. There is no DHCP flapping. The mapping between the IP’s and MAC’s never change, or at least did not change on these runs.
5. Fuel version
[root@fuel ~]# fuel --fuel-version
api: '1'
auth_required: true
feature_groups:
- mirantis
- advanced
openstack_version: pike-12.0
release: '12.0'
Hi Dominic,
I found the same issue. This fault is related with Virtual NIC feature on HPE ProLiant iLO 5 with firmware 1.40 and later.
After load bootstrap image, the Virtual NIC will be added as usb0 interface and this usb0 will have same MAC address for all computes.
# Suspect on fuel node validator scripts behavior on: python2. 7/site- packages/ nailgun/ api/v1/ validators/ node.py
/usr/lib/
that if the MAC address of NIC interfaces are identical, they are not added in the fuel node list as being discovered.
While the usb0 have same MAC address across all computes which lead to fuel node discovery failure.
# Workaround procedure with iLO5 GUI:
1. Click Security in the navigation tree.
The Access Settings page is displayed.
2. Click Edit next to the iLO category.
The Edit iLO Settings page opens.
3. Unselect the Virtual NIC check box, and then click OK.
iLO notifies you that pending changes require a reset to take effect.
4. If you are done updating access settings, click Reset iLO. iLO prompts you to confirm the request.
5. Click Yes, reset iLO.
It might take several minutes before you can re-establish a connection.
After the reset is complete, the Virtual NIC feature is disabled and it is detected by the host serveroperating system.
*Refer to: https:/ /support. hpe.com/ hpesc/public/ docDisplay? docId=a00064988 en_us
page 162 (Enabling the Virtual NIC feature)
# To reproduce the issue: enable Virtual NIC and Reset iLO
# Related Logs
[root@fuel ~]# fuel --fuel-version
api: '1'
auth_required: true
feature_groups: []
openstack_version: pike-12.0
release: '12.0'
[root@fuel ~]# fuel node | sort -k 1 -n ------- +------ ------- -----+- ------- -+----- ------- --+---- ------- ------- -+----- --+---- ------- ----+-- ------+ ------- --
---+---
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
29 | discover | Untitled (7b:16) | | 192.168.0.20 | b4:7a:f1:37:7b:16 | | | 1 |
[root@fuel ~]# fuel node | sort -k 1 -n ------- +------ ------- -----+- ------- -+----- ------- --+---- ------- ------- -+----- --+---- ------- ----+-- ------+ ------- --
---+---
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
29 | discover | Untitled (7b:16) | | 192.168.0.21 | b4:7a:f1:37:7b:e8 | | | 1 |
[root@fuel ~]# fuel2 node list ------- ------- ---+--- ------- +------ ------- +------ -+----- ------- --+---- ------- ------- -+----- ----+-- ------- ------- ------+ ------- -+ ------- ------- ---+--- ------- +------ ------- +------ -+----- ------- --+---- ------- ------- -+----- ----+-- ------- ------- ------+ ------- -+ ------- ------- ---+--- ------- +------ ------- +------ -+----- ------- --+---- ------- ------- -+----- ----+-- ------- ------- ------+ ------- -+
+----+-
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+-
| 29 | Untitled (7b:16) | discover | ubuntu | [] | 192.168.0.20 | b4:7a:f1:37:7b:16 | None | ProLiant DL380 Gen10 | True |
+----+-
[root@fuel ~]# fuel2 node list ------- ------- ---+--- ------- +------ ------- +---...
+----+-