Node Discovery Information being overwritten

Bug #1880286 reported by Dominic
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
New
Undecided
Unassigned

Bug Description

Issue:
Node Discovery Behavior

Synopsis:
1. “Name” assigned to the node on the discovery is not unique.
2. Node discovery information gets overwritten against the same “Name” and “Id”.
3. “Id” and “name” get updated with the information from the first node discovery after a node delete and stick to that thereafter.

Issue Re-production procedure:
1. Initial status on the fuel node listing.
[root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 1 | Untitled (1d:04) | discover | ubuntu | [] | 192.168.0.22 | 08:f1:ea:7e:fc:a0 | None | ProLiant DL360 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
2. Delete the discovered node.
[root@fuel ~]# fuel node --node-id 1 --delete-from-db --force
Nodes with ids [1] have been deleted from Fuel db.
3. Status on the fuel node listing.
• On the first listing, it is noticed that the node id and name get updated with information from the first discovery request after the above delete. Note, the “name” has the last 4 letters on the mac address from the discovered node.
• Note, the id is incremented after the delete on discovery of the first node.
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.22 | 08:f1:ea:7e:fc:a0 | None | ProLiant DL360 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
• On subsequent listings, it can be noticed from the discovery information is overwritten with the same “name” and “id”, we have 4 nodes here, all 4 discoveries cycle against the same “name” and “id”.
[root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.20 | 08:f1:ea:7e:1d:04 | None | ProLiant DL360 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+

root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.21 | 08:f1:ea:7e:dc:98 | None | ProLiant DL360 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+

root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 2 | Untitled (fc:a0) | discover | ubuntu | [] | 192.168.0.23 | 08:f1:ea:7e:ec:4d | None | ProLiant DL360 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+

4. There is no DHCP flapping. The mapping between the IP’s and MAC’s never change, or at least did not change on these runs.
5. Fuel version
[root@fuel ~]# fuel --fuel-version
api: '1'
auth_required: true
feature_groups:
- mirantis
- advanced
openstack_version: pike-12.0
release: '12.0'

Revision history for this message
Tunggul Wardhana (etunwar) wrote :
Download full text (9.0 KiB)

Hi Dominic,

I found the same issue. This fault is related with Virtual NIC feature on HPE ProLiant iLO 5 with firmware 1.40 and later.
After load bootstrap image, the Virtual NIC will be added as usb0 interface and this usb0 will have same MAC address for all computes.

# Suspect on fuel node validator scripts behavior on:
/usr/lib/python2.7/site-packages/nailgun/api/v1/validators/node.py
that if the MAC address of NIC interfaces are identical, they are not added in the fuel node list as being discovered.
While the usb0 have same MAC address across all computes which lead to fuel node discovery failure.

# Workaround procedure with iLO5 GUI:
1. Click Security in the navigation tree.
The Access Settings page is displayed.

2. Click Edit next to the iLO category.
The Edit iLO Settings page opens.

3. Unselect the Virtual NIC check box, and then click OK.
iLO notifies you that pending changes require a reset to take effect.

4. If you are done updating access settings, click Reset iLO. iLO prompts you to confirm the request.
5. Click Yes, reset iLO.
It might take several minutes before you can re-establish a connection.
After the reset is complete, the Virtual NIC feature is disabled and it is detected by the host serveroperating system.

*Refer to: https://support.hpe.com/hpesc/public/docDisplay?docId=a00064988en_us
  page 162 (Enabling the Virtual NIC feature)

# To reproduce the issue: enable Virtual NIC and Reset iLO

# Related Logs

[root@fuel ~]# fuel --fuel-version
api: '1'
auth_required: true
feature_groups: []
openstack_version: pike-12.0
release: '12.0'

[root@fuel ~]# fuel node | sort -k 1 -n
---+----------+------------------+---------+--------------+-------------------+-------+---------------+--------+---------
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
29 | discover | Untitled (7b:16) | | 192.168.0.20 | b4:7a:f1:37:7b:16 | | | 1 |

[root@fuel ~]# fuel node | sort -k 1 -n
---+----------+------------------+---------+--------------+-------------------+-------+---------------+--------+---------
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
29 | discover | Untitled (7b:16) | | 192.168.0.21 | b4:7a:f1:37:7b:e8 | | | 1 |

[root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| id | name | status | os_platform | roles | ip | mac | cluster | platform_name | online |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+
| 29 | Untitled (7b:16) | discover | ubuntu | [] | 192.168.0.20 | b4:7a:f1:37:7b:16 | None | ProLiant DL380 Gen10 | True |
+----+------------------+----------+-------------+-------+--------------+-------------------+---------+----------------------+--------+

[root@fuel ~]# fuel2 node list
+----+------------------+----------+-------------+---...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.