Network check fails with "Something is wrong with response data from node" error

Bug #1597199 reported by Dmitriy Kruglov
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
slava valyavskiy
Mitaka
Fix Released
Critical
slava valyavskiy

Bug Description

Detailed bug description:
Network verification fails on an env with bonded interfaces with "Something is wrong with response data from node with id=1." kind of errors.
https://product-ci.infra.mirantis.net/view/10.0_swarm/job/10.0.system_test.ubuntu.bonding_ha/76/testReport/(root)/deploy_bonding_neutron_vlan/deploy_bonding_neutron_vlan/

Steps to reproduce (please refer to the referenced auto system test for more details):
1. Create a cluster, Neutron VLAN
2. Add 3 nodes with controller role
3. Add 1 node with compute role and 1 node with cinder role
4. Setup bonding for all interfaces (including admin interface bonding)
5. Run network verification

Expected result: the env passes network verification.
Actual result: network verification fails with "Something is wrong with response data from node..."
error.

Description of the environment:
MOS 10.0-mitaka, ISO build #27.

From the nailgun/receiverd.log
2016-06-28 18:00:23.862 INFO [7f3fdfb12740] (receiver) RPC method check_dhcp_resp received: {"status": "ready", "progress": 100, "task_uuid": "59cbbf26-3e91-4855-83a7-fadd5621bb57", "nodes": [{"status": "ready", "data": [{"server_id": "10.109.34.2", "iface": "enp0s3", "yiaddr": "10.109.34.9", "mac": "64:d9:ee:a6:ca:5c", "server_ip": "10.109.34.2", "dport": 67, "message": "offer", "gateway": "0.0.0.0"}], "uid": "1"}, {"status": "ready", "data": [{"server_id": "10.109.34.2", "iface": "enp0s3", "yiaddr": "10.109.34.10", "mac": "64:d9:ee:a6:ca:5c", "server_ip": "10.109.34.2", "dport": 67, "message": "offer", "gateway": "0.0.0.0"}], "uid": "3"}, {"status": "ready", "data": [{"server_id": "", "iface": "", "yiaddr": "", "mac": "", "server_ip": "", "dport": "", "message": "", "gateway": ""}], "uid": "5"}, {"status": "ready", "data": [{"server_id": "", "iface": "", "yiaddr": "", "mac": "", "server_ip": "", "dport": "", "message": "", "gateway": ""}], "uid": "4"}, {"status": "ready", "data": [{"server_id": "", "iface": "", "yiaddr": "", "mac": "", "server_ip": "", "dport": "", "message": "", "gateway": ""}], "uid": "2"}]}
2016-06-28 18:00:23.896 DEBUG [7f3fdfb12740] (receiver) Mac addr on master node 64:d9:ee:a6:ca:5c
2016-06-28 18:00:23.896 WARNING [7f3fdfb12740] (receiver) Failed to compare mac address from response data (row = {u'server_id': u'', u'iface': u'', u'yiaddr': u'', u'mac': u'', u'server_ip': u'', u'dport': u'', u'message': u'', u'gateway': u''}) from node with id=5. Original error:
 failed to detect EUI version: u''
2016-06-28 18:00:23.897 WARNING [7f3fdfb12740] (receiver) Failed to compare mac address from response data (row = {u'server_id': u'', u'iface': u'', u'yiaddr': u'', u'mac': u'', u'server_ip': u'', u'dport': u'', u'message': u'', u'gateway': u''}) from node with id=4. Original error:
 failed to detect EUI version: u''
2016-06-28 18:00:23.897 WARNING [7f3fdfb12740] (receiver) Failed to compare mac address from response data (row = {u'server_id': u'', u'iface': u'', u'yiaddr': u'', u'mac': u'', u'server_ip': u'', u'dport': u'', u'message': u'', u'gateway': u''}) from node with id=2. Original error:
 failed to detect EUI version: u''

Revision history for this message
Dmitriy Kruglov (dkruglov) wrote :
Revision history for this message
Dmitriy Kruglov (dkruglov) wrote :
Changed in fuel:
status: New → Confirmed
Dmitry Klenov (dklenov)
Changed in fuel:
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
tags: added: area-python
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Alex Schultz (alex-schultz)
Changed in fuel:
assignee: Alex Schultz (alex-schultz) → Fuel Sustaining (fuel-sustaining-team)
description: updated
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → l23network (l23network)
Changed in fuel:
assignee: l23network (l23network) → slava valyavskiy (slava-val-al)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/339180

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/339180
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=f48e82737db73fdf10249ad12030e2bd918a2675
Submitter: Jenkins
Branch: master

commit f48e82737db73fdf10249ad12030e2bd918a2675
Author: slava <email address hidden>
Date: Thu Jul 7 21:42:25 2016 +0300

    Skip dhcp_check report if no mac of server is present

    Change-Id: Id23f9e9761f80b6e61d3c1cf52f3c7595deb3e11
    Closes-Bug: #1597199

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/371482

no longer affects: fuel/future
tags: added: blocker-for-qa
tags: added: swarm-fail
Revision history for this message
Alisa Tselovalnikova (atselovalnikova) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/mitaka)

Reviewed: https://review.openstack.org/371482
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=25af561581bea440a94b465c86db6a58fd747092
Submitter: Jenkins
Branch: stable/mitaka

commit 25af561581bea440a94b465c86db6a58fd747092
Author: slava <email address hidden>
Date: Thu Jul 7 21:42:25 2016 +0300

    Skip dhcp_check report if no mac of server is present

    Change-Id: Id23f9e9761f80b6e61d3c1cf52f3c7595deb3e11
    Closes-Bug: #1597199
    (cherry picked from commit f48e82737db73fdf10249ad12030e2bd918a2675)

tags: added: on-verification
Revision history for this message
ElenaRossokhina (esolomina) wrote :

Verified
CUSTOM_VERSION=snapshot #284
MAGNET_LINK=magnet:?xt=urn:btih:bfec808dd71ff42c5613a3527733d9012bb1fabc&dn=MirantisOpenStack-9.0.iso&tr=http%3A%2F%2Ftracker01-bud.infra.mirantis.net%3A8080%2Fannounce&tr=http%3A%2F%2Ftracker01-scc.infra.mirantis.net%3A8080%2Fannounce&tr=http%3A%2F%2Ftracker01-msk.infra.mirantis.net%3A8080%2Fannounce&ws=http%3A%2F%2Fvault.infra.mirantis.net%2FMirantisOpenStack-9.0.iso
FUEL_QA_COMMIT=19502443f1ed873aa0ee004e96293069fae7a9a9
UBUNTU_MIRROR_ID=ubuntu-2016-09-14-213640
CENTOS_MIRROR_ID=centos-7.2.1511-2016-05-31-083834
MOS_UBUNTU_MIRROR_ID=9.0-2016-09-19-122321
MOS_CENTOS_OS_MIRROR_ID=os-2016-06-23-135731
MOS_CENTOS_PROPOSED_MIRROR_ID=proposed-2016-09-19-152353
MOS_CENTOS_UPDATES_MIRROR_ID=updates-2016-06-23-135916
MOS_CENTOS_HOLDBACK_MIRROR_ID=holdback-2016-06-23-140047
MOS_CENTOS_HOTFIX_MIRROR_ID=hotfix-2016-09-16-084322
MOS_CENTOS_SECURITY_MIRROR_ID=security-2016-06-23-140002

tags: removed: on-verification
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-web 10.0.0rc1

This issue was fixed in the openstack/fuel-web 10.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-web 10.0.0

This issue was fixed in the openstack/fuel-web 10.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.