vRouter discarding ARP replies coming from MX

Bug #1551382 reported by amit surana
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Fix Committed
Critical
Anand H. Krishnan
R2.21.x
Fix Committed
Critical
Anand H. Krishnan
R2.22.x
Fix Committed
Critical
Anand H. Krishnan
R3.0
Fix Committed
Critical
Anand H. Krishnan
Trunk
Fix Committed
Critical
Anand H. Krishnan

Bug Description

3.0 2722.

Basic test of pinging from Mx to VM is failing. vRouter is dropping ARP replies coming from the MX. Only the 'discards' stats are going up. This used to work one/two builds back.

09:51:02.703428 10:0e:7e:be:79:00 > 90:e2:ba:4c:65:58, ethertype IPv4 (0x0800), length 126: 172.16.184.200 > 172.16.180.10: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 35, exp 0, [S], ttl 64) 192.168.0.42 > 192.168.0.19: ICMP echo request, id 32014, seq 58, length 64
09:51:02.733064 10:0e:7e:be:79:20 > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 408: LLDP, length 394: oblock-mini-5-qfx1
09:51:02.899256 90:e2:ba:4c:65:58 > 10:0e:7e:be:79:00, ethertype IPv4 (0x0800), length 92: 172.16.180.10.50094 > 172.16.184.200.4789: VXLAN, flags [I] (0x08), vni 19
02:72:00:32:f6:f1 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.0.42 tell 192.168.0.19, length 28
09:51:02.899267 90:e2:ba:4c:65:58 > 90:e2:ba:50:a9:d8, ethertype IPv4 (0x0800), length 124: 172.16.180.10 > 172.16.180.16: GREv0, proto MPLS unicast (0x8847), length 90: MPLS (label 4103, exp 0, [S], ttl 64)
        0x0000: 0000 0000 4500 004e 3386 0000 4011 86e3 ....E..N3...@...
        0x0010: ac10 b40a ac10 b40a c3ae 12b5 003a 0000 .............:..
        0x0020: 0800 0000 0000 1300 ffff ffff ffff 0272 ...............r
        0x0030: 0032 f6f1 0806 0001 0800 0604 0001 0272 .2.............r
        0x0040: 0032 f6f1 c0a8 0013 0000 0000 0000 c0a8 .2..............
        0x0050: 002a .*
09:51:02.901412 10:0e:7e:be:79:00 > 90:e2:ba:4c:65:58, ethertype IPv4 (0x0800), length 92: 172.16.184.200.0 > 172.16.180.10.4789: VXLAN, flags [I] (0x08), vni 19
28:c0:da:fd:2f:f0 > 02:72:00:32:f6:f1, ethertype ARP (0x0806), length 42: Reply 192.168.0.42 is-at 28:c0:da:fd:2f:f0, length 28
09:51:03.599248 90:e2:ba:4c:65:58 > 10:0e:7e:be:79:00, ethertype IPv4 (0x0800), length 92: 172.16.180.10.50094 > 172.16.184.200.4789: VXLAN, flags [I] (0x08), vni 19
02:72:00:32:f6:f1 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.0.42 tell 192.168.0.19, length 28
09:51:03.599262 90:e2:ba:4c:65:58 > 90:e2:ba:50:a9:d8, ethertype IPv4 (0x0800), length 124: 172.16.180.10 > 172.16.180.16: GREv0, proto MPLS unicast (0x8847), length 90: MPLS (label 4103, exp 0, [S], ttl 64)
        0x0000: 0000 0000 4500 004e 3389 0000 4011 86e0 ....E..N3...@...
        0x0010: ac10 b40a ac10 b40a c3ae 12b5 003a 0000 .............:..
        0x0020: 0800 0000 0000 1300 ffff ffff ffff 0272 ...............r
        0x0030: 0032 f6f1 0806 0001 0800 0604 0001 0272 .2.............r
        0x0040: 0032 f6f1 c0a8 0013 0000 0000 0000 c0a8 .2..............
        0x0050: 002a .*
09:51:03.599706 10:0e:7e:be:79:00 > 90:e2:ba:4c:65:58, ethertype IPv4 (0x0800), length 92: 172.16.184.200.0 > 172.16.180.10.4789: VXLAN, flags [I] (0x08), vni 19
28:c0:da:fd:2f:f0 > 02:72:00:32:f6:f1, ethertype ARP (0x0806), length 42: Reply 192.168.0.42 is-at 28:c0:da:fd:2f:f0, length 28

amit surana (asurana-t)
description: updated
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/18038
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18038
Committed: http://github.org/Juniper/contrail-vrouter/commit/e5e8b85bf3c90db29cd7d63b71e5fbf3d29c5e01
Submitter: Zuul
Branch: R3.0

commit e5e8b85bf3c90db29cd7d63b71e5fbf3d29c5e01
Author: Anand H. Krishnan <email address hidden>
Date: Tue Mar 1 00:35:52 2016 +0530

Post GRO, label in fmd can't be used to check whether the packet
was tunneled or not

Once the packet is submitted for GRO, all datapath information is
lost. Post GRO, only values that are saved in the packet are the
vif and the nexthop. vif is a recent addition to the saved
information that helped us to identify which interface the packet
came from originally. Once the vif value was set properly, the logic
that checked whether the packet should be trapped to agent or not
based on the presence of label (basically fabric arp responses should
be trapped or not), misbehaved since label information is not saved
pre-GRO and hence not available in the metadata post GRO. For now,
fix the specific logic by checking whether the egress vrf is different
from the ingress vrf, which will be the case since physical interface
vrf will not be the same as vm's vrf.

Change-Id: Iba000889039bc8a5020fc11a462ba1b1a68ce1c8
Closes-BUG: #1551382

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/18368
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18368
Committed: http://github.org/Juniper/contrail-vrouter/commit/7481ce1b1289a545a865f00a9e74637444f50385
Submitter: Zuul
Branch: master

commit 7481ce1b1289a545a865f00a9e74637444f50385
Author: Anand H. Krishnan <email address hidden>
Date: Tue Mar 1 00:35:52 2016 +0530

Post GRO, label in fmd can't be used to check whether the packet
was tunneled or not

Once the packet is submitted for GRO, all datapath information is
lost. Post GRO, only values that are saved in the packet are the
vif and the nexthop. vif is a recent addition to the saved
information that helped us to identify which interface the packet
came from originally. Once the vif value was set properly, the logic
that checked whether the packet should be trapped to agent or not
based on the presence of label (basically fabric arp responses should
be trapped or not), misbehaved since label information is not saved
pre-GRO and hence not available in the metadata post GRO. For now,
fix the specific logic by checking whether the egress vrf is different
from the ingress vrf, which will be the case since physical interface
vrf will not be the same as vm's vrf.

Change-Id: Iba000889039bc8a5020fc11a462ba1b1a68ce1c8
Closes-BUG: #1551382

Do not reuse forwarding metadata post mirroring

A mirror action needs its own forwarding metadata since the forwarding
parameeters that are used for mirroring will be different from those
that are used for packet forwarding. In this particular case, the vrf
to which the packets are mirrored are different from the vrf to which
packets need to be classified, and hence the packets were dropped in the
flow module with no source route as the drop reason.

Change-Id: I21930818a5cdec560818acb79f671ce29ac8bc85
Closes-BUG: #1552101

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/18568
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18568
Committed: http://github.org/Juniper/contrail-vrouter/commit/4a635e41737a14dba4cf191d50e338ffa8e01bc3
Submitter: Zuul
Branch: R2.21.x

commit 4a635e41737a14dba4cf191d50e338ffa8e01bc3
Author: Divakar <email address hidden>
Date: Fri Nov 13 19:45:28 2015 +0530

Not processing unknown unicast ARP reply packets

Currently, an ARP reply, which is an unicast packet is attempted to be
flooded in Multicast tree, if it is an unknown unicast. But as part of
L2 multicast nexthop processing, this ARP packet is processed as if this
has come from VM to Vrouter and trapped to Agent.

As a fix, an ARP reply is never processed in Vrouter, unless it is
destined to Vrouter's MAC or Vhosts mac. This is identified using
fmd_to_me in forwarding metadata.

Change-Id: I67613ab8ed21a6661bc7131ef0f9fcccca3b17c5
closes-bug: #1516026
closes-bug: #1514703

Post GRO, label in fmd can't be used to check whether the packet
was tunneled or not

Once the packet is submitted for GRO, all datapath information is
lost. Post GRO, only values that are saved in the packet are the
vif and the nexthop. vif is a recent addition to the saved
information that helped us to identify which interface the packet
came from originally. Once the vif value was set properly, the logic
that checked whether the packet should be trapped to agent or not
based on the presence of label (basically fabric arp responses should
be trapped or not), misbehaved since label information is not saved
pre-GRO and hence not available in the metadata post GRO. For now,
fix the specific logic by checking whether the egress vrf is different
from the ingress vrf, which will be the case since physical interface
vrf will not be the same as vm's vrf.

Change-Id: Iba000889039bc8a5020fc11a462ba1b1a68ce1c8
Closes-BUG: #1551382

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/20223
Submitter: Divakar Dharanalakota (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/20223
Committed: http://github.org/Juniper/contrail-vrouter/commit/91b6f0fe1fe7ad483647668fe2108f9df1993eeb
Submitter: Zuul
Branch: R2.22.x

commit 91b6f0fe1fe7ad483647668fe2108f9df1993eeb
Author: Anand H. Krishnan <email address hidden>
Date: Tue Mar 1 00:35:52 2016 +0530

Post GRO, label in fmd can't be used to check whether the packet
was tunneled or not

Once the packet is submitted for GRO, all datapath information is
lost. Post GRO, only values that are saved in the packet are the
vif and the nexthop. vif is a recent addition to the saved
information that helped us to identify which interface the packet
came from originally. Once the vif value was set properly, the logic
that checked whether the packet should be trapped to agent or not
based on the presence of label (basically fabric arp responses should
be trapped or not), misbehaved since label information is not saved
pre-GRO and hence not available in the metadata post GRO. For now,
fix the specific logic by checking whether the egress vrf is different
from the ingress vrf, which will be the case since physical interface
vrf will not be the same as vm's vrf.

Change-Id: Iba000889039bc8a5020fc11a462ba1b1a68ce1c8
Closes-BUG: #1551382

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/20241
Submitter: Divakar Dharanalakota (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/20241
Committed: http://github.org/Juniper/contrail-vrouter/commit/bba45f77c2b06b8b84c92054f3a17a67abe7969a
Submitter: Zuul
Branch: R2.20

commit bba45f77c2b06b8b84c92054f3a17a67abe7969a
Author: Anand H. Krishnan <email address hidden>
Date: Tue Mar 1 00:35:52 2016 +0530

Post GRO, label in fmd can't be used to check whether the packet
was tunneled or not

Once the packet is submitted for GRO, all datapath information is
lost. Post GRO, only values that are saved in the packet are the
vif and the nexthop. vif is a recent addition to the saved
information that helped us to identify which interface the packet
came from originally. Once the vif value was set properly, the logic
that checked whether the packet should be trapped to agent or not
based on the presence of label (basically fabric arp responses should
be trapped or not), misbehaved since label information is not saved
pre-GRO and hence not available in the metadata post GRO. For now,
fix the specific logic by checking whether the egress vrf is different
from the ingress vrf, which will be the case since physical interface
vrf will not be the same as vm's vrf.

Change-Id: Iba000889039bc8a5020fc11a462ba1b1a68ce1c8
Closes-BUG: #1551382

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.