IPv6 Fragments: If there are more than 3 fragments for the first ICMPv6 packet then the packet is dropped.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Juniper Openstack |
Won't Fix
|
High
|
Divakar Dharanalakota | ||
R3.0.3.x |
Won't Fix
|
High
|
Divakar Dharanalakota | ||
R3.2 |
Won't Fix
|
High
|
Divakar Dharanalakota | ||
R4.0 |
Won't Fix
|
High
|
Divakar Dharanalakota | ||
R4.1 |
Won't Fix
|
High
|
Divakar Dharanalakota | ||
Trunk |
Won't Fix
|
High
|
Divakar Dharanalakota |
Bug Description
Issue with flow processing, when out of order ICMPv6 fragment packets are received.
Build
———
R3.0.3.4 Build 28 Ubuntu 14.04 Mitaka
Topology
—————
Control/
Compute nodes : nodek11, nodec23
Steps
———
Create a VN and launch 2 VMs across 2 compute nodes
Send out of order ICMPv6 fragmented packets from one of VM to another
Flow processing happens for fragmented packets. But, not all fragments reach other compute node. Hence, ICMPv6 echo reply is missing.
With In order fragments everything works fine. Issue is seen with out of order fragments only.
Please see the log below:
root@nodek11:~# contrail-version
Package Version Build-ID | Repo | Package Name
-------
contrail-
contrail-
contrail-lib 3.0.3.4-28 28
contrail-nodemgr 3.0.3.4-28 28
contrail-nova-vif 3.0.3.4-28 28
contrail-
contrail-setup 3.0.3.4-28 28
contrail-utils 3.0.3.4-28 28
contrail-
contrail-
contrail-
contrail-
contrail-
nova-common 1:2015.
nova-compute 1:2015.
nova-compute-kvm 1:2015.
nova-compute-
python-contrail 3.0.3.4-28 28
python-
python-
python-nova 1:2015.
python-
root@nodek11:~# contrail-status
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_US:",
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
== Contrail vRouter ==
supervisor-vrouter: active
contrail-
contrail-
root@nodek11:~# vif --list
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload
vif0/0 OS: em1 (Speed 1000, Duplex 1)
Vrf:0 Flags:L3L2Vp MTU:1514 Ref:6
RX packets:928661 bytes:99300004 errors:0
TX packets:315286 bytes:209092111 errors:0
vif0/1 OS: vhost0
Vrf:0 Flags:L3L2 MTU:1514 Ref:3
RX packets:370669 bytes:148248611 errors:0
TX packets:974889 bytes:104096407 errors:0
vif0/2 OS: pkt0
RX packets:2255 bytes:325102 errors:0
TX packets:556699 bytes:57903207 errors:0
vif0/3 OS: tap6e876a77-f1
Vrf:1 Flags:PL3L2D MTU:9160 Ref:5
RX packets:61450 bytes:71071401 errors:0
TX packets:72761 bytes:5306683 errors:0
vif0/4350 OS: pkt3
RX packets:11628 bytes:1082027 errors:0
TX packets:11628 bytes:919235 errors:0
vif0/4351 OS: pkt1
RX packets:0 bytes:0 errors:0
TX packets:0 bytes:0 errors:0
root@nodek11:~#
root@nodek11:~# flow --match 3b7c:1f1:
Flow table(size 80609280, entries 629760)
Entries: Created 60 Added 42 Processed 60 Used Overflow entries 0
(Created Flows/CPU: 5 0 4 3 4 6 6 1 0 0 0 0 0 0 0 0 2 24 0 0 1 0 1 3 0 0 0 0 0 0 0 0)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Listing flows matching ([3b7c:
Index Source:
-------
root@nodek11:~#
Sent 5 out of order ICMPv6 Echo request fragmented packets from VM. Here is the tcpdump:
08:50:25.523345 02:6e:87:6a:77:f1 > 33:33:ff:00:00:04, ethertype IPv6 (0x86dd), length 86: 3b7c:1f1:
08:50:25.523369 02:65:ff:05:85:fa > 02:6e:87:6a:77:f1, ethertype IPv6 (0x86dd), length 86: 3b7c:1f1:
08:50:25.527323 02:6e:87:6a:77:f1 > 02:65:ff:05:85:fa, ethertype IPv6 (0x86dd), length 310: 3b7c:1f1:
08:50:25.532576 02:6e:87:6a:77:f1 > 02:65:ff:05:85:fa, ethertype IPv6 (0x86dd), length 310: 3b7c:1f1:
08:50:25.537062 02:6e:87:6a:77:f1 > 02:65:ff:05:85:fa, ethertype IPv6 (0x86dd), length 78: 3b7c:1f1:
08:50:25.541418 02:6e:87:6a:77:f1 > 02:65:ff:05:85:fa, ethertype IPv6 (0x86dd), length 310: 3b7c:1f1:
08:50:25.545775 02:6e:87:6a:77:f1 > 02:65:ff:05:85:fa, ethertype IPv6 (0x86dd), length 310: 3b7c:1f1:
Flow processing happens for fragmented packets
root@nodek11:~# flow --match 3b7c:1f1:
Flow table(size 80609280, entries 629760)
Entries: Created 62 Added 44 Processed 62 Used Overflow entries 0
(Created Flows/CPU: 5 0 5 3 4 6 6 1 0 0 0 0 0 0 0 0 3 24 0 0 1 0 1 3 0 0 0 0 0 0 0 0)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Listing flows matching ([3b7c:
Index Source:
-------
139600<=>450548 3b7c:1f1:
(Gen: 1, K(nh):13, Action:F, Flags:, S(nh):22, Stats:0/0, SPort 64665)
450548<=>139600 3b7c:1f1:
(Gen: 1, K(nh):13, Action:F, Flags:, S(nh):13, Stats:5/1318, SPort 50101)
root@nodek11:~#
Here is the scapy script:
root@ctest-
from scapy.all import *
from random import shuffle
fid=random.
I=IPv6(
ICMP=ICMPv6Echo
FH=IPv6ExtHdrFr
packets=
rand_arr = range(len(packets))
shuffle(rand_arr)
print rand_arr
import pdb;pdb.set_trace()
counter=1
for i in rand_arr:
fragment = packets[i]
print "Packet no#"+str(counter)
print "======
fragment.show() #displays each fragment
counter+=1
send(fragment)
root@ctest-
On another compute node (destination compute node), Not all fragments received.
root@nodec23:~# tcpdump -ne -i p1p1 host 10.204.216.231 -vvv -xxx
tcpdump: WARNING: p1p1: no IPv4 address assigned
tcpdump: listening on p1p1, link-type EN10MB (Ethernet), capture size 65535 bytes
08:52:53.740299 80:ac:ac:f0:a2:c1 > 00:25:90:c3:ae:b4, ethertype IPv4 (0x0800), length 356: (tos 0x0, ttl 63, id 52807, offset 0, flags [none], proto UDP (17), length 342)
10.
0x0000: 0025 90c3 aeb4 80ac acf0 a2c1 0800 4500
0x0010: 0156 ce47 0000 3f11 e4c7 0acc d8e7 0acc
0x0020: d908 cac8 c822 0142 0000 0001 1140 0265
0x0030: ff05 85fa 026e 876a 77f1 86dd 6000 0000
0x0040: 0100 2c40 3b7c 01f1 ce94 145f 0000 0000
0x0050: 0000 0003 3b7c 01f1 ce94 145f 0000 0000
0x0060: 0000 0004 3a00 0001 7e0d c359 8000 c624
0x0070: 0174 0000 4141 4141 4141 4141 4141 4141
0x0080: 4141 4141 4141 4141 4141 4141 4141 4141
0x0090: 4141 4141 4141 4141 4141 4141 4141 4141
0x00a0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00b0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00c0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00d0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00e0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00f0: 4141 4141 4141 4141 4141 4141 4141 4141
0x0100: 4141 4141 4141 4141 4141 4141 4141 4141
0x0110: 4141 4141 4141 4141 4141 4141 4141 4141
0x0120: 4141 4141 4141 4141 4141 4141 4141 4141
0x0130: 4141 4141 4141 4141 4141 4141 4141 4141
0x0140: 4141 4141 4141 4141 4141 4141 4141 4141
0x0150: 4141 4141 4141 4141 4141 4141 4141 4141
0x0160: 4141 4141
08:52:53.740336 80:ac:ac:f0:a2:c1 > 00:25:90:c3:ae:b4, ethertype IPv4 (0x0800), length 356: (tos 0x0, ttl 63, id 52809, offset 0, flags [none], proto UDP (17), length 342)
10.
0x0000: 0025 90c3 aeb4 80ac acf0 a2c1 0800 4500
0x0010: 0156 ce49 0000 3f11 e4c5 0acc d8e7 0acc
0x0020: d908 cac8 c822 0142 0000 0001 1140 0265
0x0030: ff05 85fa 026e 876a 77f1 86dd 6000 0000
0x0040: 0100 2c40 3b7c 01f1 ce94 145f 0000 0000
0x0050: 0000 0003 3b7c 01f1 ce94 145f 0000 0000
0x0060: 0000 0004 3a00 01f1 7e0d c359 4141 4141
0x0070: 4141 4141 4141 4141 4141 4141 4141 4141
0x0080: 4141 4141 4141 4141 4141 4141 4141 4141
0x0090: 4141 4141 4141 4141 4141 4141 4141 4141
0x00a0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00b0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00c0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00d0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00e0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00f0: 4141 4141 4141 4141 4141 4141 4141 4141
0x0100: 4141 4141 4141 4141 4141 4141 4141 4141
0x0110: 4141 4141 4141 4141 4141 4141 4141 4141
0x0120: 4141 4141 4141 4141 4141 4141 4141 4141
0x0130: 4141 4141 4141 4141 4141 4141 4141 4141
0x0140: 4141 4141 4141 4141 4141 4141 4141 4141
0x0150: 4141 4141 4141 4141 4141 4141 4141 4141
0x0160: 4141 4141
08:52:53.740344 80:ac:ac:f0:a2:c1 > 00:25:90:c3:ae:b4, ethertype IPv4 (0x0800), length 356: (tos 0x0, ttl 63, id 52811, offset 0, flags [none], proto UDP (17), length 342)
10.
0x0000: 0025 90c3 aeb4 80ac acf0 a2c1 0800 4500
0x0010: 0156 ce4b 0000 3f11 e4c3 0acc d8e7 0acc
0x0020: d908 cac8 c822 0142 0000 0001 1140 0265
0x0030: ff05 85fa 026e 876a 77f1 86dd 6000 0000
0x0040: 0100 2c40 3b7c 01f1 ce94 145f 0000 0000
0x0050: 0000 0003 3b7c 01f1 ce94 145f 0000 0000
0x0060: 0000 0004 3a00 00f9 7e0d c359 4141 4141
0x0070: 4141 4141 4141 4141 4141 4141 4141 4141
0x0080: 4141 4141 4141 4141 4141 4141 4141 4141
0x0090: 4141 4141 4141 4141 4141 4141 4141 4141
0x00a0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00b0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00c0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00d0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00e0: 4141 4141 4141 4141 4141 4141 4141 4141
0x00f0: 4141 4141 4141 4141 4141 4141 4141 4141
0x0100: 4141 4141 4141 4141 4141 4141 4141 4141
0x0110: 4141 4141 4141 4141 4141 4141 4141 4141
0x0120: 4141 4141 4141 4141 4141 4141 4141 4141
0x0130: 4141 4141 4141 4141 4141 4141 4141 4141
0x0140: 4141 4141 4141 4141 4141 4141 4141 4141
0x0150: 4141 4141 4141 4141 4141 4141 4141 4141
0x0160: 4141 4141
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@nodec23:~# flow -l
Flow table(size 80609280, entries 629760)
Entries: Created 66 Added 66 Processed 66 Used Overflow entries 0
(Created Flows/CPU: 18 22 16 10)(oflows 0)
Action:F=Forward, D=Drop N=NAT(S=SNAT, D=DNAT, Ps=SPAT, Pd=DPAT, L=Link Local Port)
Other:
Flags:E=Evicted, Ec=Evict Candidate, N=New Flow, M=Modified Dm=Delete Marked
TCP(r=reverse)
Index Source:
-------
268036<=>450992 3b7c:1f1:
(Gen: 1, K(nh):14, Action:F, Flags:, S(nh):21, Stats:3/930, SPort 55829)
359024<=>371184 3b7c:1f1:
(Gen: 1, K(nh):14, Action:F, Flags:, S(nh):14, Stats:1/358, SPort 50378)
371184<=>359024 3b7c:1f1:
(Gen: 1, K(nh):14, Action:F, Flags:, S(nh):21, Stats:3/930, SPort 60328)
450992<=>268036 3b7c:1f1:
(Gen: 1, K(nh):14, Action:F, Flags:, S(nh):14, Stats:0/0, SPort 49753)
root@nodec23:~#
On source compute seeing "Flow Queue Limit Exceeded” errors.
root@nodek11:~# dropstats
GARP 0
ARP no where to go 0
Invalid ARPs 0
Invalid IF 0
Trap No IF 0
IF TX Discard 0
IF Drop 0
IF RX Discard 0
Flow Unusable 0
Flow No Memory 0
Flow Table Full 0
Flow NAT no rflow 0
Flow Action Drop 18
Flow Action Invalid 0
Flow Invalid Protocol 0
Flow Queue Limit Exceeded 24
Flow Unusable (Eviction) 0
Original Packet Trapped 0
Discards 1
TTL Exceeded 0
Mcast Clone Fail 0
Cloned Original 34
Invalid NH 3
Invalid Label 0
Invalid Protocol 0
Rewrite Fail 0
Invalid Mcast Source 0
Push Fails 0
Pull Fails 0
Duplicated 0
Head Alloc Fails 0
Head Space Reserve Fails 0
PCOW fails 0
Invalid Packets 0
Misc 0
Nowhere to go 0
Checksum errors 0
No Fmd 0
Invalid VNID 0
Fragment errors 0
Invalid Source 0
Jumbo Mcast Pkt with DF Bit 0
ARP No Route 0
No L2 Route 2
Memory Failures 0
Fragment Queueing Failures 0
root@nodek11:~#
Changed in juniperopenstack: | |
milestone: | r3.0.3.4 → none |
tags: | added: operational |
Changed in juniperopenstack: | |
status: | New → Won't Fix |
summary: |
- IPv6 Fragments: Issue with flow processing, when out of order ICMPv6 - fragment packets are received. + IPv6 Fragments: If there are more than 3 fragments for the first ICMPv6 + packet then the packet is dropped. |
tags: | added: att-aic-contrail |
Review in progress for https:/ /review. opencontrail. org/36228
Submitter: Divakar Dharanalakota (<email address hidden>)