[Build CentOS release 6.5 2.10-37~icehouse.el6] Multi-Interface Sanity - vrouter crashed at NhDecode(NextHop const*, PktInfo const*, PktFlowInfo*, PktControlInfo*, PktControlInfo*, bool)

Bug #1425856 reported by Sandip Dey
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Fix Committed
High
Naveen N
R2.0
Fix Committed
High
Naveen N
R2.1
Fix Committed
High
Naveen N

Bug Description

Logs saved at :http://10.204.216.50/Docs/bugs/<bug-id>

BT
===
#0 0x000000000123fc8c in NextHop::id() const ()
#0 0x000000000123fc8c in NextHop::id() const ()
#1 0x00000000014aaedf in NhDecode(NextHop const*, PktInfo const*, PktFlowInfo*, PktControlInfo*, PktControlInfo*, bool) ()
#2 0x00000000014ab24a in RouteToOutInfo(AgentRoute const*, PktInfo const*, PktFlowInfo*, PktControlInfo*, PktControlInfo*) ()
#3 0x00000000014ad37f in PktFlowInfo::IngressProcess(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#4 0x00000000014adeb8 in PktFlowInfo::Process(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#5 0x00000000014b91e7 in FlowHandler::Run() ()
#6 0x00000000014b511f in Proto::ProcessProto(boost::shared_ptr<PktInfo>) ()
#7 0x00000000014b7543 in boost::_mfi::mf1<bool, Proto, boost::shared_ptr<PktInfo> >::operator()(Proto*, boost::shared_ptr<PktInfo>) const ()
#8 0x00000000014b6f04 in bool boost::_bi::list2<boost::_bi::value<Proto*>, boost::arg<1> >::operator()<bool, boost::_mfi::mf1<bool, Proto, boost::shared_ptr<PktInfo> >, boost::_bi::list1<boost::shared_ptr<PktInfo>&> >(boost::_bi::type<bool>, boost::_mfi::mf1<bool, Proto, boost::shared_ptr<PktInfo> >&, boost::_bi::list1<boost::shared_ptr<PktInfo>&>&, long) ()
#9 0x00000000014b67ba in bool boost::_bi::bind_t<bool, boost::_mfi::mf1<bool, Proto, boost::shared_ptr<PktInfo> >, boost::_bi::list2<boost::_bi::value<Proto*>, boost::arg<1> > >::operator()<boost::shared_ptr<PktInfo> >(boost::shared_ptr<PktInfo>&) ()
#10 0x00000000014b62a9 in boost::detail::function::function_obj_invoker1<boost::_bi::bind_t<bool, boost::_mfi::mf1<bool, Proto, boost::shared_ptr<PktInfo> >, boost::_bi::list2<boost::_bi::value<Proto*>, boost::arg<1> > >, bool, boost::shared_ptr<PktInfo> >::invoke(boost::detail::function::function_buffer&, boost::shared_ptr<PktInfo>) ()
#11 0x00000000014a9c2f in boost::function1<bool, boost::shared_ptr<PktInfo> >::operator()(boost::shared_ptr<PktInfo>) const ()
#12 0x00000000014b7928 in QueueTaskRunner<boost::shared_ptr<PktInfo>, WorkQueue<boost::shared_ptr<PktInfo> > >::RunQueue() ()
#13 0x00000000014b7760 in QueueTaskRunner<boost::shared_ptr<PktInfo>, WorkQueue<boost::shared_ptr<PktInfo> > >::Run() ()
#14 0x0000000001a84676 in TaskImpl::execute() ()
#15 0x00002b4ae65b018a in tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all (this=0x2b4b40000900, parent=...,
    child=0x41e26c0)
    at /ecbuilds/PipeLine/sb/third_party/tbb40_20111130oss/src/tbb/custom_scheduler.h:449
#16 0x00002b4ae65a7033 in tbb::internal::arena::process (this=0x41a8980, s=...)
    at /ecbuilds/PipeLine/sb/third_party/tbb40_20111130oss/src/tbb/arena.cpp:99
#17 0x00002b4ae65a5906 in tbb::internal::market::process (this=0x41a6500,
    j=...)
    at /ecbuilds/PipeLine/sb/third_party/tbb40_20111130oss/src/tbb/market.cpp:393
#18 0x00002b4ae65a04bc in tbb::internal::rml::private_worker::run (
    this=0x41a7200)
    at /ecbuilds/PipeLine/sb/third_party/tbb40_20111130oss/src/tbb/private_server.cpp:263
#19 0x00002b4ae65a0362 in tbb::internal::rml::private_worker::thread_routine (

Tags: sanity vrouter
Revision history for this message
Naveen N (naveenn) wrote :

Though there was only one vgw in setup exporting 0/0 route, agent got a ecmp route for 0/0 route,
thought both next hop and label are same,there encap types differ.

<message from=\"<email address hidden>\" to=\"nodei5/bgp-peer\">
    <event xmlns=\"http://jabber.org/protocol/pubsub\">
        <items node=\"1/1/default-domain:admin:vgwvn:vgwvn\">
            <item id=\"0.0.0.0/0\">
                <entry>
                    <nlri>
                        <af>1</af>
                        <safi>1</safi>
                        <address>0.0.0.0/0</address>
                    </nlri>
                    <next-hops>
                        <next-hop>
                            <af>1</af>
                            <address>192.168.192.5</address>
                            <label>17</label>
                            <tunnel-encapsulation-list>
                                <tunnel-encapsulation>gre</tunnel-encapsulation>
                            </tunnel-encapsulation-list>
                        </next-hop>
                        <next-hop>
                            <af>1</af>
                            <address>192.168.192.5</address>
                            <label>17</label>
                            <tunnel-encapsulation-list>
                                <tunnel-encapsulation>gre</tunnel-encapsulation>
                                <tunnel-encapsulation>udp</tunnel-encapsulation>
                            </tunnel-encapsulation-list>
                        </next-hop>
                    </next-hops>
                    <version>1</version>
                    <virtual-network>default-domain:admin:vgwvn</virtual-network>
                    <sequence-number>0</sequence-number>
                    <security-group-list />
                    <local-preference>100</local-preference>
                </entry>
            </item>
        </items>

If there is a ecmp of vgw and traffic originates first from outside to contrail-domain VM,
agent was trying to get policy enabled next hop for reverse flow setup which would not
be created for vgw.
Fix for to take policy enabled next hop for reverse flow only destination is vm interface.

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/7875
Committed: http://github.org/Juniper/contrail-controller/commit/f000a7fcd9c99fff809c87d3f30ed7e162ace1a8
Submitter: Zuul
Branch: R2.1

commit f000a7fcd9c99fff809c87d3f30ed7e162ace1a8
Author: Naveen N <email address hidden>
Date: Thu Feb 26 05:57:20 2015 -0800

* In a scenario where there are 2 vgw exporting same subnet route,
and traffic session gets initiated from external domain to
VM, then for setting reverse flow agent was trying to get policy
enabled interface nexthop for vgw, and for vgw we would only
create policy disabled nh. Fixing the same to pick
policy disabled nexthop for vgw destination
Closes-bug:#1425856

Change-Id: Ic0c61be47997a2c9ebe3196394cef21468fce8ca

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/7905
Committed: http://github.org/Juniper/contrail-controller/commit/d0414d2e6eeae350ecf4189f894c475e0ef00d21
Submitter: Zuul
Branch: R2.0

commit d0414d2e6eeae350ecf4189f894c475e0ef00d21
Author: Naveen N <email address hidden>
Date: Fri Feb 27 01:58:02 2015 -0800

* In a scenario where there are 2 vgw exporting same subnet route,
and traffic session gets initiated from external domain to
VM, then for setting reverse flow agent was trying to get policy
enabled interface nexthop for vgw, and for vgw we would only
create policy disabled nh. Fixing the same to pick
policy disabled nexthop for vgw destination
Closes-bug:#1425856

Change-Id: I23462aec80cc7fc351c376705f2fb2dc062acf16

information type: Proprietary → Public
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : master
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged
Download full text (5.5 KiB)

Reviewed: https://review.opencontrail.org/8032
Committed: http://github.org/Juniper/contrail-controller/commit/9d44b8f521cf02f70ecb7cd628754c64b8de3b9a
Submitter: Zuul
Branch: master

commit 9d44b8f521cf02f70ecb7cd628754c64b8de3b9a
Author: Hari <email address hidden>
Date: Tue Mar 3 21:29:48 2015 -0800

Merge the following commits from R2.1 branch (final).

Perf improvements for Logical Interface delete handling

Follow up on commit b4af4d67cef56ac85da5bb0d50bb55f882c70538 for delete
handling

When a link delete is notified to config listener, the IFMapNode in link
can be NULL. We used to ignore links with left/right node deleted.

Modified the link handler to pass the node-type when left/right node is
deleted. This way, the IFLinkToReq handler can take action based on the
type of node being deleted
(cherry picked from commit f1d7816dbdc886859caec5152d2b436ad4fb2a96)

Conflicts:
src/vnsw/agent/test-xml/test_xml_oper.cc

Holding the UUID from ID_PERMS in config DBState in DB Add and using it
for Config Delete
closes-bug: #1420209
(cherry picked from commit 6a9b4be6a0dcfb3980bd072a57f92761a548eb9c)

* Send ARP request packet on VMI interface instead of parent interface
such that vlan gets added, based on transmit vlan id
* If vlan-id is 0, then its treated as untagged interface, in agent
while framing the packet we were setting the tag in packet 0
avoiding the same same setting vlan id as invalid and programming
it as vlan-id 0 only in vrouter
Closes-bug:#1423122
(cherry picked from commit 38f1405b27e05e5d31a2ae6297ca42d915b368bd)

* If arp route is deleted twice, then we arp nexthop was not
getting deleted, since upon each arp route delete we enqueue
a request to modify nexthop to invalid and this might
result in arp entry creation, hence use operation RESYNC
instead of add/change
* Check for vrf while parsing on subnet address change, since
we might modify next vrf subnet routes if prefixx matches
Test case for same
Closes-bug:#1423502
(cherry picked from commit a65d557c27d15de5d7d0bf30fdc5a20fa93acf42)

Conflicts:
src/vnsw/agent/test/test_route.cc

reverting the unwated checkin of modifying the notify_count_ to bool

(cherry picked from commit 975f507d81ce89106814f4fced1d14b454dcdd5a)

* Whenever ICMP error packet is trapped for flow setup,
parse the inner payload and frame the flow key
Closes-bug:#1423848
(cherry picked from commit 81ee64728201b1d8234d790772ab3035874c1f5d)

* ARP route behind gateway should have proxy flag set
such that arp request on VM would be proxied by vrouter
Closes-bug:#1423122
(cherry picked from commit e72443e1a73d3372944981f15941e97a5a9f439f)

Remove unused UVEs

Remove UVE generated on add/delete of a link
Remove UVE generated on change of physical-device-vn entry.
Added a UT for physical-device-vn to validate add/delete of VMI to VN link

(cherry picked from commit 452b9341668f0a89455cc5e1847c0c7d70f0d3b2)

Holding the metadata of link while storing it in defer list. While
adding the link to Graph from defer list, the metadata also added back
to graph which makes the dependency tracker to work correctly

(cherry picked from commit 43d7a71e1c486a3c326d16e16134c25312b4953f)

Optimize config handling for SG and...

Read more...

Changed in juniperopenstack:
status: New → Fix Committed
Changed in juniperopenstack:
importance: Undecided → High
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.