DPDK: agent core on one of nodes Cannot access memory at address 0x303e22676e69727c
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Juniper Openstack | Status tracked in Trunk | |||||
R3.0 |
Fix Committed
|
High
|
Hari Prasad Killi | |||
Trunk |
Fix Committed
|
High
|
Hari Prasad Killi |
Bug Description
On Upgrading from 3.0.2 Build 43 - 44 kilo , see agent crash on one node with the below trace
Cannot access memory at address 0x303e22676e69727c
eading symbols from ./contrail-
warning: core file may not match specified executable file.
[New LWP 30591]
[New LWP 30595]
[New LWP 30592]
[New LWP 4576]
[New LWP 11792]
[New LWP 11790]
[New LWP 25339]
[New LWP 4809]
[New LWP 25338]
[New LWP 30593]
[New LWP 30572]
[New LWP 11793]
[New LWP 28614]
[New LWP 30589]
[New LWP 30588]
[New LWP 30590]
[New LWP 11791]
[New LWP 30594]
Cannot access memory at address 0x303e22676e69727c
Cannot access memory at address 0x303e22676e697274
(gdb) bt
#0 0x00007fe9c6b73c37 in ?? ()
#1 0x00007fe9c6b77028 in ?? ()
#2 0x0000000000000020 in ?? ()
#3 0x0000000000000000 in ?? ()
(gdb)
Core: /cs-shared/
Version : 3.0.3.44 Kilo Ubunu 14.04.4
information type: | Proprietary → Public |
description: | updated |
see it again on upgrade from 43 to 45 libc_sigaction (sig=4032, act=0xfd7, oact=0x6) at ../sysdeps/ unix/sysv/ linux/x86_ 64/sigaction. c:53 tree_insert_ and_rebalance( bool, std::_Rb_ tree_node_ base*, std::_Rb_ tree_node_ base*, std::_Rb_ tree_node_ base&) () x86_64- linux-gnu/ libstdc+ +.so.6 64-linux- gnu/libc. so.6 src/vnsw/ agent/oper/ inet_unicast_ route.cc" , assertion=0x15d <error: Cannot access memory at address 0x15d>, teEntry: :ModifyEcmpPath (boost: :asio:: ip::address const&, unsigned char, std::set< std::string, std::less< std::string> , std::allocator< std::string> > const&, unsigned int, bool, std::string const&, std::vector<int, std::allocator<int> >, std::vector< std::string, std::allocator< std::string> > const&, PathPreference const&, unsigned int, EcmpLoadBalance const&, DBRequest&, Agent*, AgentPath*)+414> "轤\331\ 377H\213< $H\211\ 303H\205\ 377t\005\ 350\\\231\ 331\377H\ 211\337\ 350\024\ 233\331\ 377\017\ 037@", line=32635, eEntry: :DeleteComponen tNH(Agent* , AgentPath*) () eEntry: :EcmpDeletePath (AgentPath* ) () ::DeletePathFro mPeer(DBTablePa rtBase* , AgentRoute*, AgentPath*) () ::Input( DBTablePartitio n*, DBClient*, DBRequest*) () tRouteTable: :Delete( Peer const*, std::string const&, boost:: asio::ip: :address const&, unsigned char) () :DeleteIpv4Inst anceIp( bool, unsigned int, VrfEntry*) () :DeleteL3( bool, VrfEntry*, boost:: asio::ip: :address_ v4 const&, bool, bool, boost:: asio::ip: :address_ v6 const&, boost:: asio::ip: :address_ v4 const&, unsigned char, int, boost:: asio::ip: :address_ v4 const&) () :ApplyConfig( bool, bool, bool, VrfEntry*, boost:: asio::ip: :address_ v4 const&, int, bool, bool, boost:: asio::ip: :address_ v6 const&, boost:: asio::ip: :address_ v4 const&, unsigned char, bool, bool, bool, boost:: asio::ip: :address_ v4 const&, bool, bool) () :Resync( InterfaceTable const*, VmInterfaceData const*) () e::Resync( DBEntry* , DBRequest const*) () :Input( DBTablePartitio n*, DBClient*, DBRequest*) () :QueueRunner: :Run() ()
core in the same location
f7bd774ccc9 in __GI___
#1 0x00007f7bd806bd04 in std::_Rb_
from /usr/lib/
#2 0x00007f7bda4d2000 in ?? ()
#3 0x00007f7bd7896830 in ?? () from /lib/x86_
#4 0x0000000001245cd0 in ?? ()
#5 0x00000000011f2580 in ?? ()
#6 0x00007f7bcaffa340 in ?? ()
#7 0x0000000000000011 in ?? ()
#8 0x00007f7bd7745b86 in __assert_fail_base (fmt=0x0, assertion=0x0, file=<optimized out>, line=<optimized out>, function=<optimized out>) at assert.c:68
#9 0x00007f7bd7745c32 in __assert_fail_base (fmt=0x11f1bb0 "controller/
file=0xa1339e <InetUnicastRou
function=0x4 <error: Cannot access memory at address 0x4>) at assert.c:53
#10 0x00007f7bcaffa340 in ?? ()
#11 0x00007f7bcaffa320 in ?? ()
#12 0x0000000000a13d86 in InetUnicastRout
#13 0x0000000000a1433e in InetUnicastRout
#14 0x00000000009d1654 in AgentRouteTable
#15 0x00000000009d2c0d in AgentRouteTable
#16 0x0000000000a1153b in InetUnicastAgen
#17 0x0000000000a8d2c3 in VmInterface:
#18 0x0000000000a96184 in VmInterface:
#19 0x0000000000a983e4 in VmInterface:
#20 0x0000000000a987b8 in VmInterface:
#21 0x00000000009fcd3a in AgentOperDBTabl
#22 0x0000000000acec7c in AgentDBTable:
#23 0x000000000109f2b6 in DBPartition:
#24 0x00000000011b0cff in TaskImpl::execute() ()
#25 0x00007f7bd831bb3a in ?? () from /usr/...