playback crashes on LInux with 'Out of memory'
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MOVED - Percona Playback |
New
|
Undecided
|
Unassigned |
Bug Description
[In:Percona Playback Documentation]
Hi,
I am currently trying to run Playback tool but every time it gets killed by OOM due to memory exhaustion. I have tried the version, which comes with Debian Squeeze(
http://
The result is the same. Moreover, if I run Playback on the server itself, sometimes MariaDB gets killed by OOM which is pretty bad.
OS details
Linux wkwvdb-fra2 2.6.32-5-amd64 #1 SMP Fri May 10 08:43:19 UTC 2013 x86_64 GNU/Linux
and I was testing it against MariaDB 5.5:
whatever@
ii libmariadbclient18 5.5.31+
ii libmysqlclient18 5.5.31+
ii mariadb-client-5.5 5.5.31+
ii mariadb-
ii mariadb-common 5.5.31+
ii mariadb-server 5.5.31+
ii mariadb-server-5.5 5.5.31+
ii mariadb-
I used the following line to start playback:
./percona-playback --input-plugin tcpdump --tcpdump-file /data/db1.tcpdump --tcpdump-mode accurate --db-plugin libmysqlclient --mysql-host my_hidden_ip --mysql-username root --mysql-schema imdb --mysql-port 3306
The syslog has the following:
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.002307] lt-percona-play invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.005245] lt-percona-play cpuset=/ mems_allowed=0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.007048] Pid: 5047, comm: lt-percona-play Not tainted 2.6.32-5-amd64 #1
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.009311] Call Trace:
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.010304] [<ffffffff810b6
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.012217] [<ffffffff810b6
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.014165] [<ffffffff810b6
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.016026] [<ffffffff810ba
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.018278] [<ffffffff810cd
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.020411] [<ffffffff810d2
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.022230] [<ffffffff81300
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.024201] [<ffffffff812fe
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.026074] Mem-Info:
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.027076] Node 0 DMA per-cpu:
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.028456] CPU 0: hi: 0, btch: 1 usd: 0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.030230] Node 0 DMA32 per-cpu:
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.031598] CPU 0: hi: 186, btch: 31 usd: 30
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.033402] active_anon:124262 inactive_
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.033403] active_file:69 inactive_file:174 isolated_file:0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.033403] unevictable:0 dirty:1 writeback:1 unstable:0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.033404] free:1998 slab_reclaimabl
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.033405] mapped:61 shmem:0 pagetables:1814 bounce:0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.043148] Node 0 DMA free:4036kB min:60kB low:72kB high:88kB active_anon:5744kB inactive_
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.055187] lowmem_reserve[]: 0 994 994 994
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.057034] Node 0 DMA32 free:3956kB min:4000kB low:5000kB high:6000kB active_
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.069915] lowmem_reserve[]: 0 0 0 0
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.071390] Node 0 DMA: 1*4kB 6*8kB 9*16kB 10*32kB 7*64kB 4*128kB 2*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4036kB
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.075570] Node 0 DMA32: 477*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3964kB
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.107716] 2501 total pagecache pages
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.109426] 2243 pages in swap cache
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.110934] Swap cache stats: add 2608278, delete 2606035, find 60756/96530
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.113557] Free swap = 0kB
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.114871] Total swap = 1044216kB
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.120817] 262139 pages RAM
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.122226] 5393 pages reserved
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.123611] 125 pages shared
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.124970] 254553 pages non-shared
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.126417] Out of memory: kill process 5047 (lt-percona-play) score 42951 or a child
Jul 5 16:50:17 wkwvdb-fra2 kernel: [1293580.129494] Killed process 5047 (lt-percona-play)
I also ran playback with strace. It sends queries to server but seems not to properly free the memory.
Do I do something wrong or is it a bug?
Have a nice weekend,
Sergey
and the dump has been created using
tcpdump -i any port 3306 -w example.dump
as explained in http:// www.percona. com/doc/ percona- playback/ plugins/ tcpdump/ index.html