Lucid (10.04) ext4 kernel panic under extreme load
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OEM Priority Project |
Invalid
|
Undecided
|
Unassigned | ||
Ubuntu |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
This appears to affect the lucid kernel (2.6.32-33-server) regardless of the backing store, either physical or virtual. Stacking block devices
are also not a variable e.g. LVM or multipath.
To reproduce:
1) Create the following FIO test file (call it say bigfio.fio)
[global]
rw=randread
size=16g
directory=
ioengine=libaio
iodepth=4
invalidate=1
direct=1
[bgwriter]
rw=randwrite
iodepth=32
[queryA]
iodepth=1
ioengine=mmap
direct=0
thinktime=3
[queryB]
iodepth=1
ioengine=mmap
direct=0
thinktime=5
[bgupdater]
rw=randrw
iodepth=16
thinktime=40
size=128m
2) # apt-get install -y fio
3) # mkdir /var/lib/TESTDATA
4) # fio bigfio.fio
The time to failure can vary, after an hour or two the kernel panics here:
KERNEL: syms/usr/
DUMPFILE: dump/VmCore
CPUS: 4
DATE: Thu Oct 13 10:22:05 2011
UPTIME: 01:08:14
LOAD AVERAGE: 2.54, 2.21, 2.19
TASKS: 151
NODENAME: toyon
RELEASE: 2.6.32-33-server
VERSION: #72-Ubuntu SMP Fri Jul 29 21:21:55 UTC 2011
MACHINE: x86_64 (2333 Mhz)
MEMORY: 4 GB
PANIC: "[ 4092.217111] kernel BUG at /build/
PID: 314
COMMAND: "ext4-dio-unwrit"
TASK: ffff880127fc0000 [THREAD_INFO: ffff880127452000]
CPU: 0
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 314 TASK: ffff880127fc0000 CPU: 0 COMMAND: "ext4-dio-unwrit"
#0 [ffff880127453bb0] die at ffffffff8101712b
#1 [ffff880127453be0] do_trap at ffffffff8155acc4
#2 [ffff880127453c40] do_invalid_op at ffffffff81014ee5
#3 [ffff880127453ce0] invalid_op at ffffffff81013f7b
[exception RIP: kfree+300]
RIP: ffffffff81137c9c RSP: ffff880127453d90 RFLAGS: 00010246
RAX: 0200000000000000 RBX: ffffffff811dd7a7 RCX: 0000000000000021
RDX: 0000000000952318 RSI: ffffea000413f5a8 RDI: ffff88012a463300
RBP: ffff880127453db0 R8: 0000000000000000 R9: feb6f83d31fb6402
R10: 0000000000000000 R11: 0000000000000000 R12: ffff88012a463300
R13: ffff88011fc0d608 R14: ffffe8ffffc04340 R15: ffff880127453fd8
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#4 [ffff880127453db8] ext4_free_io_end at ffffffff811dd7a7
#5 [ffff880127453dd8] ext4_end_
#6 [ffff880127453e08] run_workqueue at ffffffff810814f7
#7 [ffff880127453e68] worker_thread at ffffffff81081673
#8 [ffff880127453ee8] kthread at ffffffff81085d26
#9 [ffff880127453f48] kernel_thread at ffffffff810141ea
Logs to follow...