I accidentally noticed that one of my Ubuntu 14.04 LTS servers hasn't been applying unattended updates for a while. Some digging shows that this is because update-apt-xapian-index started on August 27 is still running (and eating 100% CPU, for over two weeks)!, which blocks all other /etc/cron.daily/ scripts from running. Every night cron starts a new run, which gets blocked by a new instance of update-apt-xapian-index:
$ pstree
...
|-cron-+-16*[cron---sh---run-parts---apt---update-apt-xapi]
| |-2*[cron---sh---run-parts---apt-xapian-inde---update-apt-xapi]
| |-cron---sh---run-parts---apt-xapian-inde---update-apt-xapi---cat
| `-cron---sh---python
strace shows that it's repeatedly reading and writing the same locations in /var/cache/apt-xapian-index/index.2/postlist.DB:
pread64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pwrite64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pread64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pwrite64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pread64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pwrite64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pread64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pwrite64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pread64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pwrite64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pread64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pwrite64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pread64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pwrite64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pread64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pwrite64(83, "\0\0\0\3\0\10F\10F\0\21\30\34\10W\0209\20%\20\t\17\365\17\334\17\251\17\223\17~\17"..., 8192, 8871936) = 8192
pread64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
pwrite64(83, "\0\0\0\3\0\rH\rH\0=\30\25\r\205\30\1\27\345\27\321\27\270\27\205\27o\27Z\27\21\26"..., 8192, 8880128) = 8192
I poked around a bit with gdb and dumped the C and Python stack frames, but I'm not sure what to look for.
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: apt-xapian-index 0.45ubuntu4
ProcVersionSignature: Ubuntu 3.13.0-128.177-generic 3.13.11-ckt39
Uname: Linux 3.13.0-128-generic i686
ApportVersion: 2.14.1-0ubuntu3.25
Architecture: i386
Date: Tue Sep 12 13:21:42 2017
PackageArchitecture: all
SourcePackage: apt-xapian-index
UpgradeStatus: Upgraded to trusty on 2016-03-31 (530 days ago)
