Comment 0 for bug 1872118

Revision history for this message
Andrew Welham (andreww-ubuntu) wrote : DHCP Cluster crashes after a few hours

I have a pair of DHCP serevrs running in a cluster on ubuntu 20.04, All worked perfectly until recently, when they started stopping with code=killed, status=6/ABRT.
This is being fixed by

https://bugs.launchpad.net/bugs/1870729

However now one stops after a few hours with the following errors. One can stay on line but not both.

Syslog shows
Apr 10 17:20:15 dhcp-primary sh[6828]: ../../../../lib/isc/unix/socket.c:3361: INSIST(!sock->pending_send) failed, back trace
Apr 10 17:20:15 dhcp-primary sh[6828]: #0 0x7fbe78702a4a in ??
Apr 10 17:20:15 dhcp-primary sh[6828]: #1 0x7fbe78702980 in ??
Apr 10 17:20:15 dhcp-primary sh[6828]: #2 0x7fbe7873e7e1 in ??
Apr 10 17:20:15 dhcp-primary sh[6828]: #3 0x7fbe784e5609 in ??
Apr 10 17:20:15 dhcp-primary sh[6828]: #4 0x7fbe78621103 in ??

nothing in kern.log

apport.log shows
ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: called for pid 6828, signal 6, core limit 0, dump mode 2
ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: not creating core for pid with dump mode of 2
ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: executable: /usr/sbin/dhcpd (command line "dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf")
ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: is_closing_session(): no DBUS_SESSION_BUS_ADDRESS in environment
ERROR: apport (pid 6850) Fri Apr 10 17:20:15 2020: wrote report /var/crash/_usr_sbin_dhcpd.0.crash

/var/crash/_usr_sbin_dhcpd.0.crash shows

ProblemType: Crash
Architecture: amd64
CrashCounter: 1
Date: Fri Apr 10 17:20:15 2020
DistroRelease: Ubuntu 20.04
ExecutablePath: /usr/sbin/dhcpd
ExecutableTimestamp: 1586210315
ProcCmdline: dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf
ProcEnviron: Error: [Errno 13] Permission denied: 'environ'
ProcMaps: Error: [Errno 13] Permission denied: 'maps'
ProcStatus:
 Name: dhcpd
 Umask: 0022
 State: D (disk sleep)
 Tgid: 6828
 Ngid: 0
 Pid: 6828
 PPid: 1
 TracerPid: 0
 Uid: 113 113 113 113
 Gid: 118 118 118 118
 FDSize: 128
 Groups:
 NStgid: 6828
 NSpid: 6828
 NSpgid: 6828
 NSsid: 6828
 VmPeak: 236244 kB
 VmSize: 170764 kB
 VmLck: 0 kB
 VmPin: 0 kB
 VmHWM: 12064 kB
 VmRSS: 12064 kB
 RssAnon: 5940 kB
 RssFile: 6124 kB
 RssShmem: 0 kB
 VmData: 30792 kB
 VmStk: 132 kB
 VmExe: 592 kB
 VmLib: 5424 kB
 VmPTE: 76 kB
 VmSwap: 0 kB
 HugetlbPages: 0 kB
 CoreDumping: 1
 THP_enabled: 1
 Threads: 4
 SigQ: 0/7609
 SigPnd: 0000000000000000
 ShdPnd: 0000000000000000
 SigBlk: 0000000000000000
 SigIgn: 0000000000001000
 SigCgt: 0000000180000000
 CapInh: 0000000000000000
 CapPrm: 0000000000000000
 CapEff: 0000000000000000
 CapBnd: 0000003fffffffff
 CapAmb: 0000000000000000
 NoNewPrivs: 0
 Seccomp: 0
 Speculation_Store_Bypass: thread vulnerable
 Cpus_allowed: 3
 Cpus_allowed_list: 0-1
 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,0000
0000,00000000,00000000,00000001
 Mems_allowed_list: 0
 voluntary_ctxt_switches: 111
 nonvoluntary_ctxt_switches: 144
Signal: 6
Uname: Linux 5.4.0-21-generic x86_64
UserGroups: