Worth noting that we ran for many months before hitting this, and then:
ubuntu@juju-prod-ues-proposed-migration-machine-1:~$ dmesg -T | grep "Out of memory: Kill" | uniq
[Wed May 9 16:06:23 2018] Out of memory: Kill process 1408 (beam) score 215 or sacrifice child
[Sun May 20 03:58:24 2018] Out of memory: Kill process 19495 (beam) score 437 or sacrifice child
[Fri Jun 1 02:28:15 2018] Out of memory: Kill process 6569 (beam) score 428 or sacrifice child
[Fri Jun 8 05:37:54 2018] Out of memory: Kill process 1142 (beam) score 434 or sacrifice child
4 times in a month.
One thing that happened "sort of" around this time is
Worth noting that we ran for many months before hitting this, and then:
ubuntu@ juju-prod- ues-proposed- migration- machine- 1:~$ dmesg -T | grep "Out of memory: Kill" | uniq
[Wed May 9 16:06:23 2018] Out of memory: Kill process 1408 (beam) score 215 or sacrifice child
[Sun May 20 03:58:24 2018] Out of memory: Kill process 19495 (beam) score 437 or sacrifice child
[Fri Jun 1 02:28:15 2018] Out of memory: Kill process 6569 (beam) score 428 or sacrifice child
[Fri Jun 8 05:37:54 2018] Out of memory: Kill process 1142 (beam) score 434 or sacrifice child
4 times in a month.
One thing that happened "sort of" around this time is
https:/ /launchpad. net/ubuntu/ +source/ erlang/ 1:18.3- dfsg-1ubuntu3. 1
but I have a vague correlation here, no causation.
Does seem like rabbit's memory usage grows over time until it's eventually killed.