in all the 16 nova-ceph-multistore jobs that failed in the last 10 days the same test case got killed
$ grep Killed oom-kills.txt | cut -d ':' -f2 | sed 's|controller/logs/syslog.txt|job-output.txt|g' | xargs grep inprogress /home/gibi/.cache/logsearch/358f5a3fda114764b794323a2f5d2312/job-output.txt:2023-01-17 09:32:24.979704 | controller | {3} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/8587e6a4b6de4edb93de5c064928788e/job-output.txt:2023-01-16 12:00:25.032670 | controller | {2} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/ece0cf2ce71c4a8790a0a36529dd0a8e/job-output.txt:2023-01-14 23:15:05.647826 | controller | {1} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/f5aa5edd4d354c2685fc1f3e13d0ef77/job-output.txt:2023-01-13 23:07:03.726600 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/1447c6274e924e068578ca260c9ac2a6/job-output.txt:2023-01-13 22:00:29.545457 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/1bbcaa703b7d42c7a266fde3a6acca65/job-output.txt:2023-01-13 04:13:16.809645 | controller | {2} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/7d9ca42edc5e4bdeb17be8e8045c6468/job-output.txt:2023-01-12 22:32:21.335537 | controller | {3} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/bcb7bcbbbb3b478586906c31c6558b13/job-output.txt:2023-01-12 20:38:12.837475 | controller | {2} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/7572c2bf5e6547c0a1fc6b0f180a2e1f/job-output.txt:2023-01-12 18:17:48.522025 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/aa5cf699f8d04995b43d009e55a1accd/job-output.txt:2023-01-12 16:48:59.307916 | controller | {1} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/8bc71a0ec0d34373bd25d4f691136084/job-output.txt:2023-01-12 15:49:35.341633 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/81d7cef2e0b240f89fcfa727304d8e8d/job-output.txt:2023-01-12 15:35:03.978880 | controller | {1} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/c75eb700717b4d3c9942be1385cd45bf/job-output.txt:2023-01-11 21:26:49.653320 | controller | {1} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/fa2d7bea85ad4d29acc37d78d2adb3c3/job-output.txt:2023-01-09 18:40:40.470437 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/ba57680eb8de4bf2841ed5f6b2d8b3cc/job-output.txt:2023-01-09 18:46:32.970314 | controller | {3} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress /home/gibi/.cache/logsearch/4e19ddc2b0064e548093ad06205a7d67/job-output.txt:2023-01-09 18:33:38.593899 | controller | {0} tempest.api.compute.admin.test_volume.AttachSCSIVolumeTestJSON.test_attach_scsi_disk_with_config_drive [] ... inprogress
in all the 16 nova-ceph- multistore jobs that failed in the last 10 days the same test case got killed
$ grep Killed oom-kills.txt | cut -d ':' -f2 | sed 's|controller/ logs/syslog. txt|job- output. txt|g' | xargs grep inprogress .cache/ logsearch/ 358f5a3fda11476 4b794323a2f5d23 12/job- output. txt:2023- 01-17 09:32:24.979704 | controller | {3} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 8587e6a4b6de4ed b93de5c06492878 8e/job- output. txt:2023- 01-16 12:00:25.032670 | controller | {2} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ ece0cf2ce71c4a8 790a0a36529dd0a 8e/job- output. txt:2023- 01-14 23:15:05.647826 | controller | {1} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ f5aa5edd4d354c2 685fc1f3e13d0ef 77/job- output. txt:2023- 01-13 23:07:03.726600 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 1447c6274e924e0 68578ca260c9ac2 a6/job- output. txt:2023- 01-13 22:00:29.545457 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 1bbcaa703b7d42c 7a266fde3a6acca 65/job- output. txt:2023- 01-13 04:13:16.809645 | controller | {2} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 7d9ca42edc5e4bd eb17be8e8045c64 68/job- output. txt:2023- 01-12 22:32:21.335537 | controller | {3} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ bcb7bcbbbb3b478 586906c31c6558b 13/job- output. txt:2023- 01-12 20:38:12.837475 | controller | {2} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 7572c2bf5e6547c 0a1fc6b0f180a2e 1f/job- output. txt:2023- 01-12 18:17:48.522025 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ aa5cf699f8d0499 5b43d009e55a1ac cd/job- output. txt:2023- 01-12 16:48:59.307916 | controller | {1} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 8bc71a0ec0d3437 3bd25d4f6911360 84/job- output. txt:2023- 01-12 15:49:35.341633 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 81d7cef2e0b240f 89fcfa727304d8e 8d/job- output. txt:2023- 01-12 15:35:03.978880 | controller | {1} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ c75eb700717b4d3 c9942be1385cd45 bf/job- output. txt:2023- 01-11 21:26:49.653320 | controller | {1} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ fa2d7bea85ad4d2 9acc37d78d2adb3 c3/job- output. txt:2023- 01-09 18:40:40.470437 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ ba57680eb8de4bf 2841ed5f6b2d8b3 cc/job- output. txt:2023- 01-09 18:46:32.970314 | controller | {3} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress .cache/ logsearch/ 4e19ddc2b0064e5 48093ad06205a7d 67/job- output. txt:2023- 01-09 18:33:38.593899 | controller | {0} tempest. api.compute. admin.test_ volume. AttachSCSIVolum eTestJSON. test_attach_ scsi_disk_ with_config_ drive [] ... inprogress
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/
/home/gibi/