We see this in upstream openstack CI testing, viewable here:
http://logs.openstack.org/07/251407/2/check/gate-tempest-dsvm-full/144f7fc/logs/libvirt/libvirtd.txt.gz#_2015-11-30_18_20_18_168
2015-11-30 18:20:18.168+0000: 31539: error : qemuMonitorIO:656 : internal error: End of file from monitor 2015-11-30 18:20:18.168+0000: 31539: debug : qemuMonitorIO:710 : Error on monitor internal error: End of file from monitor 2015-11-30 18:20:18.168+0000: 31539: debug : qemuMonitorIO:731 : Triggering EOF callback 2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessHandleMonitorEOF:300 : Received EOF on 0x7fa310011240 'instance-00000066' 2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessHandleMonitorEOF:318 : Monitor connection to 'instance-00000066' closed without SHUTDOWN event; assuming the domain crashed 2015-11-30 18:20:18.168+0000: 31539: debug : virObjectEventNew:643 : obj=0x7fa340aab850 2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessStop:4235 : Shutting down vm=0x7fa310011240 name=instance-00000066 id=150 pid=17830 flags=0
This was the domain log:
http://logs.openstack.org/07/251407/2/check/gate-tempest-dsvm-full/144f7fc/logs/libvirt/qemu/instance-00000066.txt.gz
I noticed this:
char device redirected to /dev/pts/1 (label charserial1) qemu-system-x86_64: /build/qemu-5LgLIn/qemu-2.0.0+dfsg/block.c:3491: bdrv_error_action: Assertion `error >= 0' failed. 2015-11-30 18:20:18.168+0000: shutting down
This is a volume-backed VM. I think around the time that this fails, we should be trying to plug a virtual interface.
Possibly also helpful:
http://logs.openstack.org/07/251407/2/check/gate-tempest-dsvm-full/144f7fc/logs/screen-n-net.txt.gz#_2015-11-30_18_19_45_252
2015-11-30 18:19:45.251 DEBUG oslo_concurrency.processutils [req-8911e8c7-2466-408f-832e-af4b78e9adec tempest-TestVolumeBootPattern-2142876884 tempest-TestVolumeBootPattern-1970238908] CMD "sudo nova-rootwrap /etc/nova/rootwrap.conf ebtables --concurrent -t nat -D PREROUTING --logical-in br100 -p ipv4 --ip-src 10.1.0.3 ! --ip-dst 10.1.0.0/20 -j redirect --redirect-target ACCEPT" returned: 255 in 0.147s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:297
We see this in upstream openstack CI testing, viewable here:
http:// logs.openstack. org/07/ 251407/ 2/check/ gate-tempest- dsvm-full/ 144f7fc/ logs/libvirt/ libvirtd. txt.gz# _2015-11- 30_18_20_ 18_168
2015-11-30 18:20:18.168+0000: 31539: error : qemuMonitorIO:656 : internal error: End of file from monitor leMonitorEOF: 300 : Received EOF on 0x7fa310011240 'instance-00000066' leMonitorEOF: 318 : Monitor connection to 'instance-00000066' closed without SHUTDOWN event; assuming the domain crashed ew:643 : obj=0x7fa340aab850 :4235 : Shutting down vm=0x7fa310011240 name=instance- 00000066 id=150 pid=17830 flags=0
2015-11-30 18:20:18.168+0000: 31539: debug : qemuMonitorIO:710 : Error on monitor internal error: End of file from monitor
2015-11-30 18:20:18.168+0000: 31539: debug : qemuMonitorIO:731 : Triggering EOF callback
2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessHand
2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessHand
2015-11-30 18:20:18.168+0000: 31539: debug : virObjectEventN
2015-11-30 18:20:18.168+0000: 31539: debug : qemuProcessStop
This was the domain log:
http:// logs.openstack. org/07/ 251407/ 2/check/ gate-tempest- dsvm-full/ 144f7fc/ logs/libvirt/ qemu/instance- 00000066. txt.gz
I noticed this:
char device redirected to /dev/pts/1 (label charserial1) qemu-5LgLIn/ qemu-2. 0.0+dfsg/ block.c: 3491: bdrv_error_action: Assertion `error >= 0' failed.
qemu-system-x86_64: /build/
2015-11-30 18:20:18.168+0000: shutting down
This is a volume-backed VM. I think around the time that this fails, we should be trying to plug a virtual interface.
Possibly also helpful:
http:// logs.openstack. org/07/ 251407/ 2/check/ gate-tempest- dsvm-full/ 144f7fc/ logs/screen- n-net.txt. gz#_2015- 11-30_18_ 19_45_252
2015-11-30 18:19:45.251 DEBUG oslo_concurrenc y.processutils [req-8911e8c7- 2466-408f- 832e-af4b78e9ad ec tempest- TestVolumeBootP attern- 2142876884 tempest- TestVolumeBootP attern- 1970238908] CMD "sudo nova-rootwrap /etc/nova/ rootwrap. conf ebtables --concurrent -t nat -D PREROUTING --logical-in br100 -p ipv4 --ip-src 10.1.0.3 ! --ip-dst 10.1.0.0/20 -j redirect --redirect-target ACCEPT" returned: 255 in 0.147s execute /usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/processutils. py:297