Ansible implementation of Nova doesn't work on systemd based systems

Bug #1474969 reported by Steven Dake
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Steven Dake

Bug Description

The nova compute container starts and exits immediately indicating no hypervisor can be found.

Revision history for this message
Steven Dake (sdake) wrote :

virsh list of the compute container while running libvirt via strace:

) = 1 ([{fd=11, revents=POLLIN}])
accept(11, {sa_family=AF_LOCAL, NULL}, [2]) = 21
getsockname(21, {sa_family=AF_LOCAL, sun_path="/var/run/libvirt/libvirt-sock"}, [32]) = 0
fcntl(21, F_GETFD) = 0
fcntl(21, F_SETFD, FD_CLOEXEC) = 0
fcntl(21, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(21, F_SETFL, O_RDWR|O_NONBLOCK) = 0
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN}], 9, 4294967295) = 1 ([{fd=21, revents=POLLIN}])
read(21, "\0\0\0\34", 4) = 4
read(21, " \0\200\206\0\0\0\1\0\0\0B\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24
futex(0x7f9511995334, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f9511995330, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f95119953cc, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f95119953c8, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f9511995308, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f95119a3bf0, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 0
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN|POLLOUT}], 9, 4294967295) = 1 ([{fd=21, revents=POLLOUT}])
write(21, "\0\0\0\330 \0\200\206\0\0\0\1\0\0\0B\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0&"..., 216) = 216
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN}], 9, 4294967295) = 1 ([{fd=21, revents=POLLIN|POLLHUP}])
read(21, "", 4) = 0
gettid() = 1
sendmsg(4, {msg_name(29)={sa_family=AF_LOCAL, sun_path="/run/systemd/journal/socket"}, msg_iov(32)=[{"MESSAGE", 7}, {"=", 1}, {"End of file while reading data: "..., 50}, {"\n", 1}, {"PRIORITY", 8}, {"=", 1}, {"3", 1}, {"\n", 1}, {"LIBVIRT_SOURCE", 14}, {"=", 1}, {"util.error", 10}, {"\n", 1}, {"CODE_FILE", 9}, {"=", 1}, {"rpc/virnetsocket.c", 18}, {"\n", 1}, {"CODE_LINE", 9}, {"=", 1}, {"1574", 4}, {"\n", 1}, {"CODE_FUNC", 9}, {"=", 1}, {"virNetSocketReadWire", 20}, {"\n", 1}, {"LIBVIRT_DOMAIN", 14}, {"=", 1}, {"7", 1}, {"\n", 1}, {"LIBVIRT_CODE", 12}, {"=", 1}, {"38", 2}, {"\n", 1}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 204
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 0
close(21) = 0
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 8, 4294967295

Revision history for this message
Steven Dake (sdake) wrote :
Download full text (10.6 KiB)

virsh list of the libvirt container while running libvirt via strace.

note this virsh command executes properly:

) = 1 ([{fd=11, revents=POLLIN}])
accept(11, {sa_family=AF_LOCAL, NULL}, [2]) = 21
getsockname(21, {sa_family=AF_LOCAL, sun_path="/var/run/libvirt/libvirt-sock"}, [32]) = 0
fcntl(21, F_GETFD) = 0
fcntl(21, F_SETFD, FD_CLOEXEC) = 0
fcntl(21, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(21, F_SETFL, O_RDWR|O_NONBLOCK) = 0
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN}], 9, 4294967295) = 1 ([{fd=21, revents=POLLIN}])
read(21, "\0\0\0\34", 4) = 4
read(21, " \0\200\206\0\0\0\1\0\0\0B\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24
futex(0x7f9511995334, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f9511995330, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f95119953cc, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f95119953c8, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f9511995308, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f95119a3bf0, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
futex(0x7f95119a3bf0, FUTEX_WAIT_PRIVATE, 2, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f95119951c0, FUTEX_WAKE_PRIVATE, 1) = 1
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN|POLLOUT}], 9, 4294967295) = 1 ([{fd=21, revents=POLLOUT}])
write(21, "\0\0\0$ \0\200\206\0\0\0\1\0\0\0B\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\1"..., 36) = 36
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN}], 9, 4294967295) = 1 ([{fd=21, revents=POLLIN}])
read(21, "\0\0\0 ", 4) = 4
read(21, " \0\200\206\0\0\0\1\0\0\0<\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\n", 28) = 28
futex(0x7f9511995334, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f9511995330, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f95119953cc, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f95119953c8, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7f9511995308, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f95119a3bf0, FUTEX_WAKE_PRIVATE, 1) = 1
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN}], 9, 5000) = 1 ([{fd=6, revents=POLLIN}])
read(6, "\0", 1) = 1
poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=21, events=POLLIN|POLLOUT}], 9, 5000) = 2 ([{fd=6, revents=POLLIN}, {fd=21, revents=POLLOUT}])
read(6, "\0", 1) = 1
write(21, "\0\0\0 \0\200\206\0\0\0\1\0\0\0<...

Revision history for this message
Steven Dake (sdake) wrote :

docker inspect of libvirt shows:

        "PidMode": "",

Revision history for this message
Sam Yaple (s8m) wrote :

This issue isn't specific to systemd based systems. The libvirt container is now broken on Ubuntu as well.

Unsure of the cause at the moment, but the error message was about Failing to connect to the dbus socket. This issue is not related to pid mode nor is it an issue with the nova_compute container.

The issue is with nova_libvirt

Revision history for this message
Steven Dake (sdake) wrote :

I think running with pidmode=host will mask the problems, not sure why.

Steven Dake (sdake)
Changed in kolla:
status: New → Fix Committed
importance: Undecided → Critical
assignee: nobody → Steven Dake (sdake)
milestone: none → liberty-2
Sam Yaple (s8m)
Changed in kolla:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.