https websockets not working in 2.5 or 2.6

Bug #1589923 reported by Ben Aitchison
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
QEMU
Fix Released
Undecided
Daniel Berrange
Arch Linux
New
Undecided
Unassigned
qemu (Ubuntu)
Fix Released
High
Unassigned
Xenial
Incomplete
Undecided
Unassigned
Yakkety
Won't Fix
Undecided
Unassigned

Bug Description

% gdb --args ./x86_64-softmmu/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=/etc/pki/libvirt-le,websocket=5701

GNU gdb (GDB) 7.11
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./x86_64-softmmu/qemu-system-x86_64...done.
(gdb) run
Starting program: /home/ben/qemu/qemu-2.6.0/x86_64-softmmu/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=/etc/pki/libvirt-le,websocket=5701
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffe16f6700 (LWP 12767)]
[New Thread 0x7fffde2d4700 (LWP 12768)]
[New Thread 0x7fffd3fff700 (LWP 12769)]
Initializing VNC server with x509 no auth
Client sioc=0x55555874d6b0 ws=1 auth=1 subauth=0
New client on socket 0x55555874d6b0
vnc_set_share_mode/0x55555874d6b0: undefined -> connecting
TLS Websocket connection required
Start TLS WS handshake process
Handshake failed TLS handshake failed: The TLS connection was non-properly terminated.
Closing down client sock: protocol error
vnc_set_share_mode/0x55555779f510: connecting -> disconnected
Client sioc=0x55555873c6a0 ws=1 auth=1 subauth=0
New client on socket 0x55555873c6a0
vnc_set_share_mode/0x55555873c6a0: undefined -> connecting
TLS Websocket connection required
Start TLS WS handshake process
TLS handshake complete, starting websocket handshake
Websocket negotiate starting
Websock handshake complete, starting VNC protocol
Write Plain: Pending output 0x555557b91c60 size 4096 offset 12. Wait SSF 0
Wrote wire 0x555557b91c60 12 -> 12

Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault.
0x0000000000000001 in ?? ()
(gdb) thread apply all bt

Thread 4 (Thread 0x7fffd3fff700 (LWP 12769)):
#0 0x00007fffef35a09f in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
#1 0x0000555555a20bd9 in qemu_cond_wait (cond=cond@entry=0x5555587267e0,
    mutex=mutex@entry=0x555558726810) at util/qemu-thread-posix.c:123
#2 0x00005555559770ab in vnc_worker_thread_loop (queue=queue@entry=0x5555587267e0)
    at ui/vnc-jobs.c:228
#3 0x00005555559775e8 in vnc_worker_thread (arg=0x5555587267e0) at ui/vnc-jobs.c:335
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 3 (Thread 0x7fffde2d4700 (LWP 12768)):
#0 0x00007fffef35a09f in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
#1 0x0000555555a20bd9 in qemu_cond_wait (cond=<optimized out>,
---Type <return> to continue, or q <return> to quit---
    emu_global_mutex>) at util/qemu-thread-posix.c:123
#2 0x0000555555715edf in qemu_tcg_wait_io_event (cpu=0x5555564ee840) at /home/ben/qemu/qemu-2.6.0/cpus.c:1015
#3 qemu_tcg_cpu_thread_fn (arg=<optimized out>) at /home/ben/qemu/qemu-2.6.0/cpus.c:1161
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 2 (Thread 0x7fffe16f6700 (LWP 12767)):
#0 0x00007fffea438229 in syscall () from /usr/lib/libc.so.6
#1 0x0000555555a20ee8 in futex_wait (val=<optimized out>, ev=<optimized out>) at util/qemu-thread-posix.c:292
#2 qemu_event_wait (ev=ev@entry=0x55555641ece4 <rcu_call_ready_event>) at util/qemu-thread-posix.c:399
#3 0x0000555555a2f2ae in call_rcu_thread (opaque=<optimized out>) at util/rcu.c:250
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 1 (Thread 0x7ffff7f5bb00 (LWP 12763)):
#0 0x0000000000000001 in ?? ()
#1 0x00005555559efb53 in qio_task_free (task=0x555558212140) at io/task.c:58
#2 0x00005555559efd89 in qio_task_complete (task=task@entry=0x555558212140) at io/task.c:145
#3 0x00005555559ef5aa in qio_channel_websock_handshake_send (ioc=0x55555873c6a0, condition=<optimized out>,
    user_data=0x555558212140) at io/channel-websock.c:289
#4 0x00007fffecf59c8a in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#5 0x000055555598a6b3 in glib_pollfds_poll () at main-loop.c:213
#6 os_host_main_loop_wait (timeout=<optimized out>) at main-loop.c:258
#7 main_loop_wait (nonblocking=<optimized out>) at main-loop.c:506
#8 0x00005555556e1fbd in main_loop () at vl.c:1934
#9 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4656

Changed in qemu:
assignee: nobody → Daniel Berrange (berrange)
Paul White (paulw2u)
affects: ubuntu → qemu (Ubuntu)
Revision history for this message
Daniel Berrange (berrange) wrote :
Changed in qemu (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in qemu:
status: New → Confirmed
Revision history for this message
Thomas Huth (th-huth) wrote :
Revision history for this message
Daniel Berrange (berrange) wrote :

Fixed in stable-2.6 branch in

commit 510531ea442a02048b1837fcf574d03559b38c9e
Author: Daniel P. Berrange <email address hidden>
Date: Tue Jun 7 12:27:51 2016 +0100

    io: remove mistaken call to object_ref on QTask

    The QTask struct is just a standalone struct, not a QOM Object,
    so calling object_ref() on it is not appropriate. This results
    in mangling the 'destroy' field in the QTask struct, causing
    the later call to qtask_free() to try to call the function
    at address 0x1, with predictably segfault happy results.

    There is in fact no need for ref counting with QTask, as the
    call to qtask_abort() or qtask_complete() will automatically
    free associated memory.

    This fixes the crash shown in

      https://bugs.launchpad.net/qemu/+bug/1589923

    Reviewed-by: Eric Blake <email address hidden>
    Signed-off-by: Daniel P. Berrange <email address hidden>
    (cherry picked from commit bc35d51077b33e68a0ab10a057f352747214223f)
    Signed-off-by: Michael Roth <email address hidden>

Thomas Huth (th-huth)
Changed in qemu:
status: Confirmed → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Fixed in >=2.7 and thereby >=Zesty.
Needs to be considered for SRUs now.

Changed in qemu (Ubuntu):
status: Triaged → Fix Released
Changed in qemu (Ubuntu Xenial):
status: New → Triaged
Changed in qemu (Ubuntu Yakkety):
status: New → Triaged
tags: added: server-next
Changed in qemu (Ubuntu Yakkety):
status: Triaged → Won't Fix
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I added this to my list a while ago but now closed out the immediate tasks for artful, so I'm coming back here.
Writing repro steps which will be needed for the SRU.

# Brute force create dummy keys for this
openssl req -out ca.pem -new -x509
openssl genrsa -out server.key 1024
openssl req -key server.key -new -out server.req
echo 00 > file.srl
openssl x509 -req -in server.req -CA ca.pem -CAkey privkey.pem -CAserial file.srl -out server-cert.pem
openssl genrsa -out client.key 1024
openssl req -key client.key -new -out client.req
openssl x509 -req -in client.req -CA ca.pem -CAkey privkey.pem -CAserial file.srl -out client-cert.pem
ln -s ca.pem ca-cert.pem
ln -s server.key server-key.pem

# run qemu with x509 websocket
/usr/bin/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=$(pwd),websocket=5707

It is not crashing and I can even connect with krdc (likely also other clients) against it without breaking.

But then I'd think your command above for the crash was the one for the crash in 2.6 which was yakkety.

But what is left to fix is the dropped connection [1] in 2.5 (Xenial).

I tried to read a better testcase out of that mail thread but failed, if you'd have a better setup description to still trigger this blocked handshake that eventually fails once the signal sets data - that would be great.
The actual report I found [2] is actually this bug bridges onto the ML so no more info there for me.

[1]: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01884.html
[2]: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01917.html

Changed in qemu (Ubuntu Xenial):
status: Triaged → Incomplete
tags: removed: server-next
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.