libdrm-intel1 2.4.60 breaks Valve/Steam games

Users with optimus systems are reporting that
many games fail to run in libdrm-intel is upgraded
from 2.4.56 to 2.4.58. (And, downgrading to 2.4.56
fixes the issue.

Steam bug:
https://github.com/ValveSoftware/steam-for-linux/issues/3506

Debian bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=768045

I don't have hardware to confirm this issue.

I notice that between 2.4.56 and 2.4.58 libdrm changed
some symbol visibility settings.

Works fine on my Optimus machine with libdrm 2.4.58. Dota2 runs fine on both GPUs!
I am not using bumblebee or primus, so i guess the problem lies there!

Anyway, keeping this bug open for a while.

(In reply to Tobias Klausmann from comment #1)
> Works fine on my Optimus machine with libdrm 2.4.58. Dota2 runs fine on both
> GPUs!
> I am not using bumblebee or primus, so i guess the problem lies there!
>
> Anyway, keeping this bug open for a while.

Out of curiosity, what version nVidia driver are you using?

I dont use the NVIDIA blob, but the nouveau driver...anyhow the intel driver is working on prime (Optimus) systems if it does not get bugged by bumblebee/primus with libdrm 2.4.58. Go report a bug against these apps.

Can confirm this issue locally on my current system.

Running Steam through the "primusrun" Bumblebee wrapper to start games on the nVidia card leads to the following error after starting a game:

malloc: unknown:0: assertion botched
free: called with unallocated block argument
last command: (null)
Aborting...Aborted (core dumped)

I can also confirm that downgrading libdrm-intel1 to 2.4.56 fixes the issue and allows the game to run on the nVidia card just fine. Furthermore, starting Steam without "primusrun" (thereby causing the game to start on the Intel card) also does not cause this issue.

Should any testing be desired, I'd be happy to oblige.

I confirm this is still present in 2.4.59+git20150225.1f73578d-0ubuntu0ricotz~utopic.

Guys can you bisect this ? Afaics it will take ~6 steps to track the issue.

This is what I came up with when bisecting

ae8edc7544e566084f7b958eb93c9109b471ca30 is the first bad commit
commit ae8edc7544e566084f7b958eb93c9109b471ca30
Author: Tvrtko Ursulin <email address hidden>
Date: Thu Jun 19 15:52:03 2014 +0100

    intel: Add support for userptr objects

    Allow userptr objects to be created and used via libdrm_intel.

    At the moment tiling and mapping to GTT aperture is not supported
    due hardware limitations across different generations and uncertainty
    about its usefulness.

    v2: Improved error handling in feature detection per review comments.

    v3: Rebase on top of the drm_public addition, minor whitespace addition.

    Reviewed-by: Damien Lespiau <email address hidden>
    Signed-off-by: Damien Lespiau <email address hidden> (v3)
    Signed-off-by: Tvrtko Ursulin <email address hidden> (v1,v2)

:040000 040000 5b1f4eecbdd1cf2c57d7da9388384c17e8448bee b41b6afa071dc408dcaf7a4c6dcfecc2d7413c73 M intel

Adding the author to the CC list.

Vedran thanks for bisecting this :-)
I'm not sure how this patch can cause such an issue. Either something funny has happened during bisect or there is another underlying problem.

Summary of the problem so far:

Optimus system running Intel mesa and the proprietary Nvidia drivers. Running Steam (games) with either bumblebee, optirun or primusrun seems to cause a crash.

Steam Log:
Game update: AppID 550 "Left 4 Dead 2", ProcID 9399, IP 0.0.0.0:0
...
malloc: unknown:0: assertion botched
free: called with unallocated block argument
last command: (null)
Aborting...Game removed: AppID 550 "Left 4 Dead 2", ProcID 9399

Bisecting libdrm shows ae8edc7544e as the offending commit.

If an Intel dev has a similar machine they can try to reproduce it locally. Otherwise a back-trace should help at least a bit.

I while back I was able to use gdb with steam. Not sure if it still works
* Close all steam games and the steam client
* $ export GAME_DEBUGGER=gdb
* $ steam

Note I'm not looking into the issue, just trying to get some information for the Intel devs.

Thanks Emil! Unfortunaltey I am not familiar with Optimus nor do I have appropriate hardware.

Could someone provide a backtrace (with symbols) of the binary which ends up saying "malloc: unknown:0: assertion botched", etc? Or maybe even Valgrind it? Although that may be pretty advanced.

I don't think this commit is the actual culprit, but it is possible that the failing binary uses UserPtr if it detects it, and has a bug in those optional code paths.

Another interesting test would be to try non-working libdrm on a kernel without UserPtr support.

I don't have pre 3.16 kernel easily available.

export GAME_DEBUGGER=gdb still makes the game crash before it runs it with the debugger.

I've tried various things to make it work (modifying the game startup shell script) but without success.

When I put the older version of libdrm (86b37c61c78edd1353a3f76f678c39e2ec168771, before Tvrtkos change), gdb runs normally for the game. So the steam binary actually crashes, not the game itself.

It's funny because steam binary is 32bit, Dota 2 binary is 64 bit, and only changing the 64bit version of libdrm_intel.so affects this bug.

I've tried debugging steam with export DEBUGGER=gdb, but it seems spawn multiple threads and I'm not experienced with multithreaded debugging.

I did manage to run steam with valgrind, I'll add an attachment now.

Created attachment 113924
valgrind steam run, error should near the end of the file

valgrind steam run, error should near the end of the file

I think we'll need to pull in someone from Steam since Valgrind reports some mismatched frees in vgui2_s.so, steamclient.so and steamui.so.

There are not debug symbols so impossible to guess what area of the code is that.

Then this also looks pretty bad:

ERROR: ld.so: object '/home/vedran/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/vedran/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored

Please if someone knows any Steam devs CC them here?

(In reply to Tvrtko Ursulin from comment #12)
> I think we'll need to pull in someone from Steam since Valgrind reports some
> mismatched frees in vgui2_s.so, steamclient.so and steamui.so.
>
> There are not debug symbols so impossible to guess what area of the code is
> that.
>
I'm assuming Jordan can coordinate that with Valve. Afaics he is the one assigned for the issue over at github.

> Then this also looks pretty bad:
>
> ERROR: ld.so: object
> '/home/vedran/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from
> LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
> ERROR: ld.so: object
> '/home/vedran/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from
> LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored
>
Those should be harmless, as steam LD_PRELOADs a bunch of libraries. Afaik steam is 32bit, while Vedran was running a 64bit system - thus the messages.

There are a couple of other interesting bits in the log:
 - Alarming heap stats (allocated ~770 MiB, in-use at exit ~220MiB)
 - Invalid memory access in various modules - invalid reads in various steam runtime components, alarming number of invalid writes in i965_dri.so.

The latter one perhaps should be looked into closer ?

> Please if someone knows any Steam devs CC them here?
Afaict unless a person is known* by bugzilla one cannot CC them. For example one cannot CC your @intel email, so I've opted for the known @linux.intel one.

(In reply to Emil Velikov from comment #13)
> - Invalid memory access in various modules - invalid reads in various steam
> runtime components, alarming number of invalid writes in i965_dri.so.
>
> The latter one perhaps should be looked into closer ?

Probably not, that is just valgrind not understanding GEM and its ioctls. You need to use a libdrm compiled with --enable-valgrind to suppress the false positives.