Firefox 89: webrender breaks videos on Raspberry Pi

Created attachment 9225141
Firefox 89.0 black video artifacts.png

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0

Steps to reproduce:

Updated to 89.0

Actual results:

Playing any video video content which contains black causes red and blue artifacts in where the lack should be.

Expected results:

Expect to have seen black instead of red and blue. A picture/screenshot is worth a thousand words and is attached.

Name Firefox
Version 89.0
Build ID 20210527174632
Distribution ID canonical
User Agent Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0
OS Linux 4.9.241-77 #1 SMP PREEMPT Mon Mar 29 23:04:14 -03 2021

The [Bugbug](https://github.com/mozilla/bugbug/) bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Created attachment 9225490
firefox_video.png

I'm also having this issue, also on aarch64, a Pinebook Pro:

I'm seeing RPi users with a similar issue.

Name Firefox
Version 89.0
Build ID 20210602080605
Distribution ID manjaro
User Agent Mozilla/5.0 (X11; Linux aarch64; rv:89.0) Gecko/20100101 Firefox/89.0
OS Linux 5.12.9-1-MANJARO-ARM #1 SMP Thu Jun 3 08:06:55 UTC 2021

Created attachment 9225491
about_support_raw.json

Here's my about:support as raw json.

Disabling WebRender as suggested here gets rid of the artifacting, but presumably this is still a problem.

https://support.mozilla.org/en-US/questions/1338832

"about:config => gfx.webrender.force-disabled = true"

Err to be perfectly clear, disabling WebRender, and then restarting Firefox fixed the issue, in that order.

Per comment6, this seems related with Graphic.

Yep, seems like an issue with software webrender playing videos on ARM Linux.

See also bug 1714739 (probably a dup of this)

Sotaro, any ideas?

*** Bug 1714739 has been marked as a duplicate of this bug. ***

Walter, Conrad, does this occur on other video sites too, or just youtube? And is it every video on youtube (Perhaps it only affects certain video formats)?

Jamie, I can confirm it happens on other sites, it also happens regardless of the codec as far as I can tell - normally I have everything but avc1/h264 disabled on the device in about:config just due to it being kinder on battery life, but I re-enabled vp9 (and ensured that youtube, in this case, was serving vp9) and it has the same issue. Please let me know if there's anything else you'd like me to check.

Jamie Nicol asked :

Walter, Conrad, does this occur on other video sites too, or just youtube? And is it every video on youtube (Perhaps it only affects certain video formats)?

It appears to be happening on all sites that contain videos even video adds!

Lee, it looks like there might be an issue with our YUV conversion code on ARM. Any guesses what that might be?

Another thing to try would be enabling hw-wr. You should be able to do that by setting gfx.webrender.all=true and unsetting gfx.webrender.force-disabled. I expect this will improve performance, battery life and remove the artifacts.

Created attachment 9226348
webrender_all_true_force_disable_false.png

Unfortunately no luck there, that seems to have caused Firefox to suddenly come to life with creativity/inspiration for the visual arts.

I am running the "bleeding edge stable" of the Panfrost driver with OpenGL 3.3 enabled (I believe the latter part specifically is experimental); no idea if that has any bearing on this but I'll disable that later and see if that makes any difference. Appreciate the suggestions (and possible hints at improved power consumpiton in the future on aarch64/in general by default).

Cheers, and again if there's anything else I can test I'd be happy to help.

These new artifacts with gfx.webrender.all=true and gfx.webrender.force-disabled=false occurred with both my compositor running and not (still using X11).

Ok yeah, those new artifacts could be panfrost bugs. They're probably worth reporting to upstream mesa.

Can you attach your "about:buildconfig" so we can see what compiler setup you built Firefox with?

(In reply to Jeff Muizelaar [:jrmuizel] from comment #17)
> Ok yeah, those new artifacts could be panfrost bugs. They're probably worth reporting to upstream mesa.

They could be panfrost issues but then we need to explain why the the previous version of Firefox is working fine with the same panfrost drivers.

In addition I have two ARM Linux systems. One is 18.04 LTS X11 the other is 20.04 PANFROST.

So whether there is panfrost or not the problem still exists.

I think we can safely discount PANFROST at this time.

(In reply to Jamie Nicol [:jnicol] from comment #8)
> Yep, seems like an issue with software webrender playing videos on ARM Linux.
>
> See also bug 1714739 (probably a dup of this)
>
> Sotaro, any ideas?

I could reproduce the problem with sw-wr on ARM64 Win10. Then it seems arm specific problem.

Created attachment 9226630
about_buildconfig.html

Re: mesa, I can confirm that my mesa version did not change between firefox versions. I tried playing around with downgrading things but didn't have any luck.

Here's my about:buildconfig, wasn't sure how to format so just saved the html.

Created attachment 9226716
Bug 1714511 - Use more saturated ops to avoid YUV math underflow. r?sotaro

Sotaro, if you are able to repro, can you check if my saturated math patch helps the issue for you?

(In reply to Lee Salzman [:lsalzman] from comment #23)
> Sotaro, if you are able to repro, can you check if my saturated math patch helps the issue for you?

I tried the following build. The problem was not addressed with D117599.
https://treeherder.mozilla.org/jobs?repo=try&revision=99c8dcb3ae16b3d3f19bcef6dfbc7ac783853e0f

The circumstantial evidence "seems like" something in the function here (https://searchfox.org/mozilla-central/source/gfx/wr/swgl/src/composite.h#534) is underflowing. But I would need someone who can reliably reproduce the issue to find out where in those math equations are actually doing so. I tried testing in my patch if the additions were the culprit, but that seems unlikely.

It's possible the multiplications "rbCoeffs * uv" or "gCoeffs * uv" are underflowing based on whatever values the ARM video codecs seem to be supplying. However, this is not happening on x86 at all, which is strange, and it is also happening on ARM across platform, which is further strange, since there isn't really much in the way of platform specific code here.

Sotaro, might you be able to investigate this?

Created attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

Sotaro, can you see if my vqmovun_s16 patch fixes this or not?

(In reply to Lee Salzman [:lsalzman] from comment #25)
> The circumstantial evidence "seems like" something in the function here (https://searchfox.org/mozilla-central/source/gfx/wr/swgl/src/composite.h#534) is underflowing. But I would need someone who can reliably reproduce the issue to find out where in those math equations are actually doing so. I tried testing in my patch if the additions were the culprit, but that seems unlikely.
>
> It's possible the multiplications "rbCoeffs * uv" or "gCoeffs * uv" are underflowing based on whatever values the ARM video codecs seem to be supplying. However, this is not happening on x86 at all, which is strange, and it is also happening on ARM across platform, which is further strange, since there isn't really much in the way of platform specific code here.
>
> Sotaro, might you be able to investigate this?

Taking a step back. If a certain function is suspected what are the diff's with this code between the working 88.0 and non working 89.0 version?

Since the issue affects all ARM regardless of platform the only culprit not eliminated is Firefox.

If you have a downloaded executable for me to test I'm happy to do so. My platforms are :

aarch64 Ubuntu 18.04 LTS
aarch64 Ubuntu 20.10

I can only test on one of these platforms!

(In reply to Lee Salzman [:lsalzman] from comment #27)
> Sotaro, can you see if my vqmovun_s16 patch fixes this or not?

The updated patch worked for me on arm64 Win10 :)

Pushed by <email address hidden>:
https://hg.mozilla.org/integration/autoland/rev/c77fc5fac5ae
use vqmovun_s16 for packing pixels. r=sotaro

https://hg.mozilla.org/mozilla-central/rev/c77fc5fac5ae

Tested that commit on top of the 89.0 release, on Ubuntu 21.04 on aarch64 (Raspberry Pi 4), and I can confirm the problem is fixed. Thanks a lot :lsalzman ! I'm going to cherry-pick that commit as a distro-patch for Ubuntu packages.

Comment on attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

### Beta/Release Uplift Approval Request
* **User impact if declined**: Enabling Software-WebRender on ARM platforms can lead to visual artifacts. Should not impact x86 platforms or non-SW-WR configurations.
* **Is this code covered by automated tests?**: No
* **Has the fix been verified in Nightly?**: Yes
* **Needs manual test from QE?**: No
* **If yes, steps to reproduce**:
* **List of other uplifts needed**: None
* **Risk to taking this patch**: Low
* **Why is the change risky/not risky? (and alternatives if risky)**: Software-WebRender on ARM is not deployed by default to release. Must be pref'd on. However, many users on tier-2 platforms like ARM/Linux are pref'ing the feature on, and it would be nice if we just supplied the fix to them so they don't need to manually apply it to their build.
* **String changes made/needed**:

Comment on attachment 9226932
Bug 1714511 - use vqmovun_s16 for packing pixels. r?sotaro

approved for 90.0b9

https://hg.mozilla.org/releases/mozilla-beta/rev/b50708896f2d