Text rendered incorrectly in presence of ligatures and justified text

For these tests, I am using fairly recent Dapper, containing the following:

  firefox 1.5.dfsg+1.5.0.1-1ubuntu10
  libgtk2.0-0 2.8.16-1ubuntu1
  libpango1.0-0 1.12.0-0ubuntu1

When rendering text on web pages, if a ligature is encountered (e.g. "ff", "fi", "ffi", etc) the ligature glyph gets rendered as expected but the advance of only the first character is used. This results in the characters after the ligature being drawn over the top of the ligature.

If I select text such that the selection begins or ends in the middle of the ligature, the spacing is corrected (since the ligature isn't used). This problem seems to be restricted to Firefox -- other GTK/Pango apps don't exhibit these problems.

The attachment shows a paragraph from http://blogs.gnome.org/jamesh with different text selections.

The page requests the "sans serif" font, which corresponds to "DejaVu Sans" on Dapper.

I've also been seeing this on dapper HEAD for the last week or two.

Further information:

The page in question also uses the "text-align: justify" CSS rule. With normal left text alignment, I don't see the rendering problems.

I didn't notice the bug when searching previously, but bug 36324 is most likely same problem. I'm not sure which direction you'd like to mark them duplicates (that bug is older, but this one probably gets a bit closer to the cause).

I think we'll need to revert the fontconfig changes of two weeks ago, or find out if the new DejaVu fixes. Michael?

A close look at DejaVu's website reveals version 2.4.1 should fix.

I tried the 2.4.1 release of "DejaVu Sans" with no change.

I doubt this is caused by a particular font, since:
 * Left aligned text displays correctly in firefox.
 * GTK Applications other than Firefox appear to have no trouble displaying ligatures.

This seems to point at Firefox's Pango rendering code.

This seems related to http://bugzilla.gnome.org/show_bug.cgi?id=327907

Some more information:

1. the bug occurs with the FreeSans and FreeSerif fonts too. If this is a font bug, then it is one common to all these fonts.

2. The bug is not present with MOZ_DISABLE_PANGO=1, but it also doesn't use the available ligatures then. This isn't that interesting since the non-pango renderer breaks layout for other languages.

3. I got Ray Strode (from Red Hat) to test if the bug occurred with the Fedora firefox builds, and it turns out that they also exhibit the bug. I also got confirmation of the bug on various other platforms in the comments here:
    http://blogs.gnome.org/view/jamesh/2006/04/03/0

So this doesn't look like an Ubuntu specific bug.

I did up a simple example page that demonstrates the bug here:
    http://www.gnome.org/~jamesh/firefox-ligature.html

I can confirm this; I have been seeing this bug since Dapper Flight 4, but it has never interefered with my workflow since, so I haven't paid much attention to this minor annoyance.

I talked a bit with Christopher Blizzard about the problem on IRC (#gnome-hackers):

 <halfline> jamesh: he says that when firefox does justification it goes into a "draw slow string" code path where firefox passes in it's own spacing
 <halfline> and that there are some known bugs with that layouting engine with generating the spacing
 <halfline> he says he can get on irc and talk to you about it...
 <halfline> let me tell him to get on #gnome-hackers
 --> blizzard (~blizzard@216-15-114-104.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com) has joined #gnome-hackers
 <blizzard> hello
 <jamesh> hi blizzard
 <blizzard> I'm here for a few mins :)
 <halfline> blizzard: url is here http://www.gnome.org/~jamesh/firefox-ligature.html
 <halfline> you'll only see the bug, though, if you have a font that supports ligatures for fi, ffi, etc
 <jamesh> ^^ that page asks for DejaVu Sans specifically (a font that triggers the rendering bug)
 <blizzard> jamesh: so
 <blizzard> jamesh: there's per-character code in the layout engine that does per-character spacing
 <blizzard> jamesh: and that tends to fall apart when you start using justification or letter spacing with any code that does character combinging
 <blizzard> combining
 <jamesh> I guessed something like that given the limited conditions when it gets triggered
 <blizzard> yeah
 <blizzard> if you stay away from those things
 <blizzard> selection and friends should work pretty well
 <blizzard> I fixed parts of the layout engine to support grapheme clusters a long time ago
 <blizzard> but not in all case
 <blizzard> cases
 <jamesh> blizzard: I guess part of the problem is that our default sans serif font now contains ligatures ...
 <blizzard> just the common ones
 <blizzard> ugh
 <jamesh> so a web page using text-align: justify and sans-serif font will trigger it
 <blizzard> that is teh suck
 <blizzard> sounds like it's time to try and get that bug fixed
 <jamesh> if it is difficult to fix, maybe going back to Vera as default would be advisable for Dapper
 <blizzard> could be, could be
 <halfline> i wonder if it would be easier to just tell pango to ignore the ligatures
 <blizzard> ok, food time
 <blizzard> back in a bit
 <jamesh> blizzard: thanks

I believe bug 33503 is also the same bug.

In Deer Park (experimental version of Firefox) there appears to be no problem with ligatures and DejaVu.

Tested on Dapper, Flight6.

I'm having the same problem with Firefox 1.5.dfsg+1.5.0.1-1ubuntu10 running under Dapper.

DejaVu 2.5 has just been released,
http://dejavu.sourceforge.net/
which includes a workaround to fix the ligature/justified issue.

I verified that with Dapper Flight 6 (+all updates), DejaVu 2.5 can show http://www.gnome.org/~jamesh/firefox-ligature.html
without any problem.

It seems that all the DejaVu people did is make the fi, ff etc. ligatures discretionary in version 2.5, which is not the way to go. These ligatures are, where available, mandatory for Latin-script typefaces. Discretionary is for fancy stuff like ct, st etc. ligatures.

Simos: What does James’s test page look like with the new DejaVu 2.5 fonts? Are the letter combinations in question not ligated at all (as I fear)?

Stefan: I do not have access to the Dapper box at the moment so I cannot add a screenshot.

It looks like firefox under dapper is not using ligatures with dejavu 2.5.

See, that’s what I meant. But since this is a Firefox bug, not a font bug, the change in DejaVu actually deprives us of our test case. So now we need to find another font with standard fi ff etc. ligatures to test this with.

The Junicode font (package ttf-junicode) is one such font. Unfortunately, I cannot test this myself since I am running Breezy on my only computer.

Confirmed by several people in the discussion below. Severity is major because it results in illegible text with all sorts of accessibility, security etc. implications.

I prepared two new sample pages (based on James) that use the Bitstream Vera Sans and Junicode fonts:

http://students.washington.edu/baums/tmp/firefox-ligature-2.html
http://students.washington.edu/baums/tmp/firefox-ligature-3.html

(In the case of Bitstream Vera Sans, only the fi and fl ligatures are available, so that table has only two rows.)

Stefan: note that the bug does not occur with Bitstream Vera Sans. It is not enough for the font to contain the ligature glyphs (which Vera does) -- it must include the appropriate OpenType tables telling the text renderer to use those ligatures in the appropriate places.

I haven't checked the Junicode font, so can't comment there.

I thought it did. Open Vera.ttf in Fontforge, go to View -> Show ATT, look at OpenType Tables -> GSUB -> latn -> dflt -> liga, and there is

fi <= f i
fl <= f l

If it doesn’t trigger the bug, then I don’t know why.

I was using the the otfdump utility from libotf-bin. It doesn't show any "liga" table entries for the copy of Vera.ttf on my system. It is possible that the font would contain ligature table entries if you opened and saved it with fontforge though.

Okay, I see., sorry about the confusion. Anyway, I just wanted to point out that DejaVu 2.5 no longers triggers the Firefox bug in question, but that of course doesn’t mean it has been resolved. For testing purposes one should therefore keep DejaVu 2.4 around or use any other Latin font with OpenType ligature, like for instance Junicode.

Gutsy users can you please run firefox from command line using the command "MOZ_DISABLE_PANGO=1 firefox" without the "" but it is case sensitive and let me know if it boosts your performance any.

You can just disable the ligatures in the fonts if you want, it's not hard. We disabled them in DejaVu for a long time because of this bug. But since there was no move towards fixing it we just enabled them again because we cannot cripple our fonts just because of one buggy program. If packagers want to do it, I'm not stopping them, but I'd rather see them patching a Firefox bug instead of removing features from our fonts.

MOZ_DISABLE_PANGO=1 is indeed the other way to work around the bug, but you'll lose support for a lot of non-Latin languages that way.

On Mon, 2007-09-24 at 19:04 +0000, Ben Laenen wrote:
> You can just disable the ligatures in the fonts if you want, it's not
> hard. We disabled them in DejaVu for a long time because of this bug.
> But since there was no move towards fixing it we just enabled them again
> because we cannot cripple our fonts just because of one buggy program.
> If packagers want to do it, I'm not stopping them, but I'd rather see
> them patching a Firefox bug instead of removing features from our fonts.
>

Indeed, this is Firefox's problem, and the fact that this bug hasn't
been fixed since it appeared in Dapper shows that Mozilla don't think
much of Linux issues.

The same goes for the GTK+ themeing.

Sorry this is a rant, I'd rather have WebKit.
--
Bruce Cowan <email address hidden>

> Indeed, this is Firefox's problem, and the fact that this bug hasn't
> been fixed since it appeared in Dapper shows that Mozilla don't think
> much of Linux issues.

Now now, don't be so hard on the Mozilla people. I'm using Gran Paradiso Alpha 7 right now and this bug is completely fixed. But it was fixed as a result of other major changes, so it can't go into an update of Firefox 2. I think it's just unfortunate timing.

Anyway, for Hardy Heron this bug will be long gone.

Sorry, I'll try to keep my opinion to the forums.

Bruce: there are a few things to keep in mind here:

1. The pango patch is not part of Mozilla's builds. That said, most Linux distros apply the patch since upstream Firefox fails to render a number of languages they care about.

2. The version of Firefox in Gutsy uses the same rendering engine as the one in Dapper (Gecko 1.8).

3. The Mozilla guys have been working on complex text rendering (using Pango on Linux) for Gecko 1.9, but that won't reach users til Firefox 3.0.

It seems that no one with the knowledge to fix the existing Pango patch to handle ligatures has the time to do so.

I would prefer Ben Laenen's proposal to disable ligatures in DejaVu Sans for Gutsy, since it's the last release with this bug anyway.

For whatever reason I get this now with Gutsy on the exact same profile of Firerfox that did not have this problem in Feisty.

The only workaround for me is disabling "Allow pages to use their own fonts" and force using Bitstream Vera Sans (or any other than DejaVu). Which is not really a proper fix - as is MOZ_DISABLE_PANGO=1

Feisty did use another /usr/lib/firefox/firefox script than gutsy does, no idea, why it changed. Gutsy version does not read /etc/firefox/firefoxrc and ~/.mozilla/firefox/rc anymore, the "hack" for enabling pango only for a certain set of languages is no longer applied, too. At the moment I'm using the /usr/lib/firefox/firefox script from feisty-package on my gutsy install to workaround this, I've had no time to investigate if this is causing some serious troubles, at least it seems to work fine for me.

> Feisty did use another /usr/lib/firefox/firefox script than gutsy does

Thanks for the info. I guess due to that change, I get this bug as well: https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/144002

I'll try switching back to Feisty's script too.

Here a patch removing ligatures from DejaVu fonts 2.19 if that's what you want to do for the release.

Please apply Denis Jacquerye's patch before the release candidate on Thursday!! This is a serious bug that can easily be worked around!

This bug has been worked around in ttf-dejavu with the upload of version 2.19-1ubuntu3:

ttf-dejavu (2.19-1ubuntu3) gutsy; urgency=low

  * Workaround: patched the fonts to disable ligatures due to a bug in
    firefox (LP #37828)

 -- Arne Goetje <email address hidden> Wed, 10 Oct 2007 12:25:02 +0800

Yay, thanks Mozilla.

The updates have fixed the issue here also.Thanks.

On Fri, Oct 12, 2007 at 10:48:42PM -0000, Steve Langasek wrote:
> ** Changed in: ttf-dejavu (Ubuntu)
> Status: New => Fix Released

I don't think this bug should be closed before the upload of firefox3.
This is probably the best workaround for the moment, but only that.

--
Jordi Mallach Pérez -- Debian developer http://www.debian.org/
<email address hidden> <email address hidden> http://www.sindominio.net/
GnuPG public key information available at http://oskuro.net/

Ah, nm me, I see it fixed the "ttf-dejavu" component of the bug, not firefox's.

After updating, the display looks good, but it still shifts if you try to select some individual characters or strings.

Fixed for me ;-)

I am unhappy with this workaround because the ligatures worked well in for example Incspape.
As a workaround it would have been sufficient not to use DejaVu in Firefox instead of affecting other programs that now suffer from an inferior typographical quality when using DejaVu.

This is not a bug in DejaVu but in Firefox, Firefox has still the same problem if a font with ligatures is used like Palatino Linotype for example.

/etc/apt/preferences :

Package: ttf-dejavu
Pin: version 2.19-1ubuntu2
Pin-Priority: 1000

Package: ttf-dejavu-core
Pin: version 2.19-1ubuntu2
Pin-Priority: 1000

Package: ttf-dejavu-extra
Pin: version 2.19-1ubuntu2
Pin-Priority: 1000

Thanks again Mozilla.

This has been fixed in 3.0, last comment on upstream bug from last year states its fixed as well.

This has been fixed in all bug trackers, it wasnt fixed in 1.5 as i recall due to it reaching EOLS before they were ablet o fix this. i dont remember if 1.5 got the fix or not but either way this bug has been fixed in supported browsers

marking as invalid due to no update since last year and it has been fixed everywhere.

Is anyone still seeing this issue?

Relinking bug watches.

This bug is certainly not invalid, so please don't mark it as such. There was ample evidence of the problem.

The firefox in Hardy is not exhibiting this bug. Firefox does appear to be using ligatures for editable text for fonts that have them. It does not appear to be using ligatures for other text, so the rendering problems on pages that combine fonts-with-ligatures and justified text are gone. Therefore I'm marking the bug fix released.

Having Firefox use the available ligatures would be nice, but is not required (it is another problem and probably best tracked upstream). This is enough to allow Ubuntu's default fonts to contain ligatures without causing rendering errors.

James fix released is fine but when the last comment is over 6 months ago without being updated its invalid because noone tells anyone when its been fix and i was never able to reproduce this. Also bug was was reported on 1.5. 1.5 is no longer supported , I suggest people update bugs more often. Honestly james invalid/fix released either way its closed .

No need to keep the upstream bugs as they have been fixed and again noone has responded in a long time, if its fixed in 2.0-3.0 than they implemented the fix and thus they should be closed.

This problem is still present in firefox-2 in Hardy. Though I can understand that no-one wants to waste time on fixing that anymore, now that firefox-3 is the default.

John: the upstream tasks are there to track related activity in related bug trackers. Unlinking them and changing the statuses just gives an incorrect view of their status there. If you think the upstream firefox task should be closed, bring that up at bugzilla.mozilla.org. Same goes for the Debian bug.

Kasper: I hadn't realised that firefox 2 was still found in Hardy. I guess it would make sense for a firefox 3 task to be marked fixed and a firefox 2 task open or invalid (if it was decided that no fix would be forthcoming).

James Henstridge wrote:
> John: the upstream tasks are there to track related activity in related
> bug trackers. Unlinking them and changing the statuses just gives an
> incorrect view of their status there. If you think the upstream firefox
> task should be closed, bring that up at bugzilla.mozilla.org. Same goes
> for the Debian bug.
>
> Kasper: I hadn't realised that firefox 2 was still found in Hardy. I
> guess it would make sense for a firefox 3 task to be marked fixed and a
> firefox 2 task open or invalid (if it was decided that no fix would be
> forthcoming).
>
>
James,
     So what makes you think there was a fix for this bug? If there was
a fix already upstream bug would have been marked fixed but its not. So
since there isnt a fix for it they should all be invalid. Now as for
upstream they died on the bug and since this is not a new bug and since
the firefox version that was affected at the time was 1.5 and that is no
longer supported. If i had to guess the xulrunner is what fixed this bug
but didnt fix it just works differently, that isnt really a fix. As for
upstream and debian i have already emailed my contacts at both of them
and waiting for a reply. Also please understand we(Ubuntu-MozillaTeam)
dont do bug states as everyone else does. There is plenty of info on
this bug to be "confirmed" but not fix released since there was never a
proper fix to this. Also bugs that are not updated for well over 3
months get Invalid (that goes for mozilla team and bug team) This is by
no means to fight or argue with you since it is not the place nor time
but i wanted to let you in on how Mozilla team does bugs and that this
bug was never fixed if it was upstream/debian bugs would have been closed.

Kasper, firefox-2.0 wont get a fix for this unless its bundled in with a
new security update but i doubt that Mozilla will speand a whole lot of
time on this since its not a critical bug,

--
Sincerely Yours,
    John Vivirito

https://launchpad.net/~gnomefreak
https://wiki.ubuntu.com/JohnVivirito
Linux User# 414246

John: there are obviously ligatures in the the DejaVu font in Hardy (you can see other GTK apps using them). We can also see that Firefox is not misrendering justified text anymore, so it does look like it has been fixed. We knew that the Firefox guys were rewriting the text rendering code for 3.0 (see previous comments), so it isn't surprising that the bug is now fixed in Hardy.

On Wed, May 14, 2008 at 12:44:41PM -0000, James Henstridge wrote:
> John: there are obviously ligatures in the the DejaVu font in Hardy (you
> can see other GTK apps using them). We can also see that Firefox is not
> misrendering justified text anymore, so it does look like it has been
> fixed. We knew that the Firefox guys were rewriting the text rendering
> code for 3.0 (see previous comments), so it isn't surprising that the
> bug is now fixed in Hardy.
>

 affects ubuntu/firefox-3.0
 status fixreleased

 affects ubuntu/firefox
 status wontfix

 - Alexander