ja/zh fontconfig configuration only applied when locale is set 'natively'

My understanding is that fontconfig-voodoo was removed because the relevant bits of fontconfig configuration are now directly guarded by locale, so fontconfig-voodoo is no longer necessary. If you believe you still need it, it would be helpful to describe the exact bug you're seeing.

Status changed to 'Confirmed' because the bug affects multiple users.

This problem affect me a lot, my locale is en_US.UTF-8, but I do need to read Chinese.
However the default Chinese sucks, it looks like it use other fonts, not the default ttf-wqy-microhei, which works perfect in 12.04.
The upgrade breaks font a lot...

Correct me if I'm wrong. I notice language-selector is replaced by language-selector-common, I wonder why which has the same files. but it behave different behaviour to 12.04?

On 2012-10-19 11:20, Ray Wang wrote:
> I notice language-selector is replaced by language-selector-common,

No, that's not the case. language-selector is the name of the source package. When building, it currently creates three .deb files:
  language-selector-common
  language-selector-gnome
  language-selector-kde

>If you believe you still need it, it would be helpful to describe the exact bug you're seeing.

I am using locale en_US.UTF-8, But I need to read Japanese.
So I want Japanese fonts are preferred over Chinese font.
Now, fontconfig-voodoo is missing and all Japanese text is rendered by using Chinese font which looks really horrible.

That is for me.
But I think Chinese people prefer Chinese font over Japanese font.

Locale setting and font preference cannot be the same.

For the current workaround, I do
cp /etc/fonts/conf.avail/69-language-selector-ja-jp.conf ~/.fonts.conf
and then, delete all occurrence of these 3 lines.

<test name="lang" compare="contains">
    <string>ja</string>
</test>

So Japanese font will be chosed if it use Non-Japanese font and render Japanese characters.

I really don't like this workaround and I wish simple fontconfig-voodoo back.

Please see my latest research: http://imtx.me/archives/1765.html

Oh, it is just the same as the #6.

Hello,

The LANG guards are included as a safety guard to prevent incorrect fonts being selected for users that don't need them. It's possible that they could actually be removed if it turns out that applying the fontconfig configuration doesn't cause problems for anyone (including those who do not have the fonts in question installed).

I'll put a PPA package up shortly for people to test, then we can look at getting it into Ubuntu properly.

The plan for Ubuntu 'R' is to distribute these configurations to the individual font packages instead of having them shipped centrally in language-selector.

Thanks.

I did this at ppa:laney/language-selector.

It's not right, though. Let's take a look at 69-language-selector-zh-cn.conf as
an example.

 <match target="pattern">
  <test qual="any" name="family">
   <string>serif</string>
  </test>
  <edit name="family" mode="prepend" binding="strong">
   <string>DejaVu Serif</string>
   <string>Bitstream Vera Serif</string>
   <string>HYSong</string>
   <string>AR PL UMing CN</string>
   <string>AR PL UMing HK</string>
   <string>AR PL ShanHeiSun Uni</string>
   <string>AR PL New Sung</string>
   <string>WenQuanYi Bitmap Song</string>
   <string>AR PL UKai CN</string>
   <string>AR PL ZenKai Uni</string>
  </edit>
 </match>

So if I ask for "serif" I get DejaVu and Bitstream Vera before getting any
others. This overrides the default selection for /all/ users, whereas before it
would only happen for zh_CN users due to the LANG test.

We could perhaps guard the "generic" fonts by a LANG test and leave the rest
unguarded?

Or just remove those ones altogether and add the others (in "append" mode?)?

It would help a lot if someone could test these options (you can edit the file
directly in /etc/fonts/conf.avail/) and report back what the best option is.

We need to make sure that users who do not have this locale/font don't see any
change.

>We could perhaps guard the "generic" fonts by a LANG test and leave the rest
unguarded?

Then, how could one use a particular font over other fonts?
Japanese prefer Japanese font over Chinese font.
I don't know Chinese but I think Chinese prefer Chinese font over Japanese font.

Current workaround most Japanese Ubuntu users take is uninstalling the Chinese font.
Since their problem is, Chinese font is selected over Japanese font.
Other workaround is modifying the 69-language-selector-ja-jp.conf by hand.

We need a font preference setting completely independent from current locale setting.
Just because somebody using en_US.UTF-8 doesn't mean he wish to use random Japanese/Chinese fonts for each CJK characters.

This bug was fixed in the package ubiquity - 2.13.4

---------------
ubiquity (2.13.4) raring; urgency=low

  [ Colin Watson ]
  * Huge pile of minor changes to make ubiquity compatible with pep8 1.3.

  [ Dmitrijs Ledkovs ]
  * Clear "Read release notes or update the installer" if neither action
    are possible. (LP: #1066302)
  * If there is only one disk & use_device recipe is used, change next step
    button to "Install Now". (LP: #1050044)
  * When auto-partitioning is not possible, allow to manually partition
    the same drive installation medium is on. (LP: #1053030)
  * Fix the question_dialog height-for-width (LP: #862270)
  * Fonconfig-voodoo is gone, not sure if some other "fontconfig"ury
    should be called instead. (LP: #1043031)
  * Fix partman test_windows_only: if resize option is available, use_disk
    should be available as well.
  * Fix test_translated, after moving question dialog into glade from
    construction by hand.
  * Automatic update of included source packages: base-installer
    1.122ubuntu14, debian-installer-utils 1.91ubuntu6, flash-kernel
    3.0~rc.4ubuntu28.

  [ Martin Pitt ]
  * ubiquity/gtkwidgets.py: Don't use GObject.constants, just use the
    constants from GObject as intended. Fixes a crash with pygobject
    3.7.2.
  * Use glib API from the GLib GI module instead of GObject. The latter
    has been deprecated for a long time and triggers warnings with pygobject
    3.7.2.
 -- Dmitrijs Ledkovs <email address hidden> Tue, 27 Nov 2012 11:47:01 +0000

I think we /can/ remove the LANG guards for the japanese case, as this fontconfig file (69_…_ja-jp.conf) only refers to takao fonts. What if we were to do that and to move the configurations out to the font packages they refer to? Would this cause problems for Chinese users?

I don't see such a simple solution for Chinese - zh_MO/zh_HK and zh_CN/zh_SG have the same configurations, but there are differences between the two sets. And anyway I now understand that if we remove all LANG guards then one configuration (happens to be the Chinese because that's later alphabetically) 'wins' if the user has both Japanese and Chinese fonts installed. This is a problem because the fonts share glyphs. Do I understand this right?

If you use a non-japanese locale and frequently read Japanese text, please try upgrading to ppa:laney/language-selector. This package is the same as the Quantal version but has no lang guards for the Japanese fontconfig file. That's the fix that people have been doing themselves.

Hi everybody ,

Maybe I found an another way to fixed the fonts without fontconfig-voodoo.

It's all know that modifying /etc/fonts/conf.d/49-sansserif.conf can fix the messy code in the flash Player.

Today I undesignedly changed the fonts which in 49-sansserif.conf to system fonts (make it stay the same). It works !

The fonts become consistent.

Hmm, I'm having trouble recreating a Chinese environment to try and fix this bug. I installed a Chinese raring daily iso (from http://china-images.ubuntu.com) and set my locale back to en_GB. I can see the Chinese text on ubuntu.com/download (scroll down), but it looks consistent to me.

Could somebody give me a hint as to how to create an environment which exhibits the bugs with the fontconfig configurations?

ppa:laney/language-selector removes the Japanese fontconfig files completely and puts them in the font packages themselves. Please try it out and let me know if it works/doesn't work.

... and some fixes for Chinese text. The LANG guards are still there, but I've made some other changes that might make a difference. Please check out the PPA if you can.

@lain Lane
I make some screenshots on my just freshly installed Ubuntu 13.04
It seems that I can't reproduce the issue.
en_US
https://picasaweb.google.com/118409732640227044376/Bug1043031_en_US?authuser=0&feat=directlink
zh_TW
https://picasaweb.google.com/118409732640227044376/Bug1043031_zh_TW?authuser=0&feat=directlink
zh_TW & en_US comparison
https://picasaweb.google.com/118409732640227044376/Bug1043031_zh_TW_en_US_comparison?authuser=0&feat=directlink

@Iain Lane

With your PPA, the Chinese font is still ugly under the English locale, please see the attachment. I'm using Ubuntu 13.04.

@tualatrix
Just double checking. Do you have pretty Chinese fonts installed? And are the fonts just as ugly in other applications (coming from ubuntu, not third party)? E.g. Firefox and gedit?

@xnox

Yes, I have pretty font installed: WQY Micro Hei.

As you requested, I open three windows: Firefox, Chrome, and Nautilus displayed with three the same text.

1. Firefox is a little different from the other twos, it is Bitmap font here.
2. Chrome and Nautilus have the exactly the same display with ugly font.

I have some news to update, the problem to cause the ugly font is mainly because I don't have ttf-wqy-zenhei installed and fontconfig-config's 65-nonlatin.conf still has Zen Hei only.

....
   <family>WenQuanYi Zen Hei</family> <!-- han (zh-cn,zh-tw) -->
   <family>WenQuanYi Bitmap Song</family> <!-- han (zh-cn,zh-tw) -->
...

As WQY Micro Hei is the default installed font now, we should update fontconfig-config, add one line to 65-nonlatin.conf like this:

...
   <family>WenQuanYi Micro Hei</family> <!-- han (zh-cn,zh-tw) -->
   <family>WenQuanYi Zen Hei</family> <!-- han (zh-cn,zh-tw) -->
   <family>WenQuanYi Bitmap Song</family> <!-- han (zh-cn,zh-tw) -->
...

Then it will be fixed without touch any other files, fontconfig-voodoo is not needed too.

On Thu, Jan 31, 2013 at 10:20:46AM -0000, Ding Zhou wrote:
> I have some news to update, the problem to cause the ugly font is mainly
> because I don't have ttf-wqy-zenhei installed and fontconfig-config's
> 65-nonlatin.conf still has Zen Hei only.
>
> ....
> <family>WenQuanYi Zen Hei</family> <!-- han (zh-cn,zh-tw) -->
> <family>WenQuanYi Bitmap Song</family> <!-- han (zh-cn,zh-tw) -->
> ...
>
> As WQY Micro Hei is the default installed font now, we should update
> fontconfig-config, add one line to 65-nonlatin.conf like this:
>
> ...
> <family>WenQuanYi Micro Hei</family> <!-- han (zh-cn,zh-tw) -->
> <family>WenQuanYi Zen Hei</family> <!-- han (zh-cn,zh-tw) -->
> <family>WenQuanYi Bitmap Song</family> <!-- han (zh-cn,zh-tw) -->
> ...

Ah, excellent! Can you add this to the sans-serif and "WenQuanYi Micro
Hei Mono" to the monospace section too? If that all fixes it then I will
upload a new ttf-wqy-microhei to my PPA next week that you can test
after settings 65-nonlatin back to the default. (I'd prefer to change
this and not fontconfig's default is possible, but doing that is fine as
a fallback).

Thanks for your testing. :-)

Cheers,

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

Actually, "WenQuanYi Micro Hei" has already in the sans-serif, just not in serif section.

And I add "WenQuanYi Micro Hei Mono" to monospace section, everything works fine.

Wait for your new PPA.Thanks.

Alright, there - I've added this to 65-ttf-wqy-microhei in my PPA. Please let me know how it goes! (make sure you reset 65-nonlatin back to how it was)

Cheers

@laney

Hi, After upgrading packages from your PPA, the font has been much better now, but I have to say there's still some inconsistent of the Chinese font.

Please see the screenshot, the last two characters is easy to see the difference.

On Tue, Feb 05, 2013 at 04:10:56AM -0000, Ding Zhou wrote:
> @laney
>
> Hi, After upgrading packages from your PPA, the font has been much
> better now, but I have to say there's still some inconsistent of the
> Chinese font.
>
> Please see the screenshot, the last two characters is easy to see the
> difference.

OK, so we're getting closer - that's good!

So the other fonts that we ship by default and in the Chinese l-s
configuration are "AR PL UMing CN" and "AR PL UKai" - what if you add
those to 65-nonlatin?

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

After putting these two lines into the "sans-serif" section in 65-nolatin file, the font has become really what it should be!

<family>AR PL UKai</family>
<family>AR PL UMing CN</family>

There's no effect after I put these two line in the "serif" section.

And in #25, I made a mistake, the "WenQuanYi Micro Hei" was not in "sans-serif" by default, this is I added before :)

So what need to be done next step?

On Tue, Feb 05, 2013 at 11:02:44AM -0000, Ding Zhou wrote:
> After putting these two lines into the "sans-serif" section in
> 65-nolatin file, the font has become really what it should be!
>
> <family>AR PL UKai</family>
> <family>AR PL UMing CN</family>
>
> There's no effect after I put these two line in the "serif" section.
>
> And in #25, I made a mistake, the "WenQuanYi Micro Hei" was not in
> "sans-serif" by default, this is I added before :)
>
> So what need to be done next step?

I updated fonts-arphic-uming and fonts-arphic-ukai in my PPA; please
give them a try and see if it makes any difference for you after
resettings 65-nonlatin back to default.

Cheers :-)

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

Cool, this time the font becomes the standard ukai.

OK, so if it's all consistent for you now then I'll upload these packages to raring proper. Thanks.

(if Micro Hei should indeed be preferred then it's a fairly simple matter to alter the relative priorities so that it comes first)

Uploaded language-selector and a whole load of CJK fonts. We'll deal with any fallout in Raring itself from now on. Thanks for your help.

I've upgraded the packages from raring-proposed, and only left ttf-wqy-microhei package, the 65-ttf-wqy-microhei.conf is already in /etc/fonts/conf.d, the font still the same look as #22.

Do I still need to alter some config to make microhei by default?

This bug was fixed in the package language-selector - 0.101

---------------
language-selector (0.101) raring; urgency=low

  * This upload is about fixing LP: #1043031. Namely the use of LANG guards
    introduced in 0.86 was possibly not an entirely correct solution because
    there is a significant class of users who do not use the 'native' locale
    of these fonts, yet this is what a LANG guards tests for. We need the
    fontconfig configurations to apply correctly in these cases too.
  * Remove Japanese fontconfig file; these have been moved to the individual
    font packages which will be pulled in by l-s if appropriate.
  * Switch to using .maintscript file for removing conffiles; version
    debhelper BD as appropriate.
  * 69_zh* - remove wqy-microhei versions. This package now has its own
    fontconfig configuration.
  * fontconfig/README: Update to reflect reality of the 69_* files
  * Remove 'strong' binding and unnecessary fonts from zh_* configuration.
 -- Iain Lane <email address hidden> Wed, 06 Feb 2013 15:37:33 +0000

In 65-nolatin and 69-language-selector-zh-cn.conf 'WenQuanYi Zen Hei' is used instead of 'WenQuanYi Micro Hei' .

To me Micro Hei is prettier for chinese font.

For better rendering with WenQuanYi Zen Hei I modified 25-wqy-zenhei.conf to use 'hintslight' and 'rgb' for the hinting style.

Hey guys, I have just installed Raring (13.04) after using Precise (12.04) for the last year and ran into the same problem as Ding Zhou and Ryou Ezoe above when I couldn't find fontconfig-voodoo. I frequently need to work with Simplified Chinese and, to a lesser extent, Japanese, so I always run "sudo fontconfig-voodoo -s -f zh_CN" on new systems, but despite having played around with Language Selector, I have so far been unable to get fonts to look properly (see attached screenshot). Other solutions posted here didn't seem to work either.

What is currently the recommended way of going about this?

Please note that this is a closed bug.

CJK subscribers to this bug report may still be interested in the solution to bug #1173571. We have just made Droid Sans Fallback the preferred font in 69-language-selector- for both simplified and traditional Chinese. There is also an intention to include fonts-droid by default on all machines instead of ttf-wqy-microhei.

However, please don't comment on this bug any longer. If you have concerns about the just mentioned changes, or encounter other font related problems, please file a new bug or use an open relevant bug report.