change default CJK fonts to Noto CJK

fonts-noto-cjk installs NotoSansCJK.ttc. Time for Chinese? For Japanese?

i think its time for chinese and japanese, or maybe korean as well.

Hello tomoe_musashi,

I believe Noto Sans CJK is one of the best fonts available in free/libre licenses for Japanese. However I'm not convinced to set the Noto Sans CJK as default yet and I would like to know your motivations here.

> But for Chinese and Japanese community, i think that the answer is clear.
> noto-cjk is definitely better what we had before, like fonts- wqy and fonts-droid.

For Japanese locale, what makes you so confident when compared to fonts-takao, the current default font?

> Android community had received these complains for years, finally they got them fixed on lollipop.
> Fedora also set it as default chinese font start from F21.

Can you point to a link where the discussion happened in Fedora?

> and of course, i still hope that ubuntu could drop those 69-language-selector fontconfig files, just like what F13 did.

Can you explain a bit how F13 dropped the files? In Ubuntu, language-selector no longer manages fontconfig files. Instead font packages have fontconfig files, e.g. /etc/fonts/conf.avail/65-fonts-takao-pgothic.conf in fonts-takao-pgothic. Even with Noto Sans CJK I think we still need to choose font family name per locale such as "Noto Sans CJK JP" for ja_JP locale.

On 07/06/2015 05:14 PM, Nobuto Murata wrote:> On 06/23/2015 07:42 PM, tomoe_musashi wrote:
>> and of course, i still hope that ubuntu could drop those
>> 69-language-selector fontconfig files, just like what F13 did.
>
> Can you explain a bit how F13 dropped the files? In Ubuntu, language-
> selector no longer manages fontconfig files.

Actually it does. The Japanese fontconfig files were moved to fonts-takao, but there are still a bunch of 69-language-selector-zh-* files, and I think tomoe_musashi is referring to those.

I, too, wonder which files F13 dropped. Fedora does not use language-selector.

2015-07-07 0:38 GMT+09:00 Gunnar Hjalmarsson <email address hidden>:
> On 07/06/2015 05:14 PM, Nobuto Murata wrote:> On 06/23/2015 07:42 PM, tomoe_musashi wrote:
>>> and of course, i still hope that ubuntu could drop those
>>> 69-language-selector fontconfig files, just like what F13 did.
>>
>> Can you explain a bit how F13 dropped the files? In Ubuntu, language-
>> selector no longer manages fontconfig files.
>
> Actually it does. The Japanese fontconfig files were moved to fonts-
> takao, but there are still a bunch of 69-language-selector-zh-* files,
> and I think tomoe_musashi is referring to those.

Ah, right. I thought CJK related files were moved when
fontconfig-voodoo was dropped. Thanks for the correction.

Hi Nobuto Murata,

Actually i think that fonts-takao is one of the best Japanese out there.but when its compared to fonts-noto, noto provides more weights, better CJK fonts support with multilingual, and personally, better monospace fonts paring bwtween English letters and CJK fonts, better e-reading experience with unified fonts design.

here are some links that they discussed about changing Fedora's Chinese fonts(in Chinese):
https://lists.stg.fedoraproject.org/archives/list/chinese%40lists.fedoraproject.org/thread/ATEHR3KTVLOL5WSXV2HZ4BAPPKDELXNF/
https://<email address hidden>/thread/ABGCWMJK7BDJD6QDZ4UFCHLVJXCCFV7T/

i apologize about uncertain F13 info i posted. its nothing to do with language-selector. i will edit the description.
And for those applications lacking locl GSUB feature and lang tagging, setting font family for per locale would be necessary.

Fedora 21 only applies Adobe Source Han Sans / Noto Sans CJK to Chinese locale (zh_CN and zh_TW).

Fedora does not use 69-language-selectors-*.conf because they only set one major font for one locale, and they write the relevant configuration in the corresponding fontconfig fille for the major font. That is, they set one font for Serif, Sans-serif, and Monospace (listed below). However, I don't like this method. Using 69-language-selectors-*.conf is more flexible here.

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
    <match>
        <test name="family" compare="contains">
                <string>Source Han Sans</string>
        </test>
        <edit name="autohint" mode="assign">
                <bool>false</bool>
        </edit>
        <edit name="hintstyle" mode="assign">
                <const>hintfull</const>
        </edit>
    </match>
    <match>
        <test name="lang">
            <string>zh-tw</string>
        </test>
        <test name="family">
            <string>monospace</string>
        </test>
        <edit name="family" mode="prepend">
        <string>Source Han Sans TW</string>
        <string>Source Han Sans CN</string>
        </edit>
        <edit name="family" mode="prepend" binding="strong">
        <string>DejaVu Sans Mono</string>
        </edit>
    </match>
    <match>
        <test name="lang">
            <string>zh-hk</string>
        </test>
        <test name="family">
            <string>monospace</string>
        </test>
        <edit name="family" mode="prepend">
        <string>Source Han Sans TW</string>
        <string>Source Han Sans CN</string>
        </edit>
        <edit name="family" mode="prepend" binding="strong">
        <string>DejaVu Sans Mono</string>
        </edit>
    </match>

    <alias>
        <family>Source Han Sans TW</family>
        <default>
            <family>monospace</family>
        </default>
    </alias>

    <match>
        <test name="lang">
            <string>zh-tw</string>
        </test>
        <test name="family">
            <string>serif</string>
        </test>
        <edit name="family" mode="prepend">
            <string>Source Han Sans TW</string>
            <string>Source Han Sans CN</string>
        </edit>
    </match>
    <match>
        <test name="lang">
            <string>zh-hk</string>
        </test>
        <test name="family">
            <string>serif</string>
        </test>
        <edit name="family" mode="prepend">
            <string>Source Han Sans TW</string>
            <string>Source Han Sans CN</string>
        </edit>
    </match>

    <alias>
        <family>Source Han Sans TW</family>
        <default>
            <family>serif</family>
        </default>
    </alias>

    <match>
        <test name="lang">
            <string>zh-tw</string>
        </test>
        <test name="family">
            <string>sans-serif</string>
        </test>
        <edit name="family" mode="prepend">
            <string>Source Han Sans TW</string>
            <string>Source Han Sans CN</string>
        </edit>
    </match>
    <match>
        <test name="lang">
            <string>zh-hk</string>
        </test>
   ...

According to the README pdf file (https://github.com/adobe-fonts/source-han-sans/raw/release/SourceHanSansReadMe.pdf) of Adobe Source Han Sans, and the experience within LibreOffice, I have to say that Super OTC file is not well supported in Linux desktop.

It is suggested using Regional Subset OTF files in Linux for now; however, that will take us a lot of space to keep all the OTF files. This is what Fedora provides for CN and TW locales, 7 files for about 39.6MB for TW locale and about 58.6MB for CN locale.

The other way is to endure the poor support of Super OTC and file related issues to those infected packages. And this will be a hard way for us to go. It will benefit the ecosystem better in the long run, but I don't recommend this way.

I agree that using 69-language-selectors-*.conf is more flexible IF ubuntu resulted to use Regional Subset OTF instead of Super OTC.
We should not fallback a sans-serif font to serif.
Also, they provide fixed width font for regular and bold weight font start from 1.002, as known as Source Han Sans HW or Noto Sans Mono CJK.
As their monospace fonts(english letters) are designed for SHS/Noto, i don't see the point to fallback DejaVu Sans Mono.

After reading through the threads of discussion on Noto fonts on Fedora 21/22, I'm getting a bit conservative about whether we want to change the default font for CJKV at the moment. Most of our users are still using non-hidpi displays, and it appears that using Noto fonts with that would result into unexpected results...

Status changed to 'Confirmed' because the bug affects multiple users.

Hi Aron,

Have you seen any particular unexpected results? I’ve been using Noto Sans CJK for about a month on a low-dpi display. Based on my experience, besides the multiple weights it provides, it looks way better compared to WenQuanYi Micro Hei.

WenQuanYi Micro Hei was derived from Droid Sans Fallback, which is a font particularly designed for saving space. Noto Sans CJK is a very well-designed font family with complete character set. I’d say Noto Sans CJK is no doubt the best open-source Chinese font at this time.

Comparing with WenQuanYi Micro Hei, Noto Sans is surely a better choice to me. For low-DPI display, a proper hinting configuration is needed. (if not using embedded bitmap font) This should not be a show stopper here?

MIR for fonts-noto-cjk filed as Bug #1532533.

happyaron: O again, rendering, rendering, rendering. For systems with TT Bytecode support, be5invis has a release of SHS with his sfdhanautohint (for CJK) and ttfautohint applied called [Inziu]. Follow the link and fetch inziu-20\d{10}.

These generally look better on lower ppem sizes (common for loDPI screen display), and is turned off (just not generating the bytecode) for sizes >= 25ppem (and for fairly impossible sizes like <=9ppem). Hey this discussion looks familiar doesn't it…

Oh and these are all TrueType, so expect the same problem (or benefit, for Qt users) with Kaigen Gothic—you lose some OpenType features. Generated bytecode also makes the files fat.

  [Inziu]:http://code.fosshub.com/Inziu/downloads

Anyway these sharp things will not be considered with Ubuntu's current hintslight policy. This policy is not quite loDPI optimized either so you might want to… whatever. WQY looks blurry too, you know. I thought the WQY files were hintless actually.

My decaying brain… I mean \d{6}. I should just say ‘anything that's not iosevka’.

* * *

Well the biggest blocker here is actually some application support things. Qt show poor overall handling of some OpenType features used in SHS, the most significant being LOCL used in all those Super files. Line spacing also look weird under both Noto and SHS (SHS has a few releases to fixes some of the problems under OS X but … some said they break Plasma 5.)

If compatibility looks like a issue, a (relatively crazy) solution is to preinstall Kaigen Gothic instead. It's basically a TrueType version with some features removed (sigh.) If size looks like a problem (it definitely is), try to wise with the weights to preinstall, and don't even consider adding bytecode.

The MIR has been accepted, would you mind to review the l-s changes? I'll continue to track what to do in fonts-noto-cjk.

On 2016-01-12 06:37, Aron Xu wrote:
> The MIR has been accepted, would you mind to review the l-s changes?

Not at all, I'll do that. Are we talking about Chinese only at this time?

Please note that the seeds of all the flavors need to be changed in the same manner as in your linked merge proposal for ubuntu.xenial.

> I'll continue to track what to do in fonts-noto-cjk.

One thing is that we need an equivalent to the config file in fonts-droid in order to make fonts-noto-cjk co-exist with the fonts-arphic-* packages, i.e. to prevent that UKai and UMing is picked over Noto by default when there is a non-Chinese locale. (See the discussion in bug #1227034.) I played with a possible config file, which may or may not be sufficient for the purpose:

$ cat /etc/fonts/conf.d/64-fonts-noto-cjk.conf
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
        <alias>
                <family>sans-serif</family>
                <prefer>
                        <family>Noto Sans CJK SC</family>
                        <family>Noto Sans CJK TC</family>
                </prefer>
        </alias>
</fontconfig>
$ LANG=en_US.UTF-8 fc-match -a | grep -iE 'noto|ar pl' | head -20
NotoSansCJK.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Black"
NotoSansCJK.ttc: "Noto Sans CJK TC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Black"
ukai.ttc: "AR PL UKai CN" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
ukai.ttc: "AR PL UKai TW" "Book"
uming.ttc: "AR PL UMing CN" "Light"
uming.ttc: "AR PL UMing HK" "Light"
uming.ttc: "AR PL UMing TW" "Light"

But this certainly needs to be tested by users who speak Chinese.

The language-selector changes were just uploaded. This *really* need to be carefully tested soon by some Chinese users.

@Aron: Any thoughts on the matters I raised in comment #20?

This bug was fixed in the package language-selector - 0.154

---------------
language-selector (0.154) xenial; urgency=medium

  * data/pkg_depends, fontconfig/69-language-selector-zh-*.conf:
    Default font for Chinese changed from "Droid Sans Fallback" to
    "Noto Sans CJK" (LP: #1468027).

 -- Gunnar Hjalmarsson <email address hidden> Sat, 16 Jan 2016 00:43:00 +0100

@Gunnar, I'm working on that, :)

There should be a real monospaced English character font prepended, like DejaVu Sans Mono (i think the Noto Sans Mono is too small for reading). For some applications, there are problems with the current config file.
reference: https://github.com/adobe-fonts/source-han-sans/issues/57

Also, i think the Aliases for Chinese should be modified for noto-cjk
the modified part:
<!-- Aliases for Simplified Chinese Windows fonts -->
    <alias>
        <family>SimSun</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>NSimSun</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>SimSun-18030</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>NSimSun-18030</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>宋体</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>新宋体</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>AR MingtiM GB</family>
        <accept>
            <family>HYSong</family>
            <family>AR PL UMing CN</family>
        </accept>
    </alias>
    <alias>
        <family>KaiTi</family>
        <accept>
            <family>AR PL UKai CN</family>
            <family>AR PL ZenKai Uni</family>
        </accept>
    </alias>
    <alias>
        <family>楷体</family>
        <accept>
            <family>AR PL UKai CN</family>
            <family>AR PL ZenKai Uni</family>
        </accept>
    </alias>
    <alias>
        <family>Microsoft YaHei</family>
        <accept>
            <family>Noto Sans CJK SC</family>
            <family>WenQuanYi Micro Hei</family>
            <family>WenQuanYi Zen Hei</family>
        </accept>
    </alias>
    <alias>
        <family>微软雅黑</family>
        <accept>
            <family>Noto Sans CJK SC</family>
            <family>WenQuanYi Micro Hei</family>
            <family>WenQuanYi Zen Hei</family>
        </accept>
    </alias>
<!-- Aliases for Traditional Chinese Windows fonts -->
    <alias>
        <family>MingLiU</family>
        <accept>
            <family>AR PL UMing TW</family>
        </accept>
    </alias>
    <alias>
        <family>細明體</family>
        <accept>
            <family>AR PL UMing TW</family>
        </accept>
    </alias>
    <alias>
        <family>PMingLiU</family>
        <accept>
            <family>AR PL UMing TW</family>
        </accept>
    </alias>
    <alias>
        <family>新細明體</family>
        <accept>
            <family>AR PL UMing TW</family>
        </accept>
    </alias>
    <alias>
        <family>AR MingtiM BIG-5</family>
        <accept>
            <family>AR PL UMing TW</family>
       ...

Thanks for your comments, tomoe_musashi.

On 2016-01-16 05:32, tomoe_musashi wrote:
> There should be a real monospaced English character font prepended, like
> DejaVu Sans Mono (i think the Noto Sans Mono is too small for reading).
> For some applications, there are problems with the current config file.
> reference: https://github.com/adobe-fonts/source-han-sans/issues/57

Do you mean like this:

                <edit name="family" mode="prepend" binding="strong">
                        <string>DejaVu Sans Mono</string>
                        <string>Noto Sans Mono CJK SC</string>
                        <string>WenQuanYi Zen Hei Mono</string>
                        <string>HYSong</string>
                        ...
                </edit>

> Also, i think the Aliases for Chinese should be modified for noto-cjk

Ok, will do.

@Gunnar Hjalmarsson, exactly like that. Thank you for your hard work.

For the prepend list, separated conf. for TC and SC is needed as different localized glyphs now supported by Noto sans. It would be nice if suitable "real" mono space EN fonts are prepended.

On 2016-01-17 04:51, tomoe_musashi wrote:
> @Gunnar Hjalmarsson, exactly like that.

Thanks for confirming.

On 2016-01-17 04:53, Yuan Chao wrote:
> For the prepend list, separated conf. for TC and SC is needed as
> different localized glyphs now supported by Noto sans.

Right, we've had different fonts for TC and SC for many years, so that's already in place.

On 2016-01-17 05:12, Gunnar Hjalmarsson wrote:
> Right, we've had different fonts for TC and SC for many years, so
> that's already in place.

s/fonts/confs/

The build of language-selector with the latest changes failed.
http://irclogs.ubuntu.com/2016/01/17/%23ubuntu-devel.html#t21:05

So please be patient; sooner or later it will end up in the archive. ;)

This bug was fixed in the package language-selector - 0.155

---------------
language-selector (0.155) xenial; urgency=medium

  * fontconfig/69-language-selector-zh-*.conf:
    Prepend "DejaVu Sans Mono" for monospace (LP: #1468027).
  * fontconfig/30-cjk-aliases.conf:
    Include "Noto Sans CJK" where applicable (LP: #1468027).

 -- Gunnar Hjalmarsson <email address hidden> Sun, 17 Jan 2016 17:30:00 +0100

fc-cache -f -v is insanely slow with fonts-noto-cjk, upwards of 20 minutes on slower atom systems!!!

Just played with the config file for fonts-noto-cjk that @Gunnar posted above.
Its works great in non-Chinese locale, but it lacks monospace font support.
this should be added:
        <alias>
                 <family>monospace</family>
                 <prefer>
                         <family>Noto Sans Mono CJK SC</family>
                         <family>Noto Sans Mono CJK TC</family>
                 </prefer>
         </alias>

Thanks, tomoe_musashi. I wrote a patch to get it in.

Just tried 16.04 nightly, I think the CJK font is too thin compared to Ubuntu Regular. Which weight for Noto Sans CJK do we use here? I played a little in LibreOffice, I believe the “Regular” weight should be used by default.

Font weight isn't specified in the fontconfig files. For simplified Chinese, fc-match lists them in this order:

$ LC_CTYPE=zh_CN.UTF-8 fc-match -a | head -7
NotoSansCJK.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Black"

Possibly it means that "DemiLight" is used by default. How would you specify font weight in fontconfig?

Using Regular here

$ LC_CTYPE=zh_TW.UTF-8 fc-match -a | head -7
SourceHanSansTW-Regular.otf: "思源黑體 TW" "Regular"
SourceHanSansCN-Regular.otf: "思源黑体 CN" "Regular"
SourceHanSansJP-Regular.otf: "Source Han Sans JP" "Regular"
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSansCondensed.ttf: "DejaVu Sans" "Condensed"

currently no much problem about it.

Regarding monospace, a +1 for musashi on prepending, since I have seen SHS HW doing terrible at distinguishing 0O 1Il. If someone can verify that these chars don't look bad in Noto Mono CJK, I guess that works too.

Note that you should always only prepend half-width fonts, where ``wcwidth(foochar) == 1`` means the glyph takes exactly 1/2em, or character alignment might still go wild.

Gave up with fontforge and put the files into https://nodebox.github.io/opentype.js/font-inspector.html.

Everything in the font looks perfectly normal now, but I am not sure if it's the good old fontFamily-fontSubfamily & preferredFamily-preferredSubfamily that confused fontconfig.

Adding fontconfig to bug.

Fontconfig bug reported upstream. And again, everything is fine with Noto CJK.

gryzzli, see the two links for something about fc-match speed:

* https://github.com/adobe-fonts/source-han-sans/issues/85#issuecomment-76082963
* https://bugs.freedesktop.org/show_bug.cgi?id=64766

Thanks for your work, Mingye. What you are doing is unfortunately above my head. ;)

Want to say, though, that the config file you mentioned in the upstream bug report doesn't have anything to do with it. It's not in the Ubuntu archive yet.

One fontconfig .conf file, which is in use at this time, is:

/etc/fonts/conf.avail/69-language-selector-zh-cn.conf

That file makes the "fc-match --verbose" command output the details for the demilight font.

OTOH, also in a session with a non-Chinese locale, demilight is the first font in a fc-match listing:

$ LC_CTYPE=en_US.UTF-8 fc-match -a | grep 'Noto Sans CJK SC'
NotoSansCJK.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Black"

Maybe there is a possible simple solution, after all.

As an experiment I added a line to 69-language-selector-zh-cn.conf:

--- 69-language-selector-zh-cn.conf.old
+++ 69-language-selector-zh-cn.conf
@@ -27,6 +27,7 @@
             <string>zh-cn</string>
         </test>
   <edit name="family" mode="prepend" binding="strong">
+ <string>Noto Sans CJK SC Regular</string>
    <string>Noto Sans CJK SC</string>
    <string>WenQuanYi Zen Hei</string>
    <string>HYSong</string>

That made a difference:

$ LC_CTYPE=zh_CN.UTF-8 fc-match -a | head -7
NotoSansCJK.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Black"

gunnarhj:
> Want to say, though, that the config file you mentioned in the upstream bug report doesn't have anything to do with it. It's not in the Ubuntu archive yet.

Umm… The good news is they seen to point to the same problem, and on JeffBai's machine that's almost how the config was written (without language matching, etc.). Anyway that works as a good test case...

> + <string>Noto Sans CJK SC Regular</string>

Nice workaround. Do you mind testing other weights too?

$ LC_CTYPE=zh_CN.UTF-8 fc-match sans-serif:weight=bold -a | head -7

If bold works, you can assume others work too, I think. If you really want to make sure, you can try out all the possible weight values in https://www.freedesktop.org/software/fontconfig/fontconfig-user.html, shown under <const>.

Yes, "Bold" works as well. The explanation is that each font has two family names: One which is prepended with the font weight and one which is not.

$ fc-list : family | grep 'Noto Sans CJK SC'
Noto Sans CJK SC,Noto Sans CJK SC Light
Noto Sans CJK SC,Noto Sans CJK SC Regular
Noto Sans CJK SC,Noto Sans CJK SC Medium
Noto Sans CJK SC,Noto Sans CJK SC Bold
Noto Sans CJK SC,Noto Sans CJK SC Black
Noto Sans CJK SC,Noto Sans CJK SC Thin
Noto Sans CJK SC,Noto Sans CJK SC DemiLight

There is a built-in priority order in NotoSansCJK.ttc by default, where "DemiLight" has been given first priority. Maybe there is a thought behind it. Controlling it via fontconfig .conf files is possibly not a workaround; possibly that's how it's supposed to work.

The question, I think, is if we *want* to change the default priority order.

So updates from fontconfig for those who doesn't want to wait for the bugzilla crawler..

Fontconfig 2.11.1 is ancient, and support for demilight has been added in https://bugs.freedesktop.org/show_bug.cgi?id=81453, which ended up in those oddly-numbered releases like 2.11.91 or so. The current latest release found in https://www.freedesktop.org/software/fontconfig/release/ is 2.11.94.

And the wiki page pointing to 2.11.1 was never updated since 2014-03-07.

Hmm... It's your decision to decide whether to update fc to such an undocumented version.

So hi everyone...

After letting Arthur use my computer for quite a while... well whatever.

I've updated to 2.11.94 on AOSC OS, and here's what I've got.

jeffbai [ ~ ] $ LC_CTYPE=zh_CN.UTF-8 fc-match -a | head -31
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"
n019003l.pfb: "Nimbus Sans" "Regular"
n019004l.pfb: "Nimbus Sans" "Bold"
n019023l.pfb: "Nimbus Sans" "Italic"
n019024l.pfb: "Nimbus Sans" "Bold Italic"
NotoSans-Regular.ttc: "Noto Sans UI" "Regular"
NotoSansUI-Regular.ttf: "Noto Sans UI" "Regular"
NotoSans-Bold.ttc: "Noto Sans UI" "Bold"
NotoSansUI-Bold.ttf: "Noto Sans UI" "Bold"
NotoSans-Italic.ttc: "Noto Sans UI" "Italic"
NotoSansUI-Italic.ttf: "Noto Sans UI" "Italic"
NotoSans-BoldItalic.ttc: "Noto Sans UI" "Bold Italic"
NotoSansUI-BoldItalic.ttf: "Noto Sans UI" "Bold Italic"
NotoSans-Regular.ttc: "Noto Sans" "Regular"
NotoSans-Regular.ttf: "Noto Sans" "Regular"
NotoSans-Bold.ttc: "Noto Sans" "Bold"
NotoSans-Bold.ttf: "Noto Sans" "Bold"
NotoSans-Italic.ttc: "Noto Sans" "Italic"
NotoSans-Italic.ttf: "Noto Sans" "Italic"
NotoSans-BoldItalic.ttc: "Noto Sans" "Bold Italic"
NotoSans-BoldItalic.ttf: "Noto Sans" "Bold Italic"
NotoSansCJK-Medium.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK SC" "Black"
NotoSansCJK-Regular.ttc: "Noto Sans Mono CJK SC" "Regular"
NotoSansMonoCJKsc-Regular.otf: "Noto Sans Mono CJK SC" "Regular"
NotoSansCJK-Bold.ttc: "Noto Sans Mono CJK SC" "Bold"
NotoSansMonoCJKsc-Bold.otf: "Noto Sans Mono CJK SC" "Bold"

Things are getting weird aren't they. But I do need to point out that the "Nimbus" gimbles are from 30-metric-aliases.conf.

Fontconfig 2.11.9x are actually devel versions. https://www.freedesktop.org/wiki/Software/fontconfig/Devel/

These patches are relevant:

ffda7c0
bf9df5a
be6506c
# bug 82228 introduced by ffda7c0
01bb697
80edacc

Ok, Mingye, it seems as if I stand corrected, and that it is an issue in the version of fontconfig in the Ubuntu archive.

Nevertheless, Ubuntu 16.04 LTS will be released in a few weeks. Considering that it's "feature freeze", I doubt there is a plan to update fontconfig for 16.04.

At the same time I happened to find a way to work around the issue and make "Regular" the default weight via the .conf files, which is what Xhacker stated as desirable.

What do others think?

I support to specify font weight directly in fontconfig.

And It is OK to specify which font you want including font weight, so I think it is a normal usage case not a workaround anyway.

I uploaded language-selector with the change we discussed and attached a modified fonts-noto-cjk patch. This way it will be easier for people to see what "Regular" as default weight instead of "DemiLight" looks like.

@Mingye: I'm not able to tell if adding all those patches to the Ubuntu fontconfig package would give us the same result. But it would require quite some work.

This bug was fixed in the package language-selector - 0.160

---------------
language-selector (0.160) xenial; urgency=medium

  * fontconfig/69-language-selector-zh-*.conf:
    Make "Regular" the default font weight for "Noto Sans CJK"
    (LP: #1468027).

 -- Gunnar Hjalmarsson <email address hidden> Sat, 12 Mar 2016 08:17:00 +0100

Let me file the fc demi report separately then. A fix is always better than a hack, and with Noto Sans CJK hacked there are still a whole bunch of Notos and other weight-rich fonts.

Just realized my PPA pkg is based on 2.11.1-0ubuntu6 instead of current ubuntu8, hmm. Can't see xenial on bzr... Whatever.

On 2016-03-12 18:46, Mingye Wang wrote:
> Let me file the fc demi report separately then. A fix is always
> better than a hack, and with Noto Sans CJK hacked there are still a
> whole bunch of Notos and other weight-rich fonts.

A separate fc demi report is a good idea IMO.

As regards other Notos: at the moment we have other default fonts in Ubuntu for Japanese and Korean (may be changed in 16.04+), so that's not an emergency for the 16.04 release.

> Can't see xenial on bzr...

Right, that is true for many packages. Don't know why. I have made patches using diff in some cases.

BTW, may this be a related bug?
https://bugs.launchpad.net/ubuntu-translations/+bug/1540063

On 2016-03-13 00:46, Kenichi Ito wrote:
> BTW, may this be a related bug?
> https://bugs.launchpad.net/ubuntu-translations/+bug/1540063

Don't think so. One step in debugging bug #1540063 may be to run a live session in Japanese, open a terminal window and run this command:

fc-match -s | head -10

The first font in the output ought to be "Takao Pゴシック" "Regular".

> BTW, may this be a related bug?
> https://bugs.launchpad.net/ubuntu-translations/+bug/1540063
It looks like some (traditional?) Chinese font (kai?) is picked up instead of the "Takao Pゴシック" Japanese font requested. Due to the lack of the glyphs, it's further fallbacked to other(?) "ゴシック" Japanese font. Should be due to some mis-configuration of the font alias. Doesn't look like a related bug to me.

@Gunnar @Yuan, i think it is a related bug. If the latest patch for fonts-noto-cjk is released,
We will see that the Noto Sans CJK SC is picked up instead of TakaoPGothic in the first screen of ubiquity installation(before choosing "Try Ubuntu" and specifying the language).

Yes, tomoe_musashi, they may well be related somehow. I'm not sure, though, that "Noto Sans CJK SC" would be picked in case of a Japanese locale, considering that "Noto Sans CJK JP" exists.

I decided to let language-selector-common provide the contents of the proposed fonts-noto-cjk patch, and hence removed the latter. I also added a couple of Japanese line to it.

Just uploaded language-selector, so we will soon be able to test.

This bug was fixed in the package language-selector - 0.161

---------------
language-selector (0.161) xenial; urgency=medium

  * fontconfig/64-language-selector-prefer.conf,
    debian/language-selector-common.links:
    Prevent that UKai and UMing is picked over default Chinese and
    Japanese fonts when the locale is something else but Chinese or
    Japanese (LP: #1468027, LP: #1540063).

 -- Gunnar Hjalmarsson <email address hidden> Sun, 13 Mar 2016 14:02:00 +0100

With the new .conf file, the interesting fonts appear in this order for me when running with a non-CJKV locale on a system where the Chinese language support (with the fonts-arphic-* packages) is present:

$ fc-match -a sans-serif | grep -iE 'noto|takao|ar pl' | head -22
NotoSansCJK.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK SC" "Black"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Regular"
NotoSansCJK.ttc: "Noto Sans CJK TC" "DemiLight"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Medium"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Thin"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Light"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Bold"
NotoSansCJK.ttc: "Noto Sans CJK TC" "Black"
fonts-japanese-gothic.ttf: "TakaoPGothic" "Regular"
TakaoPGothic.ttf: "TakaoPGothic" "Regular"
ukai.ttc: "AR PL UKai CN" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
ukai.ttc: "AR PL UKai TW" "Book"
uming.ttc: "AR PL UMing CN" "Light"
uming.ttc: "AR PL UMing HK" "Light"
uming.ttc: "AR PL UMing TW" "Light"
$ fc-match -a monospace | grep -iE 'noto|takao|ar pl' | head -11
NotoSansCJK.ttc: "Noto Sans Mono CJK SC" "Regular"
NotoSansCJK.ttc: "Noto Sans Mono CJK SC" "Bold"
NotoSansCJK.ttc: "Noto Sans Mono CJK TC" "Regular"
NotoSansCJK.ttc: "Noto Sans Mono CJK TC" "Bold"
TakaoGothic.ttf: "TakaoGothic" "Regular"
ukai.ttc: "AR PL UKai CN" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
ukai.ttc: "AR PL UKai TW" "Book"
uming.ttc: "AR PL UMing CN" "Light"
uming.ttc: "AR PL UMing HK" "Light"
uming.ttc: "AR PL UMing TW" "Light"

I don't know how to figure out if this works, or if the Chinese "Noto Sans CJK" fonts messes it up somehow when rendering Japanese contents.

Is Source Han Sans (Noto Sans) going to be the default fonts in Ubuntu 16.04?
If it is true, it will be nice.

Noto Sans will be default for Chinese in 16.04.

In an attempt to fix bug #1540063 better I'm playing with variants of the new 64-language-selector-prefer.conf file via a PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/japanese-fonts

However, there is a risk that the current test version of the file affects rendering of Chinese contents adversely under a non-Chinese locale. Therefore I'd appreciate if a few Chinese users could install language-selector from the PPA and let us know how Chinese characters look in a session with a non-Chinese locale.

ping

On 2016-03-15 02:39, Gunnar Hjalmarsson wrote:
> ... I'd appreciate if a few Chinese users could install
> language-selector from the PPA and let us know how Chinese
> characters look in a session with a non-Chinese locale.

Anyone?

Just played with the new 64-language-selector-prefer.conf from PPA in a live-session with English locale.
It didn't affect language-specified content, SChinese contents are in Noto SC and TChinese contents are in Noto TC(Firefox)
non-lanuage-specified content will just render according to the new config file.
Although i think its not a workaround to fix the ubiquity bug, but its better and acceptable.
Here comes a problem, in non-CJK locale, Firefox uses TakaoPGohtic to render non-lang-specified content, even with the 64-language-selector-prefer.conf config file, it also need to be changed to Noto else it breaks the consistency.

That's good news, tomoe_musashi. Thanks for confirming! Just uploaded to the archive.

On 2016-03-16 14:16, tomoe_musashi wrote:
> Here comes a problem, in non-CJK locale, Firefox uses TakaoPGohtic
> to render non-lang-specified content, even with the
> 64-language-selector-prefer.conf config file, it also need to be
> changed to Noto else it breaks the consistency.

I don't understand. Why on earth would Firefox use TakaoPGothic with a non-CJK locale?

@Gunnar, i don't know, seems like some other things are affecting the font fallback,
which makes Firefox didn't work as expected(64-language-selector-prefer.conf).
Currently if you type any kanji(or chinese words) on the firefox bar, or viewing some non-lang-specified contents with kanji, its rendering with TakaoPGothic.
But i found that modifying 65-nonlatin.conf would fix the problem, is it relevant?

This bug was fixed in the package language-selector - 0.162

---------------
language-selector (0.162) xenial; urgency=medium

  * fontconfig/64-language-selector-prefer.conf:
    Make Japanese contents be rendered using "Noto Sans CJK JP" in
    case of a non-CJKV locale (LP: #1540063, LP: #1468027).

 -- Gunnar Hjalmarsson <email address hidden> Wed, 16 Mar 2016 15:07:00 +0100

I'm confused.

What I did in this latest version (which just reached the archive) was putting "Noto Sans CJK JP" before the Chinese equivalents. If non-language-specified content (I assume you are talking about Chinese characters) had been rendered using "Noto Sans CJK JP", it would have made sense. (But would still have been a problem, I suppose...)

I'd rather not touch 65-nonlatin.conf, if possible. It belongs to the fontconfig-config, and I know that upstream is disinclined to modify it. But what kind of change to that file would make a difference?

Maybe we are trying to do the impossible. :(

I'm getting back about Chinese and Japanese rendering with a non-CJKV locale. We need to reach a conclusion as regards the default configuration.

Currently 64-language-selector-prefer.conf looks like this:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
        <alias>
                <family>sans-serif</family>
                <prefer>
                        <family>Noto Sans CJK JP Regular</family>
                        <family>Noto Sans CJK JP</family>
                        <family>Noto Sans CJK SC Regular</family>
                        <family>Noto Sans CJK SC</family>
                        <family>Noto Sans CJK TC Regular</family>
                        <family>Noto Sans CJK TC</family>
                 </prefer>
        </alias>
        <alias>
                <family>monospace</family>
                <prefer>
                        <family>Noto Sans Mono CJK JP</family>
                        <family>Noto Sans Mono CJK SC</family>
                        <family>Noto Sans Mono CJK TC</family>
                </prefer>
        </alias>
</fontconfig>

That configuration seems to be an improvement for Japanese (bug #1540063). I'm assuming that the Japanese lines in the file have nothing to do with 'the Takao in Firefox issue' which tomoe_musashi mentioned in comment #81. (Please let me know if I'm wrong here. If I'm not, I take it that we can keep the current version of 64-language-selector-prefer.conf in Ubuntu 16.04.)

Possibly 'the Takao in Firefox issue' is a Firefox bug.

@tomoe_musashi: You mentioned that modifying 65-nonlatin.conf may fix that issue. Can you please let us know about the change you made?

FYI I filed bug #1560548, but I don't think it needs to be fixed in 16.04.

But this 64-language-selector-prefer.conf could lead to regression for
Chinese, unfortunately.

> On Mar 23, 2016, at 00:36, Gunnar Hjalmarsson <email address hidden> wrote:
>
> I'm getting back about Chinese and Japanese rendering with a non-CJKV
> locale. We need to reach a conclusion as regards the default
> configuration.
>
> Currently 64-language-selector-prefer.conf looks like this:
>
> <?xml version="1.0"?>
> <!DOCTYPE fontconfig SYSTEM "fonts.dtd">
> <fontconfig>
> <alias>
> <family>sans-serif</family>
> <prefer>
> <family>Noto Sans CJK JP Regular</family>
> <family>Noto Sans CJK JP</family>
> <family>Noto Sans CJK SC Regular</family>
> <family>Noto Sans CJK SC</family>
> <family>Noto Sans CJK TC Regular</family>
> <family>Noto Sans CJK TC</family>
> </prefer>
> </alias>
> <alias>
> <family>monospace</family>
> <prefer>
> <family>Noto Sans Mono CJK JP</family>
> <family>Noto Sans Mono CJK SC</family>
> <family>Noto Sans Mono CJK TC</family>
> </prefer>
> </alias>
> </fontconfig>
>
> That configuration seems to be an improvement for Japanese (bug
> #1540063). I'm assuming that the Japanese lines in the file have nothing
> to do with 'the Takao in Firefox issue' which tomoe_musashi mentioned in
> comment #81. (Please let me know if I'm wrong here. If I'm not, I take
> it that we can keep the current version of 64-language-selector-
> prefer.conf in Ubuntu 16.04.)
>
> Possibly 'the Takao in Firefox issue' is a Firefox bug.
>
> @tomoe_musashi: You mentioned that modifying 65-nonlatin.conf may fix
> that issue. Can you please let us know about the change you made?
>
> FYI I filed bug #1560548, but I don't think it needs to be fixed in
> 16.04.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1468027
>
> Title:
> change default CJK fonts to Noto CJK
>
> Status in Fontconfig:
> Fix Released
> Status in kubuntu-meta package in Ubuntu:
> In Progress
> Status in language-selector package in Ubuntu:
> Fix Released
> Status in lubuntu-meta package in Ubuntu:
> In Progress
> Status in ubuntu-meta package in Ubuntu:
> Fix Released
> Status in xubuntu-meta package in Ubuntu:
> In Progress
>
> Bug description:
> just realize that fonts-noto-cjk is available in the repository, finally its packaged.
> i don't really know about korean community.
> But for Chinese and Japanese community, i think that the answer is clear.
> noto-cjk is definitely better what we had before, like fonts- wqy and fonts-droid.
> Android community had received these complains for years, finally they got them fixed on lollipop.
> Fedora also set it as default chinese font start from F21.
> and of course, i still hope that ubuntu could drop those 69-language-selector fontconfig files, just like what F13 did.
>
> To manage notifications about this bug go to:
> https...

On 2016-03-22 18:20, Aron Xu wrote:
> But this 64-language-selector-prefer.conf could lead to regression
> for Chinese, unfortunately.

Can you please be more specific? How? Do you see something which tomoe_musashi did not see?

If I understand correctly, when the user language is set to English, thus none of 69-language specific configurations are active, this configuration would take effect. Current configuration will make the ja variant take precedence over any of the Chinese ones, while the default of noto looks to be Chinese SC - which was exactly the cause of the issue for Japanese. Overriding it with ja as the first one will lead to similar issue to Chinese characters and I believe that's not what we want.

Yes, when the user language is e.g. English, the
69-language-selector-zh-*.conf files are not in effect.

But if I understand it correctly, the default (if that's the correct way to say it) of Noto is not Chinese, but Japanese. At least Japanese is listed first in a fc-match listing, when no fontconfig .conf files are in effect; please see:

https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1540063/comments/16

So possibly fonts-noto-cjk may result in proper rendering out of the box for any CJK language, as long as no fontconfig .conf file messes it up. That's why I found it worth a try to put those Japanese lines first in 64-language-selector-prefer.conf.

tomoe_musashi reported in comment #81 that if does not affect Chinese rendering. (The issue he mentioned - *Takao* used in Firefox for non-lang-specified content - appears to be unrelated.)

So, Aron, can you please test?

Sure, if the Japanese lines in 64-language-selector-prefer.conf affect Chinese rendering adversely, we should move them downwards (and with that make it worse for Japanese...). But there is no reason to prefer Chinese over Japanese just for the sake of it, is there?

It seems that Medium is easier for reading, will be the font be Regular or Medium?

Why is the font still so thin in Firefox? I have installed noto sans cjk on Ubuntu 14.04.4, here is my setting.

/etc/fonts/conf.d/69-language-selector-zh-tw.conf
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>

 <!-- Set fonts selection order for Chinese users -->
 <match target="pattern">
  <test qual="any" name="family">
   <string>serif</string>
  </test>
        <test name="lang">
            <string>zh-tw</string>
        </test>
  <edit name="family" mode="prepend" binding="strong">
   <string>Noto Sans CJK TC Medium</string>
   <string>Noto Sans CJK SC Medium</string>
          <string>Noto Sans CJK JP Medium</string>
          <string>Noto Sans CJK KR Medium</string>
   <string>AR PL UMing TW</string>
   <string>AR PL UMing HK</string>
   <string>AR PL New Sung</string>
   <string>HYSong</string>
   <string>WenQuanYi Bitmap Song</string>
   <string>AR PL UKai TW</string>
   <string>AR PL UKai HK</string>
   <string>AR PL ZenKai Uni</string>
   <string>DejaVu Serif</string>
   <string>Bitstream Vera Serif</string>
  </edit>
 </match>
 <match target="pattern">
  <test qual="any" name="family">
   <string>sans-serif</string>
  </test>
        <test name="lang">
            <string>zh-tw</string>
        </test>
  <edit name="family" mode="prepend" binding="strong">
                        <string>Noto Sans CJK TC Medium</string>
                        <string>Noto Sans CJK SC Medium</string>
                        <string>Noto Sans CJK JP Medium</string>
                        <string>Noto Sans CJK KR Medium</string>
   <string>Droid Sans Fallback</string>
   <string>WenQuanYi Zen Hei</string>
   <string>AR PL UMing TW</string>
   <string>AR PL UMing HK</string>
   <string>AR PL New Sung</string>
   <string>HYSong</string>
   <string>AR PL UKai TW</string>
   <string>AR PL UKai HK</string>
   <string>AR PL ZenKai Uni</string>
   <string>DejaVu Sans</string>
   <string>Bitstream Vera Sans</string>
  </edit>
 </match>
 <match target="pattern">
  <test qual="any" name="family">
   <string>monospace</string>
  </test>
        <test name="lang">
            <string>zh-tw</string>
        </test>
  <edit name="family" mode="prepend" binding="strong">
                        <string>Noto Sans CJK TC Medium</string>
                        <string>Noto Sans CJK SC Medium</string>
                        <string>Noto Sans CJK JP Medium</string>
                        <string>Noto Sans CJK KR Medium</string>
   <string>Droid Sans Fallback</string>
   <string>WenQuanYi Zen Hei Mono</string>
   <string>AR PL UMing TW</string>
   <string>AR PL UMing HK</string>
   <string>AR PL New Sung</string>
   <string>HYSong</string>
   <string>AR PL UKai TW</string>
   <string>AR PL UKai HK</string>
   <string>AR PL ZenKai Uni</string>
   <string>DejaVu Sans Mono</string>
   <string>Bitstream Vera Sans Mono</string>
  </edit>
 </match>
 <match>
         <test name="family" compare="contains">
                 <string>Noto Sans CJK</string>
         </test>
         <edit name="autohint" mode="assign">
                 <bool>false</bool>
         </edit>
         <edit name="hintstyle" mode="assign">
                 <const>hin...

On 2016-03-26 04:43, 賴家亨 wrote:
> I have installed noto sans cjk on Ubuntu 14.04.4, ...

If you want to help with the default configuration for 16.04, it would be better if you tried 16.04 via a daily build ISO.

From the latest 16.04 beta2 release, I tested the font display with firefox. Still according to the inspection tool, Noto Sans DemiLight is picked up first for TC and JP contents. (EN locale in live env.)

Thanks for that report, Yuan Chao. It indicates that bug #1556457 is more urgent than we (I) first thought.

May I ask: Is "Regular" picked up first with a ja_JP or zh_TW locale?

On 2016-03-28 21:56, Gunnar Hjalmarsson wrote:
> May I ask: Is "Regular" picked up first with a ja_JP or zh_TW
> locale?

Correction: with a zh_* locale.

(With a ja_JP locale, TakaoPGothic is still the default.)

> May I ask: Is "Regular" picked up first with a ja_JP or zh_TW
> locale?
Yes, (only tested with live env) according to fc-match "Regular" is picked up over "DemiLight" in both zh_TW and zh_CN. TakaoPGothic is the default for ja_JP.

However, for Firefox, this hack in /etc/fonts/conf.d/69-language-selector-zh-??.conf doesn't seem to work. According to the inspection tool of Firefox, "AR PL UMing CN/TW", "Noto Sans CJK SC/TC Bold" and "DemiLight" are picked up.

UMing is second to TakaoPGothic so could be a problem in ja_JP on serif? (UMing does not have enough glyph coverage in JP)

Thanks, Yuan Chao. Let me try so summarize then:

There is an issue with fontconfig, so it by default picks "Demilight" over "Regular" (bug #1556457). We have tried to work around this problem in 69-language-selector-zh-??.conf and
64-language-selector-prefer.conf (the latter for e.g. an English locale). The workaround is not effective in Firefox, though.

I'm out of ideas. Is there anything we can do but waiting for bug #1556457 to be fixed? (Assuming that will address the Firefox issue...)

On 2016-03-30 12:30, Yuan Chao wrote:
> UMing is second to TakaoPGothic so could be a problem in ja_JP on
> serif? (UMing does not have enough glyph coverage in JP)

Possibly, but if so it wouldn't be a new problem. With the Japanese locale, and with full Japanese language support installed, TakaoPMincho is there for serif.

As a test I built fontconfig 2.11.94 in a PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/fontconfig-test2

I don't know yet if it would be possible to have it accepted for the 16.04 archive. What I know is that it would fix 'the Demilight issue' without the current workaround in the language-selector .conf files.

Would appreciate if some of you could test it. I'm especially interested in knowing if this fontconfig version also fixes the undesirable Firefox behaviour wrt font weights.

@Gunnar, i can confirm that the ppa fixes the "DemiLight" issue.
it also fixes some cases that rendering with undesirable font weights.

On 2016-04-03 06:02, tomoe_musashi wrote:
> @Gunnar, i can confirm that the ppa fixes the "DemiLight" issue. it
> also fixes some cases that rendering with undesirable font weights.

Great, thanks! Then I'll try to get it in.

Fixed in lubuntu-meta 0.64.

Hi again,

We met some hesitation in upgrading to fontconfig 2.11.94 this late in the cycle, so together with Mingye Wang I have also built 2.11.1 in a PPA, where the necessary upstream commits have been patched:

https://launchpad.net/~gunnarhj/+archive/ubuntu/fontconfig-test

So, as a base for decision, I'd appreciate very much if you could install and test version 2.11.1-0ubuntu9~ppa. Does it fix the Chinese rendering issues as well as 2.11.94 seems to do?

Update: fontconfig 2.11.1-0ubuntu9 is building, and will soon be in the Ubuntu archive. So if you haven't already, there is no reason now to bother with the PPA I mentioned in comment #107. Better test the real thing instead. :)

This bug was fixed in the package language-selector - 0.164

---------------
language-selector (0.164) xenial; urgency=medium

  * fontconfig/69-language-selector-zh-*.conf:
  * fontconfig/64-language-selector-prefer.conf:
    Dropped workaround to make "Regular" the default font weight for
    Noto Sans CJK; fixed in fontconfig 2.11.1-0ubuntu9 (LP: #1468027).
  * debian/control:
    Bump Standards-Version to 3.9.7.

 -- Gunnar Hjalmarsson <email address hidden> Tue, 05 Apr 2016 10:35:00 +0200

Since fontconfig 2.11.94 is on its way into the archive, I'd like to mention this "call for testing" message I posted to the ubuntu-devel mailing list:

https://lists.ubuntu.com/archives/ubuntu-devel/2016-April/039303.html

This bug was fixed in the package xubuntu-meta - 2.205

---------------
xubuntu-meta (2.205) xenial; urgency=medium

  * Refreshed dependencies
  * Added fonts-noto-cjk to desktop-recommends (LP: #1468027)

 -- Sean Davis <email address hidden> Wed, 06 Apr 2016 21:59:33 -0400

I tried this on Firefox and it seems to work fine.

But, when I tried it on Chromium, it still doesn't work.

Thanks, Shih-Yuan. Thin? Why Thin?

Same for me with Google Chrome. Don't Chrome and Chromium care about fontconfig, but do it their own way?

Any ideas how this issue could be addressed?

Not sure. Some fontconfig tracing env vars might be useful for the Skia LegacyCreateTypeface part I guess...

* * *

In addition to the weight matching problem, the screenshot for Chromium in #113 gives the JP variant instead of TW. Perhaps I should try to get someone to reproduce this in other distros and file a report upstream... Not sure about what the title will be.

> Don't Chrome and Chromium care about fontconfig, but do it their own way?

Chrome seems to provide some level of per-script font handling[1]. Actually Google Chrome gives a link to an extension called ‘Advanced Font Settings’ (caclkomlalccbpcdllchkeecicepbmbm) in its font menu...

  [1]: https://developer.chrome.com/extensions/fontSettings

Regarding the JP problem, it seems that Google Chrome is trying to do the script recognition itself. This happens on my Windows 10 machine too -- Chinese comments on GitHub and Tweets are all rendered using ‘Inziu Roboto JP’, my default Japanese script font set with ‘Advanced Font Settings’. Webpages with explicit language/script declarations like https://zh.wikipedia.org/zh-tw/ work fine since they don't need any recognition by Chrome (will anyone confirm this on Ubuntu?)

It seems that some Chrome font settings can be pre-hacked by editing webkit.webprefs.fonts. The 'Zyyy' (default script) one found in chrome settings is stored in the file 'Preferences' -- perhaps some settings for other scripts can be filled in here too?

On 2016-04-07 21:24, Mingye Wang wrote:
> In addition to the weight matching problem, the screenshot for
> Chromium in #113 gives the JP variant instead of TW.

I didn't even notice "JP" first. This made me fear that
/etc/fonts/conf.avail/64-language-selector-prefer.conf has something to do with it. But the issue with too thin characters in Chrome/Chromium is there also with a TC locale:

$ locale | grep LC_CTYPE
LC_CTYPE=zh_TW.UTF-8
$ fc-match
NotoSansCJK.ttc: "Noto Sans CJK TC" "Regular"
$

So it's probably unrelated to that.

Isn't this issue really weird? Shouldn't Google make sure that their own web browser handles their own fonts properly?

We have a solution to the Chrome/Chromium and "Thin" issue in sight at bug #1575555. A modified version of fonts-noto-cjk is available in this PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/fonts-noto-cjk

It would be great if a few Chinese and Japanese users could install fonts-noto-cjk (and possibly fonts-noto-cjk-extras) from the PPA and let us know the result. Please report your observations on the other bug.

Using subset OTFs is suggested for OS exept MacOS and Windows by the README file from Source Han Sans (same as Noto Sans CJK).

However, as I stated previously, I believe using Super OTC is a way to find where the apps in Linux world which has some problems dealing with this "super font," and that helps shape a better Linux world in the future. People find the problem, report the problem, and the problem be fixed one day.

Both ways are fine though. The system still has some places to be improved to support Super OTC better while Subset OTFs require more space and more packages. To be the first to find the problems and work with upstreams to fix them? Or just do as the official recommendation to get the better result and serve users better first?

Any thoughts on this?

On 2016-04-30 06:40, Cheng-Chia Tseng wrote:
> Using subset OTFs is suggested for OS exept MacOS and Windows by the
> README file from Source Han Sans (same as Noto Sans CJK).

According to the README.formats file, which is included in the fonts-noto-cjk source package, OTC "works" on recent versions of Mac and Linux but not Windows. So they don't seem to be identical.

> However, as I stated previously, I believe using Super OTC is a way
> to find where the apps in Linux world which has some problems dealing
> with this "super font," and that helps shape a better Linux world in
> the future. People find the problem, report the problem, and the
> problem be fixed one day.

Hmm.. I see now that you anticipated problems with "Super OTC" [1] long ago. At that time I hadn't a clue what you were talking about. :(

What you say is correct, but we shouldn't use the whole community of Ubuntu users as a test panel, should we?

> To be the first to find the problems and work with upstreams to fix
> them? Or just do as the official recommendation to get the better
> result and serve users better first?

Why not do both? I filed this ticket:
https://github.com/googlei18n/noto-cjk/issues/65

Hopefully it's the right place.

Can you possibly confirm that the PPA version of fonts-noto-cjk addresses the issues we have found up to now (at the expense of more disk space)?

[1] Super OTC is the font format installed with the version of
    fonts-noto-cjk which is currently in the Ubuntu archive.

@Gunnar, i can confirm that the PPA package addresses the Google Chrome issue.

Thanks, tomoe_musashi. Did you try both without and with fonts-noto-cjk-extras?

@Gunnar, i tried both without and with fonts-noto-cjk-extras installed, and the patch addresses the chromium issue.

Here is the screenshot within virtualbox, without extras.

This is the screenshot within virtualbox, with extras.

They appeared to be the same, and the weight of the CJK characters is not the thin ones.

Thanks, tomoe_musashi and Cheng-Chia Tseng. It confirms the conclusion from my rudimentary tests, and strengthens my belief that the proposed solution is a reasonable step to take.

Sorry to bother you again, but...

I tested to replace the single "super" OTC file with 7 weight specific OTC files, and it seems like this is sufficient to fix "the Thin issue" in Chrome/Chromium. So I have uploaded a simpler proposal to the PPA, without the additional fonts-noto-cjk-extras package.

I would of course appreciate if some could test it, to make sure I didn't overlook something.

https://launchpad.net/~gunnarhj/+archive/ubuntu/fonts-noto-cjk

(Please remember to uninstall the now obsolete, and possibly conflicting, fonts-noto-cjk-extras.)

I just tested the latest PPA and it fix the "Thin issue" in Chrome/Chromium, in a clean live mode of a virtual machine.
It's great that the package install all 7 weights with similar required disk space as the Super OTC one.

This is what it looks like in virtualbox. It seems no problem with chrome when using the latest package as well.

Thanks again, tomoe_musashi and Cheng-Chia Tseng!

On 2016-05-03 18:49, tomoe_musashi wrote:
> It's great that the package install all 7 weights with similar
> required disk space as the Super OTC one.

Yeah, indeed. And it indicates that the bug resides in the "super" OTC file.

On 2016-05-04, Gunnar Hjalmarsson (gunnarhj) wrote:
> Yeah, indeed. And it indicates that the bug resides in the "super" OTC file.
I thought the problem is on google-chrome as Firefox and other applications handles super OTC well?

On 2016-05-08 11:23, Yuan Chao wrote:
> On 2016-05-04, Gunnar Hjalmarsson (gunnarhj) wrote:
>> Yeah, indeed. And it indicates that the bug resides in the "super"
>> OTC file.
>
> I thought the problem is on google-chrome as Firefox and other
> applications handles super OTC well?

What we found out is that Google's "super" OTC, unlike the other packaging formats which provide the very same Noto Sans CJK glyphs, doesn't work as expected with Google's web browser (or Chromium). So yeah, alternatively it may be Chrome/Chromium which are buggy.

Sorry to bother you again but there is a new update for the Noto CJK fonts.
Google and Adobe released the Serif fonts for CJK about several weeks ago
Noto Serif CJK:
http://www.google.com/get/noto/help/cjk/
It is a better serif font in consistency and glyph standard.
Ubuntu should seed the new Noto Serif CJK over the current

@tomoe_musashi: Can you please file a new bug about Noto Serif CJK instead of using this one which was closed long ago.

There is some related packaging discussion at Debian:

https://bugs.debian.org/862276

http://lists.alioth.debian.org/pipermail/pkg-fonts-devel/2017-May/thread.html#19499

I think it's wise to wait until we see the result of that discussion before we change anything in Ubuntu.