fc-match Ryumin should return Mincho(serif) font in ja locale

Example PDF document.

The PDF document above is a template of tax declaration paper from Japanese National Tax Agency. It specifies both Ryumin(serif) and Gothic-BBB(sans-serif), but Evince uses one sans-serif font "TakaoPGothic" to substitute both.

hm, it happens only with Japanese locale.

$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "TakaoPMincho" "Regular"

$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

ok, it turned out that 65-fonts-takao-pgothic.conf is too strong. We need to figure out good balance between 65-fonts-takao-pgothic.conf and 65-fonts-takao-mincho.conf to return the correct substituted font for Ryumin in ja locale.

lp:ubuntu/fonts-takao seems out-of-date unfortunately. I'm attaching debdiff instead.

I don't think it causes any side effects for other locale. Because all binding="strong" are guarded by ja locale or ja specific font. But to be safe, adding CJK testers as subscriber to notify this change beforehand.

Status changed to 'Confirmed' because the bug affects multiple users.

Nobuto, thank you very much for your work on making Ubuntu support Japanese fonts properly. I can always learn so much in this respect from you and share your passion for visual aesthetics here.

As you may know, I usually remain with the LTS releases and only run the development release from time to time. As such, I hope the fix for this ticket will be backported to trusty as well. Furthermore, my system is mainly English-based. I hope you won't mind me adding my own experiences here.

I can basically confirm your findings, except that my US-locale returns a different font.

$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
ipamp.ttf: "IPAPMincho" "Regular"
$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

I tried the test PDF in different viewers and noticed there were some alignment issues in the viewer built into Firefox (please refer to my screenshot). I suspect that is another, separate issue?

One of the things I sometimes notice is that Chinese at times seems to have precedence over Japanese. For someone like me, that is unfortunate. Are you aware of a list of known issues or some website to check if one's configuration has any errors? This area of Ubuntu seems to change from release to release so it is hard to keep up (with a lot of improvement being made, at least that has been my experience). I do not know as much as you do, so at best, I will notice the effects but I would be utterly unable to dig into the causes. That's why your work is so much appreciated.

You titled the ticket " fc-match Ryumin should return Mincho(serif) font in ja locale". That name is so obviously Japanese that I wonder why the returned value should be locale-specific. From the superficial information I have it seems that optimizations for Japanese are sometimes only made only for Japanese locale, leaving people like me who only display and write Japanese in a non-Japanese locale out in the rain. I understand that sometimes this might have to do with political sensitivities and the power of numbers.

After applying your patches I can confirm that fc-match now returns a Mincho font.

$ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "Takao P明朝" "Regular"
$ env LC_ALL=en_US.UTF-8 fc-match Ryumin
fonts-japanese-mincho.ttf: "TakaoPMincho" "Regular"

I cannot see any visual change in the test file in evince, though. I will have to admit I would not be good at spotting any difference, though. What words should now display in the more brush-like Mincho font?

@Rolf,

> As you may know, I usually remain with the LTS releases and only run the development release from time to time. As such, I hope the fix for this ticket will be backported to trusty as well.

If this fix lands into vivid successfully and no regression reported in vivid, we can consider to backport it to trusty as well. However I'm under the impression that this Ryumin thing is not so critical.

> Furthermore, my system is mainly English-based. I hope you won't mind me adding my own experiences here.
>
> I can basically confirm your findings, except that my US-locale returns a different font.
>
> $ env LC_ALL=en_US.UTF-8 fc-match Ryumin
> ipamp.ttf: "IPAPMincho" "Regular"
> $ env LC_ALL=ja_JP.UTF-8 fc-match Ryumin
> fonts-japanese-gothic.ttf: "Takao Pゴシック" "Regular"

My patch fixes this behavior, the result for Ryumin will be always the same in all locales.

> I tried the test PDF in different viewers and noticed there were some alignment issues in the viewer built into Firefox (please refer to my screenshot). I suspect that is another, separate issue?

Yes, that's separate issue in Mozilla's pdf.js I think. The PDF I attached is rather complicated, I think it's natural rendering is broken in some viewers.

> One of the things I sometimes notice is that Chinese at times seems to have precedence over Japanese. For someone like me, that is unfortunate. Are you aware of a list of known issues or some website to check if one's configuration has any errors? This area of Ubuntu seems to change from release to release so it is hard to keep up (with a lot of improvement being made, at least that has been my experience). I do not know as much as you do, so at best, I will notice the effects but I would be utterly unable to dig into the causes. That's why your work is so much appreciated.

> You titled the ticket " fc-match Ryumin should return Mincho(serif) font in ja locale". That name is so obviously Japanese that I wonder why the returned value should be locale-specific. From the superficial information I have it seems that optimizations for Japanese are sometimes only made only for Japanese locale, leaving people like me who only display and write Japanese in a non-Japanese locale out in the rain. I understand that sometimes this might have to do with political sensitivities and the power of numbers.

Nowadays Ubuntu switches font substitutions by locales. For example, `fc-match serif` returns "DejaVu Serif" for most locales, "TakaoPMincho" for ja locale. You can specify per-user font locale settings(actually it's LC_CTYPE) in ~/.pam_environment or ~/.profile.
https://help.ubuntu.com/community/EnvironmentVariables#Session-wide_environment_variables

Can you please put "LC_CTYPE=ja_JP.UTF-8" in your ~/.pam_environment, re-login and see how it works?

> I cannot see any visual change in the test file in evince, though. I will have to admit I would not be good at spotting any difference, though. What words should now display in the more brush-like Mincho font?

Right. I thought Ryumin was used in the doc more often, but it wasn't. We need to prepare more proper sample doc if we try SRU ;)

> We need to prepare more proper sample doc if we try SRU ;)

Attached more simple sample.

Thanks, I've sponsored the change to vivid, let's see what the feedback is there

This bug was fixed in the package fonts-takao - 003.02.01-9ubuntu3

---------------
fonts-takao (003.02.01-9ubuntu3) vivid; urgency=medium

  * 65-fonts-takao-gothic.conf, 65-fonts-takao-pgothic.conf,
    65-fonts-takao-mincho.conf:
    - Use generic font names instead of localized names for readability
    - Add 'binding="strong"' inside all sections guarded by locale for
      consistency
    - Add 'binding="strong"' for ja specific fonts "GothicBBB" and
      "Ryumin" to substitute those fonts reliably (LP: #1430339)
 -- Nobuto Murata <email address hidden> Wed, 11 Mar 2015 01:45:37 +0900

Just for later reference, attached display sample.

I am on bionic now and FWIW, the sample supplied by Shibata-San still looks more like his "before fixed" screenshot, at least in the browser (I am using Vivaldi). Is this really fixed? Or is this just a browser-specific bug?