please change wenquanyi micro hei back with 69-language-selector-zh-tw.conf

The attachment "this is the patch to make WenQuanYi Micro Hei back to zh-tw configuration" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Must to change it!
Set "WenQuanYi Micro Hei" to be instead of "WenQuanYi Zen Hei" in all place.

@Laney: Would appreciate your advice on how to deal with this.

I'm disturbed that "most" Chinese users are unhappy despite my extensive attempts to consult in order to get the configuration right for raring.

I think we should figure out how to make this stick in ttf-wqy-microhei's fontconfig files instead of in l-s. This is supposed to happen already. Perhaps adding binding=strong?

Well, in Chinese reader's eyes, it is much more readible and pretty for WQY Micro Hei than WQY Zen Hei.

The strokes of each character provided by WQY Zen Hei are firstly compozed by programs, then those strokes later adjusted by manual as it is not that readible. WQY Micro Hei is derived from Google Droid Sans Fallback, and being added some character it did not provide.

Although WQY Zen Hei does has higher coverage on Han characters, WQY Micro Hei is still prefered as the first high quility freefont for Chinese font so far. I don't know why we now palce WQY Zen Hei or Droid Sans Fallback as the first font with those zh-*.conf. What is the main reason? I would suggest using Micro Hei instead of Zen Hei in this case.

I agree with clean up those fontconfig setting in language support. Of course, we can assign or bind Micro Hei / Micro Hei Mono to existing Sans or Monospace with is matched by fontconfig settings. It is fine with this way as you think that this is the right and best way.

Current Chinese font in 13.04 really hurt my eyes, WQY Micro Hei should really have higher precedence.

@Laney: I just spent some time checking what is changed with those fontconfig files.

I found that Micro Hei related config listed in 69-language-selector-zh-tw.conf before raring is merged into 65-ttf-wqy-microhei.conf. Those binding things listed at the last part of the conf are outdated and should be removed. The binding setting was to mimic the look and feel provided by previous Ubuntu versions which use DejaVu Sans as default font. (Yeah, I wrote those lines) However, we use "ubuntu" font now. That configuration is not needed anymore.

Here is the overview of my analysis about the font configuration now.

1. the setting in 65-ttf-wqy-microhei.conf does not strongly affect the Chinese characters provided by those pseudo font family such as serif, sans-serif and monospace.

Here is the priority of the conf affecting the Chinese characters provided by sans-serif,
*I rm 69-language-selector-zh-tw.conf and found that sans-serif was then WenQuanYi Zen Hei (mainly decided by 64-wqy-zenhei.conf and partially 65-nonlatin.conf).
*I rm 64-wqy-zenhei.conf and found that sans-serif was later AR PL UKai (mainly decided by 65-fonts-arphic-ukai.conf).
*I rm 65-fonts-arphic-ukai.conf and then it would be AR PL UMing (mainly decided by 65-fonts-arphic-uming.conf).
*I rm 65-fonts-arhpic-uming.conf and it later became WenQuanYi Zen Hei (mainly decided by 65-nonlatin.conf).
*I rm 65-nanlatin.conf and it became WenQuanYi MicroHei finally.

Bottom line is that it is not that easy if we want to make things work in 65-ttf-wqy-microhei.conf. Maybe we have to figure out a better setting or change the priority.

2. 69-language-selector-zh-tw.conf has higher priority in this case about the Chinese characters provided by sans-serif than those conf with smaller prefix number.

Improvement proposal

0. it is better to clean up those unnecessary lines in 65-ttf-wqy-microhei.conf
1a. change 69-language-selector-zh-tw.conf directly to use WenQuanYi Micro Hei instead of WenQuanYi Zen Hei.
1b. make 65-ttf-wqy-microhei.conf setting affect stronger. Change the settings or tune the prefix number or somewhat.

I believe that the easiest way to fix this issue is 1a.

Well, I just don't figure out why you have to move to WenQuanYi Zen Hei for Chinese characters. Moreover, you seem to have no convincing reason about the idea behind this decision making. The visual appearance provided by WenQuanYi Micro Hei is absolutely the best choice from the eyes of Chinese readers.

This is my point of view.
Hope that we will have better configuration for Chinese users in the futer.

I think that my title is not well described and does not fit my idea well, so I change that.

Maybe that mislead readers to think that I was asking to add those outdated settings about Micro Hei back in language-selector confs, but that's not. To be clear, I just want to make Micro Hei back as the default Chinese character displaying font for sans-serif and monospace as it was.

Another way to improve the unreadible Chinese characters is to install Droid Sans Fallback and use it instead of WenQuanYi Zen Hei.

The reason to use Droid Sans Fallback instead of WenQaun Yi Micro Hei is that WenQuanYi almostly stopped the development progress for about three years while Droid Sans Fallback does not. Plus, WenQuanYi Micro Hei is at first derived from Droid Sans. Now Droid Sans Fallback covers much more characters than WenQuanYi Micro Hei. Moreover, the quality provided by Droid Sans is out of professional font designers, while WenQuanYi is mostly from online public invovement.

So in this way, the improvement prososal is

1. make Droid Sans Fallback as the first Chinese character providing font listed in all 69-language-selector-zh-xx.conf. (Yeah, I have checked that only part of the confs list Droid Sans Fallback at first)
2. make Droid Sans Fallback font a dependent package for language-support of all Chinese locales. (the font is not installed by default in raring in all ubuntu derives)

Please give some feedbacks, and I belive we, all Traditional Chinese users will be appreciate that.

Thanks for your efforts on this matter, Cheng-Chia Tseng.

I have subscribed Ding Zhou to this bug report, since he was consulted by Iain Lane about the Raring changes.

On 2013-05-26 11:34, Cheng-Chia Tseng wrote:
> 1. make Droid Sans Fallback as the first Chinese character providing
> font listed in all 69-language-selector-zh-xx.conf. (Yeah, I have
> checked that only part of the confs list Droid Sans Fallback at
> first)

Your primary concern seems to be traditional Chinese, and in -zh-tw.conf Droid Sans Fallback is already listed first. So if I understand it correctly, to deal with traditional Chinese we would just need to move Droid Sans Fallback to the top in -zh-hk.conf and add the line

fn:zh-hant::fonts-droid

to pkg_depends.

Would it possibly make sense to make those changes for traditional Chinese, while keeping WenQuanYi Zen Hei for simplified Chinese?

I will check that if the -zh-tw.conf provided in raring (which put Droid Sans Fallback into first one) works fine or not if we install Droid Sans Fallback recently.

>Would it possibly make sense to make those changes for traditional Chinese, while keeping WenQuanYi Zen Hei for simplified Chinese?

I think that it is OK to keep WenQuanYi Zen Hei for simplified Chinese as long as no user complains about this change.

Looking forward to the result of that test.

After my testing, the good news is that insatalling fonts-droid will provide better visual appearence and more readible than WQY zen-hei in Chinese charaters.

However, 69-language-selector-zh-tw.conf has a weird UNACCEPTABLE side effect. It matches DejaVu Serif for "sans-serif" and "monospace" first.

The result of 'fc-match -s serif' returns
uming.ttc: "AR PL UMing TW" "Light"
DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
uming.ttc: "AR PL UMing HK" "Light"
Kinnari.ttf: "Kinnari" "Medium"
wqy-zenhei.ttc: "文泉驛正黑" "Regular"
[...]
We can see that AR PL UMing TW is the first font for serif. It is not that good because we would like to see DejaVu Serif to present Western text more than AP PL UMing. The Western text gylphs come from AR PL UMing are not as good as those gylphs Western fonts provide. However, the result is acceptable.

The result of 'fc-match -s sans-serif' returns
DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
wqy-zenhei.ttc: "文泉驛正黑" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
[...]
This is completly unacceptable because DejaVu Serif and DejaVu Serif Bold are not sans-serif style font.

The result of 'fc-match -s monospace' returns
DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"
DejaVuSansMono-Bold.ttf: "DejaVu Sans Mono" "Bold"
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
wqy-zenhei.ttc: "文泉驛等寬正黑" "Regular"
uming.ttc: "AR PL UMing TW" "Light"
uming.ttc: "AR PL UMing HK" "Light"
n022003l.pfb: "Nimbus Mono L" "Regular"
[...]
This is completly unacceptable because DejaVu Serif and DejaVu Serif Bold are not monospace style font.

After rm 69-language-selector-zh-tw.conf,

The result of 'fc-match -s serif' returns
uming.ttc: "AR PL UMing TW" "Light"
DejaVuSerif.ttf: "DejaVu Serif" "Book"
DejaVuSerif-Bold.ttf: "DejaVu Serif" "Bold"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
uming.ttc: "AR PL UMing HK" "Light"
Kinnari.ttf: "Kinnari" "Medium"
wqy-zenhei.ttc: "文泉驛正黑" "Regular"
[...]
The reslt is the same with haveing 69-language-selector-zh-tw.conf, so it implies that 69-lanaguage-selctor-zh-tw.conf does not paly a big role in this "serif" case.

The result of 'fc-match -s sans-serif' returns
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
wqy-zenhei.ttc: "文泉驛正黑" "Regular"
n019003l.pfb: "Nimbus Sans L" "Regular"
Waree.ttf: "Waree" "Book"
ukai.ttc: "AR PL UKai CN" "Book"
ukai.ttc: "AR PL UKai HK" "Book"
uming.ttc: "AR PL UMing CN" "Light"
[...]
DejaVu Sans is sans-serif style font. The result is acceptable.

The result of 'fc-match -s monospace' returns
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"
DejaVuSansMono-Bold.ttf: "DejaVu Sans Mono" "Bold"
wqy-zenhei.ttc: "文泉驛等寬正黑" "Regular"
n022003l.pfb: "Nimbus Mono L" "Regular"
[...]
DejaVu Sans Mono is monosp...

Hi, could anyone who could fix or improve the configurations review my proposal?

Thanks for the tests and analyses!

I have prepared a language-selector branch and a ttf-wqy-microhei branch with the changes you propose. Please check them out carefully to help prevent misunderstandings.

There is a but, though. ttf-wqy-microhei is already in the main pocket of the Ubuntu archive, while fonts-droid is currently in universe. Before committing the proposed language-selector changes, fonts-droid needs to get included in main through a special process: https://wiki.ubuntu.com/MainInclusionProcess

This is doable, of course, but before we start a MIR application, I'd like to ask if basically the same appearance can be achieved by using wqy-microhei instead, i.e. as you initially suggested. If it can, the wqy-microhei path seems to be preferable, after all.

I think that the changes you made are right.

I STRONGLY recommend fonts-droid instead of wqy-microhei because WenQuanYi project is not having any major progress or new releases in last three years. Plus, wqy-microhei fail to display Korean characters. It is not solved for some years since the day the bug was reported.

If we stick to wqy-microhei, I think that we still have to move to fonts-droid one day in the future for the reasons above.

Gunnar Hjalmarsson (gunnarhj) wrote on 2013-05-29:
>Would it possibly make sense to make those changes for traditional Chinese, while keeping WenQuanYi Zen Hei for simplified Chinese?

I think it is much more readible and pretty for WQY Micro Hei than WQY Zen Hei in simplified Chinese reader's eyes too.

So, Please change WenQuanYi Micro Hei back. or move to fonts-droid.

Could we start the MIP for fonts-droid these days to finish this issue?

The fonts-droid binary is now included in the fonts-android source package, which is already in main, so no MIR application is needed.

I have uploaded changes to language-selector, which make fonts-droid the preferred font for both traditional and simplified Chinese. We are early in the 14.04 development cycle, so there is time to make further changes if proved necessary.

This bug was fixed in the package language-selector - 0.118

---------------
language-selector (0.118) trusty; urgency=low

  * data/pkg_depends:
    ttf-wqy-zenhei replaced with fonts-droid for Chinese (LP: #1173571).
  * fontconfig/69-language-selector-zh-hk.conf, -zh-cn.conf, -zh-sg.conf:
    Droid Sans Fallback moved to top.
  * fontconfig/69-language-selector-zh-tw.conf:
    Outdated DejaVu bindings removed (LP: #1173571).
 -- Gunnar Hjalmarsson <email address hidden> Wed, 06 Nov 2013 14:58:00 +0100

ttf-wqy-microhei is seeded too. Should this be changed?

ttf-wqy-microhei (from ttf-wqy-microhei) is seeded in:
  edubuntu: dvd
  kubuntu-active: daily-live
  kubuntu: daily-live
  lubuntu: daily, daily-live, daily-preinstalled
  ubuntu-gnome: daily-live
  ubuntu: daily-live
  ubuntukylin: daily-live
  ubuntustudio: dvd
  xubuntu: daily-live

fonts-android is not in main.

laney@raleigh> rmadison -S -s trusty fonts-android
fonts-android | 1:4.3-1 | trusty/universe | source
fonts-droid | 1:4.3-1 | trusty/universe | all
fonts-roboto | 1:4.3-1 | trusty/universe | all

This bug was fixed in the package ttf-wqy-microhei - 0.2.0-beta-1.1ubuntu4

---------------
ttf-wqy-microhei (0.2.0-beta-1.1ubuntu4) trusty; urgency=low

  * debian/65-ttf-wqy-microhei.conf:
    Outdated DejaVu bindings removed (LP: #1173571).
 -- Gunnar Hjalmarsson <email address hidden> Wed, 06 Nov 2013 03:23:00 +0100

Right, Iain, I was wrong when I said in comment #20 that fonts-android is already in main, and have now filed a MIR application as bug #1249132.

On 2013-11-07 12:12, Iain Lane wrote:
> ttf-wqy-microhei is seeded too. Should this be changed?

Seeding fonts-droid instead of ttf-wqy-microhei is reasonably a logical consequential step, and seems to be needed to complete the MIR process... However, I'd suggest that we wait a couple of months to see if Droid Sans Fallback proves to work smoothly and gets broadly accepted by Chinese users.

fonts-droid will now be seeded instead of ttf-wqy-microhei in the Ubuntu trusty ISOs.
http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/ubuntu.trusty/revision/2189

Keeping the Ubuntu Seeds task open, since we should make this change for other distros if it proves to be well received in Ubuntu.

The fonts that follow Taiwan standard are fonts-arphic-uming, fonts-arphic-ukai, fonts-moe-standard-kai, fonts-moe-standard-song fonts-cns11643-kai and fonts-cns11643-sung.
I think we should avoid using other fonts in 69-language-selector-zh-tw.conf.

For more information, please see the following links. (traditional Chinese)
http://blog.justfont.com/?p=5305
http://www.arphic.com/tw/news/epaper/agent/201108/font_technology.htm

But Droid Sans can give way better screen experience than the alternatives you pointed above. And I believe it's more reasonable to default to those fonts in printing materials, i.e. in libreoffice.

@fourdollars Actually, CJKUnifonts (AR PL UMing and AR PL UKai) do not follow Taiwan standard too. That's because they both were composed of Traditional style and Simplified style fonts before.

AR PL UMing is compoesed of 文鼎細上海宋繁 and 文鼎報宋簡 while AR PL UKai is composed of 文鼎中楷繁 and 文鼎中楷簡.

For example, 角 from AR PL UMing TW is using Simplified style. That's the proof.

fonts-moe-standard and fonts-cns11643 fonts are not maintained by open source community and have no promising future to improve by open source community.

@happyaron Yes, I agree with you.

@zerng07: I think CJKUnifonts does want to follow Taiwan's standard (CNS11643), China's standard (GB18030), and other countries' standard also.
However this project seema to stop the development since 2010 (http://www.freedesktop.org/wiki/Software/CJKUnifonts/).
CJKUnifonts is a truely open source project and it also has many bugs needed to fix.
fonts-moe-standard and fonts-cns11643 are the fonts released by Taiwan government, and they are licensed by CC-BY-ND (Non-Free).
The Debian package maintainers are both Taiwanese. I am the maintainer of fonts-cns11643.
As you said some glyphs of AR PL UMing/UKai don't follow Taiwan standard. I agree with you.
But I think they are bugs and we should help to fix them as a FLOSS member.
I also agree that WenQuanYi Micro Hei looks better than AR PL UMing/UKai.
But remember, WenQuanYi Micro Hei follows GB18030 and it is not compliant to CNS11643.
I think we should use AR PL UMing/UKai as the default fonts for Taiwan users, and help the development of CJKUnifonts to follow Taiwan standard.
Or we can help the development of WenQuanYi to include the support of CNS11643.
However at this point I think AR PL UMing/UKai is better than WenQuanYi Micro Hei to follow Taiwan standard.

BTW, there are other fonts: fonts-cwtex-* following Taiwan standard and licensed by GPL-2+.

@fourdolars:
I agree with the part helping the development of CJKUnifonts. However, it is no sense to use AR PL UMing for all styles as AR PL UMing is a serif style font and can only be used in serif part. BTW, the configuration is using AR PL UMing now.

So the issue here is which font we should use for sans-serif and monospace part for Chinese. Monospace part is no difference for Chinese to use sans-serif or monospace here because we fallback from DejaVu Sans Mono and Chinese characters are monospaced already.

It is always to use open font first that we can improve them by the power of the community. The existing sans-serif style open Chinese fonts are Droid Sans Fallback, WenquanYi Microhei and Zenhei, cwTeX Yuan and Hei Bold, and cwTeX Q Yuan and Hei. The reason why I suggested Droid was written above as Wenquanyi project has stopped development for many years while Droid is still being maintained and releasing new versions. cwTeX fonts are not covering as many characters as droid does, we still have to fallback to Droid to get better Chinese support, but cwTeX and Droid does not share the same font height and width which is not a good result though.

Using open fonts makes us have the right to improve the fonts. Droid Sans Fallback has good coverage and uses Apache license, so it should reduce the work for the community to improve in contrast to cwTeX fonts and it's derived fonts.

However, we still have other improvement way for other sans-serif style Chinese font. One is to fork Wenquanyi Microhei which some of my friends are planning to do, and the other is to expand the development of cwTeX or cwTeX Q fonts projects.

If you don't want to see Droid Sans Fallback which follows GB standard, we can found a project or create a subproject under CJKUnifonts to modify it. It might be a quicker way to have more efficiency.

If you cannot accept Droid Sans Fallback however, we can think about choosing cwTeX Q Hei instead and help to improve the font in community way. It will takes more time and work to have the good quality and high quantity as droid font does though.

For the information about typography issue to match Chinese font with English font, please see:
1. http://blog.justfont.com/2013/12/popular-typography/
2. http://breezymove.blogspot.tw/2013/11/cantarell.html This post uses Cantarell as example, but share the same idea with this issue.

Just FYI, one of the reasons why WQY Microhei is not maintained is that, at some time a Droid Sans Fallback update released by Google contains many more characters than in Microhei, while the project's initial goal was to expand the original Droid Sans to get better coverage.

It's currently impossible to get the best appearance for character if no hand-crafted hinting is involved for each type face, even WQY project had that pushed forward a large step while making Microhei, the project leader still said that quality of the font will never match those fonts made by commercial companies because of the limited availability of volunteer manpower.

I suggest to use the fonts as the following order.
Ubuntu, AR PL, cwTeX, MOE, and then CNS11643.
Pro: Better looking in English and better following of Taiwan standard.
Con: The programs that don't have font-config suppprt will be not able to see Chinese characters.
If we find the programs don't support font-config, we go to provide patcheset to upstream one by one.

Your suggestion will create another issue on typography that ubuntu font is sans-serif style while AR PL UMing and CNS11643 are serif style. Plus, AR PL UMing covers more characters than cwTeX and MOE, there is low probability fallbacking to them.

And we have already been using AR PL UMing for serif style in the configuration.

How about to find another English font to replace Ubuntu font in serif style?

I don't actually get what you mean here. But I guess that you are talking about it is better to use Western serif font listed before AR PL UMing for serif font family.

69-language-selector-zh-tw.conf does not affect the result about serif font family strongly. It is always using AR PL UMing first before DejaVu Serif no matter how you change 69-language-selector-zh-tw.conf. I doubt there are some other configurations in combination may have assign AR PL UMing higher priority. And if you want Western font listed before AR PL UMing we have to investigate further to solve the issue.

You can see comment #14 for my previous test on changing zh-tw.conf.

Yes, it is better to use Western serif font listed before AR PL UMing for serif font family if we want to have a better looking for Western characters.
In short, my point is to use those fonts in better compliance with Taiwan's standard instead of those are not.
I am not willing to have better Chinese characters but sacrifice the compliance of Taiwan's standard.
Of course, that is only my personal opinion. I can not represent all Taiwan users.
BTW, I prefer free fonts than non-free. Free means we have chances to fix problems.

OK, I will try to figure out which configuration play a big role to list AR PL UMing as the first one in serif style when I am free, and then file another bug report for that issue. It might around Chinese new year though. :P

There is another way to fix this problem.
That is make another Droid Sans Taiwanese font just like the way that Droid Sans Japanese font does.

Bug #1270647 is possibly related.

In reply to comment 41:

The 2 reports are not entirely related. But I hope this report is only limited to the scope of 69-language-selector-zh-tw.conf. The case for Hong Kong is more serious, in that *none* of the fonts described above have enough coverage of Hong Kong Supplementary Character Set (the only one good enough is AR PL UMing HK). Hong Kong users currently don't even have such luxury to discuss whether a font looks good or not.

I'll check Droid fallback font to see what's missing and provide comment on whether it's good enough for use in Hong Kong as well.

Actually, so far this bug has resulted in the changes stated in comment #21, #24 and #26, so Droid Sans Fallback is indeed used in zh-hk already.

http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/language-selector/trusty/view/head:/fontconfig/69-language-selector-zh-hk.conf

It's also installed by default for all users of the Ubuntu desktop. Hope you'll find that we are on the right track. ;-)

See that. Probably right now it's the best arrangement for Hong Kong users as well given currently (limited) resource.

fonts-droid has been included in the Ubuntu seed for a few months now, and I have proposed that flavor distros change their seeds in the same way.

@Aron: I couldn't find any ubuntukylin.trusty branch in ubuntu-seeds. Where did you hide it? ;-)

@gunnarhj, Ubuntu Kylin doesn't have seed, we only use default-settings package to customize.

Thanks for the explanation, Jack. Then I suppose that fonts-droid is already shipped in Ubuntu Kylin just like in standard Ubuntu.

An issue with an undesired font being selected has been reported at bug #1227034. I would recommend everyone with an interest in the topic to get involved in that bug.

Closing this bug report now.