Random Chinese character missing with Ubuntu font

Another screenshot of missing character in news-scope.

This seems to be rather a Qt bug than a toolkit one. All we do in 50 GU system is we calculate the pixels. Penn Chen, could you check the same with a simple QML test to be sure we're on the same page?

Sorry, typo, Penk Check I meant...

I'll mark it as Incomplete as it is not confirmed that is part of UITK.

---------- Copying Timo's request from the duplicate bug below ----------

As an additional data gathering, could you test if it's the same on wily images ie ubuntu-touch/devel/ubuntu ? wily has Qt 5.4.2 so additional fixes compared to vivid's Qt 5.4.1. If the problem would be fixed on wily, we could try to find out the exact fix and backport.

Is it possible to somehow reproduce this on desktop, for example a QML file that tries to display the same characters?

It'd be best to be able to submit a bug report to upstream bug tracker https://bugreports.qt.io/ , but they'd need a test case they can test themselves with plain Qt without Ubuntu additions.

I can not reproduce this issue in QtQuick.Controls with ApplicationWindow and toolBar. Test app attached, you'll need qml-module-qtquick-controls installed of course.

However, this can be easily reproduced in Ubuntu.Components 1.1 with MainView and Page title, like this:

MainView {
    objectName: "test"
    applicationName: "com.ubuntu.chinese-font"

    id: mainView
    width: units.gu(50)
    height: units.gu(67)
    useDeprecatedToolbar: false
    Component.onCompleted: pageStack.push(mainPage);
    PageStack {
        id: pageStack
        Page {
            id: mainPage
            objectName: "mainPage"
            title: i18n.tr("系统设置")
            visible: true
        }
    }
}

Screenshots from left to right:

1. normal splash screen without issue
2. issue happened with test QML in comment #7, and
3. normal app quick-controls.qml without issue

I managed to reproduce this on desktop with the following code:

import QtQuick 2.3
import Ubuntu.Components 1.2

MainView {
    objectName: "test"
    applicationName: "com.ubuntu.chinese-font"

    id: mainView
    width: units.gu(50)
    height: units.gu(67)
    // useDeprecatedToolbar: false
    Component.onCompleted: pageStack.push(mainPage);
    Page {
        id: mainPage
        objectName: "mainPage"
        title: i18n.tr("系统设置")
        visible: true

        Label {
            anchors.centerIn: parent
            text: "系统设置"
            fontSize: "x-large" // different fontSizes do not have the bug
        }
    }
}

As reported in the dupe bug https://bugs.launchpad.net/ubuntu/+source/qtdeclarative-opensource-src/+bug/1462982
the bug *only* occurs with a grid unit size of 23, so run the app with:
GRID_UNIT_PX=23 qmlscene test.qml

As you can see, I managed to reproduce it with a Label (not in the header). The bug only appears when fontSize: "x-large" is set.

It also happens with Text and certain font sizes:

        Text {
            text: "系统设置"
            font.pixelSize: 1.714*14*(23/8)
        }

where font.pixelSize here follows the same computation that goes from "x-large" fontSize to the actual pixelSize.

The 23 in the calculation is the value of GRID_UNIT_PX. When using 23 in the font.pixelSize computation, the actual value of GRID_UNIT_PIX does not matter any more for reproducing the bug.

The bug shows also when using Text, but only when Ubuntu.Components is imported:

import QtQuick 2.4

// uncomment this line to show the bug:
import Ubuntu.Components 1.2

Item {
    width: 800
    height: 600

    Text {
        anchors.centerIn: parent
        text: "系统设置"
        font.pixelSize: 69
    }
}

I reproduced the bug independent from the GU size and without importing Ubuntu.Components:

import QtQuick 2.0
Item {
    width: 800
    height: 600

    Text {
        anchors.centerIn: parent
        text: "系统设置 one two"
        font.family: "Ubuntu"
        font.pixelSize: 69
        font.weight: Font.Light
    }
}

There seems to be an issue with the Ubuntu font, but only with weight Light and pixelSize 69.

I can confirm this happens on font.family: "Ubuntu", the default fallback font of Chinese seems to be "Droid Sans".

Here's the output of Qt.fontFamilies() on our Chinese image:

qml: DejaVu Sans,DejaVu Sans Mono,DejaVu Serif,Droid Arabic Kufi,Droid Arabic Naskh,Droid Naskh Shift Alt,Droid Sans,Droid Sans Arabic,Droid Sans Armenian,Droid Sans Ethiopic,Droid Sans Fallback,Droid Sans Georgian,Droid Sans Hebrew,Droid Sans Japanese,Droid Sans Mono,Droid Serif,Liberation Mono,Liberation Sans,Liberation Sans Narrow,Liberation Serif,Lohit Devanagari,Monospace,NanumBarunGothic,NanumGothic,NanumMyeongjo,Sans Serif,Serif,Symbola,TakaoPGothic,Ubuntu,Ubuntu Condensed,Ubuntu Mono,文泉驿微米黑,文泉驿等宽微米黑

Status changed to 'Confirmed' because the bug affects multiple users.

So it only happens if all conditions are met:
Font is Ubuntu and Qt falls back to Droid Sans Fallback (which is from package fonts-wqy-microhei)
Pixelsize is 69 as computed from the 50GUs and Grid Unit size of 23
Font weight is Light

Random characters are not shown.

Seems a bug in Qt's font fallback logic - similar report at https://bugreports.qt.io/browse/QTBUG-32470

Can someone try on Wily?

Does this happen with other strings in Chinese as well? Can you give a few more examples?
Or is it specific to these 4 characters?

FWIW this shows missing characters as well, very few fonts have a Light version.

       Text {
            text: "系统设置 Bookman Light 69px"
            font.family: "URW Bookman L"
            font.pixelSize: 69
            font.weight: Font.Light
        }

@Tim,

Yes, please take comment #2 for an example, "新闻".

Copy bug description https://bugreports.qt.io/browse/QTBUG-47547 and paste here.

if developer specifies font family with Non-Regular style, potential risk to encounter Chinese characters are missing in some cases .
E.g

import QtQuick 2.4
import QtQuick.Controls 1.3

ApplicationWindow {
    visible: true
    width: 1280
    height: 800

    Text {
        anchors.centerIn: parent
        text: "系统设置"
        font.family: "Ubuntu"
        font.pixelSize: 69;
        font.weight: Font.Light
    }
}

Regarding the code snippets above.
For Chinese characters, QFontDatabase will use fallback font family("Droid Sans Fallback") with "Light" style when add font Glyph.
 since QFontDatabase checks if default style(Regular) of fallback font family "Droid Sans Fallback" is same as font style specified in qml(Font.Light). And QFontDatabase::isSmoothlyScalable will return false in such a case, which results in scene graph will
use QSGDefaultGlyphNode when createGlyphNode is called. Issue happens.

However If scene graph doesn't prefer NativeGlyphNode and create GlyphNode based on Distance Field technique. It works fine.
https://drive.google.com/open?id=0B2H9ECPSSfqIMzhvU1hPUWNlWnc

Need to only check styleKey when foundry->styles == specified style
Will see feedback from Qt guys.
Thanks.

PS: more test cases and patch can be found on https://bugreports.qt.io/browse/QTBUG-47547
Also I think we can override bool QBasicFontDatabase::fontsAlwaysScalable() method in qlinuxfbfontdatabase.cpp under qtbase/src/plugins/platforms/ for platform independent purpose.

Status changed to 'Confirmed' because the bug affects multiple users.

nice work Gary

According to Qt guys feedback https://bugreports.qt.io/browse/QTBUG-47547
Bug is marked as P2(Important, Not urgent, should be fixed, but will not stop the release), which doesn't fit our OTA release cycle since we can't wait for next release of Qt.
Thanks to Yc for correcting the "affects distribution/package"
qtdeclarative-opensource-src (Ubuntu) ---> qtbase-opensource-src (Ubuntu)
I try to submit my merge proposal for qtbase-opensource-src, but it doesn't work out since I am not a members of Ubuntu branches.
So I attach the patch here and send an email for stakeholder about this.

The attachment "enable_fonts_always_smoothly.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

I'll start preparing the patch for landing to wily & vivid-overlay. There's another qtbase landing in QA queue so that must go in first.

@Gary: I can take your patch from here regarding getting it on to images. A big thank you for developing the patch! Upstream may have other suggestions on how to solve the problem and contribute an improved patch to https://codereview.qt-project.org/ directly - but we can carry the current small workaround patch until there's a better solution.

I'm rebasing the patch for the Qt 5.4.2/5.4.1 as attached.

Could you please test on vivid+overlay with the silo 33 (https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-033/). The debdiff is http://paste.ubuntu.com/12013270/. For example: citrain device-upgrade 33 0000 ubuntu

Check with dpkg -s libqt5core5a that the version is 5.4.1+dfsg-2ubuntu8

Please share your test results. On my mako I can't see a difference. The example in comment #17 seems to always work my mako. But the test code from comment #12 always tends to have one letter missing, with or without the PPA.

So arale/krillin results would be welcome. I don't see regressions, so if even one case is fixed with the updated qtbase I can land this, but I can't see differences on my mako.

adb push Main.qml /home/phablet
adb shell
qmlscene Main.qml --desktop_file_hint=/usr/share/applications/mediaplayer-app.desktop

@Ethan,

Please help to verify this fix with arale. Thanks.

On desktop I can see the PPA fixing the issue, see the attached screenshots.

@Timo
I run the citrain to install your patch, and then try the test qml but the missing letter is still missing.

phablet@ubuntu-phablet:~$ system-image-cli -i
current build number: 73
device name: arale
channel: ubuntu-touch/rc-proposed/meizu.zh-proposed
last update: 2015-08-10 06:45:07
version version: 73
version ubuntu: 20150807.1
version device: 20150709-8965e37
version custom: 20150729-849-849-9

＠Timo
I realize that my Arale didn't install correct version of packages after perform citrain to upgrade system.
 I tried several times but it didn't install or upgrade those packages on landing-33 ppa. So the version of on the Arale is 5.4.1+dfsg-2ubuntu7. May need your help to check if anything wrong of test steps.

Steps
1. Connect my phone to a destop
2 perform 'citrain device-upgrade 33 1111 ubuntu'
3. Phone will be reboot after that
4. Check the version of package and it shows 5.4.1+dfsg-2ubuntu7

Tried the manual way to upgrade the packages , but the test is still fail on my Arale
I check the package version is correct 5.4.1+dfsg-2ubuntu8 (pastebin; ttps://pastebin.canonical.com/137235/ )

I believe the current workaround is not ok for landing as it only fixes the desktop case while leaving the phone issue, for some reason, same as before.

Basically, this patch fixes both.
The main reason why it *only* fixes PC, but not for touch is that we forget to re-compile qtubuntu(libqpa-ubuntumirclient.so) to make this work.Compiling qtbase automatically enable qpa(linuxfb,eglfs...) linked against patch applied qtbase module. So we can't reproduce this issue on PC(linuxfb).
However as a standalone qpa plugin,ubuntumirclient is not native supported,which means it's not in qtbase/src/plugins/platforms).So ubuntumirclient can't find the new added symbol(QFontconfigDatabase::fontsAlwaysScalable()).And it didn't work out on touch.

Please check the attached picture. I test and verify my patch on bq.

Verified that a no-change qtubuntu rebuild fixes the issue! The silo has now been moved to QA's hands for verification.

Upgrade the packages from silo 33, the fix works on Arale.
Thanks for fixing the bug.

Screenshot for reference

Fixed in vivid-overlay PPA:

---
qtbase-opensource-src (5.4.1+dfsg-2ubuntu8) vivid; urgency=medium

  [ Gary Wang ]
  * debian/patches/enable_fonts_always_smoothly.patch:
    - Workaround a glyph rendering problem by forcing distance field
      rendering (LP: #1462982) (LP: #1475205)

 -- Timo Jyrinki <email address hidden> Wed, 05 Aug 2015 08:50:19 +0000

This bug was fixed in the package qtbase-opensource-src - 5.4.2+dfsg-2ubuntu3

---------------
qtbase-opensource-src (5.4.2+dfsg-2ubuntu3) trusty; urgency=medium

  [ Gary Wang ]
  * debian/patches/enable_fonts_always_smoothly.patch:
    - Workaround a glyph rendering problem by forcing distance field
      rendering (LP: #1462982) (LP: #1475205)

 -- Timo Jyrinki <email address hidden> Wed, 05 Aug 2015 11:44:25 +0300

qtbase-opensource-src (5.5.1+dfsg-6ubuntu3) xenial; urgency=medium

  * debian/patches/enable-tests.patch:
    - Replace the non-DFSG-free RFC files with public domain content.
    - Adjust tests to the new files.
    (LP: #1522442)
  * debian/copyright: mention the replacement.
  * Switch to opt-in for architectures to run tests on.

 -- Timo Jyrinki <email address hidden> Mon, 07 Dec 2015 14:02:11 +0000

qtbase-opensource-src (5.5.1+dfsg-6ubuntu2) xenial; urgency=high

  * Skip largefile test on s390x too, in addition to amd64.
  * Make tests to keep going till the end with -k.

 -- Dimitri John Ledkov <email address hidden> Fri, 04 Dec 2015 16:54:28 +0000

qtbase-opensource-src (5.5.1+dfsg-6ubuntu1) xenial; urgency=medium

  * New upstream release. (LP: #1437206) (LP: #1450137) (LP: #1474313)
    (LP: #1470700) (LP: #1504631) (LP: #1423659) (LP: #1474775) (LP: #1508945)
  * Replace load_testability_from_env_var.patch with upstreamed
    Add-environment-variable-support-for-testability-lib.patch
  * Drop patches in upstream:
    - Correct-typo-in-the-Gujarati-openType-identififer.patch
  * Rebase enable-tests.patch. Disable one failing QtWidgets test.
  * Build depend on GStreamer 1.0 and add a configure option for it.
  * Update symbols.
  * Mark/unmark private symbols.
  * Replace two Ubuntu patches with upstreamed patches:
    - Drop disable-generic-plugin-when-others-available.patch, replace with
      Add-an-option-to-skip-the-generic-bearer-engine.patch
    - Drop qopenglframebufferobject_powervrworkaround.patch, replace with
      Blacklist-PowerVR-Rogue-G6200-v1.3-from-supporting-B.patch
  * debian/patches/Make-sure-networkAccessibilityChanged-is-emitted.patch:
    - Include a network fix from Qt 5.5 branch (merged after 5.5.1)
      (LP: #1470700)
  * debian/patches/Use-Node-name-if-Node-logicalModuleName-is-empty-for.patch:
    - Fix a qdoc issue (LP: #1447182)
  * Remove disable_overlay_scrollbars.diff as overlay scrollbars were dropped.
  * debian/patches/Prefer-QT_PLUGIN_PATH-over-compiled-in-paths.patch:
    - Backport. Prefer QT_PLUGIN_PATH over compiled-in paths (LP: #1519927)
  * debian/patches/Fix-crash-on-exit-caused-by-QStringLiterals.patch
    - Backport. Fix a crasher on exit (LP: #1436973)
  * Replace our workaround for font rendering with new backported upstream
    patches:
    - Add debian/patches/Fix-falsely-reported-style-for-fallback-font.patch
    - Add debian/patches/Remove-historical-4-padding-in-QFontEngine-alphaMapF.patch
    - Remove debian/patches/enable_fonts_always_smoothly.patch
      (LP: #1475205)

 -- Timo Jyrinki <email address hidden> Tue, 01 Dec 2015 06:16:35 +0000