Ubuntu
enigmail package

thunderbird does not warn anymore about unencodable glyphs in recipient name (regression, UTF-8)

Bug #145429 reported by Rolf Leggewie
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Enigmail
Won't Fix
Medium
Mozilla Thunderbird
Invalid
High
enigmail (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

Binary package hint: mozilla-thunderbird

Version 1.5 of Thunderbird in feisty used to warn me when I tried to send a mail to my friend 武, for example, that the character cannot be encoded in my standard ISO-8859-1 encoding and whether I want to send the mail in UTF-8. The latest version in gutsy (version 2.0.0.6 (20070924)) fails to do that and just sends the message which results in the name being replaced by question mark(s).

This is still present in hardy.

See original description
Tags: regression
Revision history for this message
In , M-wada (m-wada) wrote :

Attach mail to this bag thru "Add an attachment" link, please.
1) Save mail to ".eml" file. 2) Attach the file (mime-type=text/plain)

> 2. Use Firefox and go to http://foxkeh.jp/
Source of this page.
  <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="ja" lang="ja">
  <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<title> text is written in Japanese characters.

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=262876)
This is the received mail, ???? as subject, since it is not UTF encoded.

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #0)
> Subject: =?ISO-8859-1?Q?=3F=3F=3F=3F=3F*=3F=3F=3F?=

> 4. Now a mail compose window opens in Thunderbird, the subject is in Japanese,
> the Body is only the URL
What will happen when next steps are added after step 4?
4.1) Update subject field manually. (Insert a space, then remove the space)
4.2) Save as Draft

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263011)
This happens when I add and remove a space character and save in Drafts, same result: ????

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263012)
This happens, when I add some Latin text to the Japanese subject and save as Draft, again: ????

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #4)
> This happens when I add and remove a space character and save in Drafts
 (In reply to comment #5)
> This happens, when I add some Latin text to the Japanese subject

Thanks for quick test.
Can I ask you one more test to clarify?
 1. Open the web site by Firefox. (http://foxkeh.jp/)
 2. View page source, and select text between <title> and </title>(title text),
    and Command+C(copy to clip board).
 3. Open mail compose window of Thunderbird, and Command+V to subject(paste),
    and fill To:, and fill mail body text(ascii char only, such as "aaa"),
    then save as Draft.
Problem is recreated?

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263039)
I copied Japanese title from Firefox source in the TB subject line, same result: ????

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #7)
> I copied Japanese title from Firefox source in the TB subject line, same
> result: ????

Thanks for quick test. Your test results say that Firefox has no relation.
When Tb 2.0.0.0(en-US version, Japanese MS Win-XP SP2) and new profile and default encoding of compose=iso-8859-1, Tb issued warning as expected in this test case.
Mac OS X only problem? Or PPC Mac OS X only problem?
Or problem when locale=unicode? (If so, may occur on Win/Linux of locale=unicode too.)

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #7)
> Created an attachment (id=263039) [details]
> I copied Japanese title from Firefox source in the TB subject line
 Oh, forgot important question.
When the Japanese text is copied to subject, displayed characters in subject is same as one in source view of Firefox?
(Binary representation of unicode for the text may varies while operations, for example, composed <-> decomposed, and it can occur on last 3 characters.)
If you are uncertain on "same characters", please attach screen shot.

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263152)
Copy and paste of Japanese characters between Firefox and Thunderbird works without problems, display is correct

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #7)
Another question(and test request) to clarify.
> I copied Japanese title from Firefox source in the TB subject line,
 Depends on "default character encoding" setting for compose?
 1.other iso-8859-x such as iso-8859-15, 2.windows-1252, 3.koi8-r

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263200)
ISO-8859-15 encoding, does not work

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263201)
KOI8-R encoding, does not work

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263202)
ISO-8859-2 encoding (Central European), does not work

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263203)
ISO-2022-JP encoding (Japanese), works correctly

Revision history for this message
In , Rotis (rotis) wrote :

Created an attachment (id=263204)
UTF-8 encoding (Unicode), works correctly

Revision history for this message
In , Rotis (rotis) wrote :

Your requested Windows-1252 encoding doesn't exist on Thunderbird 2.0.0.0 (Mac, PPC). There is only Windows-1251 (Cyrillic) and Windows-1255 (Hebrew) of those Windows encodings.

Revision history for this message
In , M-wada (m-wada) wrote :

Thanks for test. Last test request from me to clarify.
Is problem recreated with Tb 1.5 and latest-trunk?
(New prof, Dummy POP3 account, No Global Inbox, "Compose & Save Drafts" only and ignore other bugs, iso-8859-1 only is sufficient)

Can someone(or you) test with different locale of Mac OS X?
(It's difficult for me to test with other locale than current on Japanese Win...)

Revision history for this message
In , Darren (darren-dcook) wrote :

I see the same problem, using Thunderbird 1.5.0.10 on Fedora Core 5. It doesn't sound like it is fixed, so perhaps I can give another data point, and also give weight to the opinion that it is a serious bug.

On a mailing list I replied to a message that had a subject, containing Japanese, encoded as:

Re: =?UTF-8?B?5pel4oaS6Iux44CA57+76Kiz44O76YCa6Kiz44CAY291cnNlcyA=?=
 =?UTF-8?B?b3IgY2VydGlmaWNhdGlvbnM=?=

I clicked reply, and in the mail when I composed it I could see the Japanese. But when I sent it the mail used:

Re: =?ISO-8859-1?Q?=3F=3F=3F=3F=3F=3F=3F=3Fcourses_or_certif?= =?ISO-8859-1?Q?ications?=

I.e. The Japanese has turned into lots of question marks. I see the same in my saved folder.

I have iso-8859-1 as my default email language. When I hit reply, then change the mail encoding to UTF-8, and save/send it, it is fine, no corruption.

Usually when I have UTF-8 characters in the *body* of the email it will prompt me, saying "would you like to send this email as UTF-8?".

So, I see two solutions:

SOLUTION ONE: Whatever code scans the body to detect characters different from the current encoding, in order to give the user that warning, must also scan all the headers as well.

SOLUTION TWO: If a header was utf-8 encoded, and the mail is replied-to/forwarded then the header should stay in utf-8 encoding, and not be converted to default encoding. (I.e. the header may be a different encoding to the body.)

Revision history for this message
In , Petr Hroudný (petr-hroudny) wrote :

Only solution one can work, since it's also possible that the user added some UTF-8 characters into Subject field when composing the reply. You can't rely on the original Subject field for encoding.

Revision history for this message
In , M-wada (m-wada) wrote :

(In reply to comment #19)
> Re: =?UTF-8?B?5pel4oaS6Iux44CA57+76Kiz44O76YCa6Kiz44CAY291cnNlcyA=?=
> =?UTF-8?B?b3IgY2VydGlmaWNhdGlvbnM=?=
 ==>
> Re: =?ISO-8859-1?Q?=3F=3F=3F=3F=3F=3F=3F=3Fcourses_or_certif?=
> =?ISO-8859-1?Q?ications?=

Same result as bug opener's case.
Confirming and changing to OS=All.

Revision history for this message
In , Vseerror (vseerror) wrote :

WADA, did you intend to leave HW=mac and bug unconfirmed?

Revision history for this message
In , M-wada (m-wada) wrote :

Sorry, I seems to have failed to click "Confirm bug".

Revision history for this message
Rolf Leggewie (r0lf) wrote : [gutsy] latest thunderbird does not warn anymore about unencodable glyphs in recipient name (regression, UTF-8)

Binary package hint: mozilla-thunderbird

Version 1.5 of Thunderbird used to warn me when I tried to send a mail to my friend 武, for example, that the character cannot be encoded in my standard ISO-8859-1 encoding and whether I want to send the mail in UTF-8. The latest version in gutsy (version 2.0.0.6 (20070924)) fails to do that and just sends the message which results in the name being replaced by question mark(s).

Revision history for this message
In , Mozilla-bugs-nogin (mozilla-bugs-nogin) wrote :

See also bug #90584 (dup?)

Revision history for this message
In , Mozilla-bugs-nogin (mozilla-bugs-nogin) wrote :

Updating the title to better reflect the problem - when mail is being sent, we check the body of the message to make sure all the characters in the body can be represented using the selected character encoding; the problem is that no such check is done for the subject.

Revision history for this message
In , Petr Hroudný (petr-hroudny) wrote :

This works fine for me (TB 2.0.0.12 and later).

When I add a single character not available in selected encoding to the Subject line, TB asks "Send in UTF-8"

Could you please recheck and eventually close this bug?

Revision history for this message
In , Darren (darren-dcook) wrote :

I can confirm it still happens as of thunderbird 2.0.0.14 (ubuntu 7.10)
I just created a new email with subject set to "テスト" (Japanese, meaning "test"), and only ascii in the body. When I send it doesn't prompt about encoding, and it arrives with the subject changed to "???".

Revision history for this message
In , Petr Hroudný (petr-hroudny) wrote :

Strange. Just tested with TB 2.0.0.14 (Fedora7). Encoding set to Western (8859-1).

Created new email with Subject: "テスト" and only a single word "test" in the body.

Clicking on Send immediately brings up "Send in UTF-8" dialog. The same happens
on Windows version.

Must be something wrong in your installation or preferences.

Revision history for this message
In , Darren (darren-dcook) wrote :

Default encoding for me is also iso-8859-1.
Any suggestions for what preference might be affecting it? I've had a hunt through and couldn't see anything that might be related. I'm sending text-only email.
I had "For messages that contain 8-bit characters, use 'quoted-printable' MIME encoding" unchecked. I tried checking that and sending again, but it made no difference.

In environmental variables, I have:
  LANG=en_GB.UTF-8

I have the enigmail plugin installed.

Let me know if you can think of any other setting you want me to check.

Revision history for this message
In , Petr Hroudný (petr-hroudny) wrote :

Have you tried it without enigmail plugin?

Rolf Leggewie (r0lf)
description: updated
Revision history for this message
Rolf Leggewie (r0lf) wrote :

still the same in hardy TB 2.0

Revision history for this message
In , Mozilla-bugs-nogin (mozilla-bugs-nogin) wrote :

*** Bug 369067 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Bugzilla-mozilla-org-rolf (bugzilla-mozilla-org-rolf) wrote :

I confirm this problem and it applies to all headers, not just the subject. The name of your dear Japanese friend will be all ????, too. Changed the title accordingly. It is also a regression. TB in feisty used to work correctly. It must have been something 1.5 or earlier.

Quoting
https://bugs.launchpad.net/ubuntu/+source/mozilla-thunderbird/+bug/145429

"Version 1.5 of Thunderbird in feisty used to warn me when I tried to send a
mail to my friend 武, for example, that the character cannot be encoded in my
standard ISO-8859-1 encoding and whether I want to send the mail in UTF-8. The
latest version in gutsy (version 2.0.0.6 (20070924)) fails to do that and just
sends the message which results in the name being replaced by question
mark(s)."

Revision history for this message
In , Bugzilla-mozilla-org-rolf (bugzilla-mozilla-org-rolf) wrote :

Indee(In reply to comment #30)
> Have you tried it without enigmail plugin?

Indeed, enigmail seems to be the culprit. With enigmail turned off, everything is fine. With the plugin turned the detection seems to go either wrong or not be called at all.

Bug Watch Updater (bug-watch-updater)
Changed in thunderbird:
status: Unknown → Confirmed
Revision history for this message
Rolf Leggewie (r0lf) wrote :

problem is known and discussed upstream. confirming.

Changed in mozilla-thunderbird:
importance: Undecided → Medium
status: New → Confirmed
Bug Watch Updater (bug-watch-updater)
Changed in thunderbird:
status: Unknown → Confirmed
Revision history for this message
In , Mcow (mcow) wrote :

<email address hidden>: do you also have Enigmail installed?

Revision history for this message
Rolf Leggewie (r0lf) wrote :

turns out that enigmail is the culprit

Revision history for this message
In , Bugzilla-mozilla-org-rolf (bugzilla-mozilla-org-rolf) wrote :

*** Bug 346657 has been marked as a duplicate of this bug. ***

Revision history for this message
Rolf Leggewie (r0lf) wrote :

still present as of today with uptodate karmic packages.

Revision history for this message
In , Patrick-mozilla-enigmail (patrick-mozilla-enigmail) wrote :

If it's really Enigmail, then it's fixed in current trunk for Thunderbird 3.0.

Bug Watch Updater (bug-watch-updater)
Changed in enigmail:
status: Unknown → Confirmed
Revision history for this message
Patrick Brunschwig (pbrunschwig) wrote :

That's fixed for Thunderbird 3.0

Bug Watch Updater (bug-watch-updater)
Changed in enigmail:
status: Confirmed → Won't Fix
Revision history for this message
Rolf Leggewie (r0lf) wrote :

yes, it's fixed in lucid. thanks.

Changed in enigmail (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
In , Bugzilla-mozilla-org-rolf (bugzilla-mozilla-org-rolf) wrote :

yes, it's indeed fixed. thank you, guys.

Closing

Revision history for this message
In , Vseerror (vseerror) wrote :

not fixed without identifying bug#. sorry. it's either WFM or invalid

Bug Watch Updater (bug-watch-updater)
Changed in thunderbird:
status: Confirmed → Invalid
Bug Watch Updater (bug-watch-updater)
Changed in thunderbird:
importance: Unknown → High
Bug Watch Updater (bug-watch-updater)
Changed in enigmail:
importance: Unknown → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.