Mismatching PO comments cause translation issues: translations present in PO not visible in GUI

L10n experts can you please confirm you have this problem too?

I observed this problem with both GTK and web clients.
Lionel.

Hello,

The issue is fixed in following branch:
Branch : lp:~openerp-dev/openobject-addons/6.0-opw-572985-ado with
Revision-id : <email address hidden> and
Revision-no : 5150

Soon it will be reviewed this and merged with stable addons.

Regards,
Amit

Hi,

The problem actually comes from a limitation of Launchpad's translation system. We had never hit it before because there was only one fully-synchronized translatable series of OpenERP. The problem is that PO templates entries are not only shared between series, but their comments are shared as well, regardless of the destination series.

The obvious advantage of sharing the POT entries between series is that translators can actually translate the terms only once in any series, and they will automatically be translated in other series where the same terms appear. That is absolutely vital. However the POT comments are carried over with the translation, and therefore are being exported in all series, even those where the POT comments were different for that same msgid entry.

As the most recent series (trunk or latest stable) are most likely to see their POT files updated, they will ultimately override all PO comments, and in the case where the term is used in different places, this could make it disappear in some places for older series. We can keep fixing PO comments left and right as in the current merge proposal, but this will not really fix the issue, only move it or delay it.

We discussed this with LP developers and changing this in Launchpad is not trivial, and goes deep in the db structure of the translations system. The POT comments being really meant as mere comments, their system cannot really be blamed for mixing them a little bit - they're just supposed to help translators - it's only a problem because OpenERP treats them as more than just comments.

So the quickest way to fix this correctly is to fix our framework globally, possibly in all affected series.
I can think of two options at this point:

1. Override PO comments with POT comments.
When loading PO files, we could first pre-load the corresponding POT file, and use the comments from the POT file instead of those from the PO files. As LP does not touch the POT files, the right PO comments (from the right series) will still be in the POT template. Downside: this represents a no-negligible amount of code, parsing time and risk, especially as it will be needed in stable series too.

2. Treat the translation type a hint, not an exclusive condition.
In ir.translation._get_source(), the ``types`` parameter is currently treated as an exclusive filter: if no translation is found within the given types, no result is returned at all. We could change this to make ``types`` a non-exclusive hint of the type(s) that are preferred when looking for translations. If a translation of the right type(s) is found it it will be returned with higher priority, but if none is found, candidates with another type will still be returned.
This seems to carry little risk as the translation type is really just a hint... we have no reason to believe that a translation with the wrong type is not going to be correct. In fact we have no way of giving a different translation for the same term at the moment, so they cannot be different even if we wanted.

I tend to prefer solution 2 as much simpler and less risky, so I whipped up a quick patch (attached) that should apply cleanly on any 6.0/6.1/trunk bra...

Thanks for your care.
I also prefer the second approach, it sounds a bit more foolproof.
However, there seems to be a slight problem with your patch, it seems to sens the translation of the help texts instead of that of the field names.
Lionel.

Any progress regarding this bug?
Lionel.

Here is a script I quickly wrote to work around the problem. It only processes French and only in our own directory layout, but it should be trivial to adapt to other needs.
This is of course not a solution to this bug.
Lionel.

Sorry, I posted the wrong file. Here's the right one.

I gave this annoying bug a second look, and it turns out that Olivier Dony's patch has just a small error (it selects the worst translation instead of the best).
I pushed fixed branches for merge in 6.0 and trunk.
Please, review them ASAP.
Lionel Sausin.

As the second solution proposed by Olivier Dony seems better (and finallly not that much invasive), we have started to implement it here https://code.launchpad.net/~openerp-dev/openobject-server/6.1-fix-po-targets-933496-vmt/+merge/130143.

Dear Vo Minh Thu,
Just so I understand: you're implementing the first solution then, not the second one aren't you?
 "1. Override PO comments with POT comments."
 "2. Treat the translation type a hint, not an exclusive condition." => that was Olivier's patch wasn't it?
Lionel.

Lionel, you're right, it is the first one: use the POT comments in addition to the PO one (maybe it shoule be `instead of`, not `in addition to`).

Dear Vo Minh Thu,
Does that mean translations will disappear if the POT if not up-to-date, or if there's no POT file?
I thinking of extra or custom modules here, where the developer may have exported only the translations he needs.
Lionel.

Lionel,

I will check the behavior (just in case) but a missing POT, or even a corrupt one, should do no harm. (You can check too but I will try to do it right :)

Translations should never disappear: only be updated. The POT will only add translations (those for which the comments are not in the PO).

I think this never landed in the trunk, was any progress made lately ?
Lionel Sausin

I tried to port Vo Minh Thu's fix to v7.0/trunk, and from my (very) quick and limited testing it seems to work.
I proposed for merging into trunk.
Lionel.

Dear Vo Minh Thu,
This bug is still marked unfixed, and with v8 being released soon, v7 will become affected too won't it?
I'm going to make a proposal for OCB v7.0, but I don't have the time to port it (again) to official v8 yet.
Would you please be so kind as to have a look at this ?

Lionel,

I heard Vo Minh Thu doesn't work for OpenERP SA anymore.

this bug is assigned to OPW team, so I suppose they got your message anyway.

best!

I have checked deeply the patch, and it's correct. The problem with still untranslated terms comes from pot files: they are incorrect. Problem has been detected for example on purchase module from 6.1 addons. You have these comments in pot file for the msgid "Purchases" (term for the main menu):

#. module: purchase
#: model:ir.actions.act_window,name:purchase.action_purchase_line_product_tree

In es.po, you can find though this other ones:

#. module: purchase
#: model:ir.actions.act_window,name:purchase.action_purchase_line_product_tree
#: model:ir.actions.act_window,name:purchase.open_board_purchase
#: model:ir.ui.menu,name:purchase.menu_board_purchase
#: model:ir.ui.menu,name:purchase.menu_purchase_config

Which is logical, because these comments are synchronized from the translation focus branch (7.0 in this case), but there's must be one comment on pot file that is missing:

#: model:ir.ui.menu,name:base.menu_purchase_root

With this one and the patch, "Purchases" main menu is translated.

Exporting pot file from OpenERP interface, resulting file is equal, without that comment, so the problem seems to be with the export. I'm going to open another bug for reflecting this.

Regards.

I've filled this bug:

lp:1294887

The maintenance team did it's job by escalating this and now we have a fix committed.
I'm removing the assigned team in the hope that someone who can commit to stable branches will get in charge.

By the way, I didn't make a MP for trunk, I suppose fixes for 7.0 are ported to trunk once in a while - just ask if you want an explicit MP to trunk.

Adding aeroo-reports as an affected project since the report_aeroo module redefines translate.py