FFe: Sync tesseract-* 3.02.01-1 (universe) from Debian sid (main)

Jeff, it's not yet FF (it's at 21:00 UTC today), so it does not need to be an FFe. At least, not yet!

I have subscribed ubuntu-sponsors in the hope that someone on that team have some time to look at this before the actual freeze.

To be 100% clear, this is really a sync request for tesseract-*

As one of the authors, I'd love to see users get easy (packaged) access to a recent release of Tesseract. 3.02 is a big update with lots of improvements: page layout, multi-language mode, bidirectional OCR, increased accuracy, and many more languages.

Taking a look

Thank you for trying to keep Ubuntu up to date. It seems that the patch from Ubuntu wasn't included in this upstream release: http://launchpadlibrarian.net/75401530/tesseract_2.04-2.1_2.04-2.1ubuntu1.diff.gz Would it be possible to do a merge or would you like me to do this?

>Would it be possible to do a merge or would you like me to do this?

The fix has just been merged into upstream subversion. I will add also it to Debian right now, as a maintainer patch. Anything else?

Okay, tesseract_3.02.01-2 has just been uploaded to Debian to include this patch.

This certainly needs a freeze exception at this point, so please nobody sponsor until it is granted.

I see on the linked Debian bug that sikuli has reported worse performance with this new series. Is that not a concern or is their usecase no longer as well supported?

Could you please give an explicit list of all packages to be synced?

I must admit to being a bit concerned about the way that ocropus was broken without apparently warning its maintainer too, especially given that there is no replacement available yet.

>I see on the linked Debian bug that sikuli has reported worse performance with this new series.
>Is that not a concern or is their usecase no longer as well supported?

Tesseract upstream is in communication with Sikuli upstream. a 10% drop in recognition
performance is considered acceptable by Sikuli upstream. Additionally, future releases
of Sikuli may remove that penalty now that the two upstreams are in communication.
Here is the relevant quote from Sikuli upstream Tsung-Hsiang (Sean) Chang.

  "The main reason we aren't not switching to tesseract 3 in an official release is
  that its recognition performance is worse than 2.04 in our dataset. (Not very bad,
  about 10% worse as I recall.) So I think it's fine to wrap the tesseract 3 branch for
  Debian sid."

>Could you please give an explicit list of all packages to be synced?

Appended.

>I must admit to being a bit concerned about the way that ocropus was broken
>without apparently warning its maintainer too, especially given that there is
>no replacement available yet.

This is a reasonable concern. I assume you are referring to
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=659597

From an etiquette perspective, I have been in email communication (8 threads
in the last 16 days) with Jeffrey Ratcliffe, who is the first listed maintainer for
both Tesseract and Ocropus. I have been in bug tracking communication (7 bugs)
with Jakub Wilk over the same period. Jakub has been incredibly helpful by filing
those packaging bugs. That said, my strategy was to - with blessing from my
co-maintainer - bring Tesseract 3 into Debian unstable, then find and fix problems
as quickly as possible. I apologize for causing surprise.

However, that leaves the issue of Ocropus. If Ubuntu 12.04 accepts Tesseract 3, it
will lose Ocropus. I respect Ubuntu's decision whichever way it goes. Please
consider the number of users affected on either side, and also Ocropus upstream
Tom Breuel's comments.

   "the version of OCRopus that has been packaged is completely outdated. OCRopus
  is now a set of Python libraries with a little bit of C++ in each. The complete final
  package structure isn't settled yet, but I want different components to be fairly
  independent of each other. Now, during my sabbatical, I've finally had time to actually
  work on it more than just a little on the side. The best thing for Debian probably would
  be to discontinue the current packaging for OCRopus and start over again when the
  new release is out."

Thank you for your consideration.

=========

Full list of source packages to remove:

ocropus
tesseract-ocr-deu-f

Full list of non-source packages to remove (maybe this goes away automatically):

tesseract-ocr-dev

Full list of source packages to sync (note lack of tesseract-lat-lid):

sikuli
tesseract
tesseract-afr
tesseract-ara
tesseract-aze
tesseract-bel
tesseract-ben
tesseract-bul
tesseract-cat
tesseract-ces
tesseract-chi-sim
tesseract-chi-tra
tesseract-chr
tesseract-dan
tesseract-deu
tesseract-deu-frak
tesseract-ell
tesseract-eng
tesseract-enm
tesseract-epo
tesseract-equ
tesseract-est
tesseract-eus
tesseract-fin
tesseract-fra
tesseract-frk
tesse...

Trying again with proper formatting.

>I see on the linked Debian bug that sikuli has reported worse
>performance with this new series. Is that not a concern or is their
>usecase no longer as well supported?

Tesseract upstream is in communication with Sikuli upstream. a 10%
drop in recognition performance is considered acceptable by Sikuli
upstream. Additionally, future releases of Sikuli may remove that
penalty now that the two upstreams are in communication. Here is the
relevant quote from Sikuli upstream Tsung-Hsiang (Sean) Chang.

  "The main reason we aren't not switching to tesseract 3 in an
  official release is that its recognition performance is worse than
  2.04 in our dataset. (Not very bad, about 10% worse as I recall.) So
  I think it's fine to wrap the tesseract 3 branch for Debian sid."

>Could you please give an explicit list of all packages to be synced?

Appended.

>I must admit to being a bit concerned about the way that ocropus was
>broken without apparently warning its maintainer too, especially
>given that there is no replacement available yet.

This is a reasonable concern. I assume you are referring to
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=659597

From an etiquette perspective, I have been in email communication (8
threads in the last 16 days) with Jeffrey Ratcliffe, who is the first
listed maintainer for both Tesseract and Ocropus. I have been in bug
tracking communication (7 bugs) with Jakub Wilk over the same
period. Jakub has been incredibly helpful by filing those packaging
bugs. That said, my strategy was to - with blessing from my
co-maintainer - bring Tesseract 3 into Debian unstable, then find and
fix problems as quickly as possible. I apologize for causing surprise.

However, that leaves the issue of Ocropus. If Ubuntu 12.04 accepts
Tesseract 3, it will lose Ocropus. I respect Ubuntu's decision
whichever way it goes. Please consider the number of users affected on
either side, and also Ocropus upstream Tom Breuel's comments.

   "the version of OCRopus that has been packaged is completely
  outdated. OCRopus is now a set of Python libraries with a little bit
  of C++ in each. The complete final package structure isn't settled
  yet, but I want different components to be fairly independent of
  each other. Now, during my sabbatical, I've finally had time to
  actually work on it more than just a little on the side. The best
  thing for Debian probably would be to discontinue the current
  packaging for OCRopus and start over again when the new release is
  out."

Thank you for your consideration.

I would probably prefer to see this happen in 12.10, in the hope that the problems with sikuli and ocropus are resolved in the interim.

But I won't block you from proceeding in 12.04 if you want to, so the FFe is approved by me.

I'm leaving as New and subscribing the sponsors for you. Comment #9 shows how to proceed. I'd appreciate a courtesy check that an archive administrator doesn't mind performing the work that this will cause, and please make sure to watch all changed packages for bug reports.

Please have this done by Beta 2 freeze, March 22.

I can't do the removal, but removing one package shouldn't be an issue. I don't mind doing the New processing for syncs from Debian.

Is the ocropus we have now working? If not it seems this should go ahead.

Good news, the new tesseract has now entered Debian testing.

>I would probably prefer to see this happen in 12.10, in the hope that
>the problems with sikuli and ocropus are resolved in the interim.

Sikuli is good to go. See version 1.0~x~rc3.tesseract3-dfsg1-1 in Debian
Unstable. It is still waiting to enter Debian testing.

Ocropus is broken in Debian, it can neither build nor run. This will
not change until upstream completes a major rewrite.

>Is the ocropus we have now working? If not it seems this should go ahead.

If I am reading the bug reports correctly, the current Ubuntu ocropus
is working, but is obsolete as per Debian and Ubuntu bug reports and
upstream commentary.

This really does look like a (new) Tesseract vs (old) Ocropus
tradeoff. There are a stack of people peering over my shoulder
muttering about why the former is more important, let me know if
you would like to hear from them.

https://bugs.launchpad.net/ubuntu/+source/ocropus/+bug/500527

(When I said Sikuli is good to go, I mean that it builds, runs, and upstream is happy. There has not been additional performance tuning.)

Looking at popcon, I think tesseract is used ~an order of magnitude more than
ocropus.

>I'm leaving as New and subscribing the sponsors for you. Comment #9
>shows how to proceed. I'd appreciate a courtesy check that an archive
>administrator doesn't mind performing the work that this will cause, and
>please make sure to watch all changed packages for bug reports.

I'm not sure who "you" is in this context. If "you" means Jeff, I am
a Debian Developer but otherwise not affiliated with Ubuntu
development. Happy to watch for bug reports but I don't know how
to talk to archive administrators.

https://wiki.ubuntu.com/UbuntuDevelopment/PackageArchive#Removing_Packages explains the process to get packages removed.

Syncing these packages.

All syncs (tesseract, tesseract-*, sikuili) filed.

ocropus removal requested in LP: #941136

Stefano, this is beyond fantastic. Just checking - does the sync include these removals, as mentioned in comment #9?

RM: tesseract-ocr-deu-f -- ROM package renamed to tesseract-ocr-deu-frak
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660677

RM: tesseract-ocr-dev -- ROM replaced by libtesseract-dev
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660680

> does the sync include these removals, as mentioned in comment #9?

The removals are subject of a separate process that only a few people can do.
They'll get done, but usually take longer.

Ah, I missed tesseract-ocr-deu-f, thanks. Removal requseted in LP: #941292.