Getting (more) manufacturer-supplied PostScript PPDs onto the Ubuntu desktop CDs

There are two possibilities:

- Add an appropriate dependency to ubuntu-desktop. For that this bug has to be moved to ubuntu-desktop and to be assigned to the maintainer of ubuntu-desktop (I assume that ubuntu-desktop is a meta package only containing dependencies and no files).

- Join the packages foomatic-db and linuxprinting.org-ppds. Upstream they are one package named foomatic-db.

WDYT should be done?

I see no reason why they should be separate

We are currently examining the model of splitting linuxprinting.org-ppds to include at least the most important PPDs on the CD, as we most probably will not have space for all PPDs.

At least all PPDs of the linuxprinting.org-ppds package will be moved into main.

Final decision is that all PPDs go onto the CD.

Unfortunately, the manufacturer PPDs did not make it onto the Edgy CDs.
So I propose to ship only the most important PPDs on the CDs, as there
is no space for all PPDs.

I have made fixed Ubuntu packages of foomatic-db now. They split the
linuxprinting.org-ppds package in two, one named linuxprinting.org-ppds
containting only the most important PPDs (3.5 MB) and a second one named
linuxprinting.org-ppds-extra with all the other PPDs (10 MB). The first
one will go onto the Feisty CDs, the second one will be provided in main
for optional download.

Get the source package files from

http://www.freestandards.org/~till/tmp/ubuntu/feisty/foomatic-db/

and the new binary packages from

http://www.freestandards.org/~till/tmp/ubuntu/feisty/foomatic-db/binary/

They should work on both Edgy and Feisty.

Can someone from the Ubuntu Core Developer Team set the seeds so that

- linuxprinting.org-ppds goes onto the desktop CDs of Feisty and is in Main
- linuxprinting.org-ppds-extra goes does not go onto the desktop CDs of Feisty but is in Main
- linuxprinting.org-ppds-extra can go onto the server CDs, as there seems to be space

Then this issue will be completely fixed for Feisty.

DO NOT ASSIGN BUGS TO TEAMS IN THIS MANNER

On Thu, Nov 23, 2006 at 07:02:29PM -0000, Till Kamppeter wrote:
> - linuxprinting.org-ppds goes onto the desktop CDs of Feisty and is in Main
> - linuxprinting.org-ppds-extra goes does not go onto the desktop CDs of Feisty but is in Main
> - linuxprinting.org-ppds-extra can go onto the server CDs, as there seems to be space

Why is linuxprinting.org-ppds-extra so large? Surely descriptions of
PostScript compatible printers should be among the simplest?

As before, please work with Martin Pitt for any changes which need to be
made for which you don't have the required privileges.

--
 - mdz

linuxprinting.org-ppds and linuxprinting.org-ppds-extra do purely contain PPDs as provided from the printer manufacturers, no printer descriptions nor any other documentation. Because of its use size I have split it off, so that the most important PPDs reside in the 3.5 MB linuxprinting.org-ppds package and can go onto the desktop CDs.

Martin, can you modify the seeds so that linuxprinting.org-ppds goes onto the desktop CDs (see my comment from 2006-11-23 20:02:29 CET)? Thanks.

On Fri, Nov 24, 2006 at 04:32:02PM -0000, Till Kamppeter wrote:
> linuxprinting.org-ppds and linuxprinting.org-ppds-extra do purely
> contain PPDs as provided from the printer manufacturers, no printer
> descriptions nor any other documentation. Because of its use size I have
> split it off, so that the most important PPDs reside in the 3.5 MB
> linuxprinting.org-ppds package and can go onto the desktop CDs.

By 'descriptions' I meant the PPDs ("PostScript Printer Descriptions", no?).
They are up to 100k each and there are hundreds of them.

Don't we have a facility to autogenerate them from a more concise form?

--
 - mdz

Some of these files are identical other than the printer names contained
within them. The package could easily be made significantly smaller.

The data for autop-generating PPDs is installed by the foomatic-db and foomatic-db-hpijs binary packages. This XML data is much smaller than the PPDs it generates. The linuxprinting.org-ppds and linuxprinting.org-ppds-extra packages are PPDs supplied by printer manufacturers. Most of them are for PostScript printers and they are exactly the same PPD files as they are used for Windows and Mac. They contain a lot of options and PostScript code to get the maximum out of the printers, the printers work with them exactly as under Windows or Mac OS.

Ricoh makes even dedicated PPDs for Linux, so that more advanced features get accessible to the Linux users, and they always keep their set of PPDs complete and up-to-date.

At Red Hat they thought already about converting PPDs into Foomatic data, but this did not work out as the Foomatic XML data format does not have equivalents for all types of items you can find in PPDs.

CUPS unfortunately does only support individually compressing each PPD witth gzip, not very efficient. Therefore PPDs appear in distributions in this form currently.

One idea would be to make use of CUPS's PPD auto-generation feature to make PPD packages and disk occupation by PPDs smaller:

1. Uncompress all PPDs (*.ppd.gz -> *.ppd)

2. Create an index file of the PPDs with output similar to the output of /usr/lib/cups/driver/foomatic list

3. Pack them all into a bzip2-compressed tarball, index file first

tar -cvjf ppds.tar.bz2 index.txt /usr/share/ppd/linuxprinting.org/

4. Put a simple script into /usr/lib/cups/driver/ which does

a. when called with "list": extracts the index from the tarball and drops it to STDOUT

b. when called with "cat <scriptname>:<ppdname>": extract the PPD <ppdname> and drop it on STDOUT

This way all PPDs get transparently available as on a physical file system, but occupy much less space on the live CD, the hard disk, or in a .deb package.

I could implement this as a build option in foomatic-db and then we could provide the PPDs in this mode.

This naturally only works out if

- tarball packaging makes the PPDs really substantially smaller

- Use of a compressed file system on live CDs does not make the changes in the compression useless.

WDYT? Should I file a spec or a wishlist bug on that?

Another idea would be having a look into CUPS DDK (http://www.cups.org/ddk/). CUPS DDK generates one or more PPDs from simple source files (.drv) with the "ppdc" utility. It also has a "ppdi" utility to make .drv files from PPDs. In addition it can join several PPDs for the same printer but different languages to one multi-language PPD. If PPD -> .drv -> PPD is loss-less, one could provide all PPDs as .drv files and let CUPS generate PPDs on the fly.

It seems that the idea with the bzip2-compressed tarball really saves space. I have downloaded

http://www.linuxprinting.org/download/foomatic/foomatic-filters-ppds-current.tar.gz

which is a 11-MB package full of PPD files. Then I gunzipped it and applied "bzip2 -9" on it the result is the following:

[root@majax c]# ls -l foomatic-filters-ppds-current.tar.bz2
-rw-r--r-- 1 root root 3201572 Nov 24 20:06 foomatic-filters-ppds-current.tar.bz2
[root@majax c]#

So we have only 3.2 MB instead of 11 in a gzipped tarball (and individually gzipped it is probably even bigger).

On Fri, Nov 24, 2006 at 06:52:11PM -0000, Till Kamppeter wrote:
> It seems that the idea with the bzip2-compressed tarball really saves
> space. I have downloaded

What about Matthew's suggestion to take advantage of PPDs which are
functionally equivalent, and only ship one copy?

--
 - mdz

Unfortunately, in the PPD concept there must be one PPD for each printer model, even if two different printers work with exactly the same driver and the same options. APPD header cannot contain 2 or more printer model identities.

A possible solution would be again an on-the-fly PPD generator for CUPS 1.2, but this time it would be more complex than the bzip2-tarball one I suggested ealier.

At first one would have to analyze the PPDs by comparing them with each other and find groups of similar/near equal PPDs. In each group one conserves one complete PPD and provides diffs to this one for the other PPDs. In addition, an index file is made as already mentioned above, but also containing infoabout which PPDs are provied as entire PPDs and which as diff. All this is packed into a bzip2ed tarball again.

Now if a PPD is requested, and is only available as diff, the diff is applied to the appropriate master PPD.

The effectiveness of this depends very much on the algorithms used to find and group similar PPDs.

Martin Pitt has updated the sseds now to take the new linuxprinting.org-ppds package (with the most important PPDs) onto the Feisty desktop CDs. Therefore I mark the ubuntu-meta case as fixed.

The further discussion in this report is very valuable to find a better solution, but the implementation would happen in the upstream foomatic-db package. Therefore I have marked this issue as an issue of foomatic-db upstream and I will link it from the Foomatic TODO list on linuxprinting.org.

To try out the system behaviour with all the PPDs in a bzipped tarball and an appropriate PPD generator (see one of my previous comments) I have created the following script:

http://www.linux-foundation.org/~till/tmp/compressppds

Download it and make it executable. Then install the linuxprinting.org-ppds and the linuxprinting.org-ppds-extra packages.

Do

time lpinfo -m > x
du -h /usr/share/ppd/postscript/linuxprinting.org
sudo ./compressppds postscript/linuxprinting.org/ openprinting-ppds /usr/share/ppd
time lpinfo -m > x
ls -l /usr/share/ppd/openprinting-ppds.tar.bz2
rm x

You see now that this saves a lot of space (3.6 MB instead of 18 MB on current Feisty), but it takes much more time to generate the PPD list (this is only extracting the .ppdlist file from the tarball, but even if .ppdlist is in the beginning the whole tarball gets read). On my box the first "lpinfo -m" command needs 6 sec and the second 19 sec.

To return to the old state do

cd /usr/share/ppd
sudo tar -xvf openprinting-ppds.tar.bz2
sudo rm postscript/linuxprinting.org/.ppdlist
sudo find postscript/linuxprinting.org -name '*.ppd' -type f -exec gzip -9 '{}' \;
sudo rm /usr/lib/cups/driver/openprinting-ppds

So the only problem of this is that tar reads the tarball always to the end, even if one extracts the very first file. Is there a method to make tar stop once it has found the requested file?