[needs-packaging] ibmpmlinux

*** This is an automated message ***

This bug is tagged needs-packaging which identifies it as a request for a new package in Ubuntu. As a part of the managing needs-packaging bug reports specification, https://wiki.ubuntu.com/QATeam/Specs/NeedsPackagingBugs, all needs-packaging bug reports have Wishlist importance. Subsequently, I'm setting this bug's status to Wishlist.

Status changed to 'Confirmed' because the bug affects multiple users.

Canonical,

This is an IBM proprietary software that we want to purse adding at Canonical partner archive for the 15.10 timeframe. There are package specific and we want to hear from you if this is acceptable, or, if we need to do something different in order to make the partner archive.

Adding some clarifying info per call with Michael Hohnbaum on 7/02:

This is proprietary package for PowerVM performance monitoring. Request is to include this package in the Partner Archive for 15.10 timeframe.

Waiting on Canonical to review this package.

Hi Mathieu,
Let me know if you need something missing here to start the review.

Hi Erwan,

No, there's nothing else required.

I've uploaded it the the archive queue, for review by an admin.

Mathieu, thank you very much for your response with an encouraging progress update.
Please tell us the detailed time line (schedule) of Ubuntu 15.10. We may provide new up-to-date tar balls to be built in before the release.

lcguo, the release schedule is available here: https://wiki.ubuntu.com/WilyWerewolf/ReleaseSchedule

Should there need to be a new version please file a new bug report, since we're in Feature Freeze, it will need to be approved by the release team, as per the documentation here: https://wiki.ubuntu.com/FreezeExceptionProcess

As for bugs (or a bugfix-only release), they don't need a freeze exception given they're just bugs :)

Hi Mathieu,

As I just checked today, ibmPMLinux package was still NOT archived in Ubuntu 1510's daily builds (http://cdimage.ubuntu.com/ubuntu-server/daily/current/wily-server-ppc64el.iso). Accordingly, I have not yet got a chance to test PMLinux package being installed from Ubuntu iso.

The time seems running out of the schedule for PMLinux to join Ubuntu 1510. I am wondering how the acceptance procedure is going on? Please let me know once green light is given from the review. There will be new PMLinux tar balls with recent feature enhancement/bug fix for update.

Getting ibmpmlinux on the ISOs is quite different from having it packaged in the archive. It will *not* initially be on the isos. There isn't currently any issue that I know of for ibmpmlinux, I'm just waiting for an archive administrator to review it and let it through. I've already asked Adam Conrad to look at it again this morning.

As for an update, it should be done as another bug report, for which there should indeed be another process of accepting the *update*, but once the version currently in the queue gets through, that's all that would be necessary for the upload.

Right, and I do agree that we don't want it to be part of the ISO at the moment.

So, ibmpmlinux requires major reworking to follow the Debian (and Ubuntu) policy for packages. I've done most of the work here to clean it up, but there still would be issues at least in the documentation, which is extensive and often mentions /var/perf.

Will PMLinux still work properly if we move all the binaries to /usr/lib/ibmpmlinux, for example? This would bring the location of files (at least for the binaries) in line with the FHS. I've updated PMLinux.cfg accordingly (and I suggest it goes to /etc/ibmpmlinux).

I'll attach here the current state of the packaging.

With all these files, you can use dpkg-source -x ibmpmlinux_3.1.0-0ubuntu1.dsc to rebuild the upstream directory with debian/; using debuild to build a source package.

As mentioned earlier, now the issue with this, while it uses FHS paths, is that a lot of the documentation still points users to /var/perf which may be confusing. Could we please have a new copy of the documentation with paths changed?

Hi Mathieu,

Thank you a lot for your time spent in reviewing and rearranging PMLinux's dirs/files layout.

Actually, Breno, Erwan and I etc. already discussed about FHS (FileSystem Hierarchy Standard) enforcement several weeks ago. The right home for PMLinux seemed to be at /opt/ibm/pm/. Accordingly, I had ever created a PMLinux package based on that new home (see middle column in the attachment). However, that newly built PMLinux package did not pass lintian’s validation either, with almost as many non-compliance complaints as for PMLinux's original .deb. In other words, lintian did not really comply with the FHS standard for itself!

Regarding your suggestion at a different location /usr/lib/ibmpmlinux/. I am afraid it might still not satisfy lintian because this path is also unknown to lintian. If that has no problem, I'd like to change to /usr/lib/ibm/pm/ for possible compatibility issues in case other IBM products may face the same frustration in future. Please review the detailed layout at 3rd column in the attachment.

In order to make PMLinux work as before, a few symbol links need to be created by PMLinux installation scripts (see bottom part in the attachment). If the links, such as /var/perf/pm pointing to PMLinux's new home, are not allowed, then PMLinux requires lots of modifications to be adapted for new layouts.

Personally, I still prefer PMLinux’s current dirs/files layout (see 1st column in the attachment). As I think, FHS’s distribution style is only good for the essential kernel/utilities of Linux system. It is overdone for Ubuntu to discipline any independent applications with FHS, because not every product acts/sits very close to core system, and mixing up their files with others makes simple things complicated. Instead, small tree layout (Windows style) has its tremendous conveniences.

"Will PMLinux still work properly if we move all the binaries to /usr/lib/ibmpmlinux, for example?"
Yes, but only if PMLinux (programs) can manage to locate PMLinux.cfg. PMLinux was designed to find the parameters file in this sequence: as specified by command line option, in subdirectory "config" under the program's parent directory, or in default directory "/var/perf/pm/config". If PMLinux.cfg not found (like sitting inside /etc without a link reference), PMLinux will run at /var/perf/pm with hard-coded default options.

"Could we please have a new copy of the documentation with paths changed?"
As commented above, if we are allowed to create a few symbol links to regain the original layout, then nothing in the documents needs correction. Otherwise, many codes & documents need to be changed and tested – There will be no quick answer in this tough circumstance.

I'm fine with putting everything under /opt as well, I simply picked /usr because other parts of the documentation (manpages) already shipped under /usr --- if we should ship under /opt, then it's perfectly doable but we should ship *everything* in /opt paths (this includes /var/opt or /etc/opt as appropriate). In this sense, it seems to be your "proposal 1" would work.

If you already have the packaging, could you send it to me (or attach it here) so I could look at it and figure out why lintian is unhappy?

Also, should changing PM_HOME to point to the new directory instead of /var/perf/pm work instead of creating symlinks? The issue we have is specifically with adding directories or files that don't match any FHS spec.

"should changing PM_HOME to point to the new directory instead of /var/perf/pm work instead of creating symlinks?"

No, it won't work. PMLinux.cfg will not take effect because it is now separated from PMLinux binaries, and NOT at default searching locations.

The solution is either one below:
1) Create some symbol links to fully regain the original layout
     So that we still run programs in /var/perf/pm/, but it becomes running them in /opt/ibm/pm/, while /opt/ibm/pm/bin/programs reading PMLinux.cfg in /opt/ibm/pm/config/ becomes finding it in /etc/opt/ibm/

2) Reprogram PMLinux with distributed FHS layout
    Change PMLinux programs to find PMLinux in /etc/ibm/pm/
    Then also need to correct documents to reflect the new locations.

Of course, I prefer 1), and only if the original implementation is not an option.

I will discuss this with an archive admin to see if having links is allowable.

Thanks Mathieu. There are only two "public" symbol links in controversy:
1) /var/adm/perfmgr -> /var/perf/pm/
    HMC/FSM/ESA services use it to find PMLinux data file for transmission.
2) /var/perf/pm -> /opt/ibm/pm/
    PMLinux refers it to regain original files layout (virtual ) against Ubuntu FHS enforcement.

Other three links involved are local (or "private"), and product owner should have flexibility to do so:
/opt/ibm/pm/tmp -> /var/opt/ibm/pm/tmp/
/opt/ibm/pm/daily -> /var/opt/ibm/pm/daily/
/opt/ibm/pm/config -> /etc/opt/ibm/pm/

As you can see here, FHS compliance only has negative impact on PMLinux and makes simple files layout complicated. There is no evident advantage for installing PMLinux under /opt/ibm/pm than in /var/perf/pm.

It is a condition of distribution in partner that things are policy-compliant. This means packages, unless they have a very good reason to, should ship files in Debian-policy compliant directories (this means following the FHS precisely and shipping in /usr/lib/$package, or even better, shipping files in the typical locations: /usr/bin or /usr/sbin for binaries, /usr/share/man for manpages, /etc for configuration, /run or /var/lib or other locations for temporary or "permanent" runtime files, etc.). The /opt hierarchy is fine too, provided that *everything* the package ships is under /opt directories, or /etc/opt or /var/opt (as per FHS).

Would it be possible to recompile just to replace /var/perf/pm with /opt/ibm/pm? Or even better, update the program to be fully FHS-compliant and not expect any particular hard-coded path? This way we could have a PMLinux.cfg in a reasonable location and use it to guide binaries as to where to find what they need, or have the binaries "just work" without any config file present?

One further issue with relocation to /opt remains the manpages; they really should go to /opt too if the binaries are there, and this poses further problems. We don't have automated man path detection; from what I gather from /etc/manpath.config, files should be placed in /opt/man/... so as to match binaries from /opt/sbin or /opt/bin; and don't really go looking down into /opt/ hierarchies. This would need testing to make sure the manpages properly work where they'd be placed.

Thank you Mathieu.
Q: "Would it be possible to recompile just to replace /var/perf/pm with /opt/ibm/pm? "
A: Although technically possible, it might raise questions on the viability of PM-Ubuntu project:
   1) For a FHS-compliant system, it is supposed to store "static" things only in /opt which may then be mounted for read only;
   2) There will be awful consequences to move home for PM (for IBM i, AIX, Linux, KVM ...) after we released the product more than a decade.

Q: "This way we could have a PMLinux.cfg in a reasonable location and use it to guide binaries ..."
A: Impossible. Program always runs first before configuration takes effect. It's the location of binaries that matters and guides where to find the configuration file, then knows where to store data and where to locate other things (such as sub-programs, text documents, temporary locks/buffers etc.).

Q: "or have the binaries 'just work' without any config file present?"
 A: Yes, PM can works well with .cfg file absent. In this circumstance, PM must run in default patterns. However, if config file does not exist, there is no way to control PM behaves differently (off the defaults), particularly, under the specific directories layout on Ubuntu.

Here I update the proposal (version 1.1, right sheet in the attachment) for your review for PM-Ubuntu with the files of static in /opt/ibm/pm and of dynamic in /var/opt/ibm/pm, except the manual's (static) still in the traditional place /usr/share/man.

Le 2015-09-23 19:18, Guo, Liang Chang a écrit :
> Thank you Mathieu.
> Q: "Would it be possible to recompile just to replace /var/perf/pm with /opt/ibm/pm? "
> A: Although technically possible, it might raise questions on the viability of PM-Ubuntu project:
> 1) For a FHS-compliant system, it is supposed to store "static" things only in /opt which may then be mounted for read only;
If this is truly an issue for you, storing files in the proper paths in
/usr/lib, and keeping just the very few binaries that are expected to
ever be run by users in /usr/bin, is quite acceptable: we've already
established that. The only thing that is being objected to is the use of
paths in /var/perf; which isn't FHS-compliant.

As I've expressed before, it is a requirement for packages in the
archive to follow Debian and Ubuntu packaging policies.

> 2) There will be awful consequences to move home for PM (for IBM i, AIX, Linux, KVM ...) after we released the product more than a decade.

Why? There is no consequence in keeping software in the generally
accepted locations that are both time-tested and widely adopted.

> Q: "This way we could have a PMLinux.cfg in a reasonable location and use it to guide binaries ..."
> A: Impossible. Program always runs first before configuration takes effect. It's the location of binaries that matters and guides where to find the configuration file, then knows where to store data and where to locate other things (such as sub-programs, text documents, temporary locks/buffers etc.).

On the contrary, this is why I am asking whether you can recompile these
binaries. To know where to find their files, these programs must have it
hard-coded somewhere. This means this value can be changed.

>
> Q: "or have the binaries 'just work' without any config file present?"
> A: Yes, PM can works well with .cfg file absent. In this circumstance, PM must run in default patterns. However, if config file does not exist, there is no way to control PM behaves differently (off the defaults), particularly, under the specific directories layout on Ubuntu.
>
> Here I update the proposal (version 1.1, right sheet in the attachment)
> for your review for PM-Ubuntu with the files of static in /opt/ibm/pm
> and of dynamic in /var/opt/ibm/pm, except the manual's (static) still in
> the traditional place /usr/share/man.

I thought I had mentioned it in the previous comment, but maybe I
forgot: there's an additional issue with keeping manual pages in
/usr/share/man if the rest is shipped in /opt. Packages shipping things
in /opt should ship *everything* in /opt, not pick and choose.

While I agree I've given you both options, it seems as though with all
the coming data, it is likely best to ship ibmpmlinux files in proper
/usr tree instead, such as in the initial packaging I provided for
review in the bug (which was still affected by the binaries' requirement
for arbitrary paths).

--
Mathieu Trudel-Lapierre <email address hidden>
Freenode: cyphermox, Jabber: <email address hidden>
4096R/DC95CA5A 36E2 CF22 B077 FEFE 725C 80D3 C7DA A946 DC95 CA5A

>> Q: "Would it be possible to recompile just to replace /var/perf/pm with /opt/ibm/pm? "
>> A: Although technically possible, it might raise questions on the viability of PM-Ubuntu project:
>> 1) For a FHS-compliant system, it is supposed to store "static" things only in /opt which may then be mounted for read only;

> If this is truly an issue for you ...

No, I don't mind at all. It is merely an issue for FHS which requires that only static stuff may be stored in /opt.

> storing files in the proper paths in /usr/lib, and keeping just the very few binaries
> that are expected to ever be run by users in /usr/bin, is quite acceptable

The thing is, IBM has PMLinux, ESA(Electronic Service Agent), RSCT(Reliable Scalable Cluster Technology) ... As I knew:
ESA's traditional home is at /opt/ibm/esa/, and
RSCT team has been struggling to move away from /usr/sbin/rsct/ for FHS compliance on Ubuntu.

I hope RSCT's home to be decided at /opt/ibm/rsct/, then many IBM optional packages may stay closely.

> The only thing that is being objected to is the use of paths in /var/perf; which isn't FHS-compliant.
> As I've expressed before, it is a requirement for packages in the archive to follow Debian and Ubuntu packaging policies.

That is fine and we shall comply with. What we keep asking is only to reserve that old address as a relocation sign for some moments.

>> 2) There will be awful consequences to move home for PM (for IBM i, AIX, Linux, KVM ...) after we released the product more than a decade.
> Why?

Worry of big trouble in cross-platform customer support

> On the contrary, this is why I am asking whether you can recompile these binaries.
> To know where to find their files, these programs must have it hard-coded somewhere. This means this value can be changed.

Technically, we just need to add an extra path in the list for configuration search. This is not a big deal in code/document revision.
The major issue remains in cross-platform technical support.

> I thought I had mentioned it in the previous comment, but maybe I forgot: there's an additional issue
> with keeping manual pages in /usr/share/man if the rest is shipped in /opt. Packages shipping things
> in /opt should ship *everything* in /opt, not pick and choose.

This "everything in" condition in FHS standard is certainly ridiculous:
First of all, Ubuntu itself has not yet complied with it: "man" does not look into /opt/man/ with current
/etc/manpath.config, while /opt/bin/ is not included in $PATH by default.

Secondly, we have evidences from PMLinux to feel it bad:
  1) when we have intimate files (like .cfg & .help) and naturally want to keep them at local,
     FHS forces to separate them far away;
  2) when we cooperatively distribute something (like PMUbuntu.8.gz) to a place acknowledged
     for system wide utilities (such as "man"), FHS still says no and we shall have to move them back to private.

> While I agree I've given you both options, it seems as though with all the coming data, it is likely
> best to ship ibmpmlinux files in proper /usr tree instead, such as in the initial packaging I provided
> for review in the bug (which was still affected by the binaries' requ...

> By the way, have you found why lintian complained PMLinux
> at /opt/ibm/pm/ ? (I sent you the tar balls on Sep 17 by email)

lintian assumes that the package is intended for inclusion in the distribution, so it expects it to comply with those parts of the FHS. Packages in the Ubuntu partner archive are allowed to ship in /opt instead (at their option), but lintian does not support analyzing such a package using the appropriate rules. The solution for this is to include lintian overrides in the package for the /opt path.

You are right that the FHS also expects writable files to be located in /etc/opt and /var/opt rather than in /opt. For applications that expect to find their files all in a self-contained directory tree, which is a common case for partner packages under /opt, the usual approach is to place the real files in /etc/opt or /var/opt and provide symlinks as necessary under /opt/<path>. It seems to me that this should meet your needs for PMLinux.cfg.

What is not acceptable is to ship files in new subdirectories under /var such as /var/perf, for precisely the same reason that you cite as problematic for placing user-writable files under /opt. The FHS is an interface contract with the admin that dictates what filesystem paths must be mounted read-write, what paths must be local vs. remote, what paths contain data that the admin will want to back up, and so forth. While the use of /opt is allowed for partner packages (but not for packages in the primary Ubuntu archive), the overall FHS policy regarding file placement must be conformed to.

Are there any technical blockers to placing the package's contents in /opt/ibm/pm, with symlinks to /etc/opt/ibm/pm and /var/opt/ibm/pm for the writable parts?

Thank you very much, Steve.

> ... but lintian does not support analyzing such a package using the appropriate rules.

Then, Ubuntu community should not reject a package destined for /opt, based on lintian's analysis output until that tool is fixed to comply with all FHS specifications.

> Are there any technical blockers to placing the package's contents in
> /opt/ibm/pm, with symlinks to /etc/opt/ibm/pm and /var/opt/ibm/pm
> for the writable parts?

Yes, but only Ubuntu system (iso) is the blocker. Administrator (root) can NOT run (& man) commands in /opt/(s)bin (& /opt/man) without the path because /opt/(s)bin are not included in $PATH by default when they exist. This is another Ubuntu bug (in user profile initialization) for unconditional enforcement of the FHS standard while the system is actually not ready for.

Note: 1) As I tested, SuSE has supported /opt/(s)bin in $PATH if the directories exist, at least since sles10; Fedora/Red Hat/Ubuntu are not yet;
      2) "man" will work fine because it was designed in the configuration to look into /opt/man once /opt/(s)bin appear in $PATH.

On Tue, Sep 29, 2015 at 08:58:45PM -0000, Guo, Liang Chang wrote:
> Thank you very much, Steve.

> > ... but lintian does not support analyzing such a package using the
> appropriate rules.

> Then, Ubuntu community should not reject a package destined for /opt,
> based on lintian's analysis output until that tool is fixed to comply
> with all FHS specifications.

We have not done so. The previous revision of the package has been rejected
*because it does not comply with the FHS*, not because lintian generates
warnings about it. The lintian output is a guide, not decisive.

> > Are there any technical blockers to placing the package's contents in
> > /opt/ibm/pm, with symlinks to /etc/opt/ibm/pm and /var/opt/ibm/pm
> > for the writable parts?

> Yes, but only Ubuntu system (iso) is the blocker. Administrator (root)
> can NOT run (& man) commands in /opt/(s)bin (& /opt/man) without the
> path because /opt/(s)bin are not included in $PATH by default when they
> exist. This is another Ubuntu bug (in user profile initialization) for
> unconditional enforcement of the FHS standard while the system is
> actually not ready for.

Sorry, but I really don't know what you're looking for. The previous
version of the package submitted for review shipped all files in the
/var/perf/pm directory, which means the admin would have to manually add
/var/perf/pm/bin to their path to use ibmpmlinux; but that didn't appear to
be a blocker for you at that time. Why should it be a blocker now that we
are asking for ibmpmlinux to be installed to /opt instead?

I acknowledge without reservation that the Ubuntu support for /opt is
suboptimal. But there is no work scheduled to address this; so treating
/opt/bin path integration as a blocker for ibmpmlinux is merely going to
cause delays in being able to include ibmpmlinux in the partner archive.
Isn't it preferrable to have this software in the partner archive, with or
without /opt/bin path integration, than to not have it in the archive?

> Why should it be a blocker now that we are asking for ibmpmlinux to be installed to /opt instead?

PMLinux has only one menu command for all of the user interfaces, "/var/perf/pm/config.PMLnx", which has a symbol link at /usr/sbin/config.PMLnx for root working well without $PATH setting.

Now, as per Mathieu's request "putting *everything* under /opt" " to fulfill FHS standard, we have to move that public link into /opt/sbin, which will ruin the interface access, so does PMLinux man help if we also relocate PMLinux.8.gz from /usr/share/man to /opt/man.

> Isn't it preferrable to have this software in the partner archive, with or
> without /opt/bin path integration, than to not have it in the archive?
Because Ubuntu is not ready for basic /opt support, what I hope is Ubuntu team still allows PMLinux continue to use /usr/sbin and /usr/share/man.

I've spent a few hours reviewing the package carefully; here are the few things we'll need at this point for the software to be in an appropriate form for inclusion in the Ubuntu partner archive:

- ship a sysv init script in /etc/init.d/ (which should work both for 15.10 and all the way to 14.04)
- ship all files in their paths under /opt/ibm/pm; except for PMLinux.cfg which should be under /etc/opt/ibm/pm, and /var/opt/ibm/pm/daily and /var/opt/ibm/pm/tmp for temporary/log files.
- the scripts ending in .PMLnx can be trivially changed to replace all occurences of "/var/perf/pm" with "/opt/ibm/pm" (the change also should happen in PMLinux.cfg).

We also don't appear to need the preinst, prerm, postrm files; since running verify.PMLnx or uninstall.PMLnx cause incompatible paths to be added on the filesystem, or crontab to be changed, which aren't allowable as per the packaging policy.

Hi Mathieu,
Thank you very much for your hours in the review. It sounds you almost get to the bottom of PMUbuntu.

> - ship a sysv init script in /etc/init.d/ (which should work both for 15.10 and all the way to 14.04)

This means we need to modify PMUbuntu to be a service (or job in upstart term). It's fine but will take some time.

> - ship all files in their paths under /opt/ibm/pm ...

We will follow the requests that you mentioned here, but not necessary to replace every "/var/perf/pm" with PMUbuntu's new home because we still try to maintain PM programs good for all Linux variants.

> We also don't appear to need the preinst, prerm, postrm files;

That those scriptlets are installed to /debian is not what we expect at all. The scripts are for package install/upgrade/uninstall moments only and supposed to be stored in apt/dpkg internal database.

> since running verify.PMLnx or uninstall.PMLnx cause incompatible paths to be added on the filesystem, or crontab to be changed, which aren't allowable as per the packaging policy.

If we decide to move home for PMUbuntu, then there will be no incompatible paths created by the final .PMLnx programs. PM item in crontab will also be relocated into /etc/cron.* as per your request.

Hi Mathieu,

The schedule for Ubuntu 15.10 to release is running out. Here we PMLinux team are wondering if there is time left for our package inclusion in Ubuntu 15.10 Archive repository? If time allows, we still want to submit a PMLinux update for review next week, with the major changes for FHS compliance included (detail enclosed in attachment), while leaving optional service/job enhancement for next Ubuntu release

Leaf packages (that aren't depended on by other things, and do not need to be on released images), such as ibmpmlinux may be acceptable up until the last few days before release (note however that the final release is next week). That said, the final decision will be made by an archive administrator.

To make sure ibmpmlinux can land in time for 15.10, please make sure you submit your package for review as soon as possible.

Hi Mathieu,

Thank you very much. Please review our new build ibmpmlinux_3.2.0-1.tar in the attachment which contains:
-rw-r--r-- root/root 19460 2015-10-22 11:39 ibmpmlinux_3.2.0-1.debian.tar.xz
-rw-r--r-- root/root 523655 2015-10-22 11:39 ibmpmlinux_3.2.0.orig.tar.gz
-rw-r--r-- root/root 390764 2015-10-22 11:39 ibmpmlinux_3.2.0-1_ppc64el.deb
for PMLinux Version 3.2.0-1 with all the necessary FHS compliance items implemented, as described in PMLinux compliance schedule.png in my note on 2015-10-13.

If possible, we still hope to see PMLinux package available in Ubuntu 15.10 archive.

- Version for Ubuntu should be -0ubuntu1 rather than -1.

- compat file should contain "9", not "5".

- for debian/control, please use the following line in place of the Depends: line for ibmpmlinux:
Depends: ${shlibs:Depends}, dialog (>= 0.8), bash (>= 3.0), ${misc:Depends}

- no need for preinst or postrm. Running verify or uninstall in postinst/prerm is most likely wrong.

- no application should change crontab for the root user by editing the file. If anything, it should instead ship a file in the right directories in /etc/cron.d

- rather than doing symlinks in postinst/prerm; the links should be created using dh_links, a file debian/links will have to simply contain the source and destination for the symlinks to create.

- please use debian/install containing the following, rather than copying files in debian/rules:
opt
etc
usr/share/man opt/

- you don't need file copies, creating directories or any such thing using override_dh_auto_build in debian/rules; instead, you can use the following:
override_dh_install:
 mkdir $(CURDIR)/debian/tmp
 tar -xpzf $(pkg_tgz) -C $(CURDIR)/debian/tmp
 dh_install

- It will be easiest for you to get the package name and version like this:
package := $(shell dpkg-parsechangelog -S Source)
version := $(shell dpkg-parsechangelog -S Version | sed -e 's/-.*//')
Note that here I pick just what goes in front of the - for the version number.

- please remove the line "override_dh_shlibdeps:" in debian/rules; this will fill in ${shlibs:Depends} for you automatically with the right version of libc.

- also, I suppose the PMLinux.8 manpage should be decompressed and let dh_installman deal with it in debian/rules, to get rid of this lintian warning:
W: ibmpmlinux: package-contains-timestamped-gzip opt/man/man8/PMLinux.8.gz

Hi Mathieu,

Thank you for your quick review and response. I am uploading the new tar file, with the following answers to your comments:

- Version for Ubuntu should be -0ubuntu1 rather than -1.

  The '1' in the PMLinux version "3.2.0-1" is an initial build serial number. PMLinux product is for all IBM-specific Linux platforms, some of them dislike the build number starts from '0', nor suggest a long repeated subfix (such as "-0ubuntu1") in their versioning. In fact, even on Ubuntu, I only see half of deb packages having "ubuntu" in their version strings.

- compat file should contain "9", not "5".

  Changed as suggested, although we are unwilling to tight any limitations.

- for debian/control, please use the following line in place of the Depends: line for ibmpmlinux:
Depends: ${shlibs:Depends}, dialog (>= 0.8), bash (>= 3.0), ${misc:Depends}

  Changed as suggested.

- no need for preinst or postrm. Running verify or uninstall in postinst/prerm is most likely wrong.

  That's the style converted into deb from rpm.
  It could have problem only when PMLinux is kicked in before its dependencies in system installation, which is unlikely for a leaf package.
  Sharing common programs is a good thing, so is what we do here to eliminate multiple copies existing in installation/upgrade and daily maintenance programs.

- no application should change crontab for the root user by editing the file. If anything, it should instead ship a file in the right directories in /etc/cron.d

  Again, PMLinux is compatible for all Linux variants. It has diverse options & features for users to choose. That program branches to change crontab will not be reached on Ubuntu by default.

- rather than doing symlinks in postinst/prerm; the links should be created using dh_links, a file debian/links will have to simply contain the source and destination for the symlinks to create.

  Again, that's rpm style converted into deb. It's just a way of self integrity to eliminate any special dependencies on an installation environment.
  For deb, PMLinux package already contains the major symlinks in the files list, which can then be created by dpkg in installtion, and postinst is just to verify if dpkg or rpm does it right.

- please use debian/install containing the following, rather than copying files in debian/rules:
opt
etc
usr/share/man opt/

  Added debian/install with the above content, except " opt/" (which will ruin the man page feature under current Ubuntu releases, because they still do not support (add /opt/man in searching paths) any files in /opt/man/*)

- you don't need file copies, creating directories or any such thing using override_dh_auto_build in debian/rules; instead, you can use the following:
override_dh_install:
 mkdir $(CURDIR)/debian/tmp
 tar -xpzf $(pkg_tgz) -C $(CURDIR)/debian/tmp
 dh_install

 Yes, optimized as you suggested.

- It will be easiest for you to get the package name and version like this:
package := $(shell dpkg-parsechangelog -S Source)
version := $(shell dpkg-parsechangelog -S Version | sed -e 's/-.*//')
Note that here I pick just what goes in front of the - for the version number.

 No. dpkg-parsechangelog does not support "-S" o...

Recent PMUbuntu tar file contains a few enhancements in symbol links repair up to today (2015-10-27).

The package has vastly improved, but I could still find some issues.

changelog, control, compat, etc. Basically, anything other than rules and maintainer scripts don't need to be executable files. I'm not blocking the upload on that though.

manpages appear to be compressed including timestamps, this should be avoided if possible; through the use of the -n parameter to gzip. This allows us to make reproducible builds. Again, not blocking on that.

I'm merely blocking on getting a clear idea of why there are extra messages about crontab when installing:

Unpacking dialog (1.2-20140911-1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up dialog (1.2-20140911-1) ...
Setting up ibmpmlinux (3.2.0-0ubuntu1) ...
International Business Machines(IBM) Corporation, Rochester, MN, USA
Performance Management for LINUX - PMLinux Copyright 2012-2015 (C)
verifying PMLinux at /opt/ibm/pm flexibly !!!
/var/opt/ibm/pm/daily/wichita was created
PMLinux transmission link is there, pass
Link /usr/sbin/config.PMLnx exists, pass
Register PMLinux invocation in /etc/cron ... done(0)
no crontab for root
*** WARNING *** PMLinux contact email() is NOT entered yet!!!
PMLinux 3.2.0-1 verification completed.

There also shouldn't be the creation of a link to *anything* outside of /opt in postinst scripts, certainly not /usr/sbin/config.PMLnx. I've noticed there was now an /etc/cron.hourly/PMLinux which is a nice improvement.

What worries me here is, what would be the behavior if someone already has entries in root's crontab? Will an entry be created there then? There should never be a change to root's crontab from package maintainer scripts.

Finally, you shouldn't need most of the maintainer scripts. preinst and postrm appear to duplicate work that should already be done by the package manager, which should already correctly handle upgrades where the contents of a binary might have changed. These maintainer scripts should probably be removed, they seem to only be there due to the conversion process used to get the first deb package from the original RPM.

Hi Mathieu,

Thank you for your review to the detail. I now upload PMUbuntu files with a few minor changes regarding file mode & gzip -n that you suggested.

The crontab message is normal for an uninitialized spool crontab file of root. PMUbuntu's cron entry is required to be created at /etc/cron.*, so our installation scripts need to ensure PM cron entry does not exist in spool/crontab, and delete it if found. If someone already has other entries in root's crontab, then there will be no error from crontab querying, and the scripts can continue to do whatever needed (likely nothing on Ubuntu if nobody installed PMUbuntu and adjusted the settings before).

We still need /usr/sbin/config.PMLnx until Ubuntu can fully handle /opt. By now, the support of /opt is partially broken for all Ubuntu releases. For example, with default, root profile does not contain either /opt/man for manpage search, or /opt/(s)bin for executable search ($PATH).

Maintainer scripts are also necessary for now. There are some flexibilities provided by PMUbuntu that the hard-coded package installers (dpkg/rpm) can not manipulate, such as directory layout selection and crontab/configuration rollback.

Hi Mathieu,

Following my comment #36, now we have finished the work items of dark blue in the schedule: PMUbuntu starts to be installed with an init Service/Job script of sysV, upstart, or systemd. As a service/job, the PM Data Collector daemon will be started from system boot of runlevel 2-5. Please review at your convenience.

# tar tvf ibmpmlinux_3.2.1-1_20160122.tar
-rw-r--r-- root/root 19468 2016-01-22 19:32 ibmpmlinux_3.2.1-1.debian.tar.xz
-rw-r--r-- root/root 536035 2016-01-22 19:32 ibmpmlinux_3.2.1.orig.tar.gz
-rw-r--r-- root/root 401598 2016-01-22 19:32 ibmpmlinux_3.2.1-1_ppc64el.deb

Here I just quote the description regarding this feature from /opt/ibm/pm/README:

PMLinux SERVICE/JOB

    PMLinux started to be installed as a service/job since version 3.2.0-1. This is to support manipulating PMLinux process with the
    common Linux service/job management interface, such as "service PMLinux {start|stop|status}" etc. Meanwhile, PMLinux process will
    be started from the init-rc.d sequence in system booting for every multi-user run-level, and PMLinux's cron frequency is properly
    adjusted from "hourly" to "daily" with this startup method.

    New command "initservice.PMLnx" was initiated into PMLinux package for this purpose. By default, the program probes what methods
    (of sysV, systemd or upstart) are available on the system, and follows the best one to define/enable the service/job control code
    for PMLinux application accordingly in one of the following files:
         /etc/init.d/PMLinux # sysV varieties {LSBbase|rc_d|init_d}, all obsolete
         /etc/init/PMLinux.conf # upstart, popular for earlier Debian/Ubuntu releases
         /lib/systemd/system/PMLinux.service # systemd, the latest trend for all recent Linux releases

    In general, it is not supposed to run this program manually, unless you have to select a non-default method, or destruct PMLinux
    service/job encapsulation. Moreover, it is not blessed to handle PMLinux daemon with non-service/PMLinux tools, because that may
    mess up the process status inside the service/job utilities. Anyway, PMLinux certainly remains robust as before no matter how the
    control interface is configured and utilized.

    Here are some command examples under this topic:

    (A) Construct PMLinux service/job # on the check list of verify.PMLnx (run by command line or installation scripts)
    # initservice.PMLnx # create PMLinux init script with current method, or the best if missing/OFF
    Constructing PMLinux service on [IBMpKVM3.1.0.45.0::LSBbase,rc_d,systemd*] ...
    Created symlink from /etc/systemd/system/multi-user.target.wants/PMLinux.service to /usr/lib/systemd/system/PMLinux.service.
    Register PMLinux service/job ... done
    SUCCESS! (at /lib/systemd/system/PMLinux.service)
    Redirecting to /bin/systemctl start PMLinux.service

    (B) Change PMLinux service/job with another method
    # initservice.PMLnx -J rc_d # Do (A) with an old/obsolete method "rc_d" ("systemd" seems better)
    Reset PM_INIT_SERVICEJOB=rc_d in /var/perf/pm...

Re-did a review of ibmpmlinux. Some issues still apply. I have added more details. I'm sorry if this seems complicated, I realize we're having an issue with the language barrier. Please feel free to ask for more info, or that I explain things differently.

- Version for Ubuntu should be -0ubuntu1 rather than -1. This is important, since there isn't an ibmpmlinux package in Debian currently that we sync to ubuntu. Should there be, then it would be fine to ship a package with a version -1ubuntu1. The 'ubuntu' suffix denotes it is an upload for ubuntu.
If you must use an internal revision number following a - in the filename for the include source tarball, then let's modify debian/rules to account for this. Either debian/rules should go extract the specific file needed for the version, or it should avoid that entirely and just extract any tarball named ibmpmlinux_*.tgz
Even the architecture name there is irrelevant if there won't be a build for other architectures. This would mean it's possible to remove all the variables aside from $package.

- override_dh_clean: rm -rf there will not do anything, it is unnecessary, since there will not be an ibmpmlinux directory to remove (since things are extracted to debian/tmp, which will get cleaned up automatically.

- no need for preinst or postrm. Running verify or uninstall in postinst/prerm is most likely wrong.
It is just as easy (not to say easier) to not run them at all and ship the right files from the source of the package. Tuning configuration is fine, but we should not create new directories, symlinks, and such, since that short-circuits the package installation processes from apt and dpkg. They will handle package updates and removing files just fine.
In other words, if the source already ships all the right files (including a pre-made systemd service and reasonable default config), then all the maintainer scripts (postinst, preinst, prerm, postrm) can be removed. Otherwise, I would strongly recommend not keeping preinst and postrm. Postrm is completely unnecessary if the symlinks are done using dh_link (see below).

- rather than doing symlinks in postinst/prerm; the links should be created using dh_links, a file debian/links will have to simply contain the source and destination for the symlinks to create. This is much easier to maintain anyway, as you only need to list the source and destinations for the symlinks.

- also, I suppose the PMLinux.8 manpage should be decompressed and let dh_installman deal with it in debian/rules, to get rid of this lintian warning:
W: ibmpmlinux: package-contains-timestamped-gzip opt/man/man8/PMLinux.8.gz

This may or may not be an error, but it definitely *should* be investigated. We don't program lintian checks for no reason, but to catch specific issues that come up regularly in Debian packages, such as including timestamps in tarballs, which makes it impossible to have fully reproducible builds. See https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders

Hi Mathieu,

As my regard to your comment #45, I did some minor changes for your further review.
# tar tvf ibmpmlinux_3.2.2-1_20160302.tar
-rw-r--r-- root/root 19472 2016-03-02 16:25 ibmpmlinux_3.2.2-1.debian.tar.xz
-rw-r--r-- root/root 537644 2016-03-02 16:25 ibmpmlinux_3.2.2.orig.tar.gz
-rw-r--r-- root/root 402482 2016-03-02 16:26 ibmpmlinux_3.2.2-1_ppc64el.deb

Here are my responses to your detailed concerns:

- Version for Ubuntu should be -0ubuntu1 rather than -1. This is important, since there isn't an ibmpmlinux package in Debian currently that we sync to ubuntu. Should there be, then it would be fine to ship a package with a version -1ubuntu1. The 'ubuntu' suffix denotes it is an upload for ubuntu.

Answer: Adding a long string "ubuntu" in the version field is not a good idea.
        It unnecessarily expands a package file name, which could cause troubles everywhere.
        Particularly, PMLinux programs can work fine (compatible) for all Linux variants that we support (RedHat, SuSE, Fedora, Ubuntu, and PowerKVM).
        It seems no meaning to have PMLinux pakage with a "ubuntu" tag.

- If you must use an internal revision number following a - in the filename for the include source tarball, then let's modify debian/rules to account for this. Either debian/rules should go extract the specific file needed for the version, or it should avoid that entirely and just extract any tarball named ibmpmlinux_*.tgz

Answer: We have no special versioning need for PMLinux. The release-version format "r.r.r-v" that PMLinux adapted is popular for Linux platforms. By search, there exist more than a half pakcages without "ubuntu" in their version strings on Ubuntu 16.04 and apt/dpkg can handle properly at present. We hope to maintain the same if no change is mandatory.

- Even the architecture name there is irrelevant if there won't be a build for other architectures. This would mean it's possible to remove all the variables aside from $package.

Answer: We have other builds in rpm format for ppc64 Big/little Endians. The architecture mark is critical because PMLinux packages are "global" and "exchangable": For example, PMLinux's RPM package is supposed to work well on a Ubuntu system (if rpm tool installed).

- override_dh_clean: rm -rf there will not do anything, it is unnecessary, since there will not be an ibmpmlinux directory to remove (since things are extracted to debian/tmp, which will get cleaned up automatically.

Answer: Commented it out as per your suggestion. Thanks.

- no need for preinst or postrm. Running verify or uninstall in postinst/prerm is most likely wrong.

Answer: Not really. Those function blocks are necessary even if for a package that remains still after installed, not to mention that PMLinux is "dynamic" package. In PMLinux case:
           1) preinst is to backup the old daemon programs for comparisons later in postinst for whether to restart the process(es) in upgrade.
           2) postrm is to remove PMLinux's temporary data storage and flexible configurations etc generated/changed after installed that apt/dpkg are not aware of.
        Nothing above is handlable without those scriptlets.
        Apt/Dpkg c...

Here is an upgrade of PMUbuntu with Debian support included:
# tar tvf ibmpmlinux_3.4.1-1_20160916.tar
-rw-r--r-- root/root 19848 2016-09-16 12:13 ibmpmlinux_3.4.1-1.debian.tar.xz
-rw-r--r-- root/root 536235 2016-09-16 12:13 ibmpmlinux_3.4.1.orig.tar.gz
-rw-r--r-- root/root 402410 2016-09-16 12:14 ibmpmlinux_3.4.1-1_ppc64el.deb