Please add Userspace Packages for NVDIMM support

I have updated the packages in my PPA at https://launchpad.net/~nacc/+archive/ubuntu/nvdimm

Please test and report back.

Update from NVML Team Testing from Marcin Slusarz:

Several issues found, some critical. List in severity order:
1)Libpmem-dev is not installable, because one of the files also exists in libpmem.

2)Libpmemobj-dev package doesn't install all C header files, making it completely useless.

3)Libpmemobj C++ headers and documentation are not packaged at all.

4)Librpmem is not packaged.

5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do not depend on libpmem-dev (pkg-config files won't work correctly without libpmem-dev).

6)Libraries are installed to /usr/lib instead of /usr/lib/x86_64-linux-gnu/.

7)Pkg-config files in all -dev packages point to /usr/local.
This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.

8)"NVML library" (NVM library library) in package descriptions looks a bit silly.

9)Homepage points to github, instead of http://pmem.io/pmdk/.

Thank you for the feedback, I will work on getting this resolved as
soon as I can.

-Nish

On Fri, Mar 2, 2018 at 9:39 AM, <email address hidden>
<email address hidden> wrote:
> Update from NVML Team Testing from Marcin Slusarz:
>
> Several issues found, some critical. List in severity order:
> 1)Libpmem-dev is not installable, because one of the files also exists in libpmem.
>
> 2)Libpmemobj-dev package doesn't install all C header files, making it
> completely useless.
>
> 3)Libpmemobj C++ headers and documentation are not packaged at all.
>
> 4)Librpmem is not packaged.
>
> 5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do
> not depend on libpmem-dev (pkg-config files won't work correctly without
> libpmem-dev).
>
> 6)Libraries are installed to /usr/lib instead of /usr/lib/x86_64-linux-
> gnu/.
>
> 7)Pkg-config files in all -dev packages point to /usr/local.
> This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.
>
> 8)"NVML library" (NVM library library) in package descriptions looks a
> bit silly.
>
> 9)Homepage points to github, instead of http://pmem.io/pmdk/.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1752378
>
> Title:
> Please add Userspace Packages for NVDIMM support
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+bug/1752378/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: distribution=ubuntu; sourcepackage=None; component=None; status=In Progress; importance=Medium; <email address hidden>;
> Launchpad-Bug-Tags: needs-packaging
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: bladernr nacc pragyan
> Launchpad-Bug-Reporter: Jeff Lane (bladernr)
> Launchpad-Bug-Modifier: <email address hidden> (pragyan)
> Launchpad-Message-Rationale: Assignee
> Launchpad-Message-For: nacc

Update for NDCTL Testing:
1. The ndctl package is missing the man pages.
2. The libraries and the development packages should be split into separate sub-packages i.e. ndctl, libndctl, libndctl-devel, daxctl, libdaxctl, and libdaxctl-devel

On Fri, Mar 2, 2018 at 9:39 AM, <email address hidden>
<email address hidden> wrote:
> Update from NVML Team Testing from Marcin Slusarz:
>
> Several issues found, some critical. List in severity order:
> 1)Libpmem-dev is not installable, because one of the files also exists in libpmem.

Fixed, I think. (all the -dev packages had this problem)

> 2)Libpmemobj-dev package doesn't install all C header files, making it
> completely useless.

Fixed.

> 3)Libpmemobj C++ headers and documentation are not packaged at all.

Fixed for the headers, I need to figure out the documetnation bit still.

> 4)Librpmem is not packaged.

Fixed

> 5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do
> not depend on libpmem-dev (pkg-config files won't work correctly without
> libpmem-dev).

Fixed

> 6)Libraries are installed to /usr/lib instead of /usr/lib/x86_64-linux-
> gnu/.

I think fixed.

> 7)Pkg-config files in all -dev packages point to /usr/local.
> This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.

Can you point me at the commit in master? I'm not seeing it.

> 8)"NVML library" (NVM library library) in package descriptions looks a
> bit silly.

Fixed (what verbiage does Intel actually want?)

> 9)Homepage points to github, instead of http://pmem.io/pmdk/.

Fixed

On Mon, Mar 5, 2018 at 9:30 AM, <email address hidden>
<email address hidden> wrote:
> Update for NDCTL Testing:
> 1. The ndctl package is missing the man pages.

Fixed.

> 2. The libraries and the development packages should be split into separate sub-packages i.e. ndctl, libndctl, libndctl-devel, daxctl, libdaxctl, and libdaxctl-devel

Fixed.

NVML:
1) Nope, for old packages only libpmem had this problem. Now it's even worse - none of the packages contain .so used for linking.
3) "detail" directory is still missing.
7) Already backported to stable-1.3 branch, will be part of 1.3.2: https://github.com/pmem/pmdk/commit/58988474ed0b3618e90d328dd1210ffd3490f211
8) I don't care that much as long as it makes sense :)
You can see what we came up with here: https://github.com/pmem/pmdk/blob/stable-1.3/utils/build-dpkg.sh
"NVML library" was a common mistake. In 1.4 this will no longer be a problem, because we renamed NVML to PMDK (Persistent Memory Development Kit)

On Tue, Mar 6, 2018 at 2:50 AM, Marcin Ślusarz
<email address hidden> wrote:
> NVML:
> 1) Nope, for old packages only libpmem had this problem. Now it's even worse - none of the packages contain .so used for linking.

Fixed

> 3) "detail" directory is still missing.

Should be fixed.

> 7) Already backported to stable-1.3 branch, will be part of 1.3.2: https://github.com/pmem/pmdk/commit/58988474ed0b3618e90d328dd1210ffd3490f211

Backported to the package.

> 8) I don't care that much as long as it makes sense :)
> You can see what we came up with here: https://github.com/pmem/pmdk/blob/stable-1.3/utils/build-dpkg.sh
> "NVML library" was a common mistake. In 1.4 this will no longer be a problem, because we renamed NVML to PMDK (Persistent Memory Development Kit)

Not sure why "NVML ... library" is better than "NVML library for ..." :)

1) Still nope.
Non-dev packages now also include .so files...

10) New issue: libpmem-dev now contains all libraries in nvml_dbg directory.

On Wed, Mar 7, 2018 at 8:52 AM, Marcin Ślusarz
<email address hidden> wrote:
> 1) Still nope.
> Non-dev packages now also include .so files...
>
> 10) New issue: libpmem-dev now contains all libraries in nvml_dbg
> directory.

I think these are resolved with ~ppa5.

Hi,

we switched over to a team-owned PPA:

https://launchpad.net/~canonical-server/+archive/ubuntu/nvdimm/

ppa:canonical-server/nvdimm

Please use this one from now on. At this moment, they contain the latest debs from Nish that I just re-uploaded there.

I'm going over the lintian warnings now, which include some of the issues raised already during testing.

New ndctl upload to the canonical-server PPA at https://launchpad.net/~canonical-server/+archive/ubuntu/nvdimm/:

ndctl (59.2-0ubuntu1~ppa3) bionic; urgency=medium

  * Install pkgconfig files only in the -dev library packages
  * Don't install .la files, pkgconfig is preferred
  * lib*.so go into -dev packages, lib*.so.* into non-dev
  * Update long description in -dev packages
  * Append major soname version to non-dev library packages
  * Other lintian fixes

Please test.

I'll work on the nvml package now.

I just pushed nvml_1.3.1-0ubuntu1~ppa6 to the ppa:
 nvml (1.3.1-0ubuntu1~ppa6) bionic; urgency=medium
 .
   * updated package descriptions
   * disable rpath
   * remove trigger for man-db
   * lintian: change location of bash completion file

Please test.

A big chunk of the remaining lintian errors are about copyrights. I'll work on that next, and any feedback you may have so far.

Hi, a question. What are the files under /usr/lib/$arch/nvml_dbg/ needed for? They have debugging symbols? If yes, we already generate debugging symbols in the ddeb packages, like:

$ dpkg --contents librpmem-dev-dbgsym_1.3.1-0ubuntu1~ppa6_amd64.ddeb
drwxr-xr-x root/root 0 2018-03-21 20:00 ./
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/debug/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/debug/.build-id/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/debug/.build-id/62/
-rw-r--r-- root/root 75336 2018-03-21 20:00 ./usr/lib/debug/.build-id/62/0abdc961a5cf5aa01918b22286ba0cfb6de2d0.debug
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/share/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/share/doc/
lrwxrwxrwx root/root 0 2018-03-21 20:00 ./usr/share/doc/librpmem-dev-dbgsym -> librpmem-dev

Files under nvml_dbg are builds with debugging symbols, logging, asserts and expensive checks that we normally don't want users to run with.

Would you expect end users to need the files from that nvml_dbg directory?

In the current packaging, these go into the corresponding libfoo-dev packaages, but that's not really correct from a packaging perspective. At most they should be in some new package, something like libfoo-extra-debugging-dev perhaps, but I found no precedence for such a thing.

Ubuntu already produces packages with debugging symbols for all builds, so that aspect is covered and wouldn't be lost. What these won't have is the extra logging, asserts and expensive checks that you mentioned, which I assume cannot be enabled at runtime with an environment variable for example.

I guess what I am asking is if we can drop the nvml_dbg directory from the dev packages :)

I understand your concern, but I'd like to ask you to not remove it yet, because:
- it would make responding to bug reports and support requests a bit harder
- shipped man pages mention those libraries

But I think you are right, so I opened an issue with the proposal to get rid of nvml_dbg/pmdk_dbg directory:
https://github.com/pmem/issues/issues/841

It seems almost all problems were resolved. Thanks.
The only missing part is libpmemobj++ documentation.

Ok, adding docs.

Just pushed nvml 1.3.1-0ubuntu1~ppa7 to the PPA:

nvml (1.3.1-0ubuntu1~ppa7) bionic; urgency=medium

  * Add a lintian override for the manpages.
  * Add a lintian override for dev packages regarding ldconfig because of
    the _dbg subdirectory.
  * Install docs for libpmemobj and fix its directory to match the package
    name

The ldconfig lintian override is about the _dbg directory we talked about. I don't know yet if an Archive Admin will accept that, we will find out. I added links to this bug and to the issue you opened in github.

I also renamed the libpmemobj++ documentation directory. The package is named libpmemobj-dev, but make install was installing the docs to /usr/share/libpmemobj++-dev/.

I'm going to check the debian/copyright file on ndctl now, I think it needs a bit of work. I've seen at least one case where a file has an lgpl license but d/copyright claims it's gpl.

Alternatively you could add separate package for C++ headers, pkg-config file and documentation.

You mean, create another set of packages for the *_dbg/ files? That would mean a mirror package for every libfoo<soname> and libfoo-dev package, right?

I suggested it in the context of "libpmemobj++ documentation directory" problem.

I pushed another ndctl build to the ppa, but the only change is an update to the debian/copyright file:
ndctl (59.2-0ubuntu1~ppa4) bionic; urgency=medium

  * d/copyright update

Hi,

it just occurred to me that we never had a comment in this bug about the packaged software actually works correctly with the hardware it was meant for.

Could you please add a comment here saying that these packages are working correctly with your hardware? And that the devel packages are correct in the sense that you can link your applications using them?

Thanks

The ndctl package is working for me, I verified listing namespaces and converting namespaces between a few different modes.

The only cosmetic issue is that the version does not match the package version. By default git-version tries to do the right thing based on the tag, but if you're building without the git tree it may fall back to the default defined version.

E.g.:

$ dpkg-query -l ndctl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-=============-=============-========================================
ii ndctl 59.2-0ubuntu1 amd64 Utility for managing the libnvdimm subsy

$ ndctl version
59.1+

Fyi, you can create an emulated persistent memory environment with the memmap=ss!nn command line option. For example I tested this build with: memmap=512M!4G

Ok, what about the nvml packages? Any testers out there?

I'll take a look at the version issue, but we should not block on it as that can be fixed later.

I asked some people internally to test those packages. I think we'll get results tomorrow.

While waiting on that I tried to build pmemfile project (https://github.com/pmem/pmemfile) and I got this:
-- Checking for module 'libpmemobj>=1.3'
-- Requested 'libpmemobj >= 1.3' but version of libpmemobj is

This is because "version" field in all .pc files is empty.

This is actually a bug in NVML/PMDK. It seems nobody tried to use pkg-config files with minimum version specified for packages built out of git tree.

Temporarily you can work around it by creating ".version" file in the top level directory with "1.3.1" as content.

Could you open a github issue for this and link it here please?

I fixed this today. stable-1.3 fix:
https://github.com/pmem/pmdk/commit/5cbd2be4ae1ae7541da5975c9071fa36b53cd835

Testing those packages is taking longer than I expected. I'll report back as soon as I get the results.
However without minimum version check, pmemfile builds fine and its tests pass, so I don't expect full test suite will find anything.

(FYI, I'll be out of office until Wednesday)

Hi Marcin,

I'm planning on splitting the *_dbg directories out into their own packages. Including runtime libraries from /usr/lib/$ARCH/*_dbg/ in the -dev package wasn't very popular.

I'm still working out the details, but it should be more or less like this (using libpmem as an example):
libpmem1: continues as is
libpmem-dev: no longer includes /usr/lib/x86_64-linux-gnu/nvml_dbg; has small patch to manpages explaining that nvml_dbg is in a new package in Ubuntu systems
libpmem1-debug: ships /usr/lib/x86_64-linux-gnu/nvml_dbg/lib*.so.*
libpmem-dev-debug: ships /usr/lib/x86_64-linux-gnu/nvml_dbg/{lib*.a,lib*.so} and requires libpmem1-debug

Is the separate dev package necessary? I.e., do you need the special debug library build when linking with your applications and wanting the extra logging? Or can apps keep linking against the normal library, and just set LD_LIBRARY_PATH when they need extra logging? I assume the latter, so the dev-debug package wouldn't be necessary.

Enjoy your holidays. I'm off on Friday (March 30th) but back on Monday.

Ok, new layout done: https://pastebin.ubuntu.com/p/8bV9CWFy3r/

Pushing to the ppa:

nvml (1.3.1-0ubuntu1~ppa8) bionic; urgency=medium

  * Drop the manpage lintian override and fix the problem instead.
  * Create new -debug packages for the *_dbg/ libraries, document it
    in README.source and drop the ldconfig lintian override.
  * Patch the devel manpages explaining that the extra debug libraries
    are in the -debug packages.

There was no consideration done for the upgrade from previous versions of these packages, you may have to remove and reinstall if you get conflicts or stray files.

These are the git repositories I've been working with, I don't remember if I mentioned this before:

nvml: https://code.launchpad.net/~ahasenack/ubuntu/+source/nvml/+git/nvml
ndctl: https://code.launchpad.net/~ahasenack/ubuntu/+source/ndctl/+git/ndctl

And now I uploaded ppa9 with the pkgconfig fix you committed upstream:

 nvml (1.3.1-0ubuntu1~ppa9) bionic; urgency=medium
 .
   * d/p/SRCVERSION-for-out-of-git-tree-builds.patch: fix the version field in
     the pkgconfig files (see
     https://bugs.launchpad.net/ubuntu/+bug/1752378/comments/31 and
     https://bugs.launchpad.net/ubuntu/+bug/1752378/comments/34)

I don't think versioning is working correctly when this is built from a git tree that is not upstream. For example, I build this from my packaging branch, and I get:

ubuntu@bionic-nvdimm:~$ pmempool --version
pmempool debian/1.3.1-0ubuntu1_ppa5-43-g1b3d0af

ubuntu@bionic-nvdimm:~$ cat /usr/lib/x86_64-linux-gnu/pkgconfig/libvmmalloc.pc
version=debian/1.3.1-0ubuntu1_ppa5-43-g1b3d0af
libdir=/usr/lib/x86_64-linux-gnu
prefix=/usr
includedir=${prefix}/include

Name: libvmmalloc
Description: libvmmalloc library from NVML project
Version: ${version}
URL: http://pmem.io/nvml
Requires:
Libs: -L${libdir} -lvmmalloc
Cflags: -I${includedir}

We'll see what the PPA gets.

When installing from the PPA, the version in the pkgconfig file seems fine:
ubuntu@bionic:~$ cat /usr/lib/x86_64-linux-gnu/pkgconfig/libpmem.pc
version=nvml-1.3.1
libdir=/usr/lib/x86_64-linux-gnu
prefix=/usr
includedir=${prefix}/include

Name: libpmem
Description: libpmem library from NVML project
Version: ${version}
URL: http://pmem.io/nvml
Requires:
Libs: -L${libdir} -lpmem
Cflags: -I${includedir}

I don't think there is a need for -dev-debug packages. The interface between debug and release packages is the same by design, so nobody should ever need to link to debug libraries.

Both debian/1.3.1-0ubuntu1_ppa5-43-g1b3d0af and nvml-1.3.1 are wrong. It should say 1.3.1. It seems we'll have to create source packages with .version file or keep it in tree.

We tracked one test failure in ppa7 version to debug libraries not having symbols, but I see ppa10 already have them. I don't see it mentioned in Changelog. What happened?

(obj_convert test injects crashes in some specific places using gdb and see if consecutive open handles such pool correctly)

I suspect a dh_* script just did the right thing once I moved the dbg libraries to their own -debug packages.

What should I do to get the right version in the pkgconfig files?

For now you can create .version file in the top level directory. We are evaluating various options.

nvml (1.3.1-0ubuntu1~ppa11) bionic; urgency=medium

  * Create missing .version file based on the upstream version listed in
    the debian/changelog topmost entry.

That should work for a build from a directory that is not a git repository.

I've been working on more lintian fixes raised by the AA review. Here is today's batch:
nvml (1.3.1-0ubuntu1~ppa12) bionic; urgency=medium

  * Install pmemobj docs into a html subdir
  * Enable parallel build (dh $@ --parallel)
  * Several lintian fixes:
    - binary-control-field-duplicates-source
    - duplicate-short-description
    - extended-description-is-probably-too-short
    - restore lintian override about ldconfig for the debug libraries, but
      this time in the -debug packages and not -dev packages.
    - using-first-person-in-description
    - extended-description-is-probably-too-short
    - debian-watch-does-not-check-gpg-signature
    - d/p/spelling-fixes.patch: assorted spelling fixes
    - useless-autogenerated-doxygen-file
    - hardening-no-bindnow

That last one, hardening-no-bindnow, meant I changed the build flags. This is what is used now:
CPPFLAGS : -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT
CFLAGS : -g -O2 -fdebug-prefix-map=/home/ubuntu/nvdimm/nvml/nvml-git=. -fstack-protector-strong -Wformat -Werror=format-security -std=gnu99 -Wall -Werror -pthread -DSRCVERSION="debian/1.3.1-0ubuntu1_ppa5-92-g79c0594" -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -DJEMALLOC_LIBVMEM -fvisibility=hidden
LDFLAGS : -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,-z,relro -Wl,--fatal-warnings

Before it was:
CPPFLAGS : -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT
CFLAGS : -g -O2 -fdebug-prefix-map=/home/ubuntu/nvdimm/nvml/nvml-git=. -fstack-protector-strong -Wformat -Werror=format-security -std=gnu99 -Wall -Werror -pthread -DSRCVERSION="debian/1.3.1-0ubuntu1_ppa5-91-g0b93979" -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -DJEMALLOC_LIBVMEM -fvisibility=hidden
LDFLAGS : -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -Wl,--fatal-warnings

(don't mind the version, that's just in my local builds)

There is another flags-related lintian warning that I have to address:
I: libvmem1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libvmem.so.1.0.0
N:
N: This package provides an ELF binary that lacks the use of fortified libc
N: functions. Either there are no potentially unfortified functions called
N: by any routines, all unfortified calls have already been fully validated
N: at compile-time, or the package was not built with the default Debian
N: compiler flags defined by dpkg-buildflags. If built using
N: dpkg-buildflags directly, be sure to import CPPFLAGS.
N:
N: NB: Due to false-positives, Lintian ignores some unprotected functions
N: (e.g. memcpy).
N:
N: Refer to https://wiki.debian.org/Hardening and
N: https://bugs.debian.org/673112 for details.
N:
N: Severity: normal, Certainty: wild-guess
N:
N: Check: binaries, Type: binary, udeb

But since this is about the debug libraries, I think I will just add a note to READBE.source and/or override it.

Another set of lintian fixes:

nvml (1.3.1-0ubuntu1~ppa13) bionic; urgency=medium

  * Switch debhelper compat level to 11:
    - d/rules: parallel build is default in dh level 11, so drop it from
      the rules file.
  * Add doc-base control file for the libpmemobj-dev package to register
    its html documentation.
  * New doc package for libpmemobj, moving the documentation that was
    previously in the dev package (fixes lintian
    arch-dep-package-has-big-usr-share)

nvml (1.3.1-0ubuntu1~ppa14) bionic; urgency=medium

  * More lintian fixes:
    - package-does-not-install-examples
    - package-contains-vcs-control-file

Remaining lintian complaints:
I: nvml source: out-of-date-standards-version 3.9.8 (released 2016-04-06) (current is 4.1.4)
I: nvml source: testsuite-autopkgtest-missing
P: nvml source: debian-watch-does-not-check-gpg-signature
I: libvmem1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libvmem.so.1.0.0
I: libvmem1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libvmem.so.1.0.0
I: libvmmalloc1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libvmmalloc.so.1.0.0
I: libpmem1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libpmem.so.1.0.0
I: libpmemobj1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libpmemobj.so.1.0.0
I: librpmem1: no-symbols-control-file usr/lib/x86_64-linux-gnu/librpmem.so.1.0.0
I: libpmemblk1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libpmemblk.so.1.0.0
I: libpmemobj1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libpmemobj.so.1.0.0
I: libpmemlog1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libpmemlog.so.1.0.0
I: librpmem1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/librpmem.so.1.0.0
I: libpmem1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libpmem.so.1.0.0
I: libpmempool1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libpmempool.so.1.0.0
I: libpmemlog1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libpmemlog.so.1.0.0
I: libpmemblk1-debug: hardening-no-fortify-functions usr/lib/x86_64-linux-gnu/nvml_dbg/libpmemblk.so.1.0.0
I: libvmmalloc1: no-symbols-control-file usr/lib/x86_64-linux-gnu/libvmmalloc.so.1.0.0

Big ones remaining are standards-version, testsuite, no symbols file

no-symbols-control-file addressed in ~ppa15

nvml (1.3.1-0ubuntu1~ppa15) bionic; urgency=medium

  * Create symbols files for the libraries

nvml (1.3.1-0ubuntu1~ppa16) bionic; urgency=medium

  * update README.source regarding the hardening-no-fortify-functions lintian
    check for the -debug packages and why it is not being addressed.
  * include all pmempool manpages
  * add dep8 tests

There is only one remaining lintian error now that has not been addressed yet in nvml:
N:
N: The source package refers to a Standards-Version older than the one that
N: was current at the time the package was created (according to the
N: timestamp of the latest debian/changelog entry). Please consider
N: updating the package to current Policy and setting this control field
N: appropriately.
N:
N: If the package is already compliant with the current standards, you
N: don't have to re-upload the package just to adjust the Standards-Version
N: control field. However, please remember to update this field next time
N: you upload the package.
N:
N: See /usr/share/doc/debian-policy/upgrading-checklist.txt.gz in the
N: debian-policy package for a summary of changes in newer versions of
N: Policy.
N:
N: Refer to
N: https://www.debian.org/doc/packaging-manuals/upgrading-checklist.txt for
N: details.
N:
N: Severity: wishlist, Certainty: certain
N:
N: Check: standards-version, Type: source

Uploaded twice today:
nvml (1.3.1-0ubuntu1~ppa18) bionic; urgency=medium

  * Bump standards-version to 4.1.4. No changes necessary.

 -- Andreas Hasenack <email address hidden> Tue, 17 Apr 2018 19:29:25 +0000

nvml (1.3.1-0ubuntu1~ppa17) bionic; urgency=medium

  * Do not build examples. Something changed in the bionic toolchain and the
    examples no longer build. See
    https://lists.ubuntu.com/archives/ubuntu-devel/2018-April/040289.html

 -- Andreas Hasenack <email address hidden> Tue, 17 Apr 2018 18:01:50 +0000

There was some toolchain change in ubuntu bionic in the past few days that broke the build:

https://lists.ubuntu.com/archives/ubuntu-devel/2018-April/040289.html

I don't know yet what that is about, so I disabled some example building in the meantime. I've seen in the logs that some examples are still built, but at least the failure doesn't happen anymore.

~ppa18 is my new Ubuntu Archive Admin review candidate, which I will submit shortly. It would be great if you could give it another round of testing.

I'll now switch back to ndctl and bring that one into compliance.

As the upstream has change the project name to `pmdk` we will be changing the source package to match. If there are any objections, please speak now. :)

Reference:

http://pmem.io/2017/12/11/NVML-is-now-PMDK.html

I tested ~ppa18 with PMDK in-tree tests and I realized that removal of .so files from nvml_dbg directory we discussed earlier breaks the assumption that you can just set LD_LIBRARY_PATH and everything will work correctly WHEN libvmmalloc is involved.
This works:
LD_PRELOAD=libvmmalloc.so ./app
This doesn't:
LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/nvml_dbg LD_PRELOAD=libvmmalloc.so ./app
Of course you can set LD_PRELOAD to libvmmalloc.so.1, but it's a bit annoying that you have to do that when library built from source works as intended...

One more issue which I haven't noticed earlier - missing Valgrind support.

Bit of background: PMDK supports both regular upstream Valgrind tools (memcheck, helgrind and drd) and our pmemcheck tool (available at https://github.com/pmem/valgrind).
To enable support for all tools you have to use EXTRA_CFLAGS="-DUSE_VALGRIND" option for make. We know that no distribution will package our special version of Valgrind, so we made an option to build PMDK with support for the upstream tools only, but it requires listing them all:
EXTRA_CFLAGS="-DUSE_VG_MEMCHECK -DUSE_VG_HELGRIND -DUSE_VG_DRD"

In 1.4 we fixed this by incorporating Valgrind headers into the tree and enabling support for all tools without any flag, independent of installed Valgrind version. If someone would want to use pmemcheck they would have to build our Valgrind fork, without rebuilding PMDK.
Eventually we will upstream pmemcheck, but that's a long term activity.

In the mean time, for 1.3 packages I'd suggest to go with just upstream Valgrind tools.

I'm almost done packaging 1.4, as per comment #51. We would prefer to stick with 1.4 unless you object. Keeping in mind that upgrading to 1.4 after 1.3.1 is released into bionic would be much harder, as it changes the source name.

With that in mind, do I need any changes in 1.4 regarding what you said about valgrind? Of course, you haven't seen my packages yet, but in principle?

About the LD_LIBRARY_PATH issue with debug builds, why would you set LD_PRELOAD to the libfoo.so symlink (instead of libfoo.so.1), even when using pmdk (formelly nvml) installed from source? Just because "it works" and then you don't have to remember the major lib version, or update docs whenever that changes? .so symlinks are not meant to be used as runtime libraries, but I can appreciate it's a bit convenient.

Packaging 1.4 sounds great :). You don't need to do anything for Valgrind support in 1.4.

Nobody had considered that libvmmalloc.so would not be available. Our tests use libvmalloc.so (that's how I found this issue - https://github.com/pmem/pmdk/pull/2873) and today I found out that even our libvmmalloc man page tells you to use libvmmalloc.so.
I'll fix both issues on PMDK side.

First 1.4 package uploaded:
pmdk (1.4-0ubuntu1~ppa1) bionic; urgency=medium

  * New upstream release
  * Drop debian/patches/fix_pkg_config_paths.patch, already applied
  * Drop debian/patches/manpage-macro.patch, no longer needed
  * Updated debian/patches/manpage-debug-packages.patch for version 1.4
  * Drop debian/patches/SRCVERSION-for-out-of-git-tree-builds.patch, already
    applied
  * Updated debian/patches/spelling-fixes.patch for version 1.4
  * Drop debian/patches/do-not-build-examples.patch, no longer needed
  * d/p/drop-shebang-from-bash-completion.patch: fix script-not-executable
    lintian error for the bash completion file.
  * d/t/control: fix test dependency
  * d/rules: workaround #1765851 by uncompressing the manpages right after
    auto_install so dh_compress and dh_installman don't get confused

 -- Andreas Hasenack <email address hidden> Mon, 23 Apr 2018 15:12:50 +0000

I made no attempt at creating an upgrade path from the nvml-named packages. If you were using them before, I suggest to purge them and reinstall these instead.

Also just uploaded a new ndctl: 60.1. This one still has lintian warnings that need to be fixed, and I also have to update debian/copyright because some files changed or were removed.

Not a big issue, but I don't think shipping object files and Visual Studio project files with examples makes much sense :). Also shipped Makefiles are not self-contained (they source Makefile.inc, which is not included in the package)

Right :/

I was trying to find a way to not build the examples, but even removing the "examples" directory from src/Makefile didn't do it, something else kept triggering its build. Do you have a tip on how I can avoid building the examples?

This is what I was trying:
--- a/src/Makefile
+++ b/src/Makefile
@@ -38,12 +38,12 @@

 TARGETS = libpmem libvmem libpmemblk libpmemlog libpmemobj libpmempool\
        libvmmalloc tools
-ALL_TARGETS = $(TARGETS) common librpmem examples benchmarks
+ALL_TARGETS = $(TARGETS) common librpmem benchmarks
 SCOPE_DIRS = $(TARGETS) common librpmem rpmem_common

 DEBUG_RELEASE_TARGETS = common libpmem libvmem libpmemblk libpmemlog libpmemobj\
        libpmempool libvmmalloc librpmem
-RELEASE_TARGETS = tools examples benchmarks
+RELEASE_TARGETS = tools benchmarks

 CLEAN_NO_JE_TARGETS = $(ALL_TARGETS) rpmem_common test
 CLEAN_TARGETS = $(CLEAN_NO_JE_TARGETS) jemalloc

Question: should I build pmdk with ndctl support, i.e., with the daxio package? Set NDCTL_ENABLE=y?

When I do that, however, for some reason libpmemobj and others get RPATH set again, even though I'm passing NORPATH=1:

(...)
cc -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,-z,relro -Wl,--fatal-warnings -Wl,--warn-common -Wl,-rpath=/usr/local/lib -L../nondebug/libpmemobj/.. -shared -Wl,--version-script=libpmemobj.map,-soname,libpmemobj.so.1 -o ../nondebug/libpmemobj/../libpmemobj.so.1.0.0 ../nondebug/libpmemobj/file.o ../nondebug/libpmemobj/file_posix.o
(...)

Please don't build pmdk 1.4 with NDCTL_ENABLE=y. This variable enables much more than daxio and we are not ready to support that yet.

If you want to disable building examples you have to remove both examples and benchmarks from src/Makefile ALL_TARGETS/RELEASE_TARGETS.

BTW, why do you even need to that? Why are they included in development packages?

They are considered documentation and trigger a lintian warning if not installed: https://lintian.debian.org/tags/package-does-not-install-examples.html

Thanks for the NDCTL warning, I won't enable it.

pmdk (1.4-0ubuntu1~ppa2) bionic; urgency=medium

  * Do not build examples and benchmarks. The former are documentation
    aids, and benchmarks trigger example builds.

I tested pmdk 1.4 ppa1 with pmdk in-tree tests and with some small changes all tests pass.
I'm in the process of upstreaming those changes (PR #2873 #2875 #2885 #2886) and once they are all on master I'll backport them to stable-1.4 branch.

(next week I'm on vacation)

ndctl (60.1-0ubuntu1~ppa2) bionic; urgency=medium

  * Added DEP8 tests for ndctl and daxctl.

Could someone please test the ndctl packages while I address the remaining of the lintian checks?

ndctl (60.1-0ubuntu1~ppa3) bionic; urgency=medium

  * Updated d/copyright
  * Packages descriptions updates
  * Bump standards-version to 4.1.4. No changes necessary.

Ok, barring any testing problems, I'll submit these two packages to another Archive Admin review next week.

NVML is renamed PMDK. the source link is changed. I have updated the description for clear.

Another review point came up: do you still need the bundled jemalloc library? Your README says that you have changes on top of upstream's 3.6.0. Are these changes not worth pushing back upstream?

Would you also recommend a minimal testing.sh file to run at least some of the pmdk unit tests during build? The build environment is usually a VM, but could be an unprivileged lxd container as well. As we speak I'm giving a try with the supplied testing.sh.example one (renamed to testing.sh)

Hm, that is taking ages to run, I think I'll stick to the dep8 tests I wrote.

jemalloc: We can't upstream those changes, because jemalloc changed too much since we forked it. pmemcto makes this even harder problem, because any changes to jemalloc on-media layout would have to invalidate pmemcto pools and we don't have any mechanism to automatically do that.

testing: Default configuration uses non-pmem code path, which is slow. You can disable it (unset NON_PMEM_FS_DIR), enable pmem path (PMEM_FS_DIR) and if you don't have DAX available set PMEM_FS_DIR_FORCE_PMEM to 1. I'd also recommend disabling tests for static libraries (TEST_BUILD="debug nondebug") and use "pcheck" target with -j$(($(nproc)*2)) instead of "check" (it doesn't support -j option).

FYI, yesterday I pushed some changes to the 1.4 branch to make it possible to run in-tree tests against installed libraries. Those changes will be part of 1.4.1 release.

I'm getting a timeout in util_file_create/TEST2:
(...)
util_file_create/TEST0: SETUP (check/pmem/debug)
util_file_create/TEST0: PASS [00.068 s]
util_file_create/TEST0: SETUP (check/pmem/nondebug)
util_file_create/TEST0: PASS [00.062 s]
util_file_create/TEST1: SETUP (check/pmem/debug)
util_file_create/TEST1: PASS [00.054 s]
util_file_create/TEST1: SETUP (check/pmem/nondebug)
util_file_create/TEST1: PASS [00.050 s]
util_file_create/TEST2: SETUP (check/pmem/debug)
RUNTESTS: stopping: util_file_create/TEST2 timed out, TEST=check FS=any BUILD=debug
Makefile:472: recipe for target 'check' failed

That test results in stuck processes:
30263 pts/0 R 22:28 ./util_file_create 0x4000 0x7FFFFFFFFFFFFFFF:/tmp//test_util_file_create2😘⠝⠧⠍⠇ɗPMDKӜ⥺🙋/testfile
13054 pts/0 R 4:25 ./util_file_create 0x4000 0x7FFFFFFFFFFFFFFF:/tmp//test_util_file_create2😘⠝⠧⠍⠇ɗPMDKӜ⥺🙋/testfile

They are very busy doing this:
pwrite64(7, "\0", 1, 798772768766) = 1
pwrite64(7, "\0", 1, 798772772862) = 1
pwrite64(7, "\0", 1, 798772776958) = 1
pwrite64(7, "\0", 1, 798772781054) = 1
pwrite64(7, "\0", 1, 798772785150) = 1
pwrite64(7, "\0", 1, 798772789246) = 1
pwrite64(7, "\0", 1, 798772793342) = 1
pwrite64(7, "\0", 1, 798772797438) = 1
pwrite64(7, "\0", 1, 798772801534) = 1

I'll try in a VM next (that run was in a lxd container).

I used this testing config:
        echo "PMEM_FS_DIR=/tmp" > src/test/testconfig.sh
        echo "PMEM_FS_DIR_FORCE_PMEM=1" >> src/test/testconfig.sh
        echo "TEST_BUILD=\"debug nondebug\"" >> src/test/testconfig.sh
        echo "TM=1" >> src/test/testconfig.sh
        dh_auto_test -- SKIP_SYNC_REMOTES=y pcheck

Note, there is no pmem or dax hardware here.

pcheck would always fail at this stage:
make[6]: Entering directory '/home/ubuntu/git/pmdk-git/src/test/tools/pmemspoil'
cc -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,-z,relro -Wl,--warn-common -Wl,--fatal-warnings -L../../../../src/tools/../../src/../src/nondebug -o pmemspoil.static-debug spoil.o common.o output.o pme
mblk_priv.o ../../../../src/tools/../../src/../src/debug/libpmemcommon.a ../../../../src/tools/../../src/../src/debug/libpmem.a -pthread -ldl
output.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
../../../../src/tools/Makefile.inc:292: recipe for target 'pmemspoil.static-debug' failed
make[6]: *** [pmemspoil.static-debug] Error 1
make[6]: Leaving directory '/home/ubuntu/git/pmdk-git/src/test/tools/pmemspoil'

Without parallelization it worked:
        echo "PMEM_FS_DIR=/tmp" > src/test/testconfig.sh
        echo "PMEM_FS_DIR_FORCE_PMEM=1" >> src/test/testconfig.sh
        echo "TEST_BUILD=\"debug nondebug\"" >> src/test/testconfig.sh
        echo "TM=1" >> src/test/testconfig.sh
        # dh_auto_test calls make test, which only builds the test suite
        # make check (and pcheck, for parallel check) actually runs it
        dh_auto_test -- SKIP_SYNC_REMOTES=y
        make SKIP_SYNC_REMOTES=y check

21min to build with that in a VM. Not too bad. I'll add that to the package and skip the test if it's building in a lxc container.

I'm looking into util_file_create failure.

WRT pcheck, you had hit some bug in our build system. I know what's causing the error you are seeing (make tries to build pmemspoil twice, both processes write to the same file, which means creation of corrupted object file and that leads to failure in linking), but I don't know where is the bug yet. I'll look into this.
To work around it run "make test" first and only then use pcheck with -jX.

util_file_create timed out because you used file system (PMEM/NON_PMEM_FS_DIR) that does not support fallocate syscall. glibc's posix_fallocate falls back to writing zeroes when fallocate is not supported, so test takes ages to complete. https://github.com/pmem/pmdk/pull/2939 adds code to detect it and fail early.

I finally found why pcheck failed (https://github.com/pmem/pmdk/pull/2961). As soon as the fix will land on master I'll backport it for the next stable release.

This was finally accepted into the archive and is available in cosmic:

https://launchpad.net/ubuntu/+source/ndctl
https://launchpad.net/ubuntu/+source/pmdk

I'll close this bug, and open specific ones against each package for an SRU into Bionic.

Thanks all for the work so far!

Do you have a preference for which version we should backport to bionic? Cosmic has these currently:
pmdk: 1.4 (latest upstream is 1.4.1)
ndctl: 60.1 (latest upstream is 61.2)

In general the latest is usually what people want, but I thought best to check. Maybe there is something super important we are missing, or something experimental that we do not want yet.

Keeping in mind that if want to update these, they need to be updated in cosmic first, and then pushed down to bionic.

Yes, for ndctl, please roll forward to latest. I'll Marcin comment on pmdk.

Thanks for all the support!

Please backport the latest version of pmdk.

Hi Andreas, I would say we need to go with the latest versions for both ndctl and pmdk to get back ported to bionic.

Ok, work on that has already started.

> Please don't build pmdk 1.4 with NDCTL_ENABLE=y. This variable enables much more
> than daxio and we are not ready to support that yet.

Should I enable ndctl now with pmdk 1.4.1 and get the daxio utility, or not yet?

In pmdk 1.4.1 NDCTL_ENABLE controls daxio only, so it's safe to build pmdk with this flag. The default value has been flipped to "detect", so if ndctl is installed in the system, there's no need to set it explicitly.

Would you guys be able to give the latest ndctl and pmdk packages from https://launchpad.net/~canonical-server/+archive/ubuntu/nvdimm/ a test run? I'm currently proposing them for an SRU into bionic, and it would be best to catch any issues before it hits the bionic-proposed pocket.

Yeah, we'll test pmdk packages.

PMDK packages look good.