Incorrect multipath device names in /etc/fstab

Bug #1424734 reported by Mauricio Faria de Oliveira
This bug report is a duplicate of:  Bug #1430074: fix handling of multipathed disks. Edit Remove
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
partman-target (Ubuntu)
Incomplete
Undecided
Unassigned

Bug Description

Problem:
-------

The multipath installation inserts incorrect device names on /etc/fstab.

The multipath partitions have different device names during the installation
and on the installed system (specifically, the disk-partition separator).

During installation: /dev/mapper/mpathXpY
On installed system: /dev/mapper/mpathX-partY

Impact:
------

The multipath partitions in /etc/fstab are not mountable/usable -- any partition
that is not specified by UUID (at least swap).

The system only boots because the root partition is specified by UUID, thus
not using the entry in /etc/fstab.
The swap partition is not made effective as swap.

This is the /etc/fstab from installation:

 ~ # grep mapper /target/etc/fstab
 /dev/mapper/mpath0p2 / ext4 errors=remount-ro 0 1
 /dev/mapper/mpath0p3 none swap sw 0 0

And this is the environment in the installed system.

 $ grep mapper /etc/fstab
 /dev/mapper/mpath0p2 / ext4 errors=remount-ro 0 1
 /dev/mapper/mpath0p3 none swap sw 0 0

 $ ls -1 /dev/mapper
 control
 mpath0
 mpath0-part1
 mpath0-part2
 mpath0-part3

 $ mount | grep ' / '
 /dev/mapper/mpath0-part2 on / type ext4 (rw,errors=remount-ro)

 $ grep root=UUID /proc/cmdline
 BOOT_IMAGE=/boot/vmlinux-3.18.0-13-generic root=UUID=35c634b0-f667-48a8-84f0-a571aa96d103 ro

 $ cat /proc/swaps
 Filename Type Size Used Priority
 $

Cause:
-----

The multipath block devices are created/named by different programs on
installation and installed system, and each one uses a different disk-partition separator.

During installation: libparted (via parted-server)
On installed system: kpartx (via udev rules)

On libparted, The 'p' separator is hard-coded in these functions:
[libparted/arch/linux.c]

 _dm_add_partition():
  if (asprintf (&vol_name, "%sp%d", dev_name, part->num) == -1)

 _device_get_part_path():
  snprintf (result, result_len, "%sp%d", devpath, num);

On kpartx udev rules, the '-part' separator is used:
[/lib/udev/rules.d/95-kpartx.rules]

 ENV{DM_STATE}=="ACTIVE", ENV{DM_UUID}=="mpath-*", \
  RUN+="/sbin/kpartx -a -p -part /dev/$name"

Solution proposal:
-----------------

Ideally, one side or another should be 'fixed' (either libparted or kpartx rules).

I wrote this contained work-around (patch attached) to a partman's finish.d/ trigger,
standing between each side, as I'm not sure of the fallout of either change.

It intercepts and changes the multipath device names inserted in /etc/fstab.

It patches '/lib/partman/finish.d/40fstab_hd_entries' because that file already
has a 'case' including '/dev/mapper', so it seemed an appropriate place.
The change is /in/ the addfstab() function, because the specified device must
actually exist until it's called, as 'mapdevfs $dev' is an argument.

With it applied, this is the new /etc/fstab during installation:

 ~ # grep mapper /target/etc/fstab
 /dev/mapper/mpath0-part2 / ext4 errors=remount-ro 0 1
 /dev/mapper/mpath0-part3 none swap sw 0 0

And on the installed system, the swap is available/functional now:

 $ grep mapper /etc/fstab
 /dev/mapper/mpath0-part2 / ext4 errors=remount-ro 0 1
 /dev/mapper/mpath0-part3 none swap sw 0 0

 $ mount | grep ' / '
 /dev/mapper/mpath0-part2 on / type ext4 (rw,errors=remount-ro)

 $ cat /proc/swaps
 Filename Type Size Used Priority
 /dev/mapper/mpath0-part3 partition 5492672 0 -1

Steps to reproduce:
------------------

You can use qemu/kvm if you don't have multipath hardware handy, just specify
2 drives with the same serial number. This is an example for ppc64el, easily
changeable for amd64 (change qemu-system & -M)

$ qemu-img create -f qcow2 vivid-multipath.qcow2 128g

$ qemu-system-ppc64 -M pseries \
 -enable-kvm \
 -nographic -nodefaults \
 -monitor stdio -serial pty \
 -m 2G \
 -net nic -net user \
 -drive file=vivid-multipath.qcow2,serial=abcdef \
 -drive file=vivid-multipath.qcow2,serial=abcdef \
 -kernel netboot/vmlinux \
 -initrd netboot/initrd.gz \
 -append 'disk-detect/multipath/enable=true grub-installer/multipath=true'

Note: to boot the installed system, remove the -kernel/-initrd/-append options.
      if the system takes long to boot in the initrd, use the 'debug' option
      in the kernel command line (current work-around for another problem).

Tags: patch
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

Adding an observation..

It's sligthly confusing to try to think of how/where fixing this generally..

Many pieces in the installer query libparted (parted-server) for a list of partitions (open_dialog PARTITIONS),
and libparted always return 'p' as disk-partition separator.

So, it won't work to just adjust the kpartx commands in /lib/partman/commit.d/31partition_multipath
to remove the block devices with 'p' separator and add them with '-part', e.g.,:
         # update partition mappings
- kpartx -d -p -part $(cat device)
+ kpartx -d -p p $(cat device)
         kpartx -a -p -part $(cat device)
because the filesystem formatting will fail immediately later, as they query parted for the block device names.

In the other hand, there are many places in /lib/partman/lib/base.sh that assume that '-part' is the separator,
so just leaving the 'p' from libparted actually hits other failures (for example, disks/partitions that are part of a multipath device are visible for being partitioned/formatted, which is a bug caused by the different separators).

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "replace-multipath-separator.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

By the way, interesting observation in this content.

The 'p' separator is only used when parted /creates/ partition (thus adding them through its own call to DM/dm_add_partition() function).

If the partitions already exist and are only formatted, parted won't have to create them, and the '-part' separator from the kpartx call in /lib/partman/commit.d/31multipath_partitions kicks in.
Also, existing partitions from other devices (even non-formatted) show up with '-part' due to that kpartx call and the absence of parted partition creation.

The remaining problem is that subsequent stages which query partitions names/path from parted will still receive 'p' separator, and fail, as the those device names don't exist (e.g., the swap formatting error above).

Maybe a solution is to remove the devices with parted's 'p'-separator with 'kpartx -d [-p p]', re-add them with 'kpartx -a -p -part', and create a symlink for a device name with 'p'.

That will make the '-part' device names available for everywhere (lots of code use them in partman.. also grub-installer, at least). It still doesn't fix this bug (as the device names for /etc/fstab come from parted.. so, with 'p'.. but helps in some way.

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

Looking at the installer, the p/-part problem is more general than just this (other components assume '-part', and are not getting them when multipath is used).

I'll try a more general solution.
Let go of this bug for now.

Changed in partman-target (Ubuntu):
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.