Comment 8 for bug 1893661

> 1-2) No, only the md-device, the container, raid level and number of devices (it's possible to create different RAID volumes with different levels in one container, however the number of devices must be the same in all).

OK.

Looking at the https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/ssd-software/Linux_VROC_6-0_User_Guide.pdf
without specifying a size, (raid never does, it uses whole devices) I can't
quite wrap my head around what creating to volumes in a container means.

If I have 4 disks, and put them into a container and then create a raid0
and a raid5 from the container ... what devices do I have?

Ah, the manual suggests size is important for the multi-container support

    -l Specifies the RAID level. The options supported are 0, 1, 5, 10.
    -z Specifies the size (in Kibibyte) of space dedicated on each disk to the RAID volume. This must be a multiple of the chunk size. For example

> 3) AFAIK after stopping and removing all arrays from the container, mdamd --zero-superblock on the member devices will remove the container itself. Need to check again.

OK. We currently do:

a) enumerate devices and spares
b) set_sync_action=idle
c) set_sync_action=frozen
d) wipe the superblock if the composed raid device (it may have metadata for a
   higher level device, like nested raid or LVM over RAID etc)
e) for each raid members + spares
      mdadm fail device
      mdadm remove device
f) mdadm stop
g) mdadm zero_device
h) wait for /dev/mdX to be released from the kernel

I think we'd need to notice /dev/mdX is part of a container, and if so after
tearing down the mdx, to then wipe the container? You mentioned the delete
curtin does isn't sufficient; If you have the curtin install.log with the
failure, that'll help sort this bit out.

https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/ssd-software/Linux_VROC_6-0_User_Guide.pdf

That has more details.

specifically mdadm -vS which stops volumes and if we support the multi-volume
setup, then removing sub-arrays is more complicated

> 4) Yes, and even the EFI partition can be on RAID. Btw, it doesn't really works with the legacy BIOS, mdadm doesn't find the raid support on our servers without EFI. I'm not sure if it's supposed to work with legacy BIOS, or it's just a firmware issue.

EFI really shouldn't be on RAID in that EFI is VFAT and I don't believe there
are EFI drivers for mdadm, maybe Intel provides a VROC/mdadm EFI driver, do
you know?

For legacy, the open question is whether grub2 can read an mdadm with metadata
imsm metadata... sounds like no.

> 5) That need to be checked, as the cleaning info comes from there.
>
> Yeah, I fear it cannot be automatically tested without actual hardware.

=(

>
>
> Maybe it would be possible to abstract out the container, like:
> - type: raid
> metadata: imsm
> container: /dev/md/imsm0
> name: imsm0
> devices:
> - /dev/nvme0n1p1
> - /dev/nvme1n1p1
>
> - type: raid
> devices:
> - /dev/md/imsm0 # but need to get the number of real devices for -n

We can do that by either repeating the values; it's a shame that the mdadm implementation doesn't just take the container name and look up the disks. It seems like a big oversite to require repeating the devices names since as you mention they have to keep the same devices and numbers.

> name: mirror0
> level: 1

Given that to create multiple volumes from the container, we'll need to
use two entries, the raid volumes will need a size.

- type: raid
  id: disk_raid_container0
  metadata: imsm
  name: /dev/md/imsm0
  devices:
    - /dev/nvme0n1p1
    - /dev/nvme1n1p1

- type: raid:
  id: disk_raid0_c0_vol0
  level: 1
  name: /dev/md/mirror0
  container: disk_raid_container0
  size_kb: <KiB value>

- type: raid:
  id: disk_raid0_c0_vol1
  level: 0
  name: /dev/md/striped0
  container: disk_raid_container0
  size_kb: <KiB value>