libsmartcols: Revert back to previous behaviour of non-shell parsable column output (lsblk -P)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
util-linux (Ubuntu) |
Fix Released
|
High
|
Matthew Ruffell | ||
Jammy |
Fix Released
|
High
|
Matthew Ruffell |
Bug Description
[Impact]
util-linux 2.37 in Jammy has introduced some new behaviour for lsblk and similar tools which depend on libsmartcols. This switched the -P / --pairs parameter from printing column names as normal, to changing the names to shell compatible names instead.
e.g. lsblk -P now outputs LOG_SEC instead of LOG-SEC.
The change broke some core tooling which rely on on the output of lsblk -P, most notably, curtin and MAAS, but I am sure there will be more applications affected.
Affected MAAS users will see the following traceback when attempting to deploy 22.04:
Traceback (most recent call last):
File "/curtin/
logical = info[parent]
KeyError: 'LOG-SEC'
'LOG-SEC'
curtin: Installation failed with exception: Unexpected error while running
This is documented in MAAS bug 1956613.
MAAS decided to fix it by changing from -P to -J, in the following commit:
https:/
Curtin now checks for MAJ_MIN, and changes it back to MAJ:MIN in:
https:/
The issue is that these commits are not tagged to any MAAS release, and users would be forced to upgrade MAAS to the latest stable release when available if they want to deploy 22.04.
There are many users out there that don't want to upgrade MAAS, so returning to the previous column output is the most desirable solution.
[Testcase]
On a Jammy install, simply run lsblk with either -P or --pairs:
$ sudo lsblk -P
...
NAME="sda" MAJ_MIN="8:0" RM="0" SIZE="465.8G" RO="0" TYPE="disk" MOUNTPOINTS=""
...
Affected installs will see MAJ_MIN.
There is a test package available in the following ppa:
https:/
If you install it, you will see MAJ:MIN, just like it is on Impish and previous.
[Where problems could occur]
We are changing the column output for -P and --pairs for the following applications:
* lsblk
* findmnt
* lsipc
If any application has been modified to depend on the new column output, it will break. I don't have any examples of something that will break, because MAAS and curtin were modified such that they would be compatible with both column name formats.
It should be noted that the manpage documents that lsblk output can change at any time:
> The default output, as well as the default output from options like --fs and --topology, is subject to change.
> So whenever possible, you should avoid using default outputs in your scripts.
> Always explicitly define expected columns by using --output columns-list and --list in environments where a stable output is required.
If a regression should occur, we will need to fix up these affected packages also.
[Other info]
The change came about when a user asked upstream to make -P / --pairs shell parsable, in Issue 1201 upstream:
https:/
Karel Zak obliged, and it was implemented in the following commit:
commit 58b510e5805d835
From: Karel Zak <email address hidden>
Date: Thu, 3 Dec 2020 12:14:10 +0100
Subject: libsmartcols: sanitize variable names on export output
Link: https:/
I wrote to Karel Zak with the regressions introduced by changing the format, and asked to revert back, and instead implement the shell parsable logic as a new parameter.
This happened in upstream issue 1594:
https:/
Karel Zak was happy to oblige again, and we now have the following commits:
338ad4a93 findmnt: commit missing flag
0f843ab64 lsblk: update --help output for -y
eba05f308 lsipc: add -y,--shell
152c17aa4 findmnt: add -y,--shell
9c7e81ff1 lslogins: add -y,--shell
25fb0638a lsblk: add -y/--shell
39679ea0c lsfd: use new libsmartcols functions
6fd0e3590 column: use new libsmartcols functions
0b3c2e80d include/
3b5db50f7 libsmartcols: change "export" behavior, add "shellvar" flag
While we got the intended behaviour, these commits won't land until util-linux 2.38, which will be after Jammy releases, and the other issue is that this changes a significant amount of code, like nearly 1k lines, and is spread over 10+ commits.
I wrote to ubuntu-devel asking for advice, on either 1) not changing anything 2) backporting the 10+ new commits, or 3) simply reverting the commit which changed the behaviour.
https:/
ubuntu-devel had strong support for option (3).
Hence, we will revert the below commit to ensure Jammy can be deployed on all existing MAAS releases.
58b510e580 libsmartcols: sanitize variable names on export output
Changed in util-linux (Ubuntu Jammy): | |
status: | New → In Progress |
importance: | Undecided → High |
assignee: | nobody → Matthew Ruffell (mruffell) |
tags: | added: jammy sts |
Attached is a debdiff for Jammy util-linux.