Ubuntu
uvtool package

libvirt.libvirtError: internal error: Child process () unexpected exit status 1: qemu-img: Image creation needs a size parameter

Bug #1459748 reported by Chris J Arges on 2015-05-28

This bug affects 5 people

Affects		Status	Importance	Assigned to	Milestone
	libvirt (Ubuntu)	Fix Released	Medium	Chris J Arges
	uvtool (Ubuntu)	Invalid	Undecided	Unassigned	Ubuntu ubuntu-15.10

Bug Description

Syncing cloud images fails on wily:

$ uvt-simplestreams-libvirt sync release=vivid arch=amd64
Traceback (most recent call last):
  File "/usr/bin/uvt-simplestreams-libvirt", line 27, in <module>
    uvtool.libvirt.simplestreams.main()
  File "/usr/lib/python2.7/dist-packages/uvtool/libvirt/simplestreams.py", line 326, in main
    args.func(args)
  File "/usr/lib/python2.7/dist-packages/uvtool/libvirt/simplestreams.py", line 269, in main_sync
    tmirror.sync(smirror, initial_path)
  File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 88, in sync
    return self.sync_index(reader, path, data, content)
  File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 236, in sync_index
    self.sync(reader, path=epath)
  File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 86, in sync
    return self.sync_products(reader, path, data, content)
  File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py", line 315, in sync_products
    self.insert_item(item, src, target, pgree, ipath_cs)
  File "/usr/lib/python2.7/dist-packages/uvtool/libvirt/simplestreams.py", line 236, in insert_item
    pool_name=LIBVIRT_POOL_NAME
  File "/usr/lib/python2.7/dist-packages/uvtool/libvirt/__init__.py", line 70, in create_volume_from_fobj
    pool_name=pool_name
  File "/usr/lib/python2.7/dist-packages/uvtool/libvirt/__init__.py", line 91, in _create_volume_from_fobj_with_size
    vol = pool.createXML(etree.tostring(new_vol), 0)
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3087, in createXML
    if ret is None:raise libvirtError('virStorageVolCreateXML() failed', pool=self)
libvirt.libvirtError: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMDQ6YW1kNjQgMjAxNTA1MTMuMQ==) unexpected exit status 1: qemu-img: /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMDQ6YW1kNjQgMjAxNTA1MTMuMQ==: Image creation needs a size parameter

Chris J Arges (arges) on 2015-05-28

summary:

- libvirt.libvirtError: internal error: Child process (/usr/bin/qemu-img
- create -f qcow2 -o compat=0.10
- /var/lib/uvtool/libvirt/images/x-uvt-b64-foo==) unexpected exit status
- 1: qemu-img: /var/lib/uvtool/libvirt/images/x-uvt-b64-foo==: Image
- creation needs a size parameter
+ libvirt.libvirtError: internal error: Child process () unexpected exit
+ status 1: qemu-img: Image creation needs a size parameter

Revision history for this message

Launchpad Janitor (janitor) wrote on 2015-05-29:

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in uvtool (Ubuntu):
status:	New → Confirmed

Robie Basak (racb) on 2015-05-29

Changed in uvtool (Ubuntu):
importance:	Undecided → High
milestone:	none → ubuntu-15.10

Revision history for this message

Chris J Arges (arges) wrote on 2015-06-03:

I don't think this is related to qemu, it may be a libvirt change.

A simpler reproducer:
echo "<volume><name>test</name><target><format type="qcow2"/></target><capacity>0</capacity></volume>" > vol.xml
virsh vol-create uvtool vol.xml

shows the error:
error: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/test) unexpected exit status 1: qemu-img: /var/lib/uvtool/libvirt/images/test: Image creation needs a size parameter

However if you run the same command in an older version this works.
The qemu-img command fails in all versions, so I suspect a libvirt change.

Revision history for this message

Ryan Harper (raharper) wrote on 2015-06-29:

In the change between vivid (1.2.12) and wily (1.2.15) the following patch from upstream was applied:

commit fbcf7da95b872ac45cabc4356fc9c06e809d0237
Author: Ján Tomko <email address hidden>
Date: Thu Jul 31 18:07:56 2014 +0200

Introduce struct _virStorageBackendQemuImgInfo

This will contain the data required for creating the qemu-img
command line without having access to the volume definition.

diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c
index fdda0dc..f5b95ec 100644

The core issue is when the qemu-img command builder includes the size string ("%llk", size_arg). In vivid, if you are not converting, it creates it (0k) in our case, which is completely valid parameter to qemu-img.

In wily (1.2.15) this is now a new condition see this diff:

- if (convert)
- virCommandAddArg(cmd, inputPath);
- virCommandAddArg(cmd, vol->target.path);
- if (!convert)
- virCommandAddArgFormat(cmd, "%lluK", size_arg);
+ if (info.inputPath)
+ virCommandAddArg(cmd, info.inputPath);
+ virCommandAddArg(cmd, info.path);
+ if (!info.inputPath && info.size_arg)
+ virCommandAddArgFormat(cmd, "%lluK", info.size_arg);

I suspect that we might need to report this upstream. If we don't pass a capacity (or pass it as 0)
then the if (!info.inputPath && info.size_arg) will always fail: if (1 && 0), and it will stay broken

I think it's safe to drop the && info.size_arg check since it's safe to call qemu-img with size 0:

(foudres) libvirt % sudo /usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
qemu-img: /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=: Image creation needs a size parameter
(foudres) libvirt % sudo /usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ= 0K
Formatting '/var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=', fmt=qcow2 size=0 compat='0.10' encryption=off cluster_size=65536 lazy_refcounts=off
(foudres) libvirt % ls -al /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
-rw-r--r-- 1 root root 196608 Jun 29 12:12 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
(foudres) libvirt % sudo qemu-img info /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
image: /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
file format: qcow2
virtual size: 0 (0 bytes)
disk size: 192K
cluster_size: 65536
Format specific information:
compat: 0.10

In the change between vivid (1.2.12) and wily (1.2.15) the following patch from upstream was applied:

commit fbcf7da95b872ac45cabc4356fc9c06e809d0237
Author: Ján Tomko <jtomko@redhat.com>
Date:   Thu Jul 31 18:07:56 2014 +0200

Introduce struct _virStorageBackendQemuImgInfo
    
    This will contain the data required for creating the qemu-img
    command line without having access to the volume definition.

diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c
index fdda0dc..f5b95ec 100644

The core issue is when the qemu-img command builder includes the size string ("%llk", size_arg).  In vivid, if you are not converting, it creates it (0k) in our case, which is completely valid parameter to qemu-img.

In wily (1.2.15) this is now a new condition  see this diff:

-    if (convert)
-        virCommandAddArg(cmd, inputPath);
-    virCommandAddArg(cmd, vol->target.path);
-    if (!convert)
-        virCommandAddArgFormat(cmd, "%lluK", size_arg);
+    if (info.inputPath)
+        virCommandAddArg(cmd, info.inputPath);
+    virCommandAddArg(cmd, info.path);
+    if (!info.inputPath && info.size_arg)
+        virCommandAddArgFormat(cmd, "%lluK", info.size_arg);

I suspect that we might need to report this upstream.  If we don't pass a capacity (or pass it as 0)
then the if (!info.inputPath && info.size_arg) will always fail:  if (1 && 0),   and it will stay broken

I think it's safe to drop the && info.size_arg check since it's safe to call qemu-img with size 0:

(foudres) libvirt % sudo /usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ= 
qemu-img: /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=: Image creation needs a size parameter
(foudres) libvirt % sudo /usr/bin/qemu-img create -f qcow2 -o compat=0.10 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ= 0K
Formatting '/var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=', fmt=qcow2 size=0 compat='0.10' encryption=off cluster_size=65536 lazy_refcounts=off 
(foudres) libvirt % ls -al /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
-rw-r--r-- 1 root root 196608 Jun 29 12:12 /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
(foudres) libvirt % sudo qemu-img info /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
image: /var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZDpzZXJ2ZXI6MTUuMTA6YW1kNjQgMjAxNTA2MjQ=
file format: qcow2
virtual size: 0 (0 bytes)
disk size: 192K
cluster_size: 65536
Format specific information:
    compat: 0.10

Revision history for this message

Ryan Harper (raharper) wrote on 2015-06-29:

libvirt_storage_create_dont_check_size_arg.patch Edit (674 bytes, text/plain)

Revision history for this message

Ubuntu Foundations Team Bug Bot (crichton) wrote on 2015-06-29:

The attachment "libvirt_storage_create_dont_check_size_arg.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags:

added: patch

Revision history for this message

Chris J Arges (arges) wrote on 2015-06-29:

This patch causes the libvirt test 'storagevolxml2argvtest' to fail. A few changes as discussed in channel also cause failures.

Revision history for this message

Chris J Arges (arges) wrote on 2015-06-30:

Test build with another attempt:
http://people.canonical.com/~arges/lp1459748/

Revision history for this message

Ryan Harper (raharper) wrote on 2015-06-30:

The rebuilt packages with the updated check resolves the sync issue for me.

Chris J Arges (arges) on 2015-06-30

Changed in uvtool (Ubuntu):
status:	Confirmed → Invalid
Changed in libvirt (Ubuntu):
status:	New → In Progress
importance:	Undecided → Medium
Changed in uvtool (Ubuntu):
importance:	High → Critical
importance:	Critical → Undecided
Changed in libvirt (Ubuntu):
assignee:	nobody → Chris J Arges (arges)

Revision history for this message

Chris J Arges (arges) wrote on 2015-07-01:

#10

Posted patches here:
http://www.redhat.com/archives/libvir-list/2015-June/msg01681.html

Chris J Arges (arges) on 2015-07-01

tags:

removed: patch

Revision history for this message

Launchpad Janitor (janitor) wrote on 2015-07-10:

#11

This bug was fixed in the package libvirt - 1.2.16-2ubuntu4

---------------
libvirt (1.2.16-2ubuntu4) wily; urgency=medium

  * d/p/{storage-allow-zero-capacity-with-non-backing-file-to.patch,
    tests-add-vol-qcow2-zerocapacity-test-to-storagevolx.patch} added to address
    (LP: #1459748). Allow zero capacity storage creation with non-backing file.

-- Chris J Arges <email address hidden> Fri, 10 Jul 2015 12:50:50 -0500