Ubuntu
autopkgtest package

issues using user and password in adt-virt-qemu

Bug #1630963 reported by Christian Ehrhardt  on 2016-10-06

6

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	autopkgtest (Ubuntu)	Fix Released	Medium	Martin Pitt

Bug Description

I found a few issues I wanted to report that I ran into trying to avoid another issue by using using --user/--password.

First I found that with debug enabled this fails:
  File "/home/ubuntu/autopkgtest-4.1/virt/autopkgtest-virt-qemu", line 171, in login_tty_and_setup_shell
    VirtSubproc.expect(term, None, 10, 'password prompt')
  File "/home/ubuntu/autopkgtest-4.1/lib/VirtSubproc.py", line 233, in expect
    adtlog.debug('expect: "%s"' % search_bytes.decode())
AttributeError: 'NoneType' object has no attribute 'decode'

This would be fixed by:
--- /home/ubuntu/autopkgtest-4.1/lib/VirtSubproc.py.orig 2016-10-06 07:36:21.330764811 -0400
+++ /home/ubuntu/autopkgtest-4.1/lib/VirtSubproc.py 2016-10-06 07:30:29.110383136 -0400
@@ -230,7 +230,8 @@

def expect(sock, search_bytes, timeout_sec, description=None, echo=False):
- adtlog.debug('expect: "%s"' % search_bytes.decode())
+ if search_bytes:
+ adtlog.debug('expect: "%s"' % search_bytes.decode())
     what = '"%s"' % (description or search_bytes or 'data')
     out = b''
     with timeout(timeout_sec,

Then second there is a non conform concat when setting up the shell for that:
File "/home/ubuntu/autopkgtest-4.1/virt/autopkgtest-virt-qemu", line 181, in login_tty_and_setup_shell
cmd = b"sudo sh -c '" + cmd + "'"
TypeError: can't concat bytes to str

Fixed by:
--- virt/autopkgtest-virt-qemu.orig 2016-10-06 07:13:55.409049291 -0400
+++ virt/autopkgtest-virt-qemu 2016-10-06 07:33:19.730639581 -0400
@@ -178,7 +178,7 @@

# if we are a non-root user, run through sudo
if args.user != 'root':
- cmd = b"sudo sh -c '" + cmd + "'"
+ cmd = b"sudo sh -c '" + cmd + b"'"

term.send(cmd)
term.send(b'\nexit\n')

Revision history for this message

Martin Pitt (pitti) wrote on 2016-10-06:

#1

Fixed in https://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=24bacc0, thanks!

Note that this will still not work if sudo needs a password, only for passwordless sudo (like in cloud instances). But I'm going to teach a-v-qemu about that now.

Changed in autopkgtest (Ubuntu):
importance:	Undecided → Medium
status:	New → Fix Committed

Revision history for this message

Martin Pitt (pitti) wrote on 2016-10-06:

#2

Not enough yet indeed, the password prompt is from login already, not from sudo (although that is still a separate issue).

Changed in autopkgtest (Ubuntu):
assignee:	nobody → Martin Pitt (pitti)
status:	Fix Committed → In Progress

Revision history for this message

Martin Pitt (pitti) wrote on 2016-10-10:

#3

I updated the commit: https://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=7077f969e

This fully fixes user/password based VMs now. Can you please check current git (you can run it straight out of there, with /path/to/checkout/runner/autopkgtest) how it works for you?

Changed in autopkgtest (Ubuntu):
status:	In Progress → Fix Committed

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2016-10-11:

#4

Hi Martin,
I'm still blocked by other things on my s390x env where all of this started.
But that shall stay part of bug 1630909.

I removed my workarounds which gave me a root login on ttyS1 and ran into the issues described here.
Then I pulled latest git and ran it and it seems I now get as far with the user/pw based login as with the root shell.

Thank you!

Here as reference what I got with debug/verbose enabled:
autopkgtest-virt-qemu: DBG: expect: found ""login prompt on ttyS0""
autopkgtest-virt-qemu: DBG: expect: "ok"
autopkgtest-virt-qemu: DBG: setup_shell(): no default shell on ttyS1
autopkgtest-virt-qemu: DBG: Shell setup: have user and password, logging in..
autopkgtest-virt-qemu: DBG: expect: "assword:"
autopkgtest-virt-qemu: DBG: expect: found ""password prompt""
autopkgtest-virt-qemu: DBG: expect: "<none>"
autopkgtest-virt-qemu: DBG: expect: found ""acked password""
autopkgtest-virt-qemu: DBG: login_tty: logged in
autopkgtest-virt-qemu: DBG: expect: "LOGIN_OK"
autopkgtest-virt-qemu: DBG: expect: found ""logged in""
autopkgtest-virt-qemu: DBG: expect: "<none>"
autopkgtest-virt-qemu: DBG: expect: found ""accepted ttyS1 shell command""
autopkgtest-virt-qemu: DBG: expect: "
logout"
autopkgtest-virt-qemu: DBG: expect: found ""b'\nlogout'""
autopkgtest-virt-qemu: DBG: expect: "ok"

Revision history for this message

Launchpad Janitor (janitor) wrote on 2016-11-05:

#5

This bug was fixed in the package autopkgtest - 4.2

---------------
autopkgtest (4.2) unstable; urgency=medium

  [ Martin Pitt ]
  * ssh: Fix result of tests that break the testbed. (LP: #1630578)
  * qemu: Fix user/password login mode without a ttyS1 root shell.
     - Concatenate bytes with each other, not a str to a byte.
     - Sync/flush console after sending password and ttyS1 shell command, wait
       until login is complete before continuing.
     - Send password to sudo for ttyS1 shell command, to also work with
       password-requiring sudo. (LP: #1630963)
  * qemu: Fix reboot in user/password login mode.
  * ssh: Add --capability option.
    This is useful to run tests which require an isolation level or have
    breaks-testbed without a setup script.
  * autopkgtest-build-lxd: Ask "lxc profile" for default bridge instead of
    /etc/default/lxd-bridge. The latter went away in recent LXD versions, and
    "lxc profile show default" also works in earlier versions.
  * Make --apt-upgrade consider a "404" error as test failure.
    Previously all errors from "apt-get update" were considered temporary
    failures, i. e. the setup command always exited with 1. But if we specify
    e. g. a nonexisting release from a distro or a PPA, this won't just go
    away by itself -- we want the test to actually fail instead of getting
    stuck in an eternal "retry on temporary failures" loop.
  * schroot: Don't fail on stderr of schroot as long as it succeeds.
    (LP: #1637898)
  * qemu: Hide detected udev file system properties on /dev/baseimage.
    (Closes: #842299)

  [ Simon McVittie ]
  * VirtSubproc: open arbitrary files in binary mode.
  * VirtSubproc: if check_exec status is nonzero, include stderr in message.
  * qemu: put the shared directory in /run. If the virtual machine's root
    filesystem is read-only, we won't be able to create /autopkgtest.
  * qemu: Move eofcat into /tmp. This avoids having to write it to /bin, which
    might be read-only. Re-create eofcat on every boot for this.
  * source_rules_command: log the result we got if it is not as expected
    (Closes: #842302)
  * Add 'needs-reboot' restriction. This allows tests to be explicit about
    needing to reboot the machine, rather than assuming that
    'isolation-machine' is enough. It is plumbed into the existing 'reboot'
    capability, which is distinct from 'isolation-machine'.
    (Closes: #842300)
  * Add --setup-commands-boot for commands that must be run at every boot.
    This can be used for doing "mount -o remount,rw /" before any dpkg
    operation, or transient setup like writing files into /run/.
    (Closes: #842091)

-- Martin Pitt <email address hidden> Tue, 01 Nov 2016 23:12:55 +0200

This bug was fixed in the package autopkgtest - 4.2

---------------
autopkgtest (4.2) unstable; urgency=medium

[ Martin Pitt ]
  * ssh: Fix result of tests that break the testbed. (LP: #1630578)
  * qemu: Fix user/password login mode without a ttyS1 root shell.
     - Concatenate bytes with each other, not a str to a byte.
     - Sync/flush console after sending password and ttyS1 shell command, wait
       until login is complete before continuing.
     - Send password to sudo for ttyS1 shell command, to also work with
       password-requiring sudo. (LP: #1630963)
  * qemu: Fix reboot in user/password login mode.
  * ssh: Add --capability option.
    This is useful to run tests which require an isolation level or have
    breaks-testbed without a setup script.
  * autopkgtest-build-lxd: Ask "lxc profile" for default bridge instead of
    /etc/default/lxd-bridge. The latter went away in recent LXD versions, and
    "lxc profile show default" also works in earlier versions.
  * Make --apt-upgrade consider a "404" error as test failure.
    Previously all errors from "apt-get update" were considered temporary
    failures, i. e. the setup command always exited with 1. But if we specify
    e. g. a nonexisting release from a distro or a PPA, this won't just go
    away by itself -- we want the test to actually fail instead of getting
    stuck in an eternal "retry on temporary failures" loop.
  * schroot: Don't fail on stderr of schroot as long as it succeeds.
    (LP: #1637898)
  * qemu: Hide detected udev file system properties on /dev/baseimage.
    (Closes: #842299)

[ Simon McVittie ]
  * VirtSubproc: open arbitrary files in binary mode.
  * VirtSubproc: if check_exec status is nonzero, include stderr in message.
  * qemu: put the shared directory in /run. If the virtual machine's root
    filesystem is read-only, we won't be able to create /autopkgtest.
  * qemu: Move eofcat into /tmp. This avoids having to write it to /bin, which
    might be read-only.  Re-create eofcat on every boot for this.
  * source_rules_command: log the result we got if it is not as expected
    (Closes: #842302)
  * Add 'needs-reboot' restriction. This allows tests to be explicit about
    needing to reboot the machine, rather than assuming that
    'isolation-machine' is enough. It is plumbed into the existing 'reboot'
    capability, which is distinct from 'isolation-machine'.
    (Closes: #842300)
  * Add --setup-commands-boot for commands that must be run at every boot.
    This can be used for doing "mount -o remount,rw /" before any dpkg
    operation, or transient setup like writing files into /run/.
    (Closes: #842091)

-- Martin Pitt <mpitt@debian.org>  Tue, 01 Nov 2016 23:12:55 +0200

Changed in autopkgtest (Ubuntu):
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.