Fallback networking doesn't handle IOError when reading sys/net/<iface>/carrier

Bug #1625766 reported by J
28
This bug affects 4 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
Unassigned
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Undecided
Unassigned
Yakkety
Fix Released
Low
Unassigned

Bug Description

Sometimes reading from /sys/class/net/<iface>/carrier returns an error and is unhangled causing fallback networking to not bring anything up.

[Original Description]

I am running Arch on a KVM vps provider. I installed using this template: Arch Linux 2016.03 64-bit (template). Everything was working fine until I decided to upgrade. I did pacman -Syu and everything upgraded without error until it restarted.

I had to manually install certain python packages. But, I kept getting more errors so I joined IRC.

Here's the log: https://irclogs.ubuntu.com/2016/09/20/%23cloud-init.html

Was told to post it Here to sum up everything

Related branches

J (jwillx)
description: updated
Ryan Harper (raharper)
description: updated
summary: - Arch issue with start up
+ Fallback networking doesn't handle IOError when reading
+ sys/net/<iface>/carrier
Changed in cloud-init:
importance: Undecided → High
status: New → Triaged
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → Medium
Changed in cloud-init:
status: Triaged → Fix Committed
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.9.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

I'm accepting this cloud-init upload into yakkety-proposed but this particular bug is missing SRU bug information, could it please be added?

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello J, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-68-gca3ae67-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Scott Moser (smoser) wrote :

This code path is involved on each boot, but on Ubuntu we never triggered the failure case. Failure case happens when some drivers were in different state. I realize this is not great, but I'm marking this as verification-done based on:

$ release=yakkety
$ lxc launch "ubuntu-daily:$release" "$name"
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-49-g9e904bb-0ubuntu1~16.10.1

$ lxc file pull $name/etc/cloud/build.info -
build_name: server
serial: 20170119

## enable proposed, update
$ m=http://archive.ubuntu.com/ubuntu;
$ echo "deb $m $release-proposed main" | lxc file push - $name/etc/apt/sources.list.d/proposed.list
$ lxc exec $name -- sh -c 'apt update -q && apt install cloud-init' </dev/null
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-68-gca3ae67-0ubuntu1~16.10.1

# clean up to make it look like first boot.
$ lxc exec $name -- sh -c 'cd /var/lib/cloud; for d in *; do [ "$d" = "seed" ] || rm -vRf "$d"; done'
$ lxc exec $name -- sh -c 'rm -f /var/log/cloud*'
$ lxc restart $name

$ lxc exec $name -- grep WARN /var/log/cloud-init.log || echo no warn
no warn

$ lxc lau

tags: added: verification-done
removed: verification-needed
Revision history for this message
Andy Whitcroft (apw) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.8-68-gca3ae67-0ubuntu1~16.10.1

---------------
cloud-init (0.7.8-68-gca3ae67-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cherry-pick: add utility for cherry picking commits from upstream
    into patches in debian/patches.
  * New upstream snapshot.
    - mounts: use mount -a again to accomplish mounts (LP: #1647708)
    - CloudSigma: Fix bug where datasource was not loaded in local search.
      (LP: #1648380)
    - when adding a user, strip whitespace from group list
      [Lars Kellogg-Stedman] (LP: #1354694)
    - fix decoding of utf-8 chars in yaml test
    - Replace usage of sys_netdev_info with read_sys_net (LP: #1625766)
    - fix problems found in python2.6 test.
    - OpenStack: extend physical types to include hyperv, hw_veb, vhost_user.
      (LP: #1642679)
    - tests: fix assumptions that expected no eth0 in system. (LP: #1644043)
    - net/cmdline: Consider ip= or ip6= on command line not only ip=
      (LP: #1639930)
    - Just use file logging by default [Joshua Harlow] (LP: #1643990)
    - Improve formatting for ProcessExecutionError [Wesley Wiedenmeier]
    - flake8: fix trailing white space
    - Doc: various documentation fixes [Sean Bright]
    - cloudinit/config/cc_rh_subscription.py: Remove repos before adding
      [Brent Baude]
    - packages/redhat: fix rpm spec file.
    - main: set TZ in environment if not already set. [Ryan Harper]
    - disk_setup: Use sectors as unit when formatting MBR disks with sfdisk.
      [Daniel Watkins] (LP: #1460715)

 -- Scott Moser <email address hidden> Mon, 19 Dec 2016 15:07:12 -0500

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello J, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: New → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello J, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Scott Moser (smoser) wrote :

$ release=xenial
$ lxc launch "ubuntu-daily:$release" "$name"
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-49-g9e904bb-0ubuntu1~16.04.4

$ lxc file pull $name/etc/cloud/build.info -
build_name: server
serial: 20170207

## enable proposed, update
$ m=http://archive.ubuntu.com/ubuntu;
$ echo "deb $m $release-proposed main" | lxc file push - $name/etc/apt/sources.list.d/proposed.list
$ lxc exec $name -- sh -c 'apt update -q && apt install cloud-init' </dev/null
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.9-0ubuntu1~16.04.2

# clean up to make it look like first boot.
$ lxc exec $name -- sh -c 'cd /var/lib/cloud; for d in *; do [ "$d" = "seed" ] || rm -vRf "$d"; done'
$ lxc exec $name -- sh -c 'rm -f /var/log/cloud*'
$ lxc restart $name

$ lxc exec $name -- grep WARN /var/log/cloud-init.log || echo no warn
no warn

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.9-0ubuntu1~16.04.2

---------------
cloud-init (0.7.9-0ubuntu1~16.04.2) xenial-proposed; urgency=medium

  * debian/update-grub-legacy-ec2: fix shell syntax error. (LP: #1662221)

cloud-init (0.7.9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/copyright: update License field to include Apache.
  * debian/update-grub-legacy-ec2: fix to include kernels whose config
    has CONFIG_XEN=y (LP: #1379080).
  * debian/patches/azure-use-walinux-agent.patch: continue relying on
    walinux agent in stable release.
  * New upstream release.
    - doc: adjust headers in tests documentation for consistency.
    - pep8: fix issue found in zesty build with pycodestyle.
    - integration test: initial commit of integration test framework
      [Wesley Wiedenmeier]
    - LICENSE: Allow dual licensing GPL-3 or Apache 2.0 [Jon Grimm]
    - Fix config order of precedence, putting kernel command line over system.
      [Wesley Wiedenmeier] (LP: #1582323)
    - pep8: whitespace fix [Scott Moser]
    - Update the list of valid ssh keys. [Michael Felt]
    - network: add ENI unit test for statically rendered routes.
    - set_hostname: avoid erroneously appending domain to fqdn
      [Lars Kellogg-Stedman] (LP: #1647910)
    - doc: change 'nobootwait' to 'nofail' in docs [Anhad Jai Singh]
    - Replace an expired bit.ly link in code comment. [Joshua Harlow]
    - user-groups: fix bug when groups was provided as string and had spaces
      [Scott Moser] (LP: #1354694)
    - when adding a user, strip whitespace from group list
      [Lars Kellogg-Stedman] (LP: #1354694)
    - fix decoding of utf-8 chars in yaml test
    - Replace usage of sys_netdev_info with read_sys_net
      [Joshua Harlow] (LP: #1625766)
    - fix problems found in python2.6 test. [Joshua Harlow]
    - Just use file logging by default [Joshua Harlow] (LP: #1643990)
    - Improve formatting for ProcessExecutionError [Wesley Wiedenmeier]
    - flake8: fix trailing white space
    - Doc: various documentation fixes [Sean Bright]
    - cloudinit/config/cc_rh_subscription.py: Remove repos before adding
      [Brent Baude]
    - packages/redhat: fix rpm spec file.
    - main: set TZ in environment if not already set. [Ryan Harper]

 -- Scott Moser <email address hidden> Mon, 06 Feb 2017 16:18:28 -0500

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.