Wily: add machine fails using kvm and lxcbr0

Bug #1510619 reported by Adam Stokes
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
juju-core
Invalid
High
Unassigned
lxc (Ubuntu)
Fix Released
High
Unassigned
Vivid
Fix Released
Undecided
Unassigned
Wily
Fix Released
Undecided
Unassigned

Bug Description

Juju fails to start a machine after add-machine takes place.

My environments.yaml:

local:
  type: local
  container: kvm
  network-bridge: lxcbr0

Status output:

ubuntu@ancient-spot:~$ juju status
environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.24.7.1
    dns-name: localhost
    instance-id: localhost
    series: wily
    state-server-member-status: has-vote
  "1":
    agent-state: pending
    instance-id: ubuntu-local-machine-1
    series: wily
    hardware: arch=amd64 cpu-cores=1 mem=512M root-disk=8192M

debug log:

http://paste.ubuntu.com/12980615/

I never see anything in the logs which indicate machine-1 is being deployed or what else could be going on with it.

tags: added: cloud-installer
Curtis Hovey (sinzui)
tags: added: kvm
tags: added: network
tags: added: add-machine
Changed in juju-core:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Ryan Harper (raharper) wrote :
Download full text (3.4 KiB)

It appears that lxc-net starts up before all of the network interfaces (eth0 in this case) are online. When it checks to see if anything is using ips in 10.0.3.X/16 nothing is (because eth0 isn't up).

Then ifup-wait-all triggers an ifup eth0 which gets an ip from the hosts 10.0.3.1 lxcbr0 bridge but since the internal lxcbr0 is on 10.0.3.1, none of the traffic gets to eth0 (or out of it).

[ 11.456968] cloud-init[558]: Cloud-init v. 0.7.7 running 'init-local' at Tue, 27 Oct 2015 16:49:58 +0000. Up 11.03 seconds.^M^M
[^[[32m OK ^[[0m] Started Initial cloud-init job (pre-networking).^M^M
cloud-init-local.service^M^M
[^[[32m OK ^[[0m] Started Container hypervisor based on LXC - boot time check.^M^M
[^[[32m OK ^[[0m] Started Wait for all "auto" /etc/ne...be up for network-online.target.^M^M
[^[[32m OK ^[[0m] Reached target Network is Online.^M^M
         Starting /etc/rc.local Compatibility...^M^M
         Starting Initial cloud-init job (metadata service crawler)...^M^M
ifup-wait-all-auto.service^M^M
[^[[32m OK ^[[0m] Started /etc/rc.local Compatibility.^M^M
rc-local.service^M^M
tmp-tmpv77mgee6.mount^M^M
[ 13.042028] cloud-init[880]: Oct 27 16:50:00 ubuntu <13>Oct 27 16:50:00 pollinate[941]: ERROR: Network communication failed [6]\n % Total % Received % Xferd Average Speed Time Time Time Current^M^M
[ 13.045474] cloud-init[880]: Dload Upload Total Spent Left Speed^M^M
[ 13.046276] cloud-init[880]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 016:50:00.039432 * Could not resolve host: entropy.ubuntu.com^M^M
[ 13.046467] cloud-init[880]: 16:50:00.039489 * Closing connection 0^M^M
[ 13.046648] cloud-init[880]: curl: (6) Could not resolve host: entropy.ubuntu.com^M^M
[ 13.050804] cloud-init[880]: Cloud-init v. 0.7.7 running 'init' at Tue, 27 Oct 2015 16:50:00 +0000. Up 12.60 seconds.^M^M
[ 13.052244] cloud-init[880]: ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++^M^M
[ 13.052431] cloud-init[880]: ci-info: +--------+------+------------------------------+---------------+-------+-------------------+^M^M
[ 13.052615] cloud-init[880]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |^M^M
[ 13.052794] cloud-init[880]: ci-info: +--------+------+------------------------------+---------------+-------+-------------------+^M^M
[ 13.052973] cloud-init[880]: ci-info: | eth0 | True | 10.0.3.90 | 255.255.255.0 | . | 52:54:00:77:2e:57 |^M^M
[ 13.053149] cloud-init[880]: ci-info: | eth0 | True | fe80::5054:ff:fe77:2e57/64 | . | link | 52:54:00:77:2e:57 |^M^M
[ 13.053368] cloud-init[880]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . |^M^M
[ 13.055164] cloud-init[880]: ci-info: | lo | True | ::1/128 | . | host | . |^M^M
[ 13.055340] cloud-init[880]: ci-info: | lxcbr0 | True | 10.0.3.1 | 255.255.255.0 | . | 1e:e1:f4:04:a6:85 |^M^M
[ 13.055516] cloud-init[880]: ci-info: | lxcbr0 | True | fe80::...

Read more...

Changed in juju-core:
status: Triaged → Invalid
Revision history for this message
Ryan Harper (raharper) wrote :

Currently looking at updatin lxc package to have lxc-net run After=network-online.target which should ensure that ifup on auto interfaces has run, meaning ethX interfaces will have obtained and IP and at that point then lxc-net can check if something is squating on 10.0.3.X network and switch to a new one.

Revision history for this message
Ryan Harper (raharper) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in lxc (Ubuntu):
status: New → Confirmed
Revision history for this message
Stéphane Graber (stgraber) wrote :

This will be fixed with 1.0.8.

Changed in lxc (Ubuntu):
status: Confirmed → In Progress
importance: Undecided → High
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxc - 1.1.5-0ubuntu1

---------------
lxc (1.1.5-0ubuntu1) xenial; urgency=medium

  * New upstream bugfix release (1.1.5)
    (LP: #1514558, LP: #1497420, LP: #1466458, LP: #1510619)
  * Drop proxy detection from the autopkgtest exercise script.

 -- Stéphane Graber <email address hidden> Mon, 09 Nov 2015 14:22:16 -0500

Changed in lxc (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Adam, or anyone else affected,

Accepted lxc into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxc/1.1.5-0ubuntu0.15.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in lxc (Ubuntu Vivid):
status: New → Fix Committed
tags: added: verification-needed
Changed in lxc (Ubuntu Wily):
status: New → Fix Committed
Revision history for this message
Chris J Arges (arges) wrote :

Hello Adam, or anyone else affected,

Accepted lxc into wily-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxc/1.1.5-0ubuntu0.15.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Chris J Arges (arges) wrote :

Hello Adam, or anyone else affected,

Accepted lxc into wily-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxc/1.1.5-0ubuntu0.15.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Chris J Arges (arges) wrote :

Hello Adam, or anyone else affected,

Accepted lxc into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxc/1.1.5-0ubuntu0.15.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxc - 1.1.5-0ubuntu0.15.04.3

---------------
lxc (1.1.5-0ubuntu0.15.04.3) vivid-proposed; urgency=medium

  * Cherry-pick from upstream:
    - Fix preserve_ns to work on < 3.8 kernels. (LP: #1516971)
    - Fix process title rewrite to not mangle the environment. (LP: #1517107)

lxc (1.1.5-0ubuntu0.15.04.2) vivid-proposed; urgency=medium

  * Cherry-pick from upstream:
    - Fix ubuntu-cloud template to detect compression algorithm instead
      of hardcoding xz. Also update list of supported releases and use trusty
      as the fallback release. (LP: #1515463)
  * Update lxc-tests description to make it clear that this package is
    meant to be used by developers and by automated testing.

lxc (1.1.5-0ubuntu0.15.04.1) vivid-proposed; urgency=medium

  * New upstream bugfix release (MRE) (1.1.5)
    (LP: #1497420, LP: #1436723, LP: #1441068, LP: #1504496,
     LP: #1466458, LP: #1510619)
  * Drop proxy detection from the autopkgtest exercise script.

 -- Stéphane Graber <email address hidden> Wed, 18 Nov 2015 13:41:23 -0500

Changed in lxc (Ubuntu Vivid):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxc - 1.1.5-0ubuntu0.15.10.3

---------------
lxc (1.1.5-0ubuntu0.15.10.3) wily-proposed; urgency=medium

  * Cherry-pick from upstream:
    - Fix preserve_ns to work on < 3.8 kernels. (LP: #1516971)
    - Fix process title rewrite to not mangle the environment. (LP: #1517107)

lxc (1.1.5-0ubuntu0.15.10.2) wily-proposed; urgency=medium

  * Cherry-pick from upstream:
    - Fix ubuntu-cloud template to detect compression algorithm instead
      of hardcoding xz. Also update list of supported releases and use trusty
      as the fallback release. (LP: #1515463)
  * Update lxc-tests description to make it clear that this package is
    meant to be used by developers and by automated testing.

lxc (1.1.5-0ubuntu0.15.10.1) wily-proposed; urgency=medium

  * New upstream bugfix release (MRE) (1.1.5)
    (LP: #1497420, LP: #1441068, LP: #1466458, LP: #1510619)
  * Drop proxy detection from the autopkgtest exercise script.

 -- Stéphane Graber <email address hidden> Wed, 18 Nov 2015 13:40:28 -0500

Changed in lxc (Ubuntu Wily):
status: Fix Committed → Fix Released
Revision history for this message
Stéphane Graber (stgraber) wrote : Update Released

The verification of the Stable Release Update for lxc has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.