Ubuntu
cloud-init package

cloudstack and azure datasources broken when using netplan/systemd-networkd

Bug #1718029 reported by Scott Moser
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Netplan
Invalid
Undecided
Unassigned
cloud-init
Fix Released
High
Unassigned
cloud-init (Ubuntu)
Fix Released
Critical
Scott Moser
nplan (Ubuntu)
Invalid
Undecided
Unassigned
systemd (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

In Ubuntu artful, cloud-init renders network configuration through netplan.
This means that there is no dhclient and thus no /var/lib/dhclient/*.leases.

Azure and CloudStack both are reading those leases file to get useful
information about the platform.

Specifically:
 * Azure reads option-245 from the dhclient response to find the IP address of the metadata service.
 * CloudStack reads the 'dhcp-server-identifier' option in the dhclient response to get the address of the virtual router (metadata service). [1]

In ubuntu this happens to be done with systemd-networkd, so cloud-init can possibly probably interact over the dbus with systemd-networkd to get information. However that is less than ideal, as ultimately cloud-init should not need to know that it systemd-networkd is involved. It should be hidden via netplan. So there should be an interface to get current networking configuratoin information from netplan including dhcp lease response info.

--
[1] http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/virtual_machines/user-data.html

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: cloud-init 0.7.9-280-ge626966e-0ubuntu1
ProcVersionSignature: Ubuntu 4.12.0-11.12-generic 4.12.5
Uname: Linux 4.12.0-11-generic x86_64
ApportVersion: 2.20.7-0ubuntu1
Architecture: amd64
CloudName: Amazon - Ec2
Date: Mon Sep 18 19:56:40 2017
PackageArchitecture: all
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)
user_data.txt:
 #cloud-config
 {}

Related branches

~raharper/cloud-init:ubuntu-devel-new-artful-release-v5
Scott Moser: Approve
Diff: 2252 lines (+1196/-148)
97 files modified
cloudinit/config/cc_ssh_authkey_fingerprints.py (+2/-2)
cloudinit/config/cc_zypper_add_repo.py (+218/-0)
cloudinit/net/dhcp.py (+42/-0)
cloudinit/net/tests/test_dhcp.py (+111/-2)
cloudinit/netinfo.py (+4/-4)
cloudinit/simpletable.py (+62/-0)
cloudinit/sources/DataSourceAltCloud.py (+2/-2)
cloudinit/sources/DataSourceCloudStack.py (+13/-4)
cloudinit/sources/DataSourceOVF.py (+47/-27)
cloudinit/sources/helpers/azure.py (+14/-6)
cloudinit/tests/helpers.py (+10/-0)
cloudinit/tests/test_simpletable.py (+100/-0)
config/cloud.cfg.tmpl (+3/-0)
debian/changelog (+25/-3)
debian/control (+0/-1)
packages/debian/copyright (+10/-15)
packages/pkg-deps.json (+0/-3)
requirements.txt (+0/-3)
systemd/cloud-final.service.tmpl (+3/-1)
tests/cloud_tests/__init__.py (+1/-1)
tests/cloud_tests/instances/nocloudkvm.py (+16/-15)
tests/cloud_tests/testcases/bugs/README.md (+0/-0)
tests/cloud_tests/testcases/bugs/lp1511485.yaml (+0/-0)
tests/cloud_tests/testcases/bugs/lp1611074.yaml (+0/-0)
tests/cloud_tests/testcases/bugs/lp1628337.yaml (+0/-0)
tests/cloud_tests/testcases/examples/README.md (+0/-0)
tests/cloud_tests/testcases/examples/TODO.md (+0/-0)
tests/cloud_tests/testcases/examples/add_apt_repositories.yaml (+0/-0)
tests/cloud_tests/testcases/examples/alter_completion_message.yaml (+0/-0)
tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.yaml (+0/-0)
tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.yaml (+0/-0)
tests/cloud_tests/testcases/examples/including_user_groups.yaml (+0/-0)
tests/cloud_tests/testcases/examples/install_arbitrary_packages.yaml (+0/-0)
tests/cloud_tests/testcases/examples/install_run_chef_recipes.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_apt_upgrade.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_commands.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_commands_first_boot.yaml (+0/-0)
tests/cloud_tests/testcases/examples/setup_run_puppet.yaml (+0/-0)
tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.yaml (+0/-0)
tests/cloud_tests/testcases/main/README.md (+0/-0)
tests/cloud_tests/testcases/main/command_output_simple.yaml (+0/-0)
tests/cloud_tests/testcases/modules/README.md (+0/-0)
tests/cloud_tests/testcases/modules/TODO.md (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_conf.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_disable_suites.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_primary.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_proxy.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_security.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_key.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_list.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_pipelining_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_pipelining_os.yaml (+0/-0)
tests/cloud_tests/testcases/modules/bootcmd.yaml (+0/-0)
tests/cloud_tests/testcases/modules/byobu.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ca_certs.yaml (+0/-0)
tests/cloud_tests/testcases/modules/debug_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/debug_enable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/final_message.yaml (+0/-0)
tests/cloud_tests/testcases/modules/keys_to_console.yaml (+0/-0)
tests/cloud_tests/testcases/modules/landscape.yaml (+0/-0)
tests/cloud_tests/testcases/modules/locale.yaml (+0/-0)
tests/cloud_tests/testcases/modules/lxd_bridge.yaml (+0/-0)
tests/cloud_tests/testcases/modules/lxd_dir.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp_pools.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp_servers.yaml (+0/-0)
tests/cloud_tests/testcases/modules/package_update_upgrade_install.yaml (+0/-0)
tests/cloud_tests/testcases/modules/runcmd.yaml (+0/-0)
tests/cloud_tests/testcases/modules/salt_minion.yaml (+0/-0)
tests/cloud_tests/testcases/modules/seed_random_command.yaml (+0/-0)
tests/cloud_tests/testcases/modules/seed_random_data.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_hostname.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_hostname_fqdn.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_expire.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_list.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_list_string.yaml (+0/-0)
tests/cloud_tests/testcases/modules/snappy.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_auth_key_fingerprints_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_auth_key_fingerprints_enable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_import_id.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_keys_generate.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_keys_provided.yaml (+0/-0)
tests/cloud_tests/testcases/modules/timezone.yaml (+0/-0)
tests/cloud_tests/testcases/modules/user_groups.yaml (+0/-0)
tests/cloud_tests/testcases/modules/write_files.yaml (+0/-0)
tests/unittests/test_datasource/test_altcloud.py (+2/-2)
tests/unittests/test_datasource/test_azure_helper.py (+95/-48)
tests/unittests/test_datasource/test_cloudstack.py (+7/-4)
tests/unittests/test_datasource/test_ovf.py (+164/-0)
tests/unittests/test_handler/test_handler_bootcmd.py (+1/-0)
tests/unittests/test_handler/test_handler_zypper_add_repo.py (+237/-0)
tests/unittests/test_handler/test_schema.py (+7/-1)
tools/build-on-freebsd (+0/-1)
tox.ini (+0/-3)
~smoser/cloud-init:bug/1718029-fix-dhcp-parsing-from-networkd
Chad Smith: Approve
Dimitri John Ledkov (community): Approve
Server Team CI bot: Approve (continuous-integration)
Diff: 629 lines (+282/-64)
6 files modified
cloudinit/net/dhcp.py (+42/-0)
cloudinit/net/tests/test_dhcp.py (+111/-2)
cloudinit/sources/DataSourceCloudStack.py (+13/-4)
cloudinit/sources/helpers/azure.py (+14/-6)
tests/unittests/test_datasource/test_azure_helper.py (+95/-48)
tests/unittests/test_datasource/test_cloudstack.py (+7/-4)
~xnox/cloud-init:networkd
On hold for merging into cloud-init:master
Dimitri John Ledkov (community): Needs Resubmitting
Server Team CI bot: Approve (continuous-integration)
cloud-init Commiters: Pending requested
Diff: 174 lines (+84/-6)
3 files modified
cloudinit/sources/DataSourceCloudStack.py (+20/-1)
cloudinit/sources/helpers/azure.py (+23/-5)
tests/unittests/test_datasource/test_azure_helper.py (+41/-0)
Revision history for this message
Scott Moser (smoser) wrote :
Changed in cloud-init (Ubuntu):
importance: Undecided → Critical
status: New → Confirmed
Revision history for this message
Steve Langasek (vorlon) wrote :

I understand that systemd in artful is already exposing these options upstream, so marking this task fixed.

Changed in systemd (Ubuntu):
status: New → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote :

And I understand Dimitri has the necessary changes to systemd in progress.

Changed in cloud-init (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

No changes in netplan required.

Changed in nplan (Ubuntu):
status: New → Invalid
Changed in netplan:
status: New → Invalid
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

For CloudStack, I need a CloudStack instance =/ can I have a xenial instance please?

Dimitri John Ledkov (xnox)
Changed in cloud-init (Ubuntu):
status: Confirmed → In Progress
Dimitri John Ledkov (xnox)
Changed in cloud-init (Ubuntu):
assignee: Dimitri John Ledkov (xnox) → Scott Moser (smoser)
Scott Moser (smoser)
Changed in cloud-init:
status: New → Fix Committed
importance: Undecided → High
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 17.1-13-g7fd04255-0ubuntu1

---------------
cloud-init (17.1-13-g7fd04255-0ubuntu1) artful; urgency=medium

  * debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.
    (LP: #1718681)
  * debian/control: drop dependency on python3-prettytable
  * New upstream snapshot.
    - systemd: remove limit on tasks created by cloud-init-final.service.
      [Robert Schweikert] (LP: #1717969)
    - suse: Support addition of zypper repos via cloud-config.
      [Robert Schweikert] (LP: #1718675)
    - tests: Combine integration configs and testcases [Joshua Powers]
    - Azure, CloudStack: Support reading dhcp options from systemd-networkd.
      [Dimitri John Ledkov] (LP: #1718029)
    - packages/debian/copyright: remove mention of boto and MIT license
    - systemd: only mention Before=apt-daily.service on debian based distros.
      [Robert Schweikert]
    - Add missing simpletable and simpletable tests for failed merge
      [Chad Smith]
    - Remove prettytable dependency, introduce simpletable [Andrew Jorgensen]
    - debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.
      [Joshua Powers] (LP: #1718681)
    - tests: remove dependency on shlex [Joshua Powers]
    - AltCloud: Trust PATH for udevadm and modprobe.
    - DataSourceOVF: use util.find_devs_with(TYPE=iso9660)
      [Ryan Harper] (LP: #1718287)
    - tests: remove a temp file used in bootcmd tests.

 -- Ryan Harper <email address hidden> Tue, 03 Oct 2017 10:59:52 -0500

Changed in cloud-init (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Scott Moser (smoser) wrote : Fixed in Cloud-init 1705804

This bug is believed to be fixed in cloud-init in 1705804. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3004

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.