Ubuntu
zfs-linux package

zfs-import-cache.service slows boot by 60 seconds

Bug #1571761 reported by Scott Moser
32
This bug affects 4 people
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Fix Released
High
Unassigned
Xenial
Fix Released
High
Scott Moser
zfs-linux (Ubuntu)
Fix Released
High
Unassigned
Xenial
Fix Released
High
Unassigned

Bug Description

Fresh uvt-kvm guest, then
$ sudo apt-get install zfsutils-linux
$ sudo reboot

The reboot will show on console waiting for tasks, then after ~ 60 seconds it continues boot.
Logging in shows:

$ sudo systemd-analyze critical-chain zfs-mount.service
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.

zfs-mount.service +81ms
└─zfs-import-cache.service @1min 786ms +272ms
  └─systemd-udev-settle.service @494ms +1min 275ms
    └─systemd-udev-trigger.service @415ms +55ms
      └─systemd-udevd-control.socket @260ms
        └─-.mount @124ms
          └─system.slice @129ms
            └─-.slice @124ms

Seems possibly related / or some discussion at https://github.com/zfsonlinux/zfs/issues/2368

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: zfsutils-linux 0.6.5.6-0ubuntu8
ProcVersionSignature: User Name 4.4.0-18.34-generic 4.4.6
Uname: Linux 4.4.0-18-generic x86_64
NonfreeKernelModules: zfs zunicode zcommon znvpair zavl
ApportVersion: 2.20.1-0ubuntu1
Architecture: amd64
Date: Mon Apr 18 16:42:35 2016
ProcEnviron:
 TERM=screen.xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: zfs-linux
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.sudoers.d.zfs: [deleted]

Related branches

lp:~cloud-init-dev/cloud-init/trunk
Chad Smith (community): Needs Fixing
Hannes: Pending requested
Diff: 51583 lines (+11/-49380) (has conflicts)
424 files modified
.bzrignore (+0/-4)
ChangeLog (+0/-761)
HACKING.rst (+0/-48)
LICENSE (+0/-674)
MANIFEST.in (+0/-8)
Makefile (+0/-82)
README (+11/-0)
TODO.rst (+0/-43)
bin/cloud-init (+0/-673)
cloudinit/__init__.py (+0/-21)
cloudinit/cloud.py (+0/-109)
cloudinit/config/__init__.py (+0/-58)
cloudinit/config/cc_apt_configure.py (+0/-292)
cloudinit/config/cc_apt_pipelining.py (+0/-57)
cloudinit/config/cc_bootcmd.py (+0/-54)
cloudinit/config/cc_byobu.py (+0/-80)
cloudinit/config/cc_ca_certs.py (+0/-104)
cloudinit/config/cc_chef.py (+0/-342)
cloudinit/config/cc_debug.py (+0/-109)
cloudinit/config/cc_disable_ec2_metadata.py (+0/-36)
cloudinit/config/cc_emit_upstart.py (+0/-69)
cloudinit/config/cc_fan.py (+0/-101)
cloudinit/config/cc_final_message.py (+0/-73)
cloudinit/config/cc_foo.py (+0/-52)
cloudinit/config/cc_growpart.py (+0/-300)
cloudinit/config/cc_grub_dpkg.py (+0/-73)
cloudinit/config/cc_keys_to_console.py (+0/-62)
cloudinit/config/cc_landscape.py (+0/-99)
cloudinit/config/cc_locale.py (+0/-37)
cloudinit/config/cc_lxd.py (+0/-176)
cloudinit/config/cc_mcollective.py (+0/-88)
cloudinit/config/cc_migrator.py (+0/-85)
cloudinit/config/cc_mounts.py (+0/-405)
cloudinit/config/cc_package_update_upgrade_install.py (+0/-99)
cloudinit/config/cc_phone_home.py (+0/-122)
cloudinit/config/cc_power_state_change.py (+0/-223)
cloudinit/config/cc_puppet.py (+0/-118)
cloudinit/config/cc_resizefs.py (+0/-185)
cloudinit/config/cc_resolv_conf.py (+0/-116)
cloudinit/config/cc_rh_subscription.py (+0/-408)
cloudinit/config/cc_rightscale_userdata.py (+0/-102)
cloudinit/config/cc_rsyslog.py (+0/-366)
cloudinit/config/cc_runcmd.py (+0/-38)
cloudinit/config/cc_salt_minion.py (+0/-59)
cloudinit/config/cc_scripts_per_boot.py (+0/-41)
cloudinit/config/cc_scripts_per_instance.py (+0/-41)
cloudinit/config/cc_scripts_per_once.py (+0/-41)
cloudinit/config/cc_scripts_user.py (+0/-42)
cloudinit/config/cc_scripts_vendor.py (+0/-43)
cloudinit/config/cc_seed_random.py (+0/-94)
cloudinit/config/cc_set_hostname.py (+0/-37)
cloudinit/config/cc_set_passwords.py (+0/-167)
cloudinit/config/cc_snappy.py (+0/-304)
cloudinit/config/cc_ssh.py (+0/-142)
cloudinit/config/cc_ssh_authkey_fingerprints.py (+0/-105)
cloudinit/config/cc_ssh_import_id.py (+0/-99)
cloudinit/config/cc_timezone.py (+0/-39)
cloudinit/config/cc_ubuntu_init_switch.py (+0/-162)
cloudinit/config/cc_update_etc_hosts.py (+0/-60)
cloudinit/config/cc_update_hostname.py (+0/-43)
cloudinit/config/cc_users_groups.py (+0/-34)
cloudinit/config/cc_write_files.py (+0/-103)
cloudinit/config/cc_yum_add_repo.py (+0/-107)
cloudinit/cs_utils.py (+0/-105)
cloudinit/distros/__init__.py (+0/-950)
cloudinit/distros/arch.py (+0/-201)
cloudinit/distros/debian.py (+0/-227)
cloudinit/distros/fedora.py (+0/-31)
cloudinit/distros/freebsd.py (+0/-417)
cloudinit/distros/gentoo.py (+0/-160)
cloudinit/distros/net_util.py (+0/-182)
cloudinit/distros/parsers/__init__.py (+0/-28)
cloudinit/distros/parsers/hostname.py (+0/-88)
cloudinit/distros/parsers/hosts.py (+0/-92)
cloudinit/distros/parsers/resolv_conf.py (+0/-169)
cloudinit/distros/parsers/sys_conf.py (+0/-113)
cloudinit/distros/rhel.py (+0/-222)
cloudinit/distros/rhel_util.py (+0/-89)
cloudinit/distros/sles.py (+0/-179)
cloudinit/distros/ubuntu.py (+0/-31)
cloudinit/ec2_utils.py (+0/-201)
cloudinit/filters/__init__.py (+0/-21)
cloudinit/filters/launch_index.py (+0/-75)
cloudinit/handlers/__init__.py (+0/-274)
cloudinit/handlers/boot_hook.py (+0/-70)
cloudinit/handlers/cloud_config.py (+0/-163)
cloudinit/handlers/shell_script.py (+0/-55)
cloudinit/handlers/upstart_job.py (+0/-119)
cloudinit/helpers.py (+0/-458)
cloudinit/importer.py (+0/-58)
cloudinit/log.py (+0/-155)
cloudinit/mergers/__init__.py (+0/-166)
cloudinit/mergers/m_dict.py (+0/-88)
cloudinit/mergers/m_list.py (+0/-89)
cloudinit/mergers/m_str.py (+0/-46)
cloudinit/net/__init__.py (+0/-771)
cloudinit/net/network_state.py (+0/-447)
cloudinit/net/udev.py (+0/-54)
cloudinit/netinfo.py (+0/-249)
cloudinit/patcher.py (+0/-58)
cloudinit/registry.py (+0/-37)
cloudinit/reporting/__init__.py (+0/-42)
cloudinit/reporting/events.py (+0/-248)
cloudinit/reporting/handlers.py (+0/-91)
cloudinit/safeyaml.py (+0/-32)
cloudinit/settings.py (+0/-68)
cloudinit/signal_handler.py (+0/-71)
cloudinit/sources/DataSourceAltCloud.py (+0/-293)
cloudinit/sources/DataSourceAzure.py (+0/-651)
cloudinit/sources/DataSourceBigstep.py (+0/-57)
cloudinit/sources/DataSourceCloudSigma.py (+0/-141)
cloudinit/sources/DataSourceCloudStack.py (+0/-253)
cloudinit/sources/DataSourceConfigDrive.py (+0/-424)
cloudinit/sources/DataSourceDigitalOcean.py (+0/-110)
cloudinit/sources/DataSourceEc2.py (+0/-211)
cloudinit/sources/DataSourceGCE.py (+0/-167)
cloudinit/sources/DataSourceMAAS.py (+0/-353)
cloudinit/sources/DataSourceNoCloud.py (+0/-336)
cloudinit/sources/DataSourceNone.py (+0/-57)
cloudinit/sources/DataSourceOVF.py (+0/-429)
cloudinit/sources/DataSourceOpenNebula.py (+0/-455)
cloudinit/sources/DataSourceOpenStack.py (+0/-173)
cloudinit/sources/DataSourceSmartOS.py (+0/-536)
cloudinit/sources/__init__.py (+0/-337)
cloudinit/sources/helpers/__init__.py (+0/-13)
cloudinit/sources/helpers/azure.py (+0/-279)
cloudinit/sources/helpers/openstack.py (+0/-499)
cloudinit/sources/helpers/vmware/__init__.py (+0/-13)
cloudinit/sources/helpers/vmware/imc/__init__.py (+0/-13)
cloudinit/sources/helpers/vmware/imc/boot_proto.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/config.py (+0/-95)
cloudinit/sources/helpers/vmware/imc/config_file.py (+0/-129)
cloudinit/sources/helpers/vmware/imc/config_namespace.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+0/-247)
cloudinit/sources/helpers/vmware/imc/config_source.py (+0/-23)
cloudinit/sources/helpers/vmware/imc/guestcust_error.py (+0/-24)
cloudinit/sources/helpers/vmware/imc/guestcust_event.py (+0/-27)
cloudinit/sources/helpers/vmware/imc/guestcust_state.py (+0/-25)
cloudinit/sources/helpers/vmware/imc/guestcust_util.py (+0/-128)
cloudinit/sources/helpers/vmware/imc/ipv4_mode.py (+0/-45)
cloudinit/sources/helpers/vmware/imc/nic.py (+0/-147)
cloudinit/sources/helpers/vmware/imc/nic_base.py (+0/-154)
cloudinit/ssh_util.py (+0/-314)
cloudinit/stages.py (+0/-844)
cloudinit/templater.py (+0/-149)
cloudinit/type_utils.py (+0/-52)
cloudinit/url_helper.py (+0/-509)
cloudinit/user_data.py (+0/-352)
cloudinit/util.py (+0/-2226)
cloudinit/version.py (+0/-27)
config/cloud.cfg (+0/-115)
config/cloud.cfg-freebsd (+0/-88)
config/cloud.cfg.d/05_logging.cfg (+0/-66)
config/cloud.cfg.d/README (+0/-3)
doc/README (+0/-4)
doc/examples/cloud-config-add-apt-repos.txt (+0/-34)
doc/examples/cloud-config-archive-launch-index.txt (+0/-30)
doc/examples/cloud-config-archive.txt (+0/-16)
doc/examples/cloud-config-boot-cmds.txt (+0/-15)
doc/examples/cloud-config-ca-certs.txt (+0/-31)
doc/examples/cloud-config-chef-oneiric.txt (+0/-90)
doc/examples/cloud-config-chef.txt (+0/-95)
doc/examples/cloud-config-datasources.txt (+0/-73)
doc/examples/cloud-config-disk-setup.txt (+0/-251)
doc/examples/cloud-config-final-message.txt (+0/-7)
doc/examples/cloud-config-gluster.txt (+0/-18)
doc/examples/cloud-config-growpart.txt (+0/-31)
doc/examples/cloud-config-install-packages.txt (+0/-15)
doc/examples/cloud-config-landscape.txt (+0/-22)
doc/examples/cloud-config-launch-index.txt (+0/-23)
doc/examples/cloud-config-lxd.txt (+0/-55)
doc/examples/cloud-config-mcollective.txt (+0/-49)
doc/examples/cloud-config-mount-points.txt (+0/-46)
doc/examples/cloud-config-phone-home.txt (+0/-14)
doc/examples/cloud-config-power-state.txt (+0/-40)
doc/examples/cloud-config-puppet.txt (+0/-51)
doc/examples/cloud-config-reporting.txt (+0/-17)
doc/examples/cloud-config-resolv-conf.txt (+0/-20)
doc/examples/cloud-config-rh_subscription.txt (+0/-49)
doc/examples/cloud-config-rsyslog.txt (+0/-46)
doc/examples/cloud-config-run-cmds.txt (+0/-22)
doc/examples/cloud-config-salt-minion.txt (+0/-53)
doc/examples/cloud-config-seed-random.txt (+0/-32)
doc/examples/cloud-config-ssh-keys.txt (+0/-46)
doc/examples/cloud-config-update-apt.txt (+0/-7)
doc/examples/cloud-config-update-packages.txt (+0/-8)
doc/examples/cloud-config-user-groups.txt (+0/-109)
doc/examples/cloud-config-vendor-data.txt (+0/-16)
doc/examples/cloud-config-write-files.txt (+0/-33)
doc/examples/cloud-config-yum-repo.txt (+0/-20)
doc/examples/cloud-config.txt (+0/-645)
doc/examples/include-once.txt (+0/-7)
doc/examples/include.txt (+0/-5)
doc/examples/kernel-cmdline.txt (+0/-18)
doc/examples/part-handler-v2.txt (+0/-38)
doc/examples/part-handler.txt (+0/-23)
doc/examples/plain-ignored.txt (+0/-2)
doc/examples/seed/README (+0/-22)
doc/examples/seed/meta-data (+0/-30)
doc/examples/seed/user-data (+0/-3)
doc/examples/upstart-cloud-config.txt (+0/-12)
doc/examples/upstart-rclocal.txt (+0/-12)
doc/examples/user-script.txt (+0/-8)
doc/merging.rst (+0/-188)
doc/rtd/conf.py (+0/-77)
doc/rtd/index.rst (+0/-31)
doc/rtd/static/logo.svg (+0/-89)
doc/rtd/topics/availability.rst (+0/-20)
doc/rtd/topics/capabilities.rst (+0/-24)
doc/rtd/topics/datasources.rst (+0/-200)
doc/rtd/topics/dir_layout.rst (+0/-81)
doc/rtd/topics/examples.rst (+0/-133)
doc/rtd/topics/format.rst (+0/-159)
doc/rtd/topics/hacking.rst (+0/-1)
doc/rtd/topics/merging.rst (+0/-5)
doc/rtd/topics/modules.rst (+0/-342)
doc/rtd/topics/moreinfo.rst (+0/-12)
doc/sources/altcloud/README.rst (+0/-87)
doc/sources/azure/README.rst (+0/-134)
doc/sources/cloudsigma/README.rst (+0/-38)
doc/sources/cloudstack/README.rst (+0/-29)
doc/sources/configdrive/README.rst (+0/-123)
doc/sources/digitalocean/README.rst (+0/-21)
doc/sources/kernel-cmdline.txt (+0/-48)
doc/sources/nocloud/README.rst (+0/-71)
doc/sources/opennebula/README.rst (+0/-142)
doc/sources/openstack/README.rst (+0/-24)
doc/sources/ovf/README (+0/-83)
doc/sources/ovf/example/ovf-env.xml (+0/-46)
doc/sources/ovf/example/ubuntu-server.ovf (+0/-130)
doc/sources/ovf/make-iso (+0/-156)
doc/sources/ovf/ovf-env.xml.tmpl (+0/-28)
doc/sources/ovf/ovfdemo.pem (+0/-27)
doc/sources/ovf/user-data (+0/-7)
doc/sources/smartos/README.rst (+0/-149)
doc/status.txt (+0/-53)
doc/userdata.txt (+0/-79)
doc/var-lib-cloud.txt (+0/-63)
doc/vendordata.txt (+0/-53)
packages/bddeb (+0/-260)
packages/brpm (+0/-277)
packages/debian/changelog.in (+0/-6)
packages/debian/cloud-init.postinst (+0/-16)
packages/debian/cloud-init.preinst (+0/-20)
packages/debian/compat (+0/-1)
packages/debian/control.in (+0/-29)
packages/debian/copyright (+0/-29)
packages/debian/dirs (+0/-6)
packages/debian/rules.in (+0/-23)
packages/debian/watch (+0/-2)
packages/redhat/cloud-init.spec.in (+0/-200)
packages/suse/cloud-init.spec.in (+0/-163)
requirements.txt (+0/-36)
setup.py (+0/-215)
systemd/cloud-config.service (+0/-16)
systemd/cloud-config.target (+0/-11)
systemd/cloud-final.service (+0/-17)
systemd/cloud-init-generator (+0/-133)
systemd/cloud-init-local.service (+0/-22)
systemd/cloud-init.service (+0/-18)
systemd/cloud-init.target (+0/-6)
sysvinit/debian/cloud-config (+0/-64)
sysvinit/debian/cloud-final (+0/-66)
sysvinit/debian/cloud-init (+0/-64)
sysvinit/debian/cloud-init-local (+0/-63)
sysvinit/freebsd/cloudconfig (+0/-35)
sysvinit/freebsd/cloudfinal (+0/-35)
sysvinit/freebsd/cloudinit (+0/-35)
sysvinit/freebsd/cloudinitlocal (+0/-35)
sysvinit/gentoo/cloud-config (+0/-13)
sysvinit/gentoo/cloud-final (+0/-11)
sysvinit/gentoo/cloud-init (+0/-12)
sysvinit/gentoo/cloud-init-local (+0/-13)
sysvinit/redhat/cloud-config (+0/-121)
sysvinit/redhat/cloud-final (+0/-121)
sysvinit/redhat/cloud-init (+0/-121)
sysvinit/redhat/cloud-init-local (+0/-124)
templates/chef_client.rb.tmpl (+0/-58)
templates/hosts.debian.tmpl (+0/-26)
templates/hosts.freebsd.tmpl (+0/-24)
templates/hosts.redhat.tmpl (+0/-24)
templates/hosts.suse.tmpl (+0/-26)
templates/resolv.conf.tmpl (+0/-30)
templates/sources.list.debian.tmpl (+0/-32)
templates/sources.list.ubuntu.tmpl (+0/-58)
test-requirements.txt (+0/-17)
tests/configs/sample1.yaml (+0/-52)
tests/data/filter_cloud_multipart.yaml (+0/-30)
tests/data/filter_cloud_multipart_1.email (+0/-11)
tests/data/filter_cloud_multipart_2.email (+0/-39)
tests/data/filter_cloud_multipart_header.email (+0/-11)
tests/data/merge_sources/expected1.yaml (+0/-1)
tests/data/merge_sources/expected10.yaml (+0/-7)
tests/data/merge_sources/expected11.yaml (+0/-5)
tests/data/merge_sources/expected12.yaml (+0/-5)
tests/data/merge_sources/expected2.yaml (+0/-3)
tests/data/merge_sources/expected3.yaml (+0/-1)
tests/data/merge_sources/expected4.yaml (+0/-2)
tests/data/merge_sources/expected5.yaml (+0/-7)
tests/data/merge_sources/expected6.yaml (+0/-9)
tests/data/merge_sources/expected7.yaml (+0/-38)
tests/data/merge_sources/expected8.yaml (+0/-7)
tests/data/merge_sources/expected9.yaml (+0/-5)
tests/data/merge_sources/source1-1.yaml (+0/-3)
tests/data/merge_sources/source1-2.yaml (+0/-5)
tests/data/merge_sources/source10-1.yaml (+0/-6)
tests/data/merge_sources/source10-2.yaml (+0/-6)
tests/data/merge_sources/source11-1.yaml (+0/-5)
tests/data/merge_sources/source11-2.yaml (+0/-3)
tests/data/merge_sources/source11-3.yaml (+0/-3)
tests/data/merge_sources/source12-1.yaml (+0/-8)
tests/data/merge_sources/source12-2.yaml (+0/-5)
tests/data/merge_sources/source2-1.yaml (+0/-6)
tests/data/merge_sources/source2-2.yaml (+0/-5)
tests/data/merge_sources/source3-1.yaml (+0/-4)
tests/data/merge_sources/source3-2.yaml (+0/-4)
tests/data/merge_sources/source4-1.yaml (+0/-3)
tests/data/merge_sources/source4-2.yaml (+0/-6)
tests/data/merge_sources/source5-1.yaml (+0/-6)
tests/data/merge_sources/source5-2.yaml (+0/-8)
tests/data/merge_sources/source6-1.yaml (+0/-5)
tests/data/merge_sources/source6-2.yaml (+0/-8)
tests/data/merge_sources/source7-1.yaml (+0/-27)
tests/data/merge_sources/source7-2.yaml (+0/-17)
tests/data/merge_sources/source8-1.yaml (+0/-7)
tests/data/merge_sources/source8-2.yaml (+0/-6)
tests/data/merge_sources/source9-1.yaml (+0/-5)
tests/data/merge_sources/source9-2.yaml (+0/-6)
tests/data/mountinfo_precise_ext4.txt (+0/-24)
tests/data/mountinfo_raring_btrfs.txt (+0/-13)
tests/data/roots/simple_ubuntu/etc/networks/interfaces (+0/-3)
tests/data/user_data.1.txt (+0/-15)
tests/data/vmware/cust-dhcp-2nic.cfg (+0/-34)
tests/data/vmware/cust-static-2nic.cfg (+0/-39)
tests/unittests/helpers.py (+0/-346)
tests/unittests/test__init__.py (+0/-220)
tests/unittests/test_builtin_handlers.py (+0/-73)
tests/unittests/test_cli.py (+0/-54)
tests/unittests/test_cs_util.py (+0/-84)
tests/unittests/test_data.py (+0/-559)
tests/unittests/test_datasource/test_altcloud.py (+0/-452)
tests/unittests/test_datasource/test_azure.py (+0/-646)
tests/unittests/test_datasource/test_azure_helper.py (+0/-421)
tests/unittests/test_datasource/test_cloudsigma.py (+0/-99)
tests/unittests/test_datasource/test_cloudstack.py (+0/-87)
tests/unittests/test_datasource/test_configdrive.py (+0/-407)
tests/unittests/test_datasource/test_digitalocean.py (+0/-127)
tests/unittests/test_datasource/test_gce.py (+0/-166)
tests/unittests/test_datasource/test_maas.py (+0/-163)
tests/unittests/test_datasource/test_nocloud.py (+0/-187)
tests/unittests/test_datasource/test_opennebula.py (+0/-300)
tests/unittests/test_datasource/test_openstack.py (+0/-347)
tests/unittests/test_datasource/test_smartos.py (+0/-585)
tests/unittests/test_distros/test_generic.py (+0/-233)
tests/unittests/test_distros/test_hostname.py (+0/-38)
tests/unittests/test_distros/test_hosts.py (+0/-41)
tests/unittests/test_distros/test_netconfig.py (+0/-321)
tests/unittests/test_distros/test_resolv.py (+0/-67)
tests/unittests/test_distros/test_sysconfig.py (+0/-82)
tests/unittests/test_distros/test_user_data_normalize.py (+0/-297)
tests/unittests/test_ec2_util.py (+0/-139)
tests/unittests/test_filters/test_launch_index.py (+0/-132)
tests/unittests/test_handler/test_handler_apt_configure.py (+0/-109)
tests/unittests/test_handler/test_handler_ca_certs.py (+0/-271)
tests/unittests/test_handler/test_handler_chef.py (+0/-192)
tests/unittests/test_handler/test_handler_debug.py (+0/-81)
tests/unittests/test_handler/test_handler_disk_setup.py (+0/-30)
tests/unittests/test_handler/test_handler_growpart.py (+0/-220)
tests/unittests/test_handler/test_handler_locale.py (+0/-67)
tests/unittests/test_handler/test_handler_lxd.py (+0/-134)
tests/unittests/test_handler/test_handler_mounts.py (+0/-133)
tests/unittests/test_handler/test_handler_power_state.py (+0/-127)
tests/unittests/test_handler/test_handler_rsyslog.py (+0/-174)
tests/unittests/test_handler/test_handler_seed_random.py (+0/-227)
tests/unittests/test_handler/test_handler_set_hostname.py (+0/-72)
tests/unittests/test_handler/test_handler_snappy.py (+0/-306)
tests/unittests/test_handler/test_handler_timezone.py (+0/-76)
tests/unittests/test_handler/test_handler_write_files.py (+0/-112)
tests/unittests/test_handler/test_handler_yum_add_repo.py (+0/-68)
tests/unittests/test_helpers.py (+0/-33)
tests/unittests/test_merging.py (+0/-257)
tests/unittests/test_net.py (+0/-130)
tests/unittests/test_pathprefix2dict.py (+0/-44)
tests/unittests/test_registry.py (+0/-28)
tests/unittests/test_reporting.py (+0/-369)
tests/unittests/test_rh_subscription.py (+0/-214)
tests/unittests/test_runs/test_merge_run.py (+0/-54)
tests/unittests/test_runs/test_simple_run.py (+0/-81)
tests/unittests/test_sshutil.py (+0/-171)
tests/unittests/test_templating.py (+0/-119)
tests/unittests/test_util.py (+0/-489)
tests/unittests/test_vmware_config_file.py (+0/-103)
tools/21-cloudinit.conf (+0/-6)
tools/Z99-cloud-locale-test.sh (+0/-98)
tools/build-on-freebsd (+0/-66)
tools/ccfg-merge-debug (+0/-90)
tools/cloud-init-per (+0/-60)
tools/hacking.py (+0/-170)
tools/make-dist-tarball (+0/-21)
tools/make-mime.py (+0/-60)
tools/make-tarball (+0/-39)
tools/mock-meta.py (+0/-454)
tools/motd-hook (+0/-35)
tools/read-dependencies (+0/-29)
tools/read-version (+0/-26)
tools/run-pep8 (+0/-22)
tools/run-pyflakes (+0/-18)
tools/run-pyflakes3 (+0/-2)
tools/tox-venv (+0/-42)
tools/uncloud-init (+0/-141)
tools/validate-yaml.py (+0/-25)
tools/write-ssh-key-fingerprints (+0/-38)
tox.ini (+0/-35)
udev/66-azure-ephemeral.rules (+0/-18)
udev/79-cloud-init-net-wait.rules (+0/-10)
udev/cloud-init-wait (+0/-70)
upstart/cloud-config.conf (+0/-9)
upstart/cloud-final.conf (+0/-10)
upstart/cloud-init-blocknet.conf (+0/-83)
upstart/cloud-init-container.conf (+0/-57)
upstart/cloud-init-local.conf (+0/-16)
upstart/cloud-init-nonet.conf (+0/-66)
upstart/cloud-init.conf (+0/-9)
upstart/cloud-log-shutdown.conf (+0/-19)
JJ Asghar: Pending requested
Diff: 0 lines
Scott Moser: Pending requested
Diff: 3061 lines (+2239/-175)
39 files modified
ChangeLog (+39/-9)
cloud-init.py (+31/-1)
cloudinit/CloudConfig/__init__.py (+1/-1)
cloudinit/CloudConfig/cc_apt_pipelining.py (+53/-0)
cloudinit/CloudConfig/cc_ca_certs.py (+3/-1)
cloudinit/CloudConfig/cc_chef.py (+5/-5)
cloudinit/CloudConfig/cc_landscape.py (+5/-0)
cloudinit/CloudConfig/cc_resizefs.py (+31/-12)
cloudinit/CloudConfig/cc_salt_minion.py (+2/-1)
cloudinit/CloudConfig/cc_update_etc_hosts.py (+1/-1)
cloudinit/DataSource.py (+4/-1)
cloudinit/DataSourceCloudStack.py (+92/-0)
cloudinit/DataSourceConfigDrive.py (+231/-0)
cloudinit/DataSourceEc2.py (+2/-84)
cloudinit/DataSourceMAAS.py (+345/-0)
cloudinit/DataSourceNoCloud.py (+75/-2)
cloudinit/DataSourceOVF.py (+1/-1)
cloudinit/SshUtil.py (+2/-0)
cloudinit/UserDataHandler.py (+2/-2)
cloudinit/__init__.py (+44/-9)
cloudinit/netinfo.py (+5/-5)
cloudinit/util.py (+253/-30)
config/cloud.cfg (+3/-1)
debian.trunk/control (+1/-0)
doc/configdrive/README (+118/-0)
doc/examples/cloud-config-chef-oneiric.txt (+90/-0)
doc/examples/cloud-config-chef.txt (+48/-6)
doc/examples/cloud-config-datasources.txt (+18/-0)
doc/examples/cloud-config.txt (+11/-0)
doc/kernel-cmdline.txt (+48/-0)
doc/nocloud/README (+55/-0)
setup.py (+1/-0)
tests/unittests/test__init__.py (+242/-0)
tests/unittests/test_datasource/test_maas.py (+153/-0)
tests/unittests/test_handler/test_handler_ca_certs.py (+4/-0)
tests/unittests/test_userdata.py (+107/-0)
tests/unittests/test_util.py (+18/-2)
tools/Z99-cloud-locale-test.sh (+92/-0)
tools/run-pylint (+3/-1)
lp:~smoser/ubuntu/yakkety/cloud-init/pkg
lp:~smoser/ubuntu/xenial/cloud-init/pkg
Revision history for this message
Scott Moser (smoser) wrote :
Scott Moser (smoser)
tags: added: systemd-boot
Revision history for this message
Scott Moser (smoser) wrote :

So:

a.) zfs-import-cache.service is 'After systemd-udev-settle.service'
b.) cloud-init installs /lib/udev/rules.d/79-cloud-init-net-wait.rules which uses IMPORT of /lib/udev/cloud-init-wait to block network device add events until cloud-init-local has written networking configuration

I'm pretty sure, then what is happening is:
1.) network device gets added and cloud-init-wait starts its 60 second wait for cloud-init-local to finish.
2.) systemd-udev-settle runs, which waits until queued all events are processed
3.) deadlock until timeout from 1

Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Martin Pitt (pitti) wrote :

There is some angles of attack here:

└─zfs-import-cache.service @1min 786ms +272ms
  └─systemd-udev-settle.service @494ms +1min 275ms

This is a case of "you asked for it, you got it", really. udev-settle is a workaround for old software that wasn't written with hotplugging in mind. It's 2016, and we really should avoid introducing new dependencies to udev-settle, as it's conceptually broken and can't work. This totally does not work for any hotplugged device, or a device which just gets detected by the kernel late, or which needs to be unwrapped from building an LVM or cryptsetup device, etc.

I think it would be much more elegant to instead create udev rules (or systemd units) which load the ZFS cache as soon as a ZFS block device is detected. Would that be possible/reasonable? I don't know anything about what import-cache does, but I hope/suppose it has some way of doing this on a per-device basis?

    └─systemd-udev-trigger.service @415ms +55ms

This is cloud-init's rules for blocking udev rules until network devices get detected and cloud-init's naming rules get applied. This is a hairy topic and Scott and I already discussed this at length. I think there's a better solution here, but I doubt we'll completely turn this around by the release. This just illustrates one of the problems of the "waiting for all hardware just to single out a particular device" approach from above -- this waits on stuff which is completely unrelated to zfs.

no longer affects: systemd (Ubuntu)
Revision history for this message
Richard Laager (rlaager) wrote :

@pitti: zfs-import-cache.service doesn't "load the ZFS cache". It imports zpools which are listed in the /etc/zfs/zpool.cache file. It is conditioned (ConditionPathExists) on the existence of /etc/zfs/zpool.cache. It seems to me that upstream ZoL is tending to deprecate the zpool.cache file.

In contrast is zfs-import-scan.service, which imports pools by device scanning. This is conditioned on /etc/zfs/zpool.cache NOT existing. So one or the other service runs, depending on whether you have a zpool.cache file.

Longer-term, I agree that we need some sort of solution that isn't "wait for all devices". Is there any prior art with btrfs or MD that's applicable here? (I know MD devices are assembled by the kernel in some circumstances. I think mdadm is involved in other cases.)

As a side note, zfs-import-scan.service hardcodes /dev/disk/by-id, which is probably not ideal. The reason for that is likely to work-around that `zpool import` is preferring /dev. See also: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1571241/comments/8

Revision history for this message
Martin Pitt (pitti) wrote : Re: [Bug 1571761] Re: zfs-import-cache.service slows boot by 60 seconds

Hello Richard,

thanks for the explanation what these units do.

Richard Laager [2016-04-19 16:29 -0000]:
> @pitti: zfs-import-cache.service doesn't "load the ZFS cache". It
> imports zpools which are listed in the /etc/zfs/zpool.cache file. It is
> conditioned (ConditionPathExists) on the existence of
> /etc/zfs/zpool.cache. It seems to me that upstream ZoL is tending to
> deprecate the zpool.cache file.
>
> In contrast is zfs-import-scan.service, which imports pools by device
> scanning. This is conditioned on /etc/zfs/zpool.cache NOT existing. So
> one or the other service runs, depending on whether you have a
> zpool.cache file.

I'm curious, how are these actually being used? They can't be relevant
for the root file system as that already needs to be set up in the
initrd. This unit does not sort itself before local-fs-pre.target
that, i. e. you also can't rely on having these done when stuff in
/etc/fstab is being mounted. The only thing that I see is that
zfs-mount.service.in is Before=local-fs.target, i. e the zfs-import-*
bits will run in parallel with mounting fstab.

> Longer-term, I agree that we need some sort of solution that isn't "wait
> for all devices"

Indeed. The concept of "wait for all devices" hasn't been well-defined
for the past 20 years at least. Is zpool import idempotent in any way,
i. e. would it be safe (and also performant) to run it whenever a
block device is seen, instead of once ever in the boot sequence? If
not, could it be made to accept a device name, and then limit its
actions to just that? Then it would be straightforward to put this
into an udev rule and become hotpluggable.

Does this mean that ZFS isn't currently supported on hotpluggable
storage? (As there's nothing that would call zpool import on them)

> Is there any prior art with btrfs or MD that's applicable here? (I
> know MD devices are assembled by the kernel in some circumstances. I
> think mdadm is involved in other cases.)

Yes, there's plenty. E. g. mdadm ships
/lib/udev/rules.d/64-md-raid-assembly.rules which adds/removes devices
from a RAID array as they come and go. LVM is a bit more complicated,
its udev rules mostly just tell a daemon (lvmetad) about new devices
which then decides when it has enough pieces (PVs) to bring up a VG.

There's also some really simple stuff like 85-hdparm.rules which just
calls a script for every encountered block device.

Thanks,
Martin

Revision history for this message
Richard Laager (rlaager) wrote :

[quoted blocks trimmed and reordered for ease of reply]

> I'm curious, how are these actually being used? They can't be relevant
> for the root file system as that already needs to be set up in the
> initrd.

Correct. The root pool (with all of its filesystems) is handled in the initrd.
These units are for regular (non-root) pools.

> This unit does not sort itself before local-fs-pre.target
> that, i. e. you also can't rely on having these done when stuff in
> /etc/fstab is being mounted. The only thing that I see is that
> zfs-mount.service.in is Before=local-fs.target, i. e the zfs-import-*
> bits will run in parallel with mounting fstab.

In parallel with mounting fstab sounds reasonable, since mounting filesystems
from fstab and mounting filesystems from ZFS (which happens by default upon
import) are analogous.

> Does this mean that ZFS isn't currently supported on hotpluggable
> storage? (As there's nothing that would call zpool import on them)

In short, yes. If (after boot) you plug in a USB drive with a zpool on it, you
would need to manually import it. Personally, I'd like to see this improved in
the future.

> Is zpool import idempotent in any way,
> i. e. would it be safe (and also performant) to run it whenever a
> block device is seen, instead of once ever in the boot sequence?

I'm not super confident, but I think it would be idempotent, at least assuming
it is not run in parallel. But I doubt it would be performant. Running
zpool import examines all devices (all zfs_member devices if compiled with
libblkid support, assuming I'm understand the code correctly).

Additionally, if you're responding to device events one-by-one, for pools with
more than one disk, there's a question of whether you should import the pool
when disks are missing (within the level of redundancy where you *can* import
the pool, of course).

> LVM is a bit more complicated,
> its udev rules mostly just tell a daemon (lvmetad) about new devices
> which then decides when it has enough pieces (PVs) to bring up a VG.

This might be the right approach for ZFS long-term. I think the best behavior
is to import a pool:
  once all devices are present ||
  (once sufficient devices are present &&
   a short timeout has expired with no new disks arriving)

Revision history for this message
Richard Laager (rlaager) wrote :

@smoser: If this is a fresh install with no zpools, you shouldn't have a zpool.cache file, and so your chain should have zfs-import-scan.service, not zfs-import-cache.service. You'd still have the same problem, I expect, but that detail is a bit worrisome.

Revision history for this message
Martin Pitt (pitti) wrote :

Hello Richard,

Richard Laager [2016-04-20 0:47 -0000]:
> In parallel with mounting fstab sounds reasonable, since mounting filesystems
> from fstab and mounting filesystems from ZFS (which happens by default upon
> import) are analogous.

Ah, I see. So do file systems from ZFS have a separate equivalent to
/etc/fstab?

> But I doubt it would be performant. Running
> zpool import examines all devices
> [...]
> Additionally, if you're responding to device events one-by-one, for pools with
> more than one disk, there's a question of whether you should import the pool
> when disks are missing

I think these two make it sufficiently "device unspecific" to make a
simple "call this in an udev rule" impractical.

> I think the best behavior
> is to import a pool:
> once all devices are present ||

It's really best to completely drop the idea of "all devices". It just
doesn't work in a world where you even can (and do) hotplug PCI
devices (e. g. to replace a faulty disk in a server). Thinking in
these terms also quickly gets you down into a rabbit hole of race
conditions and too strong locks.

> (once sufficient devices are present &&

That sounds like the right approach. That's more or less what LVM or
mdadm are doing. For some RAID modes in mdadm it's possible that you
already assemble a device before all members have been detected (e. g.
RAID-1 with 3 redundant disks), but LVM gets along with the third
member being added afterwards.

> a short timeout has expired with no new disks arriving)

That again is a part I strongly recommend to drop, as it's not
well-defined.

Thanks!

Revision history for this message
Richard Laager (rlaager) wrote :

> Ah, I see. So do file systems from ZFS have a separate equivalent to
> /etc/fstab?

Yes, each filesystem has a "mountpoint" property. This can be set directly, and is also inherited automatically. For example, I have rpool's mountpoint set to / and then rpool/home automatically inherits /home as its mountpoint. I also have rpool/home/root with a custom mountpoint set of /root. For a more complete list, see section 3.3 of my HOWTO:
https://github.com/rlaager/zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem

> It's really best to completely drop the idea of "all devices".

I think I might have done a bad job of explaining my point. Let me try again.

Imagine I have a raidz2 ("RAID-6", can survive two disks missing) with 6 disks. I want the daemon to import that pool as soon as all 6 disks are present. I also want it to import the pool if 4 (or 5) disks are present, but only after a short timeout where no new disks have been detected. That is, I do not want the pool to import in a degraded state when the first 4 disks are present, only to have the 5th and 6th show up milliseconds later.

The daemon would handle each disk one-by-one. It would track the information about discovered pools in memory. After each new disk is discovered and we've identified the pool it belongs to (C/glib pseudo-code follows):
handle_disk_arrival(disk) {
    /* TODO: read details from disk, initialize some pool object */

    if (has_sufficient_disks(pool)) {
        g_source_remove_by_funcs_user_data(import_pool, pool);
        if (has_all_disks(pool)) {
           g_timeout_add(0, import_pool, pool);
        } else {
           g_timeout_add(500, import_pool, pool);
       }
    }
}

import_pool(zpool *pool) {
    /* TODO: import the pool */
    return FALSE;
}

Revision history for this message
Martin Pitt (pitti) wrote :

Hello Richard,

Richard Laager [2016-04-20 17:59 -0000]:
> Yes, each filesystem has a "mountpoint" property.

I see, thanks. (Sorry, I'm just familiar with btrfs and subvolumes,
and they work differently). So I see how putting this into
local-fs.target in parallel with the fstab mounts makes sense.

> I want the daemon to import that pool as soon as all 6 disks are
> present. I also want it to import the pool if 4 (or 5) disks are
> present, but only after a short timeout where no new disks have been
> detected.

Aah, that makes much more sense than "wait for all devices", so that
was just a misunderstanding.

> That is, I do not want the pool to import in a degraded state when
> the first 4 disks are present, only to have the 5th and 6th show up
> milliseconds later.
>
> The daemon would handle each disk one-by-one. It would track the
> information about discovered pools in memory. After each new disk is
> discovered and we've identified the pool it belongs to (C/glib
> pseudo-code follows):

Doing this with a daemon would certainly work, it can do arbitrary
state keeping and timeouts. OTOH this is a bit of a central point of
failure/you need to write a new daemon/etc.

I haven't thought this through very thorouhgly, but I have a gut
feeling this could even work in a decentral and asynchronous way:
Whenever we get an uevent for a block device we call a helper (through
udev's RUN) which checks which zpool this belongs to and if there's at
least the minimal number of members available (but not all yet); if
so, it can start a systemd timer unit [1]
zpool-assemble@<devicename>.timer which will trigger some
zpool-assemble@<devicename>.service after, say, 500ms. But when that
helper sees (on the next uevent) that this was the last missing member
of the zpool, i. e. we now have all devices, it could immediately
start zpool-assemble@<devicename>.service. The timer then goes off
into nothing as the corresponding .service is already running (i. e.
it's harmless), or the udev helper could also explicitly stop it.

This is just a sketch, but if you are interested in exploring this I
recommend that we have a more synchronous chat (IRC, I'm pitti on
Freenode) or a hangout or so.

[1] Triggering a systemd unit from a udev rule is trivial,
    e. g. ENV{SYSTEMD_WANTS}+="zpool-assemble@%k.service" (with the
    appropriate checks for "is this a zfs block device)

Revision history for this message
Richard Laager (rlaager) wrote :

You make an excellent point about single-point-of-failure. It wouldn't necessarily have to be a new daemon. ZFS already has a daemon, so this could just be additional functionality.

I've pointed Brian Behlendorf (the upstream ZFS-on-Linux lead developer) to this bug report. It'd be nice to get some official comments from upstream.

As I said, I definitely want to see something done, to make removable media work correctly. (Though how to unmount it is a different question.)

Revision history for this message
Richard Laager (rlaager) wrote :

The relevant issue upstream seems to be:
https://github.com/zfsonlinux/zfs/issues/4178

There's no answer there yet, but I wanted to get that link posted here.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1227-0ubuntu1

---------------
cloud-init (0.7.7~bzr1227-0ubuntu1) yakkety; urgency=medium

  * New upstream snapshot.
    - fix one more unit test to run inside buildd.

 -- Scott Moser <email address hidden> Sat, 04 Jun 2016 20:55:07 -0400

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

Hello,
An SRU upload of cloud-init for 16.04 that contains a fix for this bug has been made under bug 1595302. Please track that bug if you are interested.

Changed in cloud-init (Ubuntu Xenial):
status: New → Fix Released
status: Fix Released → In Progress
importance: Undecided → High
assignee: nobody → Scott Moser (smoser)
Mathew Hodson (mhodson)
Changed in zfs-linux (Ubuntu Xenial):
importance: Undecided → High
Revision history for this message
Scott Moser (smoser) wrote :

fix is now released to xenial under bug 1595302. daily cloud-images with this newer version of cloud-init should appear in the next few days. Any image with a serial number *newer* than 20160707 should have cloud-init at 0.7.7~bzr1246-0ubuntu1~16.04.1 .

Changed in cloud-init (Ubuntu Xenial):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in zfs-linux (Ubuntu Xenial):
status: New → Confirmed
Revision history for this message
Colin Ian King (colin-king) wrote :

Since this seems to be closed against bug 1595302, I'm going to mark this bug as fix released too.

Changed in zfs-linux (Ubuntu):
status: Confirmed → Fix Released
Changed in zfs-linux (Ubuntu Xenial):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.