vms loose acess to config drive with CONF.force_config_drive=True after hard reboot

Bug #1835822 reported by sean mooney
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
sean mooney

Bug Description

The fix to bug https://bugs.launchpad.net/nova/+bug/1827492
https://review.opendev.org/#/c/659703/8

changed the behavior of nova.virt.configdrive.required_by
to depend on instance.launched_at

https://review.opendev.org/#/c/659703/8/nova/virt/configdrive.py@196

but did not reorder
https://github.com/openstack/nova/blob/86524773b8cd3a52c98409c7ca183b4e1873e2b8/nova/compute/manager.py#L1757-L1758

as a result when nova.compute.manager._update_instance_after_spawn
is called instance.launched_at is always set before we call nova.virt.configdrive.update_instance

as a result instance.config_drive will always be set to false if not set on the api.

this results in a vm that is spawned on a host with force_config_drive=True initally spawning
with a config drive but loosing it after a hard reboot.

for any deployment that uses config driver for vendor data or device role tagging because they do not deploy the metadata service this is a regressions as they cannot fall back to the metadta service.

this also might cause issue for deployment that support the deprectated file injection api that is part of the v2.1 api as the files are only stored in the config drive and are not part metadta endoint
note: i have not checked if we autoset instnace.config_drive when you use file injection or not so it may be unaffected since the breakage of the other support uscases is enough to justify this bug.

the fix is simple jsut swap the order of https://github.com/openstack/nova/blob/86524773b8cd3a52c98409c7ca183b4e1873e2b8/nova/compute/manager.py#L1757-L1758

and then instance will have there instnace.config_drive value set correctly when they first boot
and it will be sticky for the lifetime of the instance.

Revision history for this message
sean mooney (sean-k-mooney) wrote :

note i set this to medium since we were looking at backporting
https://bugs.launchpad.net/nova/+bug/1827492
and we might have data lost in the file injection case if that does not auto
set instance.config_drive seperately

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/669738

Changed in nova:
status: Confirmed → In Progress
Changed in nova:
assignee: sean mooney (sean-k-mooney) → Matt Riedemann (mriedem)
Matt Riedemann (mriedem)
Changed in nova:
assignee: Matt Riedemann (mriedem) → sean mooney (sean-k-mooney)
Matt Riedemann (mriedem)
tags: added: train-rc-potential
Changed in nova:
assignee: sean mooney (sean-k-mooney) → Matt Riedemann (mriedem)
Changed in nova:
assignee: Matt Riedemann (mriedem) → sean mooney (sean-k-mooney)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/669738
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=641feb62a33e211a3ed1f9a2309ee16d827f5fe2
Submitter: Zuul
Branch: master

commit 641feb62a33e211a3ed1f9a2309ee16d827f5fe2
Author: Sean Mooney <email address hidden>
Date: Mon Jul 8 18:35:11 2019 +0000

    make config drives sticky bug 1835822

    This change reorders the call in
    _update_instance_after_spawn so that we call
    configdrive.update_instance before we set
    instance.launched_at. This ensures that if the vm
    is booted with a config drive because the host had
    force_config_drive=true the instance will keep its
    config drive across reboots.

    This change fixes a regression introduced as part
    of fixing bug #1827492 which addressed failing
    to boot vms after changing force_config_drive=false
    to force_config_drive=true.

    Change-Id: I9194423f5f95e9799bd891548e24756131d65e76
    Related-Bug: #1827492
    Closes-Bug: #1835822

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 20.0.0.0rc1

This issue was fixed in the openstack/nova 20.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/691864

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/stein)

Change abandoned by sean mooney (<email address hidden>) on branch: stable/stein
Review: https://review.opendev.org/691864
Reason: actully based on matts last comment on https://review.opendev.org/#/c/660914/ i wont backport this unless we decide to backport that upstream

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.