instance can not resize ephemeral from mitaka to stein and master

Possibly duplicate of https://bugs.launchpad.net/nova/+bug/1558343

Well, I meant possibly related to https://bugs.launchpad.net/nova/+bug/1558343 rather

Adding this one as a candidate for RC2 unless we verify it's not a regression.

Haven't confirmed, but at first glance I don't see how this is obviously related to the config drive bug. I'll try to reproduce.

Can you post the exact boot command you used to create the instance? I'd like to know if the ephemeral was created implicitly or explicitly.

Nevermind, I can see that an implicitly-created ephemeral disk is created as eph0.

This has been the behaviour of Nova since at least change I8efd6af6706a097fb540e040a86ccbeaf131631f merged, which was September 2013. It may even be older than that, but the code before that was a bit hard to read. This isn't a recent regression, so certainly not an rc blocker. It's also not immediately obvious how to fix it robustly.

Given c#7, I think this is not a regression, so punting out of RC2

Tina Kevin,
Are you working on the fix? Please change status to Inprogress if you are, otherwise change Assigned to ->nobody.

Fix proposed to branch: master
Review: https://review.openstack.org/320759

Here's a tempest test that demonstrates this bug:

    https://review.openstack.org/#/c/338411/

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/320759
Reason: This review is > 6 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

The disk.local thing is a distraction, as that code is always a no-op in practice. The resize is done by the _create_image call.

For reference, the bug happened way before this in ComputeManager._finish_migration where we do:

        if old_instance_type_id != new_instance_type_id:
            instance_type = instance.get_flavor('new')
            self._set_instance_info(instance, instance_type)
            for key in ('root_gb', 'swap', 'ephemeral_gb'):
                if old_instance_type[key] != instance_type[key]:
                    resize_instance = True
                    break

The problem is that ephemeral disks are defined by BDMs, and not by instance.ephemeral_gb. The above code updates ephemeral_gb, but not the BDM. The libvirt driver is only looking in the BDM, so it doesn't see the resize.

The issue with automatically updating the BDM is that the BDM doesn't necessarily map directly to the flavor's ephemeral_gb.

By default, if a flavor has an ephemeral disk we will create a BDM of size flavor.ephemeral_gb for that disk when creating an instance. From this point on, the BDM is canonical.

However, a user may also specify BDMs explicitly at creation time, which means that the user may:

* Specify an ephemeral disk smaller than flavor.ephemeral_gb
* Specify multiple ephemeral disks whose total size is <= flavor.ephemeral_gb

In both of these cases, it is not at all clear what the behaviour on resize should be. In practise, the only safe thing would be to leave them alone. As there is no api to explicitly resize an ephemeral disk, and it's not clear that it's worth creating one, I don't see that there is any solution to the general case.

It is possible that we could check if there is a single, full size ephemeral disk and resize only in that case, possibly emitting a warning otherwise.

Found open reviews for this bug in gerrit, setting to In Progress.

review: https://review.openstack.org/372721 in branch: master

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/372721
Reason: This review is > 4 weeks without comment, and is not mergable in it's current state. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Hello what is the status of this bug, is someone actively working on it ?
If not, releasing it by marking it confirmed or new would be good.

Removing assignee due to lack of progress/update. I'm running into this as well

So i am also affected by this issue.

I was thinking. It sounds like we have the following items to work around.

1. ephemeral disk takes up the full allocation from flavor.
   if so upon resize we increase the current ephemeral disk to the new size

2. Multiple ephemeral disks, split into multiple disks upto the max allowed flavor size.
   In this case, could we not just create a new ephemeral disk that takes the full allocation of the additional ephemeral allocation from the new flavor.

3. Single or multi ephemeral disks which add up to less than the current flavor.
   This is the hardest one to handle i believe, i would be tempted to just build another single disk taking the full additional allocation. EG. first flavor has 200Gb Eph, but only 100Gb is used, then resized to a flavor with 400Gb eph, we keep the current 100Gb ephemeral, (as thats what they decided to use), then we would create a new eph disk out of the remainder so in this example 200Gb

Hi, Is there any updates about this bug?

@leehom (feli5)
There should be no one

@Tina Kevin
Hi！ Isn't it recommended to adjust the ephemeral disk size?

Yes

Or we can provide API to support add new ephemeral disks.
As it is supported to create multiple ephemeral disks as long as the total size not greater than the size defined in flavor.
In case there are multiple ephemeral disks. It's hard to determine which disk should be extended.

Related fix proposed to branch: master
Review: https://review.opendev.org/691915

This is so latent and complicated to resolve that I've marked it as wishlist since it would probably require a spec to describe all of the detailed issues involved in fixing this and what the options and alternatives would be.

Reviewed: https://review.opendev.org/691915
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d3e38143a7e46d06a6632b41b5484b13e81884cf
Submitter: Zuul
Branch: master

commit d3e38143a7e46d06a6632b41b5484b13e81884cf
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 29 12:50:13 2019 -0400

    Add known limitation about resize not resizing ephemeral disks

    It's a long known issue that resize does not resize ephemeral
    disks attached to the server yet we continue to get bugs about
    it and there is no one working on fixing it. As such this change
    adds a note to the API reference for the resize server action API
    mentioning the known limitation.

    Change-Id: I31ca3627f2d3ef5a8db5eecf3759cd304b059963
    Related-Bug: #1558880

Confirmed that the bug still exists on current master too.

We are affected also but the issue is on stein and master.
The context on my side the compute is on RAW type.

Reproduce steps:
* create an instance with public image and flavor with local disk in raw type.
* put the public image to private
* remove in the base image folder the image (on source and dest compute)

Expected result:
* Instance was running with a local disk are larger.

Actual result:
* Everything is resized except local disk/

The issue come from create_image if image_id don't exist and path of the disk exist no action taken on the local disk.

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/820531

I propose this patch to fix the issue: https://review.opendev.org/c/openstack/nova/+/820531

Reviewed: https://review.opendev.org/c/openstack/nova/+/820531
Committed: https://opendev.org/openstack/nova/commit/f7af85dd7431eea61cdd2b1723a5dbcc3d9471ed
Submitter: "Zuul (22348)"
Branch: master

commit f7af85dd7431eea61cdd2b1723a5dbcc3d9471ed
Author: Pierre LIBEAU <email address hidden>
Date: Mon Dec 6 15:48:57 2021 +0100

    Nova resize don't extend disk in one specific case

    Nova resize not apply extend virtual size of the instance if the image
    is not accessible by the customer (public image use to build the
    instance is now private image because this image is deprecated) and the
    source compute of the resize don't have anymore the base image.

    Related-Bug: #1558880
    Change-Id: I4d6dfca1efe10caebb017b6ec96820979018203f

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/877260

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/877284

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/877284
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/877260
Reason: stable/xena branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/xena if you want to further work on this patch.