diff -Nru nova-17.0.10/api-ref/source/os-volume-attachments.inc nova-17.0.11/api-ref/source/os-volume-attachments.inc --- nova-17.0.10/api-ref/source/os-volume-attachments.inc 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/api-ref/source/os-volume-attachments.inc 2019-07-10 21:45:12.000000000 +0000 @@ -145,6 +145,9 @@ .. note:: This action only valid when the server is in ACTIVE, PAUSED and RESIZED state, or a conflict(409) error will be returned. +Updating, or what is commonly referred to as "swapping", volume attachments +with volumes that have more than one read/write attachment, is not supported. + Normal response codes: 202 Error response codes: badRequest(400), unauthorized(401), forbidden(403), itemNotFound(404), conflict(409) diff -Nru nova-17.0.10/AUTHORS nova-17.0.11/AUTHORS --- nova-17.0.10/AUTHORS 2019-03-24 23:14:28.000000000 +0000 +++ nova-17.0.11/AUTHORS 2019-07-10 21:47:09.000000000 +0000 @@ -22,6 +22,7 @@ Aditi Raveesh Aditi Raveesh Aditya Prakash Vaja +Adrian Chiris Adrian Smith Adrian Vladu Adrien Cunin @@ -57,6 +58,7 @@ Alexander Sakhnov Alexander Schmidt Alexandra Settle +Alexandre Arents Alexandre Levine Alexandru Muresan Alexei Kornienko @@ -433,6 +435,7 @@ Florian Haas Forest Romain Francesco Santoro +Francois Palin François Charlier Frederic Lepied Gabe Westmaas @@ -996,6 +999,7 @@ Robert Pothier Robert Tingirica Rodolfo Alonso Hernandez +Rodrigo Barbieri Roey Chen Rohan Kanade Rohan Kanade @@ -1602,6 +1606,7 @@ yatin karel yatinkarel ydoyeul +yenai yongiman yuanyue yugsuo diff -Nru nova-17.0.10/ChangeLog nova-17.0.11/ChangeLog --- nova-17.0.10/ChangeLog 2019-03-24 23:14:25.000000000 +0000 +++ nova-17.0.11/ChangeLog 2019-07-10 21:47:07.000000000 +0000 @@ -1,6 +1,58 @@ CHANGES ======= +17.0.11 +------- + +* fix up numa-topology live migration hypervisor check +* Fail to live migration if instance has a NUMA topology +* Allow driver to properly unplug VIFs on destination on confirm resize +* Move get\_pci\_mapping\_for\_migration to MigrationContext +* libvirt: Do not reraise DiskNotFound exceptions during resize +* Include all network devices in nova diagnostics +* Workaround missing RequestSpec.instance\_group.uuid +* Add regression recreate test for bug 1830747 +* Restore connection\_info after live migration rollback +* DRY up test\_rollback\_live\_migration\_set\_migration\_status +* Block swap volume on volumes with >1 rw attachment +* Fix live-migration when glance image deleted +* Add functional confirm\_migration\_error test +* [stable-only] Delete allocations even if \_confirm\_resize raises (part 2) +* Delete allocations even if \_confirm\_resize raises +* Stop logging traceback when skipping quiesce +* Fix retry of instance\_update\_and\_get\_original +* Teardown networking when rolling back live migration even if shared disk +* Use migration\_status during volume migrating and retyping +* libvirt: Always disconnect volumes after libvirtError exceptions +* libvirt: Stop ignoring unknown libvirtError exceptions during volume attach +* Fix {min|max}\_version in ironic Adapter setup +* Create request spec, build request and mappings in one transaction +* xenapi/agent: Change openssl error handling +* libvirt: Avoid using os-brick encryptors when device\_path isn't provided +* Fix nova-grenade-live-migration run book for opendev migration +* Fix regression in glance client call +* OpenDev Migration Patch +* Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree +* Fix legacy-grenade-dsvm-neutron-multinode-live-migration +* libvirt: set device address tag only if setting disk unit +* libvirt: disconnect volume when encryption fails +* Add missing libvirt exception during device detach +* Error out migration when confirm\_resize fails +* prevent common kwargs from glance client failure +* Define irrelevant-files for tempest-full-py3 job +* Delete instance\_id\_mappings record in instance\_destroy +* Do not persist RequestSpec.ignore\_hosts +* Add functional regression test for bug 1669054 +* Update instance.availability\_zone on revertResize +* Add functional recreate test for bug 1819963 +* [Stable Only] hardware: Handle races during pinning +* Fix incomplete instance data returned after build failure +* Update instance.availability\_zone during live migration +* Update resources once in update\_available\_resource +* Don't persist zero allocation ratios in ResourceTracker +* Document unset/reset wrinkle for \*\_allocation\_ratio options +* Replace openstack.org git:// URLs with https:// + 17.0.10 ------- @@ -15,6 +67,7 @@ * Avoid redundant initialize\_connection on source post live migration * Make host\_manager use scatter-gather and ignore down cells * Make service all-cells min version helper use scatter-gather +* Ignore VolumeAttachmentNotFound exception in compute.manager * Lock detach\_volume * Fix a missing policy in test policy data * Handle unicode characters in migration params diff -Nru nova-17.0.10/debian/changelog nova-17.0.11/debian/changelog --- nova-17.0.10/debian/changelog 2019-08-13 00:34:12.000000000 +0000 +++ nova-17.0.11/debian/changelog 2019-07-29 15:45:29.000000000 +0000 @@ -1,3 +1,11 @@ +nova (2:17.0.11-0ubuntu1) bionic; urgency=medium + + * New stable point release for OpenStack Queens (LP: #1838288). + * d/p/xenapi-agent-change-openssl-error-handling.patch, bug_*.patch: + Dropped. Fixed in 17.0.11. + + -- Corey Bryant Mon, 29 Jul 2019 11:45:29 -0400 + nova (2:17.0.10-0ubuntu2.1) bionic-security; urgency=medium [ Sahid Orentino Ferdjaoui ] diff -Nru nova-17.0.10/debian/patches/bug_1821594_1.patch nova-17.0.11/debian/patches/bug_1821594_1.patch --- nova-17.0.10/debian/patches/bug_1821594_1.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1821594_1.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,127 +0,0 @@ -From 6c806f1ca8c14b875c454718d16ddc27dfe6ef81 Mon Sep 17 00:00:00 2001 -From: Matt Riedemann -Date: Mon, 25 Mar 2019 13:16:42 -0400 -Subject: [PATCH 1/4] Error out migration when confirm_resize fails - -If anything fails and raises an exception during -confirm_resize, the migration status is stuck in -"confirming" status even though the instance status -may be "ERROR". - -This change adds the errors_out_migration decorator -to the confirm_resize method to make sure the migration -status is "error" if an error is raised. - -In bug 1821594 it was the driver.confirm_migration -method that raised some exception, so a unit test is -added here which simulates a similar scenario. - -This only partially closes the bug because we are still -leaking allocations on the source node resource provider -since _delete_allocation_after_move is not called. That -will be dealt with in a separate patch. - -Conflicts: - nova/tests/unit/compute/test_compute_mgr.py - -NOTE(mriedem): The conflict is due to not having change -Ia05525058e47efb806cf8820410c8bc80eccca25 in Queens. - -Change-Id: Ic7d78ad43a2bad7f932c22c98944accbbed9e9e2 -Partial-Bug: #1821594 -(cherry picked from commit 408ef8f84a698f764aa5d769d6d01fd9340de2e5) -(cherry picked from commit 972d4e0eb391e83fe8d3020ff95db0e6a840a224) -(cherry picked from commit e3f69c8af0d13f0aa60e9a267f25050729f7766c) ---- - nova/compute/manager.py | 1 + - nova/tests/unit/compute/test_compute_mgr.py | 49 +++++++++++++++++++++ - 2 files changed, 50 insertions(+) - -diff --git a/nova/compute/manager.py b/nova/compute/manager.py -index 7817245a65..0dd5cbe550 100644 ---- a/nova/compute/manager.py -+++ b/nova/compute/manager.py -@@ -3675,6 +3675,7 @@ class ComputeManager(manager.Manager): - - @wrap_exception() - @wrap_instance_event(prefix='compute') -+ @errors_out_migration - @wrap_instance_fault - def confirm_resize(self, context, instance, migration): - """Confirms a migration/resize and deletes the 'old' instance. -diff --git a/nova/tests/unit/compute/test_compute_mgr.py b/nova/tests/unit/compute/test_compute_mgr.py -index 0e40939686..108f3078b2 100644 ---- a/nova/tests/unit/compute/test_compute_mgr.py -+++ b/nova/tests/unit/compute/test_compute_mgr.py -@@ -6250,12 +6250,14 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - expected_attrs=['metadata', 'system_metadata', 'info_cache']) - self.migration = objects.Migration( - context=self.context.elevated(), -+ id=1, - uuid=mock.sentinel.uuid, - instance_uuid=self.instance.uuid, - new_instance_type_id=7, - dest_compute=None, - dest_node=None, - dest_host=None, -+ source_compute='source_compute', - status='migrating') - self.migration.save = mock.MagicMock() - self.useFixture(fixtures.SpawnIsSynchronousFixture()) -@@ -6658,6 +6660,53 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - mock_resources.return_value) - do_it() - -+ @mock.patch('nova.compute.utils.add_instance_fault_from_exc') -+ @mock.patch('nova.objects.Migration.get_by_id') -+ @mock.patch('nova.objects.Instance.get_by_uuid') -+ @mock.patch('nova.compute.utils.notify_about_instance_usage') -+ @mock.patch('nova.compute.utils.notify_about_instance_action') -+ @mock.patch('nova.objects.Instance.save') -+ def test_confirm_resize_driver_confirm_migration_fails( -+ self, instance_save, notify_action, notify_usage, -+ instance_get_by_uuid, migration_get_by_id, add_fault): -+ """Tests the scenario that driver.confirm_migration raises some error -+ to make sure the error is properly handled, like the instance and -+ migration status is set to 'error'. -+ """ -+ self.migration.status = 'confirming' -+ migration_get_by_id.return_value = self.migration -+ instance_get_by_uuid.return_value = self.instance -+ -+ error = exception.HypervisorUnavailable( -+ host=self.migration.source_compute) -+ with test.nested( -+ mock.patch.object(self.compute, 'network_api'), -+ mock.patch.object(self.compute.driver, 'confirm_migration', -+ side_effect=error) -+ ) as ( -+ network_api, confirm_migration -+ ): -+ self.assertRaises(exception.HypervisorUnavailable, -+ self.compute.confirm_resize, -+ self.context, self.instance, self.migration) -+ # Make sure the instance is in ERROR status. -+ self.assertEqual(vm_states.ERROR, self.instance.vm_state) -+ # Make sure the migration is in error status. -+ self.assertEqual('error', self.migration.status) -+ # Instance.save is called twice, once to clear the resize metadata -+ # and once to set the instance to ERROR status. -+ self.assertEqual(2, instance_save.call_count) -+ # The migration.status should have been saved. -+ self.migration.save.assert_called_once_with() -+ # Assert other mocks we care less about. -+ notify_usage.assert_called_once() -+ notify_action.assert_called_once() -+ add_fault.assert_called_once() -+ confirm_migration.assert_called_once() -+ network_api.setup_networks_on_host.assert_called_once() -+ instance_get_by_uuid.assert_called_once() -+ migration_get_by_id.assert_called_once() -+ - @mock.patch('nova.scheduler.utils.resources_from_flavor') - def test_delete_allocation_after_move_confirm_by_migration(self, mock_rff): - mock_rff.return_value = {} --- -2.17.1 - diff -Nru nova-17.0.10/debian/patches/bug_1821594_2.patch nova-17.0.11/debian/patches/bug_1821594_2.patch --- nova-17.0.10/debian/patches/bug_1821594_2.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1821594_2.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,248 +0,0 @@ -From 8443758c0afabc940453d4c9b113dd8961395ab7 Mon Sep 17 00:00:00 2001 -From: Matt Riedemann -Date: Mon, 25 Mar 2019 14:02:17 -0400 -Subject: [PATCH 2/4] Delete allocations even if _confirm_resize raises - -When we are confirming a resize, the guest is on the dest -host and the instance host/node values in the database -are pointing at the dest host, so the _confirm_resize method -on the source is really best effort. If something fails, we -should not leak allocations in placement for the source compute -node resource provider since the instance is not actually -consuming the source node provider resources. - -This change refactors the error handling around the _confirm_resize -call so the big nesting for _error_out_instance_on_exception is -moved to confirm_resize and then a try/finally is added around -_confirm_resize so we can be sure to try and cleanup the allocations -even if _confirm_resize fails in some obscure way. If _confirm_resize -does fail, the error gets re-raised along with logging a traceback -and hint about how to correct the instance state in the DB by hard -rebooting the server on the dest host. - -Change-Id: I29c5f491ec20a71283190a1599e7732541de736f -Closes-Bug: #1821594 -(cherry picked from commit 03a6d26691c1f182224d59190b79f48df278099e) -(cherry picked from commit 5f515060f6c79f113f7b8107596e41056445c79f) -(cherry picked from commit 37ac54a42ec91821d62864d63c486c002608491b) ---- - nova/compute/manager.py | 124 ++++++++++++-------- - nova/tests/unit/compute/test_compute_mgr.py | 22 +++- - 2 files changed, 89 insertions(+), 57 deletions(-) - -diff --git a/nova/compute/manager.py b/nova/compute/manager.py -index 0dd5cbe550..4abcf6c541 100644 ---- a/nova/compute/manager.py -+++ b/nova/compute/manager.py -@@ -3725,7 +3725,31 @@ class ComputeManager(manager.Manager): - instance=instance) - return - -- self._confirm_resize(context, instance, migration=migration) -+ with self._error_out_instance_on_exception(context, instance): -+ old_instance_type = instance.old_flavor -+ try: -+ self._confirm_resize( -+ context, instance, migration=migration) -+ except Exception: -+ # Something failed when cleaning up the source host so -+ # log a traceback and leave a hint about hard rebooting -+ # the server to correct its state in the DB. -+ with excutils.save_and_reraise_exception(logger=LOG): -+ LOG.exception( -+ 'Confirm resize failed on source host %s. ' -+ 'Resource allocations in the placement service ' -+ 'will be removed regardless because the instance ' -+ 'is now on the destination host %s. You can try ' -+ 'hard rebooting the instance to correct its ' -+ 'state.', self.host, migration.dest_compute, -+ instance=instance) -+ finally: -+ # Whether an error occurred or not, at this point the -+ # instance is on the dest host so to avoid leaking -+ # allocations in placement, delete them here. -+ self._delete_allocation_after_move( -+ context, instance, migration, old_instance_type, -+ migration.source_node) - - do_confirm_resize(context, instance, migration.id) - -@@ -3737,63 +3761,59 @@ class ComputeManager(manager.Manager): - self.host, action=fields.NotificationAction.RESIZE_CONFIRM, - phase=fields.NotificationPhase.START) - -- with self._error_out_instance_on_exception(context, instance): -- # NOTE(danms): delete stashed migration information -- old_instance_type = instance.old_flavor -- instance.old_flavor = None -- instance.new_flavor = None -- instance.system_metadata.pop('old_vm_state', None) -- instance.save() -- -- # NOTE(tr3buchet): tear down networks on source host -- self.network_api.setup_networks_on_host(context, instance, -- migration.source_compute, teardown=True) -+ # NOTE(danms): delete stashed migration information -+ old_instance_type = instance.old_flavor -+ instance.old_flavor = None -+ instance.new_flavor = None -+ instance.system_metadata.pop('old_vm_state', None) -+ instance.save() - -- network_info = self.network_api.get_instance_nw_info(context, -- instance) -- # TODO(mriedem): Get BDMs here and pass them to the driver. -- self.driver.confirm_migration(context, migration, instance, -- network_info) -+ # NOTE(tr3buchet): tear down networks on source host -+ self.network_api.setup_networks_on_host(context, instance, -+ migration.source_compute, teardown=True) - -- migration.status = 'confirmed' -- with migration.obj_as_admin(): -- migration.save() -+ network_info = self.network_api.get_instance_nw_info(context, -+ instance) -+ # TODO(mriedem): Get BDMs here and pass them to the driver. -+ self.driver.confirm_migration(context, migration, instance, -+ network_info) - -- rt = self._get_resource_tracker() -- rt.drop_move_claim(context, instance, migration.source_node, -- old_instance_type, prefix='old_') -- self._delete_allocation_after_move(context, instance, migration, -- old_instance_type, -- migration.source_node) -- instance.drop_migration_context() -+ migration.status = 'confirmed' -+ with migration.obj_as_admin(): -+ migration.save() - -- # NOTE(mriedem): The old_vm_state could be STOPPED but the user -- # might have manually powered up the instance to confirm the -- # resize/migrate, so we need to check the current power state -- # on the instance and set the vm_state appropriately. We default -- # to ACTIVE because if the power state is not SHUTDOWN, we -- # assume _sync_instance_power_state will clean it up. -- p_state = instance.power_state -- vm_state = None -- if p_state == power_state.SHUTDOWN: -- vm_state = vm_states.STOPPED -- LOG.debug("Resized/migrated instance is powered off. " -- "Setting vm_state to '%s'.", vm_state, -- instance=instance) -- else: -- vm_state = vm_states.ACTIVE -+ rt = self._get_resource_tracker() -+ rt.drop_move_claim(context, instance, migration.source_node, -+ old_instance_type, prefix='old_') -+ instance.drop_migration_context() -+ -+ # NOTE(mriedem): The old_vm_state could be STOPPED but the user -+ # might have manually powered up the instance to confirm the -+ # resize/migrate, so we need to check the current power state -+ # on the instance and set the vm_state appropriately. We default -+ # to ACTIVE because if the power state is not SHUTDOWN, we -+ # assume _sync_instance_power_state will clean it up. -+ p_state = instance.power_state -+ vm_state = None -+ if p_state == power_state.SHUTDOWN: -+ vm_state = vm_states.STOPPED -+ LOG.debug("Resized/migrated instance is powered off. " -+ "Setting vm_state to '%s'.", vm_state, -+ instance=instance) -+ else: -+ vm_state = vm_states.ACTIVE - -- instance.vm_state = vm_state -- instance.task_state = None -- instance.save(expected_task_state=[None, task_states.DELETING, -- task_states.SOFT_DELETING]) -+ instance.vm_state = vm_state -+ instance.task_state = None -+ instance.save(expected_task_state=[None, task_states.DELETING, -+ task_states.SOFT_DELETING]) - -- self._notify_about_instance_usage( -- context, instance, "resize.confirm.end", -- network_info=network_info) -- compute_utils.notify_about_instance_action(context, instance, -- self.host, action=fields.NotificationAction.RESIZE_CONFIRM, -- phase=fields.NotificationPhase.END) -+ self._notify_about_instance_usage( -+ context, instance, "resize.confirm.end", -+ network_info=network_info) -+ compute_utils.notify_about_instance_action(context, instance, -+ self.host, action=fields.NotificationAction.RESIZE_CONFIRM, -+ phase=fields.NotificationPhase.END) - - def _delete_allocation_after_move(self, context, instance, migration, - flavor, nodename): -diff --git a/nova/tests/unit/compute/test_compute_mgr.py b/nova/tests/unit/compute/test_compute_mgr.py -index 108f3078b2..6c95c3cfdf 100644 ---- a/nova/tests/unit/compute/test_compute_mgr.py -+++ b/nova/tests/unit/compute/test_compute_mgr.py -@@ -6258,6 +6258,7 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - dest_node=None, - dest_host=None, - source_compute='source_compute', -+ source_node='source_node', - status='migrating') - self.migration.save = mock.MagicMock() - self.useFixture(fixtures.SpawnIsSynchronousFixture()) -@@ -6611,6 +6612,8 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - do_finish_revert_resize() - - def test_confirm_resize_deletes_allocations(self): -+ @mock.patch('nova.objects.Instance.get_by_uuid') -+ @mock.patch('nova.objects.Migration.get_by_id') - @mock.patch.object(self.migration, 'save') - @mock.patch.object(self.compute, '_notify_about_instance_usage') - @mock.patch.object(self.compute, 'network_api') -@@ -6621,12 +6624,15 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - @mock.patch.object(self.instance, 'save') - def do_confirm_resize(mock_save, mock_drop, mock_delete, mock_get_rt, - mock_confirm, mock_nwapi, mock_notify, -- mock_mig_save): -+ mock_mig_save, mock_mig_get, mock_inst_get): - self.instance.migration_context = objects.MigrationContext() - self.migration.source_compute = self.instance['host'] - self.migration.source_node = self.instance['node'] -- self.compute._confirm_resize(self.context, self.instance, -- self.migration) -+ self.migration.status = 'confirming' -+ mock_mig_get.return_value = self.migration -+ mock_inst_get.return_value = self.instance -+ self.compute.confirm_resize(self.context, self.instance, -+ self.migration) - mock_delete.assert_called_once_with(self.context, self.instance, - self.migration, - self.instance.old_flavor, -@@ -6682,9 +6688,10 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - with test.nested( - mock.patch.object(self.compute, 'network_api'), - mock.patch.object(self.compute.driver, 'confirm_migration', -- side_effect=error) -+ side_effect=error), -+ mock.patch.object(self.compute, '_delete_allocation_after_move') - ) as ( -- network_api, confirm_migration -+ network_api, confirm_migration, delete_allocation - ): - self.assertRaises(exception.HypervisorUnavailable, - self.compute.confirm_resize, -@@ -6698,6 +6705,11 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - self.assertEqual(2, instance_save.call_count) - # The migration.status should have been saved. - self.migration.save.assert_called_once_with() -+ # Allocations should always be cleaned up even if cleaning up the -+ # source host fails. -+ delete_allocation.assert_called_once_with( -+ self.context, self.instance, self.migration, -+ self.instance.old_flavor, self.migration.source_node) - # Assert other mocks we care less about. - notify_usage.assert_called_once() - notify_action.assert_called_once() --- -2.17.1 - diff -Nru nova-17.0.10/debian/patches/bug_1821594_3.patch nova-17.0.11/debian/patches/bug_1821594_3.patch --- nova-17.0.10/debian/patches/bug_1821594_3.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1821594_3.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,94 +0,0 @@ -From 94cae3b149b80fa16d0bf6ec6218d9554360e901 Mon Sep 17 00:00:00 2001 -From: Matt Riedemann -Date: Wed, 15 May 2019 12:27:17 -0400 -Subject: [PATCH 3/4] [stable-only] Delete allocations even if _confirm_resize - raises (part 2) - -The backport https://review.opendev.org/#/c/652153/ to fix -bug 1821594 did not account for how the _delete_allocation_after_move -method before Stein is tightly coupled to the migration status -being set to "confirmed" which is what the _confirm_resize method -does after self.driver.confirm_migration returns. - -However, if self.driver.confirm_migration raises an exception -we still want to cleanup the allocations held on the source node -and for that we call _delete_allocation_after_move. But because -of that tight coupling before Stein, we need to temporarily -mutate the migration status to "confirmed" to get the cleanup -method to do what we want. - -This isn't a problem starting in Stein because change -I0851e2d54a1fdc82fe3291fb7e286e790f121e92 removed that -tight coupling on the migration status, so this is a stable -branch only change. - -Note that we don't call self.reportclient.delete_allocation_for_instance -directly since before Stein we still need to account for a -migration that does not move the source node allocations to the -migration record, and that logic is in _delete_allocation_after_move. - -A simple unit test assertion is added here but the functional -test added in change I9d6478f492351b58aa87b8f56e907ee633d6d1c6 -will assert the bug is fixed properly before Stein. - -Change-Id: I933687891abef4878de09481937d576ce5899511 -Closes-Bug: #1821594 -(cherry picked from commit dac3239e92fc1865cacc17bbfbd2316072a9d26e) ---- - nova/compute/manager.py | 13 ++++++++++--- - nova/tests/unit/compute/test_compute_mgr.py | 9 ++++++++- - 2 files changed, 18 insertions(+), 4 deletions(-) - -diff --git a/nova/compute/manager.py b/nova/compute/manager.py -index 4abcf6c541..f8a0d47b62 100644 ---- a/nova/compute/manager.py -+++ b/nova/compute/manager.py -@@ -3747,9 +3747,16 @@ class ComputeManager(manager.Manager): - # Whether an error occurred or not, at this point the - # instance is on the dest host so to avoid leaking - # allocations in placement, delete them here. -- self._delete_allocation_after_move( -- context, instance, migration, old_instance_type, -- migration.source_node) -+ # NOTE(mriedem): _delete_allocation_after_move is tightly -+ # coupled to the migration status on the confirm step so -+ # we unfortunately have to mutate the migration status to -+ # have _delete_allocation_after_move cleanup the allocation -+ # held by the migration consumer. -+ with utils.temporary_mutation( -+ migration, status='confirmed'): -+ self._delete_allocation_after_move( -+ context, instance, migration, old_instance_type, -+ migration.source_node) - - do_confirm_resize(context, instance, migration.id) - -diff --git a/nova/tests/unit/compute/test_compute_mgr.py b/nova/tests/unit/compute/test_compute_mgr.py -index 6c95c3cfdf..5aa5ad9d2e 100644 ---- a/nova/tests/unit/compute/test_compute_mgr.py -+++ b/nova/tests/unit/compute/test_compute_mgr.py -@@ -6683,13 +6683,20 @@ class ComputeManagerMigrationTestCase(test.NoDBTestCase): - migration_get_by_id.return_value = self.migration - instance_get_by_uuid.return_value = self.instance - -+ def fake_delete_allocation_after_move(_context, instance, migration, -+ flavor, nodename): -+ # The migration.status must be 'confirmed' for the method to -+ # properly cleanup the allocation held by the migration. -+ self.assertEqual('confirmed', migration.status) -+ - error = exception.HypervisorUnavailable( - host=self.migration.source_compute) - with test.nested( - mock.patch.object(self.compute, 'network_api'), - mock.patch.object(self.compute.driver, 'confirm_migration', - side_effect=error), -- mock.patch.object(self.compute, '_delete_allocation_after_move') -+ mock.patch.object(self.compute, '_delete_allocation_after_move', -+ side_effect=fake_delete_allocation_after_move) - ) as ( - network_api, confirm_migration, delete_allocation - ): --- -2.17.1 - diff -Nru nova-17.0.10/debian/patches/bug_1821594_4.patch nova-17.0.11/debian/patches/bug_1821594_4.patch --- nova-17.0.10/debian/patches/bug_1821594_4.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1821594_4.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,127 +0,0 @@ -From b755275d2559f71ad612fbd15c135c2ad25d17e2 Mon Sep 17 00:00:00 2001 -From: Rodrigo Barbieri -Date: Wed, 8 May 2019 16:01:25 -0300 -Subject: [PATCH 4/4] Add functional confirm_migration_error test - -This test checks if allocations have been -successfully cleaned up upon the driver failing -during "confirm_migration". - -NOTE(mriedem): The _wait_until_deleted helper method -is modified here to include the changes from patch -I4ced19bd9259f0b5a50b89dd5908abe35ca73894 in Rocky -otherwise the test fails. - -Change-Id: I9d6478f492351b58aa87b8f56e907ee633d6d1c6 -Related-bug: #1821594 -(cherry picked from commit 873ac499c50125adc2fb49728d936922f9acf4a9) -(cherry picked from commit d7d7f115430c7ffeb88ec9dcd155ac69b29d7513) -(cherry picked from commit 01bfb7863c80c43538632952ec9f1728f0b412d6) ---- - nova/tests/functional/integrated_helpers.py | 3 +- - nova/tests/functional/test_servers.py | 72 +++++++++++++++++++++ - 2 files changed, 74 insertions(+), 1 deletion(-) - -diff --git a/nova/tests/functional/integrated_helpers.py b/nova/tests/functional/integrated_helpers.py -index 4fcd60f1db..941a4015e8 100644 ---- a/nova/tests/functional/integrated_helpers.py -+++ b/nova/tests/functional/integrated_helpers.py -@@ -270,10 +270,11 @@ class InstanceHelperMixin(object): - return server - - def _wait_until_deleted(self, server): -+ initially_in_error = (server['status'] == 'ERROR') - try: - for i in range(40): - server = self.api.get_server(server['id']) -- if server['status'] == 'ERROR': -+ if not initially_in_error and server['status'] == 'ERROR': - self.fail('Server went to error state instead of' - 'disappearing.') - time.sleep(0.5) -diff --git a/nova/tests/functional/test_servers.py b/nova/tests/functional/test_servers.py -index 9c35fbe2af..0d271dd0f5 100644 ---- a/nova/tests/functional/test_servers.py -+++ b/nova/tests/functional/test_servers.py -@@ -1852,6 +1852,78 @@ class ServerMovingTests(ProviderUsageBaseTestCase): - new_flavor=new_flavor, source_rp_uuid=source_rp_uuid, - dest_rp_uuid=dest_rp_uuid) - -+ def test_migration_confirm_resize_error(self): -+ source_hostname = self.compute1.host -+ dest_hostname = self.compute2.host -+ -+ source_rp_uuid = self._get_provider_uuid_by_host(source_hostname) -+ dest_rp_uuid = self._get_provider_uuid_by_host(dest_hostname) -+ -+ server = self._boot_and_check_allocations(self.flavor1, -+ source_hostname) -+ -+ self._move_and_check_allocations( -+ server, request={'migrate': None}, old_flavor=self.flavor1, -+ new_flavor=self.flavor1, source_rp_uuid=source_rp_uuid, -+ dest_rp_uuid=dest_rp_uuid) -+ -+ # Mock failure -+ def fake_confirm_migration(context, migration, instance, network_info): -+ raise exception.MigrationPreCheckError( -+ reason='test_migration_confirm_resize_error') -+ -+ with mock.patch('nova.virt.fake.FakeDriver.' -+ 'confirm_migration', -+ side_effect=fake_confirm_migration): -+ -+ # Confirm the migration/resize and check the usages -+ post = {'confirmResize': None} -+ self.api.post_server_action( -+ server['id'], post, check_response_status=[204]) -+ server = self._wait_for_state_change(self.api, server, 'ERROR') -+ -+ # After confirming and error, we should have an allocation only on the -+ # destination host -+ source_usages = self._get_provider_usages(source_rp_uuid) -+ self.assertEqual({'VCPU': 0, -+ 'MEMORY_MB': 0, -+ 'DISK_GB': 0}, source_usages, -+ 'Source host %s still has usage after the failed ' -+ 'migration_confirm' % source_hostname) -+ -+ # Check that the server only allocates resource from the original host -+ allocations = self._get_allocations_by_server_uuid(server['id']) -+ self.assertEqual(1, len(allocations)) -+ -+ dest_allocation = allocations[dest_rp_uuid]['resources'] -+ self.assertFlavorMatchesAllocation(self.flavor1, dest_allocation) -+ -+ dest_usages = self._get_provider_usages(dest_rp_uuid) -+ self.assertFlavorMatchesAllocation(self.flavor1, dest_usages) -+ -+ self._run_periodics() -+ -+ # After confirming and error, we should have an allocation only on the -+ # destination host -+ source_usages = self._get_provider_usages(source_rp_uuid) -+ self.assertEqual({'VCPU': 0, -+ 'MEMORY_MB': 0, -+ 'DISK_GB': 0}, source_usages, -+ 'Source host %s still has usage after the failed ' -+ 'migration_confirm' % source_hostname) -+ -+ # Check that the server only allocates resource from the original host -+ allocations = self._get_allocations_by_server_uuid(server['id']) -+ self.assertEqual(1, len(allocations)) -+ -+ dest_allocation = allocations[dest_rp_uuid]['resources'] -+ self.assertFlavorMatchesAllocation(self.flavor1, dest_allocation) -+ -+ dest_usages = self._get_provider_usages(dest_rp_uuid) -+ self.assertFlavorMatchesAllocation(self.flavor1, dest_usages) -+ -+ self._delete_and_check_allocations(server) -+ - def _test_resize_revert(self, dest_hostname): - source_hostname = self._other_hostname(dest_hostname) - source_rp_uuid = self._get_provider_uuid_by_host(source_hostname) --- -2.17.1 - diff -Nru nova-17.0.10/debian/patches/bug_1825882.patch nova-17.0.11/debian/patches/bug_1825882.patch --- nova-17.0.10/debian/patches/bug_1825882.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1825882.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,104 +0,0 @@ -From 63b45a87b82655824e61641016430a6613cee001 Mon Sep 17 00:00:00 2001 -From: Lee Yarwood -Date: Thu, 25 Apr 2019 14:42:09 +0100 -Subject: [PATCH] libvirt: Stop ignoring unknown libvirtError exceptions during - volume attach - -Id346bce6e47431988cce7001abcf29a9faf2936a attempted to introduce a -simple breadcrumb in the logs to highlight a known Libvirt issue. -Unfortunately this change resulted in libvirtError exceptions that -didn't match the known issue being silently ignored. - -This change corrects this by using excutils.save_and_reraise_exception -to ensure all libvirtError exceptions are logged and raised regardless -of being linked to the known issue. - -Change-Id: Ib440f4f2e484312af5f393722363846f6c95b760 -Closes-Bug: #1825882 -(cherry picked from commit bc57ae50734fa6a70115b6369e867079fb5eb4b8) -(cherry picked from commit b3dedefcc58a1fc76ba37f9f8bb1ef7d238aaceb) -(cherry picked from commit 34b4220448395f10eb2fd39d68b3f527339ab414) ---- - nova/tests/unit/virt/libvirt/test_driver.py | 40 +++++++++++++++++++++ - nova/virt/libvirt/driver.py | 14 +++++--- - 2 files changed, 49 insertions(+), 5 deletions(-) - -diff --git a/nova/tests/unit/virt/libvirt/test_driver.py b/nova/tests/unit/virt/libvirt/test_driver.py -index 8f56ad456e..5cafb59234 100644 ---- a/nova/tests/unit/virt/libvirt/test_driver.py -+++ b/nova/tests/unit/virt/libvirt/test_driver.py -@@ -7300,6 +7300,46 @@ class LibvirtConnTestCase(test.NoDBTestCase, - disk_bus=bdm['disk_bus'], device_type=bdm['device_type']) - mock_log.warning.assert_called_once() - -+ @mock.patch('nova.virt.libvirt.blockinfo.get_info_from_bdm') -+ def test_attach_volume_with_libvirt_exception(self, mock_get_info): -+ drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) -+ instance = objects.Instance(**self.test_instance) -+ connection_info = {"driver_volume_type": "fake", -+ "data": {"device_path": "/fake", -+ "access_mode": "rw"}} -+ bdm = {'device_name': 'vdb', -+ 'disk_bus': 'fake-bus', -+ 'device_type': 'fake-type'} -+ disk_info = {'bus': bdm['disk_bus'], 'type': bdm['device_type'], -+ 'dev': 'vdb'} -+ libvirt_exc = fakelibvirt.make_libvirtError(fakelibvirt.libvirtError, -+ "Target vdb already exists', device is busy", -+ error_code=fakelibvirt.VIR_ERR_INTERNAL_ERROR) -+ -+ with test.nested( -+ mock.patch.object(drvr._host, 'get_guest'), -+ mock.patch('nova.virt.libvirt.driver.LOG'), -+ mock.patch.object(drvr, '_connect_volume'), -+ mock.patch.object(drvr, '_get_volume_config'), -+ mock.patch.object(drvr, '_check_discard_for_attach_volume'), -+ mock.patch.object(drvr, '_build_device_metadata'), -+ ) as (mock_get_guest, mock_log, mock_connect_volume, -+ mock_get_volume_config, mock_check_discard, mock_build_metadata): -+ -+ mock_conf = mock.MagicMock() -+ mock_guest = mock.MagicMock() -+ mock_guest.attach_device.side_effect = libvirt_exc -+ mock_get_volume_config.return_value = mock_conf -+ mock_get_guest.return_value = mock_guest -+ mock_get_info.return_value = disk_info -+ mock_build_metadata.return_value = objects.InstanceDeviceMetadata() -+ -+ self.assertRaises(fakelibvirt.libvirtError, drvr.attach_volume, -+ self.context, connection_info, instance, "/dev/vdb", -+ disk_bus=bdm['disk_bus'], device_type=bdm['device_type']) -+ mock_log.exception.assert_called_once_with(u'Failed to attach ' -+ 'volume at mountpoint: %s', '/dev/vdb', instance=instance) -+ - @mock.patch('nova.utils.get_image_from_system_metadata') - @mock.patch('nova.virt.libvirt.blockinfo.get_info_from_bdm') - @mock.patch('nova.virt.libvirt.host.Host._get_domain') -diff --git a/nova/virt/libvirt/driver.py b/nova/virt/libvirt/driver.py -index c39720b004..42f6ec83d2 100644 ---- a/nova/virt/libvirt/driver.py -+++ b/nova/virt/libvirt/driver.py -@@ -1490,11 +1490,15 @@ class LibvirtDriver(driver.ComputeDriver): - # distributions provide Libvirt 3.3.0 or earlier with - # https://libvirt.org/git/?p=libvirt.git;a=commit;h=7189099 applied. - except libvirt.libvirtError as ex: -- if 'Incorrect number of padding bytes' in six.text_type(ex): -- LOG.warning(_('Failed to attach encrypted volume due to a ' -- 'known Libvirt issue, see the following bug for details: ' -- 'https://bugzilla.redhat.com/show_bug.cgi?id=1447297')) -- raise -+ with excutils.save_and_reraise_exception(): -+ if 'Incorrect number of padding bytes' in six.text_type(ex): -+ LOG.warning(_('Failed to attach encrypted volume due to a ' -+ 'known Libvirt issue, see the following bug ' -+ 'for details: ' -+ 'https://bugzilla.redhat.com/1447297')) -+ else: -+ LOG.exception(_('Failed to attach volume at mountpoint: ' -+ '%s'), mountpoint, instance=instance) - except Exception: - LOG.exception(_('Failed to attach volume at mountpoint: %s'), - mountpoint, instance=instance) --- -2.20.1 - diff -Nru nova-17.0.10/debian/patches/bug_1826523.patch nova-17.0.11/debian/patches/bug_1826523.patch --- nova-17.0.10/debian/patches/bug_1826523.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/bug_1826523.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,57 +0,0 @@ -From 99a63a6633f623cc4e6c9b361965a9c0935113cf Mon Sep 17 00:00:00 2001 -From: Lee Yarwood -Date: Thu, 25 Apr 2019 15:34:41 +0100 -Subject: [PATCH] libvirt: Always disconnect volumes after libvirtError - exceptions - -Building on Ib440f4f2e484312af5f393722363846f6c95b760 we should always -attempt to disconnect volumes from the host when exceptions are thrown -while attempting to attach a volume to a domain. This was previously -done for generic exceptions but not for libvirtError exceptions. - -Closes-Bug: #1826523 -Change-Id: If21230869826c992e7d0398434b9a4b255940213 -(cherry picked from commit 091a910576d9b580678f1881fffa425ab4632f48) -(cherry picked from commit 048d5b790f3da2756a0a1bf2bc015812cb24d53a) -(cherry picked from commit 786083c91fa69ba9ff90bafe80d3b83b9f9bbc69) ---- - nova/tests/unit/virt/libvirt/test_driver.py | 10 ++++++++-- - nova/virt/libvirt/driver.py | 2 ++ - 2 files changed, 10 insertions(+), 2 deletions(-) - ---- a/nova/tests/unit/virt/libvirt/test_driver.py -+++ b/nova/tests/unit/virt/libvirt/test_driver.py -@@ -7260,11 +7260,13 @@ - mock.patch.object(drvr._host, 'get_guest'), - mock.patch('nova.virt.libvirt.driver.LOG'), - mock.patch.object(drvr, '_connect_volume'), -+ mock.patch.object(drvr, '_disconnect_volume'), - mock.patch.object(drvr, '_get_volume_config'), - mock.patch.object(drvr, '_check_discard_for_attach_volume'), - mock.patch.object(drvr, '_build_device_metadata'), - ) as (mock_get_guest, mock_log, mock_connect_volume, -- mock_get_volume_config, mock_check_discard, mock_build_metadata): -+ mock_disconnect_volume, mock_get_volume_config, -+ mock_check_discard, mock_build_metadata): - - mock_conf = mock.MagicMock() - mock_guest = mock.MagicMock() -@@ -7279,6 +7281,7 @@ - disk_bus=bdm['disk_bus'], device_type=bdm['device_type']) - mock_log.exception.assert_called_once_with(u'Failed to attach ' - 'volume at mountpoint: %s', '/dev/vdb', instance=instance) -+ mock_disconnect_volume.assert_called_once() - - @mock.patch('nova.utils.get_image_from_system_metadata') - @mock.patch('nova.virt.libvirt.blockinfo.get_info_from_bdm') ---- a/nova/virt/libvirt/driver.py -+++ b/nova/virt/libvirt/driver.py -@@ -1483,6 +1483,8 @@ - else: - LOG.exception(_('Failed to attach volume at mountpoint: ' - '%s'), mountpoint, instance=instance) -+ self._disconnect_volume(context, connection_info, instance, -+ encryption=encryption) - except Exception: - LOG.exception(_('Failed to attach volume at mountpoint: %s'), - mountpoint, instance=instance) diff -Nru nova-17.0.10/debian/patches/series nova-17.0.11/debian/patches/series --- nova-17.0.10/debian/patches/series 2019-08-13 00:31:55.000000000 +0000 +++ nova-17.0.11/debian/patches/series 2019-07-29 15:45:29.000000000 +0000 @@ -2,12 +2,5 @@ skip-ssl-tests.patch arm-console-patch.patch revert-generalize-db-conf-group-copying.patch -xenapi-agent-change-openssl-error-handling.patch skip-double-word-hacking-test.patch -bug_1825882.patch -bug_1826523.patch -bug_1821594_1.patch -bug_1821594_2.patch -bug_1821594_3.patch -bug_1821594_4.patch CVE-2019-14433.patch diff -Nru nova-17.0.10/debian/patches/xenapi-agent-change-openssl-error-handling.patch nova-17.0.11/debian/patches/xenapi-agent-change-openssl-error-handling.patch --- nova-17.0.10/debian/patches/xenapi-agent-change-openssl-error-handling.patch 2019-06-10 13:20:41.000000000 +0000 +++ nova-17.0.11/debian/patches/xenapi-agent-change-openssl-error-handling.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,99 +0,0 @@ -From 5b0adaa0ca5f757bb224d1ffac0c6705b03ee2ed Mon Sep 17 00:00:00 2001 -From: Corey Bryant -Date: Thu, 07 Feb 2019 10:12:54 -0500 -Subject: [PATCH] xenapi/agent: Change openssl error handling - -Prior to this patch, if the openssl command returned a zero exit code -and wrote details to stderr, nova would raise a RuntimeError exception. -This patch changes the behavior to only raise a RuntimeError exception -when openssl returns a non-zero exit code. Regardless of the exit code -a warning will always be logged with stderr details if stderr is not -None. Note that processutils.execute will now raise a -processutils.ProcessExecutionError exception for any non-zero exit code -since we are passing check_exit_code=True, which we convert to a -Runtime error. - -Thanks to Dimitri John Ledkov and Eric Fried - for helping with this patch. - -Conflicts: - nova/virt/xenapi/agent.py - -NOTE(coreycb): The conflict is due to -Ibe2f478288db42f8168b52dfc14d85ab92ace74b not being in stable/queens. - -Change-Id: I212ac2b5ccd93e00adb7b9fe102fcb70857c6073 -Partial-Bug: #1771506 -(cherry picked from commit 1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a) -(cherry picked from commit 64793cf6f77c5ba7c9ea51662d936c7545ffce8c) -(cherry picked from commit 82de38ad4ce86c5398538a8635713a86407216d0) ---- - -diff --git a/nova/tests/unit/virt/xenapi/test_agent.py b/nova/tests/unit/virt/xenapi/test_agent.py -index 8c77445..2848fc2 100644 ---- a/nova/tests/unit/virt/xenapi/test_agent.py -+++ b/nova/tests/unit/virt/xenapi/test_agent.py -@@ -19,6 +19,7 @@ - import mock - from os_xenapi.client import host_agent - from os_xenapi.client import XenAPI -+from oslo_concurrency import processutils - from oslo_utils import uuidutils - - from nova import exception -@@ -311,6 +312,19 @@ - - mock_add_fault.assert_called_once_with(error, mock.ANY) - -+ @mock.patch('oslo_concurrency.processutils.execute') -+ def test_run_ssl_successful(self, mock_execute): -+ mock_execute.return_value = ('0', -+ '*** WARNING : deprecated key derivation used.' -+ 'Using -iter or -pbkdf2 would be better.') -+ agent.SimpleDH()._run_ssl('foo') -+ -+ @mock.patch('oslo_concurrency.processutils.execute', -+ side_effect=processutils.ProcessExecutionError( -+ exit_code=1, stderr=('ERROR: Something bad happened'))) -+ def test_run_ssl_failure(self, mock_execute): -+ self.assertRaises(RuntimeError, agent.SimpleDH()._run_ssl, 'foo') -+ - - class UpgradeRequiredTestCase(test.NoDBTestCase): - def test_less_than(self): -diff --git a/nova/virt/xenapi/agent.py b/nova/virt/xenapi/agent.py -index 9410f47..f0e9949 100644 ---- a/nova/virt/xenapi/agent.py -+++ b/nova/virt/xenapi/agent.py -@@ -21,6 +21,7 @@ - - from os_xenapi.client import host_agent - from os_xenapi.client import XenAPI -+from oslo_concurrency import processutils - from oslo_log import log as logging - from oslo_serialization import base64 - from oslo_serialization import jsonutils -@@ -422,11 +423,18 @@ - 'pass:%s' % self._shared, '-nosalt'] - if decrypt: - cmd.append('-d') -- out, err = utils.execute(*cmd, -- process_input=encodeutils.safe_encode(text)) -- if err: -- raise RuntimeError(_('OpenSSL error: %s') % err) -- return out -+ try: -+ out, err = utils.execute( -+ *cmd, -+ process_input=encodeutils.safe_encode(text), -+ check_exit_code=True) -+ if err: -+ LOG.warning("OpenSSL stderr: %s", err) -+ return out -+ except processutils.ProcessExecutionError as e: -+ raise RuntimeError( -+ _('OpenSSL errored with exit code %(exit_code)d: %(stderr)s') % -+ {'exit_code': e.exit_code, 'stderr': e.stderr}) - - def encrypt(self, text): - return self._run_ssl(text).strip('\n') diff -Nru nova-17.0.10/doc/source/admin/adv-config.rst nova-17.0.11/doc/source/admin/adv-config.rst --- nova-17.0.10/doc/source/admin/adv-config.rst 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/doc/source/admin/adv-config.rst 2019-07-10 21:45:12.000000000 +0000 @@ -20,6 +20,8 @@ performance. The Compute service provides features to improve individual instance for these kind of workloads. +.. include:: /common/numa-live-migration-warning.txt + .. toctree:: :maxdepth: 2 diff -Nru nova-17.0.10/doc/source/admin/configuring-migrations.rst nova-17.0.11/doc/source/admin/configuring-migrations.rst --- nova-17.0.10/doc/source/admin/configuring-migrations.rst 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/doc/source/admin/configuring-migrations.rst 2019-07-10 21:45:12.000000000 +0000 @@ -19,7 +19,8 @@ .. note:: Not all Compute service hypervisor drivers support live-migration, or - support all live-migration features. + support all live-migration features. Similarly not all compute service + features are supported. Consult :doc:`/user/support-matrix` to determine which hypervisors support live-migration. diff -Nru nova-17.0.10/doc/source/admin/cpu-topologies.rst nova-17.0.11/doc/source/admin/cpu-topologies.rst --- nova-17.0.10/doc/source/admin/cpu-topologies.rst 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/doc/source/admin/cpu-topologies.rst 2019-07-10 21:45:12.000000000 +0000 @@ -7,6 +7,8 @@ CPUs available to instances. These features help minimize latency and maximize performance. +.. include:: /common/numa-live-migration-warning.txt + SMP, NUMA, and SMT ~~~~~~~~~~~~~~~~~~ diff -Nru nova-17.0.10/doc/source/common/numa-live-migration-warning.txt nova-17.0.11/doc/source/common/numa-live-migration-warning.txt --- nova-17.0.10/doc/source/common/numa-live-migration-warning.txt 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/doc/source/common/numa-live-migration-warning.txt 2019-07-10 21:45:04.000000000 +0000 @@ -0,0 +1,10 @@ +.. important:: + + Unless :oslo.config:option:`specifically enabled + `, live migration is not currently + possible for instances with a NUMA topology when using the libvirt driver. + A NUMA topology may be specified explicitly or can be added implicitly due + to the use of CPU pinning or huge pages. Refer to `bug #1289064`__ for more + information. + + __ https://bugs.launchpad.net/nova/+bug/1289064 diff -Nru nova-17.0.10/doc/source/user/feature-classification.rst nova-17.0.11/doc/source/user/feature-classification.rst --- nova-17.0.10/doc/source/user/feature-classification.rst 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/doc/source/user/feature-classification.rst 2019-07-10 21:45:04.000000000 +0000 @@ -65,6 +65,8 @@ bare metal like performance, i.e. low latency and close to line speed performance. +.. include:: /common/numa-live-migration-warning.txt + .. feature_matrix:: feature-matrix-nfv.ini .. _matrix-hpc: diff -Nru nova-17.0.10/nova/api/openstack/compute/migrate_server.py nova-17.0.11/nova/api/openstack/compute/migrate_server.py --- nova-17.0.10/nova/api/openstack/compute/migrate_server.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/api/openstack/compute/migrate_server.py 2019-07-10 21:45:12.000000000 +0000 @@ -100,7 +100,11 @@ disk_over_commit = strutils.bool_from_string(disk_over_commit, strict=True) - instance = common.get_instance(self.compute_api, context, id) + # NOTE(stephenfin): we need 'numa_topology' because of the + # 'LiveMigrationTask._check_instance_has_no_numa' check in the + # conductor + instance = common.get_instance(self.compute_api, context, id, + expected_attrs=['numa_topology']) try: self.compute_api.live_migrate(context, instance, block_migration, disk_over_commit, host, force, async) diff -Nru nova-17.0.10/nova/api/openstack/compute/volumes.py nova-17.0.11/nova/api/openstack/compute/volumes.py --- nova-17.0.10/nova/api/openstack/compute/volumes.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/api/openstack/compute/volumes.py 2019-07-10 21:45:12.000000000 +0000 @@ -394,7 +394,8 @@ new_volume) except exception.VolumeBDMNotFound as e: raise exc.HTTPNotFound(explanation=e.format_message()) - except exception.InvalidVolume as e: + except (exception.InvalidVolume, + exception.MultiattachSwapVolumeNotSupported) as e: raise exc.HTTPBadRequest(explanation=e.format_message()) except exception.InstanceIsLocked as e: raise exc.HTTPConflict(explanation=e.format_message()) diff -Nru nova-17.0.10/nova/compute/api.py nova-17.0.11/nova/compute/api.py --- nova-17.0.10/nova/compute/api.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/compute/api.py 2019-07-10 21:45:12.000000000 +0000 @@ -57,6 +57,7 @@ from nova import context as nova_context from nova import crypto from nova.db import base +from nova.db.sqlalchemy import api as db_api from nova import exception from nova import exception_wrapper from nova import hooks @@ -886,6 +887,20 @@ # by the network quotas return base_options, max_network_count, key_pair, security_groups + @staticmethod + @db_api.api_context_manager.writer + def _create_reqspec_buildreq_instmapping(context, rs, br, im): + """Create the request spec, build request, and instance mapping in a + single database transaction. + + The RequestContext must be passed in to this method so that the + database transaction context manager decorator will nest properly and + include each create() into the same transaction context. + """ + rs.create() + br.create() + im.create() + def _provision_instances(self, context, instance_type, min_count, max_count, base_options, boot_meta, security_groups, block_device_mapping, shutdown_terminate, @@ -915,7 +930,6 @@ # spec as this is how the conductor knows how many were in this # batch. req_spec.num_instances = num_instances - req_spec.create() # Create an instance object, but do not store in db yet. instance = objects.Instance(context=context) @@ -939,7 +953,6 @@ project_id=instance.project_id, block_device_mappings=block_device_mapping, tags=instance_tags) - build_request.create() # Create an instance_mapping. The null cell_mapping indicates # that the instance doesn't yet exist in a cell, and lookups @@ -951,7 +964,14 @@ inst_mapping.instance_uuid = instance_uuid inst_mapping.project_id = context.project_id inst_mapping.cell_mapping = None - inst_mapping.create() + + # Create the request spec, build request, and instance mapping + # records in a single transaction so that if a DBError is + # raised from any of them, all INSERTs will be rolled back and + # no orphaned records will be left behind. + self._create_reqspec_buildreq_instmapping(context, req_spec, + build_request, + inst_mapping) instances_to_build.append( (req_spec, build_request, inst_mapping)) @@ -2965,10 +2985,15 @@ if strutils.bool_from_string(instance.system_metadata.get( 'image_os_require_quiesce')): raise - else: + + if isinstance(err, exception.NovaException): LOG.info('Skipping quiescing instance: %(reason)s.', - {'reason': err}, + {'reason': err.format_message()}, instance=instance) + else: + LOG.info('Skipping quiescing instance because the ' + 'operation is not supported by the underlying ' + 'compute driver.', instance=instance) # NOTE(tasker): discovered that an uncaught exception could occur # after the instance has been frozen. catch and thaw. except Exception as ex: @@ -3290,6 +3315,19 @@ self._check_quota_for_upsize(context, instance, instance.flavor, instance.old_flavor) + # The AZ for the server may have changed when it was migrated so while + # we are in the API and have access to the API DB, update the + # instance.availability_zone before casting off to the compute service. + # Note that we do this in the API to avoid an "up-call" from the + # compute service to the API DB. This is not great in case something + # fails during revert before the instance.host is updated to the + # original source host, but it is good enough for now. Long-term we + # could consider passing the AZ down to compute so it can set it when + # the instance.host value is set in finish_revert_resize. + instance.availability_zone = ( + availability_zones.get_host_availability_zone( + context, migration.source_compute)) + instance.task_state = task_states.RESIZE_REVERTING instance.save(expected_task_state=[None]) @@ -4156,6 +4194,52 @@ else: self._detach_volume(context, instance, volume) + def _count_attachments_for_swap(self, ctxt, volume): + """Counts the number of attachments for a swap-related volume. + + Attempts to only count read/write attachments if the volume attachment + records exist, otherwise simply just counts the number of attachments + regardless of attach mode. + + :param ctxt: nova.context.RequestContext - user request context + :param volume: nova-translated volume dict from nova.volume.cinder. + :returns: count of attachments for the volume + """ + # This is a dict, keyed by server ID, to a dict of attachment_id and + # mountpoint. + attachments = volume.get('attachments', {}) + # Multiattach volumes can have more than one attachment, so if there + # is more than one attachment, attempt to count the read/write + # attachments. + if len(attachments) > 1: + count = 0 + for attachment in attachments.values(): + attachment_id = attachment['attachment_id'] + # Get the attachment record for this attachment so we can + # get the attach_mode. + # TODO(mriedem): This could be optimized if we had + # GET /attachments/detail?volume_id=volume['id'] in Cinder. + try: + attachment_record = self.volume_api.attachment_get( + ctxt, attachment_id) + # Note that the attachment record from Cinder has + # attach_mode in the top-level of the resource but the + # nova.volume.cinder code translates it and puts the + # attach_mode in the connection_info for some legacy + # reason... + if attachment_record.get( + 'connection_info', {}).get( + # attachments are read/write by default + 'attach_mode', 'rw') == 'rw': + count += 1 + except exception.VolumeAttachmentNotFound: + # attachments are read/write by default so count it + count += 1 + else: + count = len(attachments) + + return count + @check_instance_lock @check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.PAUSED, vm_states.RESIZED]) @@ -4179,6 +4263,20 @@ except exception.InvalidInput as exc: raise exception.InvalidVolume(reason=exc.format_message()) + # Disallow swapping from multiattach volumes that have more than one + # read/write attachment. We know the old_volume has at least one + # attachment since it's attached to this server. The new_volume + # can't have any attachments because of the attach_status check above. + # We do this count after calling "begin_detaching" to lock against + # concurrent attachments being made while we're counting. + try: + if self._count_attachments_for_swap(context, old_volume) > 1: + raise exception.MultiattachSwapVolumeNotSupported() + except Exception: # This is generic to handle failures while counting + # We need to reset the detaching status before raising. + with excutils.save_and_reraise_exception(): + self.volume_api.roll_detaching(context, old_volume['id']) + # Get the BDM for the attached (old) volume so we can tell if it was # attached with the new-style Cinder 3.44 API. bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( diff -Nru nova-17.0.10/nova/compute/manager.py nova-17.0.11/nova/compute/manager.py --- nova-17.0.10/nova/compute/manager.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/compute/manager.py 2019-07-10 21:45:12.000000000 +0000 @@ -28,6 +28,7 @@ import base64 import binascii import contextlib +import copy import functools import inspect import sys @@ -3696,6 +3697,7 @@ @wrap_exception() @wrap_instance_event(prefix='compute') + @errors_out_migration @wrap_instance_fault def confirm_resize(self, context, instance, migration): """Confirms a migration/resize and deletes the 'old' instance. @@ -3745,10 +3747,68 @@ instance=instance) return - self._confirm_resize(context, instance, migration=migration) + with self._error_out_instance_on_exception(context, instance): + old_instance_type = instance.old_flavor + try: + self._confirm_resize( + context, instance, migration=migration) + except Exception: + # Something failed when cleaning up the source host so + # log a traceback and leave a hint about hard rebooting + # the server to correct its state in the DB. + with excutils.save_and_reraise_exception(logger=LOG): + LOG.exception( + 'Confirm resize failed on source host %s. ' + 'Resource allocations in the placement service ' + 'will be removed regardless because the instance ' + 'is now on the destination host %s. You can try ' + 'hard rebooting the instance to correct its ' + 'state.', self.host, migration.dest_compute, + instance=instance) + finally: + # Whether an error occurred or not, at this point the + # instance is on the dest host so to avoid leaking + # allocations in placement, delete them here. + # NOTE(mriedem): _delete_allocation_after_move is tightly + # coupled to the migration status on the confirm step so + # we unfortunately have to mutate the migration status to + # have _delete_allocation_after_move cleanup the allocation + # held by the migration consumer. + with utils.temporary_mutation( + migration, status='confirmed'): + self._delete_allocation_after_move( + context, instance, migration, old_instance_type, + migration.source_node) do_confirm_resize(context, instance, migration.id) + def _get_updated_nw_info_with_pci_mapping(self, nw_info, pci_mapping): + # NOTE(adrianc): This method returns a copy of nw_info if modifications + # are made else it returns the original nw_info. + updated_nw_info = nw_info + if nw_info and pci_mapping: + updated_nw_info = copy.deepcopy(nw_info) + for vif in updated_nw_info: + if vif['vnic_type'] in network_model.VNIC_TYPES_SRIOV: + try: + vif_pci_addr = vif['profile']['pci_slot'] + new_addr = pci_mapping[vif_pci_addr].address + vif['profile']['pci_slot'] = new_addr + LOG.debug("Updating VIF's PCI address for VIF %(id)s. " + "Original value %(orig_val)s, " + "new value %(new_val)s", + {'id': vif['id'], + 'orig_val': vif_pci_addr, + 'new_val': new_addr}) + except (KeyError, AttributeError): + with excutils.save_and_reraise_exception(): + # NOTE(adrianc): This should never happen. If we + # get here it means there is some inconsistency + # with either 'nw_info' or 'pci_mapping'. + LOG.error("Unexpected error when updating network " + "information with PCI mapping.") + return updated_nw_info + def _confirm_resize(self, context, instance, migration=None): """Destroys the source instance.""" self._notify_about_instance_usage(context, instance, @@ -3757,63 +3817,66 @@ self.host, action=fields.NotificationAction.RESIZE_CONFIRM, phase=fields.NotificationPhase.START) - with self._error_out_instance_on_exception(context, instance): - # NOTE(danms): delete stashed migration information - old_instance_type = instance.old_flavor - instance.old_flavor = None - instance.new_flavor = None - instance.system_metadata.pop('old_vm_state', None) - instance.save() - - # NOTE(tr3buchet): tear down networks on source host - self.network_api.setup_networks_on_host(context, instance, - migration.source_compute, teardown=True) + # NOTE(danms): delete stashed migration information + old_instance_type = instance.old_flavor + instance.old_flavor = None + instance.new_flavor = None + instance.system_metadata.pop('old_vm_state', None) + instance.save() - network_info = self.network_api.get_instance_nw_info(context, - instance) - # TODO(mriedem): Get BDMs here and pass them to the driver. - self.driver.confirm_migration(context, migration, instance, - network_info) + # NOTE(tr3buchet): tear down networks on source host + self.network_api.setup_networks_on_host(context, instance, + migration.source_compute, teardown=True) + network_info = self.network_api.get_instance_nw_info(context, + instance) + + # NOTE(adrianc): Populate old PCI device in VIF profile + # to allow virt driver to properly unplug it from Hypervisor. + pci_mapping = (instance.migration_context. + get_pci_mapping_for_migration(True)) + network_info = self._get_updated_nw_info_with_pci_mapping( + network_info, pci_mapping) - migration.status = 'confirmed' - with migration.obj_as_admin(): - migration.save() + # TODO(mriedem): Get BDMs here and pass them to the driver. + self.driver.confirm_migration(context, migration, instance, + network_info) - rt = self._get_resource_tracker() - rt.drop_move_claim(context, instance, migration.source_node, - old_instance_type, prefix='old_') - self._delete_allocation_after_move(context, instance, migration, - old_instance_type, - migration.source_node) - instance.drop_migration_context() + migration.status = 'confirmed' + with migration.obj_as_admin(): + migration.save() - # NOTE(mriedem): The old_vm_state could be STOPPED but the user - # might have manually powered up the instance to confirm the - # resize/migrate, so we need to check the current power state - # on the instance and set the vm_state appropriately. We default - # to ACTIVE because if the power state is not SHUTDOWN, we - # assume _sync_instance_power_state will clean it up. - p_state = instance.power_state - vm_state = None - if p_state == power_state.SHUTDOWN: - vm_state = vm_states.STOPPED - LOG.debug("Resized/migrated instance is powered off. " - "Setting vm_state to '%s'.", vm_state, - instance=instance) - else: - vm_state = vm_states.ACTIVE + rt = self._get_resource_tracker() + rt.drop_move_claim(context, instance, migration.source_node, + old_instance_type, prefix='old_') + instance.drop_migration_context() + + # NOTE(mriedem): The old_vm_state could be STOPPED but the user + # might have manually powered up the instance to confirm the + # resize/migrate, so we need to check the current power state + # on the instance and set the vm_state appropriately. We default + # to ACTIVE because if the power state is not SHUTDOWN, we + # assume _sync_instance_power_state will clean it up. + p_state = instance.power_state + vm_state = None + if p_state == power_state.SHUTDOWN: + vm_state = vm_states.STOPPED + LOG.debug("Resized/migrated instance is powered off. " + "Setting vm_state to '%s'.", vm_state, + instance=instance) + else: + vm_state = vm_states.ACTIVE - instance.vm_state = vm_state - instance.task_state = None - instance.save(expected_task_state=[None, task_states.DELETING, - task_states.SOFT_DELETING]) + instance.vm_state = vm_state + instance.task_state = None + instance.save(expected_task_state=[None, task_states.DELETING, + task_states.SOFT_DELETING]) - self._notify_about_instance_usage( - context, instance, "resize.confirm.end", - network_info=network_info) - compute_utils.notify_about_instance_action(context, instance, - self.host, action=fields.NotificationAction.RESIZE_CONFIRM, - phase=fields.NotificationPhase.END) + self._notify_about_instance_usage( + context, instance, "resize.confirm.end", + network_info=network_info) + compute_utils.notify_about_instance_action(context, instance, + self.host, action=fields.NotificationAction.RESIZE_CONFIRM, + phase=fields.NotificationPhase.END) def _delete_allocation_after_move(self, context, instance, migration, flavor, nodename): @@ -5344,8 +5407,16 @@ 'mountpoint': bdm['mount_device']}, instance=instance) if bdm['attachment_id']: - self.volume_api.attachment_delete(context, - bdm['attachment_id']) + # Try to delete the attachment to make the volume + # available again. Note that DriverVolumeBlockDevice + # may have already deleted the attachment so ignore + # VolumeAttachmentNotFound. + try: + self.volume_api.attachment_delete( + context, bdm['attachment_id']) + except exception.VolumeAttachmentNotFound as exc: + LOG.debug('Ignoring VolumeAttachmentNotFound: %s', + exc, instance=instance) else: self.volume_api.unreserve_volume(context, bdm.volume_id) compute_utils.notify_about_volume_attach_detach( @@ -5669,9 +5740,9 @@ # new style attachments (v3.44). Once we drop support for old style # attachments we could think about cleaning up the cinder-initiated # swap volume API flows. - is_cinder_migration = ( - True if old_volume['status'] in ('retyping', - 'migrating') else False) + is_cinder_migration = False + if 'migration_status' in old_volume: + is_cinder_migration = old_volume['migration_status'] == 'migrating' old_vol_size = old_volume['size'] new_volume = self.volume_api.get(context, new_volume_id) new_vol_size = new_volume['size'] @@ -6084,7 +6155,7 @@ return [] def _cleanup_pre_live_migration(self, context, dest, instance, - migration, migrate_data): + migration, migrate_data, source_bdms): """Helper method for when pre_live_migration fails Sets the migration status to "error" and rolls back the live migration @@ -6101,13 +6172,18 @@ :param migrate_data: Data about the live migration, populated from the destination host. :type migrate_data: Subclass of nova.objects.LiveMigrateData + :param source_bdms: BDMs prior to modification by the destination + compute host. Set by _do_live_migration and not + part of the callback interface, so this is never + None """ self._set_migration_status(migration, 'error') # Make sure we set this for _rollback_live_migration() # so it can find it, as expected if it was called later migrate_data.migration = migration self._rollback_live_migration(context, instance, dest, - migrate_data) + migrate_data=migrate_data, + source_bdms=source_bdms) def _do_live_migration(self, context, dest, instance, block_migration, migration, migrate_data): @@ -6147,20 +6223,23 @@ 'to be plugged on the destination host %s.', dest, instance=instance) self._cleanup_pre_live_migration( - context, dest, instance, migration, migrate_data) + context, dest, instance, migration, migrate_data, + source_bdms) except eventlet.timeout.Timeout: msg = 'Timed out waiting for events: %s' LOG.warning(msg, events, instance=instance) if CONF.vif_plugging_is_fatal: self._cleanup_pre_live_migration( - context, dest, instance, migration, migrate_data) + context, dest, instance, migration, migrate_data, + source_bdms) raise exception.MigrationError(reason=msg % events) except Exception: with excutils.save_and_reraise_exception(): LOG.exception('Pre live migration failed at %s', dest, instance=instance) self._cleanup_pre_live_migration( - context, dest, instance, migration, migrate_data) + context, dest, instance, migration, migrate_data, + source_bdms) self._set_migration_status(migration, 'running') @@ -6174,12 +6253,14 @@ # cleanup. post_live_migration = functools.partial(self._post_live_migration, source_bdms=source_bdms) + rollback_live_migration = functools.partial( + self._rollback_live_migration, source_bdms=source_bdms) LOG.debug('live_migration data is %s', migrate_data) try: self.driver.live_migration(context, instance, dest, post_live_migration, - self._rollback_live_migration, + rollback_live_migration, block_migration, migrate_data) except Exception: LOG.exception('Live migration failed.', instance=instance) @@ -6568,7 +6649,8 @@ @wrap_instance_fault def _rollback_live_migration(self, context, instance, dest, migrate_data=None, - migration_status='error'): + migration_status='error', + source_bdms=None): """Recovers Instance/volume state from migrating -> running. :param context: security context @@ -6580,6 +6662,10 @@ if not none, contains implementation specific data. :param migration_status: Contains the status we want to set for the migration object + :param source_bdms: BDMs prior to modification by the destination + compute host. Set by _do_live_migration and not + part of the callback interface, so this is never + None """ if (isinstance(migrate_data, migrate_data_obj.LiveMigrateData) and @@ -6605,11 +6691,19 @@ # NOTE(tr3buchet): setup networks on source host (really it's re-setup) self.network_api.setup_networks_on_host(context, instance, self.host) + source_bdms_by_volid = {bdm.volume_id: bdm for bdm in source_bdms + if bdm.is_volume} + + # NOTE(lyarwood): Fetch the current list of BDMs and delete any volume + # attachments used by the destination host before rolling back to the + # original and still valid source host volume attachments. bdms = objects.BlockDeviceMappingList.get_by_instance_uuid( context, instance.uuid) for bdm in bdms: if bdm.is_volume: # remove the connection on the destination host + # NOTE(lyarwood): This actually calls the cinderv2 + # os-terminate_connection API if required. self.compute_rpcapi.remove_volume_connection( context, instance, bdm.volume_id, dest) @@ -6618,13 +6712,21 @@ # attachment_id to the old attachment of the source # host. If old_attachments is not there, then # there was an error before the new attachment was made. + # TODO(lyarwood): migrate_data.old_vol_attachment_ids can + # be removed now as we can lookup the original + # attachment_ids from the source_bdms list here. old_attachments = migrate_data.old_vol_attachment_ids \ if 'old_vol_attachment_ids' in migrate_data else None if old_attachments and bdm.volume_id in old_attachments: self.volume_api.attachment_delete(context, bdm.attachment_id) bdm.attachment_id = old_attachments[bdm.volume_id] - bdm.save() + + # NOTE(lyarwood): Rollback the connection_info stored within + # the BDM to that used by the source and not the destination. + source_bdm = source_bdms_by_volid[bdm.volume_id] + bdm.connection_info = source_bdm.connection_info + bdm.save() self._notify_about_instance_usage(context, instance, "live_migration._rollback.start") @@ -6641,6 +6743,18 @@ self.compute_rpcapi.rollback_live_migration_at_destination( context, instance, dest, destroy_disks=destroy_disks, migrate_data=migrate_data) + elif utils.is_neutron(): + # The port binding profiles need to be cleaned up. + with errors_out_migration_ctxt(migration): + try: + self.network_api.setup_networks_on_host( + context, instance, teardown=True) + except Exception: + with excutils.save_and_reraise_exception(): + LOG.exception( + 'An error occurred while cleaning up networking ' + 'during live migration rollback.', + instance=instance) self._notify_about_instance_usage(context, instance, "live_migration._rollback.end") diff -Nru nova-17.0.10/nova/compute/resource_tracker.py nova-17.0.11/nova/compute/resource_tracker.py --- nova-17.0.10/nova/compute/resource_tracker.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/compute/resource_tracker.py 2019-07-10 21:45:12.000000000 +0000 @@ -558,7 +558,6 @@ cn = self.compute_nodes[nodename] self._copy_resources(cn, resources) self._setup_pci_tracker(context, cn, resources) - self._update(context, cn) return # now try to get the compute node record from the @@ -568,7 +567,6 @@ self.compute_nodes[nodename] = cn self._copy_resources(cn, resources) self._setup_pci_tracker(context, cn, resources) - self._update(context, cn) return if self._check_for_nodes_rebalance(context, resources, nodename): @@ -587,7 +585,6 @@ {'host': self.host, 'node': nodename, 'uuid': cn.uuid}) self._setup_pci_tracker(context, cn, resources) - self._update(context, cn) def _setup_pci_tracker(self, context, compute_node, resources): if not self.pci_tracker: @@ -616,10 +613,31 @@ stats.digest_stats(resources.get('stats')) compute_node.stats = stats - # update the allocation ratios for the related ComputeNode object - compute_node.ram_allocation_ratio = self.ram_allocation_ratio - compute_node.cpu_allocation_ratio = self.cpu_allocation_ratio - compute_node.disk_allocation_ratio = self.disk_allocation_ratio + # Update the allocation ratios for the related ComputeNode object + # but only if the configured values are not the default 0.0; the + # ComputeNode._from_db_object method takes care of providing default + # allocation ratios when the config is left at the 0.0 default, so + # we'll really end up with something like a + # ComputeNode.cpu_allocation_ratio of 16.0, not 0.0. We want to avoid + # resetting the ComputeNode fields to 0.0 because that will make + # the _resource_change method think something changed when really it + # didn't. + # TODO(mriedem): Will this break any scenarios where an operator is + # trying to *reset* the allocation ratios by changing config from + # non-0.0 back to 0.0? Maybe we should only do this if the fields on + # the ComputeNode object are not already set. For example, let's say + # the cpu_allocation_ratio config was 1.0 and then the operator wants + # to get back to the default (16.0 via the facade), and to do that they + # change the config back to 0.0 (or just unset the config option). + # Should we support that or treat these config options as "sticky" in + # that once you start setting them, you can't go back to the implied + # defaults by unsetting or resetting to 0.0? Sort of like how + # per-tenant quota is sticky once you change it in the API. + for res in ('cpu', 'disk', 'ram'): + attr = '%s_allocation_ratio' % res + conf_alloc_ratio = getattr(self, attr) + if conf_alloc_ratio != 0.0: + setattr(compute_node, attr, conf_alloc_ratio) # now copy rest to compute_node compute_node.update_from_virt_driver(resources) diff -Nru nova-17.0.10/nova/conductor/manager.py nova-17.0.11/nova/conductor/manager.py --- nova-17.0.10/nova/conductor/manager.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/conductor/manager.py 2019-07-10 21:45:12.000000000 +0000 @@ -965,8 +965,7 @@ elif recreate: # NOTE(sbauza): Augment the RequestSpec object by excluding # the source host for avoiding the scheduler to pick it - request_spec.ignore_hosts = request_spec.ignore_hosts or [] - request_spec.ignore_hosts.append(instance.host) + request_spec.ignore_hosts = [instance.host] # NOTE(sbauza): Force_hosts/nodes needs to be reset # if we want to make sure that the next destination # is not forced to be the original host @@ -1318,13 +1317,6 @@ 'build_instances', updates, exc, request_spec) - # TODO(mnaser): The cell mapping should already be populated by - # this point to avoid setting it below here. - inst_mapping = objects.InstanceMapping.get_by_instance_uuid( - context, instance.uuid) - inst_mapping.cell_mapping = cell - inst_mapping.save() - # In order to properly clean-up volumes when deleting a server in # ERROR status with no host, we need to store BDMs in the same # cell. @@ -1340,6 +1332,18 @@ with nova_context.target_cell(context, cell) as cctxt: self._create_tags(cctxt, instance.uuid, tags) + # NOTE(mdbooth): To avoid an incomplete instance record being + # returned by the API, the instance mapping must be + # created after the instance record is complete in + # the cell, and before the build request is + # destroyed. + # TODO(mnaser): The cell mapping should already be populated by + # this point to avoid setting it below here. + inst_mapping = objects.InstanceMapping.get_by_instance_uuid( + context, instance.uuid) + inst_mapping.cell_mapping = cell + inst_mapping.save() + # Be paranoid about artifacts being deleted underneath us. try: build_request.destroy() diff -Nru nova-17.0.10/nova/conductor/tasks/live_migrate.py nova-17.0.11/nova/conductor/tasks/live_migrate.py --- nova-17.0.10/nova/conductor/tasks/live_migrate.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/conductor/tasks/live_migrate.py 2019-07-10 21:45:12.000000000 +0000 @@ -14,6 +14,7 @@ import oslo_messaging as messaging import six +from nova import availability_zones from nova.compute import power_state from nova.conductor.tasks import base from nova.conductor.tasks import migrate @@ -21,6 +22,7 @@ from nova import exception from nova.i18n import _ from nova import objects +from nova.objects import fields as obj_fields from nova.scheduler import utils as scheduler_utils from nova import utils @@ -55,6 +57,7 @@ def _execute(self): self._check_instance_is_active() + self._check_instance_has_no_numa() self._check_host_is_up(self.source) if should_do_migration_allocation(self.context): @@ -99,6 +102,10 @@ # node name off it to set in the Migration object below. dest_node = dest_node.hypervisor_hostname + self.instance.availability_zone = ( + availability_zones.get_host_availability_zone( + self.context, self.destination)) + self.migration.source_node = self.instance.node self.migration.dest_node = dest_node self.migration.dest_compute = self.destination @@ -136,6 +143,33 @@ state=self.instance.power_state, method='live migrate') + def _check_instance_has_no_numa(self): + """Prevent live migrations of instances with NUMA topologies.""" + if not self.instance.numa_topology: + return + + # Only KVM (libvirt) supports NUMA topologies with CPU pinning; + # HyperV's vNUMA feature doesn't allow specific pinning + hypervisor_type = objects.ComputeNode.get_by_host_and_nodename( + self.context, self.source, self.instance.node).hypervisor_type + + # KVM is not a hypervisor, so when using a virt_type of "kvm" the + # hypervisor_type will still be "QEMU". + if hypervisor_type.lower() != obj_fields.HVType.QEMU: + return + + msg = ('Instance has an associated NUMA topology. ' + 'Instance NUMA topologies, including related attributes ' + 'such as CPU pinning, huge page and emulator thread ' + 'pinning information, are not currently recalculated on ' + 'live migration. See bug #1289064 for more information.' + ) + + if CONF.workarounds.enable_numa_live_migration: + LOG.warning(msg, instance=self.instance) + else: + raise exception.MigrationPreCheckError(reason=msg) + def _check_host_is_up(self, host): service = objects.Service.get_by_compute_host(self.context, host) diff -Nru nova-17.0.10/nova/conf/compute.py nova-17.0.11/nova/conf/compute.py --- nova-17.0.10/nova/conf/compute.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/conf/compute.py 2019-07-10 21:45:12.000000000 +0000 @@ -415,7 +415,9 @@ NOTE: This can be set per-compute, or if set to 0.0, the value set on the scheduler node(s) or compute node(s) will be used -and defaulted to 16.0. +and defaulted to 16.0. Once set to a non-default value, it is not possible +to "unset" the config to get back to the default behavior. If you want +to reset back to the default, explicitly specify 16.0. NOTE: As of the 16.0.0 Pike release, this configuration option is ignored for the ironic.IronicDriver compute driver and is hardcoded to 1.0. @@ -442,7 +444,9 @@ NOTE: This can be set per-compute, or if set to 0.0, the value set on the scheduler node(s) or compute node(s) will be used and -defaulted to 1.5. +defaulted to 1.5. Once set to a non-default value, it is not possible +to "unset" the config to get back to the default behavior. If you want +to reset back to the default, explicitly specify 1.5. NOTE: As of the 16.0.0 Pike release, this configuration option is ignored for the ironic.IronicDriver compute driver and is hardcoded to 1.0. @@ -473,7 +477,9 @@ NOTE: This can be set per-compute, or if set to 0.0, the value set on the scheduler node(s) or compute node(s) will be used and -defaulted to 1.0. +defaulted to 1.0. Once set to a non-default value, it is not possible +to "unset" the config to get back to the default behavior. If you want +to reset back to the default, explicitly specify 1.0. NOTE: As of the 16.0.0 Pike release, this configuration option is ignored for the ironic.IronicDriver compute driver and is hardcoded to 1.0. diff -Nru nova-17.0.10/nova/conf/workarounds.py nova-17.0.11/nova/conf/workarounds.py --- nova-17.0.10/nova/conf/workarounds.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/conf/workarounds.py 2019-07-10 21:45:12.000000000 +0000 @@ -177,6 +177,32 @@ * ``[libvirt]/images_type`` (rbd) * ``instances_path`` """), + + cfg.BoolOpt( + 'enable_numa_live_migration', + default=False, + help=""" +Enable live migration of instances with NUMA topologies. + +Live migration of instances with NUMA topologies is disabled by default +when using the libvirt driver. This includes live migration of instances with +CPU pinning or hugepages. CPU pinning and huge page information for such +instances is not currently re-calculated, as noted in bug #1289064. This +means that if instances were already present on the destination host, the +migrated instance could be placed on the same dedicated cores as these +instances or use hugepages allocated for another instance. Alternately, if the +host platforms were not homogeneous, the instance could be assigned to +non-existent cores or be inadvertently split across host NUMA nodes. + +Despite these known issues, there may be cases where live migration is +necessary. By enabling this option, operators that are aware of the issues and +are willing to manually work around them can enable live migration support for +these instances. + +Related options: + +* ``compute_driver``: Only the libvirt driver is affected. +"""), ] diff -Nru nova-17.0.10/nova/db/sqlalchemy/api.py nova-17.0.11/nova/db/sqlalchemy/api.py --- nova-17.0.10/nova/db/sqlalchemy/api.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/db/sqlalchemy/api.py 2019-07-10 21:45:12.000000000 +0000 @@ -1854,6 +1854,8 @@ model_query(context, models.Migration).\ filter_by(instance_uuid=instance_uuid).\ soft_delete() + model_query(context, models.InstanceIdMapping).filter_by( + uuid=instance_uuid).soft_delete() # NOTE(snikitin): We can't use model_query here, because there is no # column 'deleted' in 'tags' or 'console_auth_tokens' tables. context.session.query(models.Tag).filter_by( @@ -2773,6 +2775,11 @@ if not uuidutils.is_uuid_like(instance_uuid): raise exception.InvalidUUID(instance_uuid) + # NOTE(mdbooth): We pop values from this dict below, so we copy it here to + # ensure there are no side effects for the caller or if we retry the + # function due to a db conflict. + updates = copy.copy(values) + if expected is None: expected = {} else: @@ -2784,8 +2791,8 @@ # updates for field in ('task_state', 'vm_state'): expected_field = 'expected_%s' % field - if expected_field in values: - value = values.pop(expected_field, None) + if expected_field in updates: + value = updates.pop(expected_field, None) # Coerce all single values to singleton lists if value is None: expected[field] = [None] @@ -2793,23 +2800,23 @@ expected[field] = sqlalchemyutils.to_list(value) # Values which need to be updated separately - metadata = values.pop('metadata', None) - system_metadata = values.pop('system_metadata', None) + metadata = updates.pop('metadata', None) + system_metadata = updates.pop('system_metadata', None) - _handle_objects_related_type_conversions(values) + _handle_objects_related_type_conversions(updates) # Hostname is potentially unique, but this is enforced in code rather # than the DB. The query below races, but the number of users of # osapi_compute_unique_server_name_scope is small, and a robust fix # will be complex. This is intentionally left as is for the moment. - if 'hostname' in values: - _validate_unique_server_name(context, values['hostname']) + if 'hostname' in updates: + _validate_unique_server_name(context, updates['hostname']) compare = models.Instance(uuid=instance_uuid, **expected) try: instance_ref = model_query(context, models.Instance, project_only=True).\ - update_on_match(compare, 'uuid', values) + update_on_match(compare, 'uuid', updates) except update_match.NoRowsMatched: # Update failed. Try to find why and raise a specific error. diff -Nru nova-17.0.10/nova/exception.py nova-17.0.11/nova/exception.py --- nova-17.0.10/nova/exception.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/exception.py 2019-07-10 21:45:12.000000000 +0000 @@ -283,6 +283,11 @@ "shelved-offloaded instances.") +class MultiattachSwapVolumeNotSupported(Invalid): + msg_fmt = _('Swapping multi-attach volumes with more than one read/write ' + 'attachment is not supported.') + + class VolumeNotCreated(NovaException): msg_fmt = _("Volume %(volume_id)s did not finish being created" " even after we waited %(seconds)s seconds or %(attempts)s" diff -Nru nova-17.0.10/nova/image/glance.py nova-17.0.11/nova/image/glance.py --- nova-17.0.10/nova/image/glance.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/image/glance.py 2019-07-10 21:45:12.000000000 +0000 @@ -160,21 +160,35 @@ self.api_server = next(self.api_servers) return _glanceclient_from_endpoint(context, self.api_server, version) - def call(self, context, version, method, *args, **kwargs): + def call(self, context, version, method, controller=None, args=None, + kwargs=None): """Call a glance client method. If we get a connection error, retry the request according to CONF.glance.num_retries. + + :param context: RequestContext to use + :param version: Numeric version of the *Glance API* to use + :param method: string method name to execute on the glanceclient + :param controller: optional string name of the client controller to + use. Default (None) is to use the 'images' + controller + :param args: optional iterable of arguments to pass to the + glanceclient method, splatted as positional args + :param kwargs: optional dict of arguments to pass to the glanceclient, + splatted into named arguments """ + args = args or [] + kwargs = kwargs or {} retry_excs = (glanceclient.exc.ServiceUnavailable, glanceclient.exc.InvalidEndpoint, glanceclient.exc.CommunicationError) num_attempts = 1 + CONF.glance.num_retries + controller_name = controller or 'images' for attempt in range(1, num_attempts + 1): client = self.client or self._create_onetime_client(context, version) try: - controller = getattr(client, - kwargs.pop('controller', 'images')) + controller = getattr(client, controller_name) result = getattr(controller, method)(*args, **kwargs) if inspect.isgenerator(result): # Convert generator results to a list, so that we can @@ -238,7 +252,7 @@ image is deleted. """ try: - image = self._client.call(context, 2, 'get', image_id) + image = self._client.call(context, 2, 'get', args=(image_id,)) except Exception: _reraise_translated_image_exception(image_id) @@ -273,7 +287,7 @@ """Calls out to Glance for a list of detailed image information.""" params = _extract_query_params_v2(kwargs) try: - images = self._client.call(context, 2, 'list', **params) + images = self._client.call(context, 2, 'list', kwargs=params) except Exception: _reraise_translated_exception() @@ -318,7 +332,8 @@ LOG.exception("Download image error") try: - image_chunks = self._client.call(context, 2, 'data', image_id) + image_chunks = self._client.call( + context, 2, 'data', args=(image_id,)) except Exception: _reraise_translated_image_exception(image_id) @@ -430,14 +445,14 @@ def _add_location(self, context, image_id, location): # 'show_multiple_locations' must be enabled in glance api conf file. try: - return self._client.call(context, 2, 'add_location', image_id, - location, {}) + return self._client.call( + context, 2, 'add_location', args=(image_id, location, {})) except glanceclient.exc.HTTPBadRequest: _reraise_translated_exception() def _upload_data(self, context, image_id, data): - self._client.call(context, 2, 'upload', image_id, data) - return self._client.call(context, 2, 'get', image_id) + self._client.call(context, 2, 'upload', args=(image_id, data)) + return self._client.call(context, 2, 'get', args=(image_id,)) def _get_image_create_disk_format_default(self, context): """Gets an acceptable default image disk_format based on the schema. @@ -457,8 +472,8 @@ # Get the image schema - note we don't cache this value since it could # change under us. This looks a bit funky, but what it's basically # doing is calling glanceclient.v2.Client.schemas.get('image'). - image_schema = self._client.call(context, 2, 'get', 'image', - controller='schemas') + image_schema = self._client.call( + context, 2, 'get', args=('image',), controller='schemas') # get the disk_format schema property from the raw schema disk_format_schema = ( image_schema.raw()['properties'].get('disk_format') if image_schema @@ -498,7 +513,7 @@ location = sent_service_image_meta.pop('location', None) image = self._client.call( - context, 2, 'create', **sent_service_image_meta) + context, 2, 'create', kwargs=sent_service_image_meta) image_id = image['id'] # Sending image location in a separate request. @@ -542,7 +557,7 @@ location = sent_service_image_meta.pop('location', None) image_id = sent_service_image_meta['image_id'] image = self._client.call( - context, 2, 'update', **sent_service_image_meta) + context, 2, 'update', kwargs=sent_service_image_meta) # Sending image location in a separate request. if location: @@ -565,7 +580,7 @@ """ try: - self._client.call(context, 2, 'delete', image_id) + self._client.call(context, 2, 'delete', args=(image_id,)) except glanceclient.exc.NotFound: raise exception.ImageNotFound(image_id=image_id) except glanceclient.exc.HTTPForbidden: diff -Nru nova-17.0.10/nova/network/neutronv2/api.py nova-17.0.11/nova/network/neutronv2/api.py --- nova-17.0.10/nova/network/neutronv2/api.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/network/neutronv2/api.py 2019-07-10 21:45:12.000000000 +0000 @@ -2647,45 +2647,14 @@ # device_id field on the port which is not what we'd want for shelve. pass - def _get_pci_devices_from_migration_context(self, migration_context, - migration): - if migration and migration.get('status') == 'reverted': - # In case of revert, swap old and new devices to - # update the ports back to the original devices. - return (migration_context.new_pci_devices, - migration_context.old_pci_devices) - return (migration_context.old_pci_devices, - migration_context.new_pci_devices) - - def _get_pci_mapping_for_migration(self, context, instance, migration): - """Get the mapping between the old PCI devices and the new PCI - devices that have been allocated during this migration. The - correlation is based on PCI request ID which is unique per PCI - devices for SR-IOV ports. - - :param context: The request context. - :param instance: Get PCI mapping for this instance. - :param migration: The migration for this instance. - :Returns: dictionary of mapping {'': } - """ - migration_context = instance.migration_context - if not migration_context: + def _get_pci_mapping_for_migration(self, instance, migration): + if not instance.migration_context: return {} - - old_pci_devices, new_pci_devices = \ - self._get_pci_devices_from_migration_context(migration_context, - migration) - if old_pci_devices and new_pci_devices: - LOG.debug("Determining PCI devices mapping using migration" - "context: old_pci_devices: %(old)s, " - "new_pci_devices: %(new)s", - {'old': [dev for dev in old_pci_devices], - 'new': [dev for dev in new_pci_devices]}) - return {old.address: new - for old in old_pci_devices - for new in new_pci_devices - if old.request_id == new.request_id} - return {} + # In case of revert, swap old and new devices to + # update the ports back to the original devices. + revert = (migration and + migration.get('status') == 'reverted') + return instance.migration_context.get_pci_mapping_for_migration(revert) def _update_port_binding_for_instance(self, context, instance, host, migration=None): @@ -2724,7 +2693,7 @@ if (vnic_type in network_model.VNIC_TYPES_SRIOV and migration is not None): if not pci_mapping: - pci_mapping = self._get_pci_mapping_for_migration(context, + pci_mapping = self._get_pci_mapping_for_migration( instance, migration) pci_slot = binding_profile.get('pci_slot') diff -Nru nova-17.0.10/nova/objects/migration_context.py nova-17.0.11/nova/objects/migration_context.py --- nova-17.0.10/nova/objects/migration_context.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/objects/migration_context.py 2019-07-10 21:45:12.000000000 +0000 @@ -12,6 +12,7 @@ # License for the specific language governing permissions and limitations # under the License. +from oslo_log import log as logging from oslo_serialization import jsonutils from oslo_utils import versionutils @@ -20,6 +21,8 @@ from nova.objects import base from nova.objects import fields +LOG = logging.getLogger(__name__) + @base.NovaObjectRegistry.register class MigrationContext(base.NovaPersistentObject, base.NovaObject): @@ -80,3 +83,32 @@ return None return cls.obj_from_db_obj(db_extra['migration_context']) + + def get_pci_mapping_for_migration(self, revert): + """Get the mapping between the old PCI devices and the new PCI + devices that have been allocated during this migration. The + correlation is based on PCI request ID which is unique per PCI + devices for SR-IOV ports. + + :param revert: If True, return a reverse mapping i.e + mapping between new PCI devices and old PCI devices. + :returns: dictionary of PCI mapping. + if revert==False: + {'': } + if revert==True: + {'': } + """ + step = -1 if revert else 1 + current_pci_devs, updated_pci_devs = (self.old_pci_devices, + self.new_pci_devices)[::step] + if current_pci_devs and updated_pci_devs: + LOG.debug("Determining PCI devices mapping using migration " + "context: current_pci_devs: %(cur)s, " + "updated_pci_devs: %(upd)s", + {'cur': [dev for dev in current_pci_devs], + 'upd': [dev for dev in updated_pci_devs]}) + return {curr_dev.address: upd_dev + for curr_dev in current_pci_devs + for upd_dev in updated_pci_devs + if curr_dev.request_id == upd_dev.request_id} + return {} diff -Nru nova-17.0.10/nova/objects/request_spec.py nova-17.0.11/nova/objects/request_spec.py --- nova-17.0.10/nova/objects/request_spec.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/objects/request_spec.py 2019-07-10 21:45:12.000000000 +0000 @@ -205,6 +205,8 @@ policies = list(filter_properties.get('group_policies')) hosts = list(filter_properties.get('group_hosts')) members = list(filter_properties.get('group_members')) + # TODO(mriedem): We could try to get the group uuid from the + # group hint in the filter_properties. self.instance_group = objects.InstanceGroup(policies=policies, hosts=hosts, members=members) @@ -450,11 +452,23 @@ # though they should match. if key in ['id', 'instance_uuid']: setattr(spec, key, db_spec[key]) - else: + elif key == 'ignore_hosts': + # NOTE(mriedem): Do not override the 'ignore_hosts' + # field which is not persisted. It is not a lazy-loadable + # field. If it is not set, set None. + if not spec.obj_attr_is_set(key): + setattr(spec, key, None) + elif key in spec_obj: setattr(spec, key, getattr(spec_obj, key)) spec._context = context if 'instance_group' in spec and spec.instance_group: + # NOTE(mriedem): We could have a half-baked instance group with no + # uuid if some legacy translation was performed on this spec in the + # past. In that case, try to workaround the issue by getting the + # group uuid from the scheduler hint. + if 'uuid' not in spec.instance_group: + spec.instance_group.uuid = spec.get_scheduler_hint('group') # NOTE(danms): We don't store the full instance group in # the reqspec since it would be stale almost immediately. # Instead, load it by uuid here so it's up-to-date. @@ -511,9 +525,9 @@ if 'instance_group' in spec and spec.instance_group: spec.instance_group.members = None spec.instance_group.hosts = None - # NOTE(mriedem): Don't persist retries or requested_destination - # since those are per-request - for excluded in ('retry', 'requested_destination'): + # NOTE(mriedem): Don't persist retries, requested_destination + # or ignored hosts since those are per-request + for excluded in ('retry', 'requested_destination', 'ignore_hosts'): if excluded in spec and getattr(spec, excluded): setattr(spec, excluded, None) diff -Nru nova-17.0.10/nova/tests/functional/db/test_compute_api.py nova-17.0.11/nova/tests/functional/db/test_compute_api.py --- nova-17.0.10/nova/tests/functional/db/test_compute_api.py 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/db/test_compute_api.py 2019-07-10 21:45:12.000000000 +0000 @@ -0,0 +1,58 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import mock + +from nova.compute import api as compute_api +from nova import context as nova_context +from nova import exception +from nova import objects +from nova import test +from nova.tests import fixtures as nova_fixtures +import nova.tests.uuidsentinel as uuids + + +class ComputeAPITestCase(test.NoDBTestCase): + USES_DB_SELF = True + + def setUp(self): + super(ComputeAPITestCase, self).setUp() + self.useFixture(nova_fixtures.Database(database='api')) + + @mock.patch('nova.objects.instance_mapping.InstanceMapping.create') + def test_reqspec_buildreq_instmapping_single_transaction(self, + mock_create): + # Simulate a DBError during an INSERT by raising an exception from the + # InstanceMapping.create method. + mock_create.side_effect = test.TestingException('oops') + + ctxt = nova_context.RequestContext('fake-user', 'fake-project') + rs = objects.RequestSpec(context=ctxt, instance_uuid=uuids.inst) + # project_id and instance cannot be None + br = objects.BuildRequest(context=ctxt, instance_uuid=uuids.inst, + project_id=ctxt.project_id, + instance=objects.Instance()) + im = objects.InstanceMapping(context=ctxt, instance_uuid=uuids.inst) + + self.assertRaises( + test.TestingException, + compute_api.API._create_reqspec_buildreq_instmapping, ctxt, rs, br, + im) + + # Since the instance mapping failed to INSERT, we should not have + # written a request spec record or a build request record. + self.assertRaises( + exception.RequestSpecNotFound, + objects.RequestSpec.get_by_instance_uuid, ctxt, uuids.inst) + self.assertRaises( + exception.BuildRequestNotFound, + objects.BuildRequest.get_by_instance_uuid, ctxt, uuids.inst) diff -Nru nova-17.0.10/nova/tests/functional/integrated_helpers.py nova-17.0.11/nova/tests/functional/integrated_helpers.py --- nova-17.0.10/nova/tests/functional/integrated_helpers.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/integrated_helpers.py 2019-07-10 21:45:12.000000000 +0000 @@ -270,10 +270,11 @@ return server def _wait_until_deleted(self, server): + initially_in_error = (server['status'] == 'ERROR') try: for i in range(40): server = self.api.get_server(server['id']) - if server['status'] == 'ERROR': + if not initially_in_error and server['status'] == 'ERROR': self.fail('Server went to error state instead of' 'disappearing.') time.sleep(0.5) diff -Nru nova-17.0.10/nova/tests/functional/regressions/test_bug_1669054.py nova-17.0.11/nova/tests/functional/regressions/test_bug_1669054.py --- nova-17.0.10/nova/tests/functional/regressions/test_bug_1669054.py 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/regressions/test_bug_1669054.py 2019-07-10 21:45:12.000000000 +0000 @@ -0,0 +1,84 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from nova import context +from nova import objects +from nova.tests.functional import integrated_helpers +from nova.tests.unit import fake_network +from nova.virt import fake + + +class ResizeEvacuateTestCase(integrated_helpers._IntegratedTestBase, + integrated_helpers.InstanceHelperMixin): + """Regression test for bug 1669054 introduced in Newton. + + When resizing a server, if CONF.allow_resize_to_same_host is False, + the API will set RequestSpec.ignore_hosts = [instance.host] and then + later in conductor the RequestSpec changes are saved to persist the new + flavor. This inadvertently saves the ignore_hosts value. Later if you + try to migrate, evacuate or unshelve the server, that original source + host will be ignored. If the deployment has a small number of computes, + like two in an edge node, then evacuate will fail because the only other + available host is ignored. This test recreates the scenario. + """ + # Set variables used in the parent class. + REQUIRES_LOCKING = False + ADMIN_API = True + USE_NEUTRON = True + _image_ref_parameter = 'imageRef' + _flavor_ref_parameter = 'flavorRef' + api_major_version = 'v2.1' + microversion = '2.11' # Need at least 2.11 for the force-down API + + def setUp(self): + super(ResizeEvacuateTestCase, self).setUp() + fake_network.set_stub_network_methods(self) + + def test_resize_then_evacuate(self): + # Create a server. At this point there is only one compute service. + flavors = self.api.get_flavors() + flavor1 = flavors[0]['id'] + server = self._build_server(flavor1) + server = self.api.post_server({'server': server}) + self._wait_for_state_change(self.api, server, 'ACTIVE') + + # Start up another compute service so we can resize. + fake.set_nodes(['host2']) + self.addCleanup(fake.restore_nodes) + host2 = self.start_service('compute', host='host2') + + # Now resize the server to move it to host2. + flavor2 = flavors[1]['id'] + req = {'resize': {'flavorRef': flavor2}} + self.api.post_server_action(server['id'], req) + server = self._wait_for_state_change(self.api, server, 'VERIFY_RESIZE') + self.assertEqual('host2', server['OS-EXT-SRV-ATTR:host']) + self.api.post_server_action(server['id'], {'confirmResize': None}) + server = self._wait_for_state_change(self.api, server, 'ACTIVE') + + # Disable the host on which the server is now running (host2). + host2.stop() + self.api.force_down_service('host2', 'nova-compute', forced_down=True) + # Now try to evacuate the server back to the original source compute. + req = {'evacuate': {'onSharedStorage': False}} + self.api.post_server_action(server['id'], req) + server = self._wait_for_state_change(self.api, server, 'ACTIVE') + # The evacuate flow in the compute manager is annoying in that it + # sets the instance status to ACTIVE before updating the host, so we + # have to wait for the migration record to be 'done' to avoid a race. + self._wait_for_migration_status(server, ['done']) + self.assertEqual(self.compute.host, server['OS-EXT-SRV-ATTR:host']) + + # Assert the RequestSpec.ignore_hosts field is not populated. + reqspec = objects.RequestSpec.get_by_instance_uuid( + context.get_admin_context(), server['id']) + self.assertIsNone(reqspec.ignore_hosts) diff -Nru nova-17.0.10/nova/tests/functional/regressions/test_bug_1830747.py nova-17.0.11/nova/tests/functional/regressions/test_bug_1830747.py --- nova-17.0.10/nova/tests/functional/regressions/test_bug_1830747.py 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/regressions/test_bug_1830747.py 2019-07-10 21:45:12.000000000 +0000 @@ -0,0 +1,131 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import mock + +from nova import context as nova_context +from nova import objects +from nova.scheduler import weights +from nova import test +from nova.tests import fixtures as nova_fixtures +from nova.tests.functional import integrated_helpers +from nova.tests.unit.image import fake as fake_image +from nova.virt import fake as fake_virt + + +class HostNameWeigher(weights.BaseHostWeigher): + # Prefer host1 over host2. + weights = {'host1': 100, 'host2': 50} + + def _weigh_object(self, host_state, weight_properties): + return self.weights.get(host_state.host, 0) + + +class MissingReqSpecInstanceGroupUUIDTestCase( + test.TestCase, integrated_helpers.InstanceHelperMixin): + """Regression recreate test for bug 1830747 + + Before change I4244f7dd8fe74565180f73684678027067b4506e in Stein, when + a cold migration would reschedule to conductor it would not send the + RequestSpec, only the filter_properties. The filter_properties contain + a primitive version of the instance group information from the RequestSpec + for things like the group members, hosts and policies, but not the uuid. + When conductor is trying to reschedule the cold migration without a + RequestSpec, it builds a RequestSpec from the components it has, like the + filter_properties. This results in a RequestSpec with an instance_group + field set but with no uuid field in the RequestSpec.instance_group. + That RequestSpec gets persisted and then because of change + Ie70c77db753711e1449e99534d3b83669871943f, later attempts to load the + RequestSpec from the database will fail because of the missing + RequestSpec.instance_group.uuid. + + This test recreates the regression scenario by cold migrating a server + to a host which fails and triggers a reschedule but without the RequestSpec + so a RequestSpec is created/updated for the instance without the + instance_group.uuid set which will lead to a failure loading the + RequestSpec from the DB later. + """ + + def setUp(self): + super(MissingReqSpecInstanceGroupUUIDTestCase, self).setUp() + # Stub out external dependencies. + self.useFixture(nova_fixtures.NeutronFixture(self)) + self.useFixture(nova_fixtures.PlacementFixture()) + fake_image.stub_out_image_service(self) + self.addCleanup(fake_image.FakeImageService_reset) + # Configure the API to allow resizing to the same host so we can keep + # the number of computes down to two in the test. + self.flags(allow_resize_to_same_host=True) + # Start nova controller services. + api_fixture = self.useFixture(nova_fixtures.OSAPIFixture( + api_version='v2.1')) + self.api = api_fixture.admin_api + self.start_service('conductor') + # Use our custom weigher defined above to make sure that we have + # a predictable scheduling sort order. + self.flags(weight_classes=[__name__ + '.HostNameWeigher'], + group='filter_scheduler') + self.start_service('scheduler') + # Start two computes, one where the server will be created and another + # where we'll cold migrate it. + self.addCleanup(fake_virt.restore_nodes) + self.computes = {} # keep track of the compute services per host name + for host in ('host1', 'host2'): + fake_virt.set_nodes([host]) + compute_service = self.start_service('compute', host=host) + self.computes[host] = compute_service + + def test_cold_migrate_reschedule(self): + # Create an anti-affinity group for the server. + body = { + 'server_group': { + 'name': 'test-group', + 'policies': ['anti-affinity'] + } + } + group_id = self.api.api_post( + '/os-server-groups', body).body['server_group']['id'] + + # Create a server in the group which should land on host1 due to our + # custom weigher. + networks = [{ + 'port': nova_fixtures.NeutronFixture.port_1['id'] + }] + server = self._build_minimal_create_server_request( + self.api, 'test_cold_migrate_reschedule', networks=networks) + body = dict(server=server) + body['os:scheduler_hints'] = {'group': group_id} + server = self.api.post_server(body) + server = self._wait_for_state_change(self.api, server, 'ACTIVE') + self.assertEqual('host1', server['OS-EXT-SRV-ATTR:host']) + + # Verify the group uuid is set in the request spec. + ctxt = nova_context.get_admin_context() + reqspec = objects.RequestSpec.get_by_instance_uuid(ctxt, server['id']) + self.assertEqual(group_id, reqspec.instance_group.uuid) + + # Now cold migrate the server. Because of allow_resize_to_same_host and + # the weigher, the scheduler will pick host1 first. The FakeDriver + # actually allows migrating to the same host so we need to stub that + # out so the compute will raise UnableToMigrateToSelf like when using + # the libvirt driver. + host1_driver = self.computes['host1'].driver + with mock.patch.dict(host1_driver.capabilities, + supports_migrate_to_same_host=False): + self.api.post_server_action(server['id'], {'migrate': None}) + server = self._wait_for_state_change( + self.api, server, 'VERIFY_RESIZE') + self.assertEqual('host2', server['OS-EXT-SRV-ATTR:host']) + + # The RequestSpec.instance_group.uuid should still be set. + reqspec = objects.RequestSpec.get_by_instance_uuid(ctxt, server['id']) + self.assertEqual(group_id, reqspec.instance_group.uuid) diff -Nru nova-17.0.10/nova/tests/functional/test_availability_zones.py nova-17.0.11/nova/tests/functional/test_availability_zones.py --- nova-17.0.10/nova/tests/functional/test_availability_zones.py 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/test_availability_zones.py 2019-07-10 21:45:12.000000000 +0000 @@ -0,0 +1,167 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from nova import context +from nova import objects +from nova import test +from nova.tests import fixtures as nova_fixtures +from nova.tests.functional import integrated_helpers +from nova.tests.unit.image import fake as fake_image +from nova.tests.unit import policy_fixture +from nova.virt import fake + + +class TestAvailabilityZoneScheduling( + test.TestCase, integrated_helpers.InstanceHelperMixin): + + def setUp(self): + super(TestAvailabilityZoneScheduling, self).setUp() + + self.useFixture(policy_fixture.RealPolicyFixture()) + self.useFixture(nova_fixtures.NeutronFixture(self)) + self.useFixture(nova_fixtures.PlacementFixture()) + + api_fixture = self.useFixture(nova_fixtures.OSAPIFixture( + api_version='v2.1')) + + self.api = api_fixture.admin_api + self.api.microversion = 'latest' + + fake_image.stub_out_image_service(self) + self.addCleanup(fake_image.FakeImageService_reset) + + self.start_service('conductor') + self.start_service('scheduler') + + # Start two compute services in separate zones. + self._start_host_in_zone('host1', 'zone1') + self._start_host_in_zone('host2', 'zone2') + + flavors = self.api.get_flavors() + self.flavor1 = flavors[0]['id'] + self.flavor2 = flavors[1]['id'] + + def _start_host_in_zone(self, host, zone): + # Start the nova-compute service. + fake.set_nodes([host]) + self.addCleanup(fake.restore_nodes) + self.start_service('compute', host=host) + # Create a host aggregate with a zone in which to put this host. + aggregate_body = { + "aggregate": { + "name": zone, + "availability_zone": zone + } + } + aggregate = self.api.api_post( + '/os-aggregates', aggregate_body).body['aggregate'] + # Now add the compute host to the aggregate. + add_host_body = { + "add_host": { + "host": host + } + } + self.api.api_post( + '/os-aggregates/%s/action' % aggregate['id'], add_host_body) + + def _create_server(self, name): + # Create a server, it doesn't matter which host it ends up in. + server_body = self._build_minimal_create_server_request( + self.api, name, image_uuid=fake_image.get_valid_image_id(), + flavor_id=self.flavor1, networks='none') + server = self.api.post_server({'server': server_body}) + server = self._wait_for_state_change(self.api, server, 'ACTIVE') + original_host = server['OS-EXT-SRV-ATTR:host'] + # Assert the server has the AZ set (not None or 'nova'). + expected_zone = 'zone1' if original_host == 'host1' else 'zone2' + self.assertEqual(expected_zone, server['OS-EXT-AZ:availability_zone']) + return server + + def _assert_instance_az(self, server, expected_zone): + # Check the API. + self.assertEqual(expected_zone, server['OS-EXT-AZ:availability_zone']) + # Check the DB. + ctxt = context.get_admin_context() + with context.target_cell( + ctxt, self.cell_mappings[test.CELL1_NAME]) as cctxt: + instance = objects.Instance.get_by_uuid(cctxt, server['id']) + self.assertEqual(expected_zone, instance.availability_zone) + + def test_live_migrate_implicit_az(self): + """Tests live migration of an instance with an implicit AZ. + + Before Pike, a server created without an explicit availability zone + was assigned a default AZ based on the "default_schedule_zone" config + option which defaults to None, which allows the instance to move + freely between availability zones. + + With change I8d426f2635232ffc4b510548a905794ca88d7f99 in Pike, if the + user does not request an availability zone, the + instance.availability_zone field is set based on the host chosen by + the scheduler. The default AZ for all nova-compute services is + determined by the "default_availability_zone" config option which + defaults to "nova". + + This test creates two nova-compute services in separate zones, creates + a server without specifying an explicit zone, and then tries to live + migrate the instance to the other compute which should succeed because + the request spec does not include an explicit AZ, so the instance is + still not restricted to its current zone even if it says it is in one. + """ + server = self._create_server('test_live_migrate_implicit_az') + original_host = server['OS-EXT-SRV-ATTR:host'] + + # Attempt to live migrate the instance; again, we don't specify a host + # because there are only two hosts so the scheduler would only be able + # to pick the second host which is in a different zone. + live_migrate_req = { + 'os-migrateLive': { + 'block_migration': 'auto', + 'host': None + } + } + self.api.post_server_action(server['id'], live_migrate_req) + + # Poll the migration until it is done. + migration = self._wait_for_migration_status(server, ['completed']) + self.assertEqual('live-migration', migration['migration_type']) + + # Assert that the server did move. Note that we check both the API and + # the database because the API will return the AZ from the host + # aggregate if instance.host is not None. + server = self.api.get_server(server['id']) + expected_zone = 'zone2' if original_host == 'host1' else 'zone1' + self._assert_instance_az(server, expected_zone) + + def test_resize_revert_across_azs(self): + """Creates two compute service hosts in separate AZs. Creates a server + without an explicit AZ so it lands in one AZ, and then resizes the + server which moves it to the other AZ. Then the resize is reverted and + asserts the server is shown as back in the original source host AZ. + """ + server = self._create_server('test_resize_revert_across_azs') + original_host = server['OS-EXT-SRV-ATTR:host'] + original_az = 'zone1' if original_host == 'host1' else 'zone2' + + # Resize the server which should move it to the other zone. + self.api.post_server_action( + server['id'], {'resize': {'flavorRef': self.flavor2}}) + server = self._wait_for_state_change(self.api, server, 'VERIFY_RESIZE') + + # Now the server should be in the other AZ. + new_zone = 'zone2' if original_host == 'host1' else 'zone1' + self._assert_instance_az(server, new_zone) + + # Revert the resize and the server should be back in the original AZ. + self.api.post_server_action(server['id'], {'revertResize': None}) + server = self._wait_for_state_change(self.api, server, 'ACTIVE') + self._assert_instance_az(server, original_az) diff -Nru nova-17.0.10/nova/tests/functional/test_servers.py nova-17.0.11/nova/tests/functional/test_servers.py --- nova-17.0.10/nova/tests/functional/test_servers.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/functional/test_servers.py 2019-07-10 21:45:12.000000000 +0000 @@ -1849,6 +1849,78 @@ new_flavor=new_flavor, source_rp_uuid=source_rp_uuid, dest_rp_uuid=dest_rp_uuid) + def test_migration_confirm_resize_error(self): + source_hostname = self.compute1.host + dest_hostname = self.compute2.host + + source_rp_uuid = self._get_provider_uuid_by_host(source_hostname) + dest_rp_uuid = self._get_provider_uuid_by_host(dest_hostname) + + server = self._boot_and_check_allocations(self.flavor1, + source_hostname) + + self._move_and_check_allocations( + server, request={'migrate': None}, old_flavor=self.flavor1, + new_flavor=self.flavor1, source_rp_uuid=source_rp_uuid, + dest_rp_uuid=dest_rp_uuid) + + # Mock failure + def fake_confirm_migration(context, migration, instance, network_info): + raise exception.MigrationPreCheckError( + reason='test_migration_confirm_resize_error') + + with mock.patch('nova.virt.fake.FakeDriver.' + 'confirm_migration', + side_effect=fake_confirm_migration): + + # Confirm the migration/resize and check the usages + post = {'confirmResize': None} + self.api.post_server_action( + server['id'], post, check_response_status=[204]) + server = self._wait_for_state_change(self.api, server, 'ERROR') + + # After confirming and error, we should have an allocation only on the + # destination host + source_usages = self._get_provider_usages(source_rp_uuid) + self.assertEqual({'VCPU': 0, + 'MEMORY_MB': 0, + 'DISK_GB': 0}, source_usages, + 'Source host %s still has usage after the failed ' + 'migration_confirm' % source_hostname) + + # Check that the server only allocates resource from the original host + allocations = self._get_allocations_by_server_uuid(server['id']) + self.assertEqual(1, len(allocations)) + + dest_allocation = allocations[dest_rp_uuid]['resources'] + self.assertFlavorMatchesAllocation(self.flavor1, dest_allocation) + + dest_usages = self._get_provider_usages(dest_rp_uuid) + self.assertFlavorMatchesAllocation(self.flavor1, dest_usages) + + self._run_periodics() + + # After confirming and error, we should have an allocation only on the + # destination host + source_usages = self._get_provider_usages(source_rp_uuid) + self.assertEqual({'VCPU': 0, + 'MEMORY_MB': 0, + 'DISK_GB': 0}, source_usages, + 'Source host %s still has usage after the failed ' + 'migration_confirm' % source_hostname) + + # Check that the server only allocates resource from the original host + allocations = self._get_allocations_by_server_uuid(server['id']) + self.assertEqual(1, len(allocations)) + + dest_allocation = allocations[dest_rp_uuid]['resources'] + self.assertFlavorMatchesAllocation(self.flavor1, dest_allocation) + + dest_usages = self._get_provider_usages(dest_rp_uuid) + self.assertFlavorMatchesAllocation(self.flavor1, dest_usages) + + self._delete_and_check_allocations(server) + def _test_resize_revert(self, dest_hostname): source_hostname = self._other_hostname(dest_hostname) source_rp_uuid = self._get_provider_uuid_by_host(source_hostname) diff -Nru nova-17.0.10/nova/tests/live_migration/hooks/ceph.sh nova-17.0.11/nova/tests/live_migration/hooks/ceph.sh --- nova-17.0.10/nova/tests/live_migration/hooks/ceph.sh 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/live_migration/hooks/ceph.sh 2019-07-10 21:45:12.000000000 +0000 @@ -1,7 +1,7 @@ #!/bin/bash function prepare_ceph { - git clone git://git.openstack.org/openstack/devstack-plugin-ceph /tmp/devstack-plugin-ceph + git clone https://git.openstack.org/openstack/devstack-plugin-ceph /tmp/devstack-plugin-ceph source /tmp/devstack-plugin-ceph/devstack/settings source /tmp/devstack-plugin-ceph/devstack/lib/ceph install_ceph @@ -10,7 +10,7 @@ $ANSIBLE subnodes --sudo -f 5 -i "$WORKSPACE/inventory" -m raw -a "executable=/bin/bash source $BASE/new/devstack/functions source $BASE/new/devstack/functions-common - git clone git://git.openstack.org/openstack/devstack-plugin-ceph /tmp/devstack-plugin-ceph + git clone https://git.openstack.org/openstack/devstack-plugin-ceph /tmp/devstack-plugin-ceph source /tmp/devstack-plugin-ceph/devstack/lib/ceph install_ceph_remote " diff -Nru nova-17.0.10/nova/tests/live_migration/hooks/run_tests.sh nova-17.0.11/nova/tests/live_migration/hooks/run_tests.sh --- nova-17.0.10/nova/tests/live_migration/hooks/run_tests.sh 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/live_migration/hooks/run_tests.sh 2019-07-10 21:45:12.000000000 +0000 @@ -41,23 +41,25 @@ #run_tempest "NFS shared storage test" "live_migration" #nfs_teardown -echo '3. test with Ceph for root + ephemeral disks' -prepare_ceph -GLANCE_API_CONF=${GLANCE_API_CONF:-/etc/glance/glance-api.conf} -configure_and_start_glance - -# Deal with grenade craziness... if [ "$GRENADE_OLD_BRANCH" ]; then - # NOTE(mriedem): Grenade runs in singleconductor mode, so it won't use - # /etc/nova/nova-cpu.conf so we have to overwrite NOVA_CPU_CONF which is - # read in configure_and_start_nova. - if ! is_service_enabled n-super-cond; then - NOVA_CPU_CONF=$NOVA_CONF - fi -fi + # NOTE(mriedem): Testing with ceph in the grenade live migration job is + # disabled because of a mess of changes in devstack from queens which + # result in the pike node running with nova.conf and the queens node + # running with nova-cpu.conf and _ceph_configure_nova (in ceph.sh) does + # not configure the nodes properly for rbd auth which makes rbd-backed + # live migration fail (because the nodes on shared storage can't + # communicate). Fixing that is non-trivial so we just skip ceph testing + # in the grenade case. + echo '2. test with Ceph is skipped due to bug 1813216' +else + echo '3. test with Ceph for root + ephemeral disks' + prepare_ceph + GLANCE_API_CONF=${GLANCE_API_CONF:-/etc/glance/glance-api.conf} + configure_and_start_glance -configure_and_start_nova -run_tempest "Ceph nova&glance test" "^.*test_live_migration(?!.*(test_volume_backed_live_migration))" + configure_and_start_nova + run_tempest "Ceph nova&glance test" "^.*test_live_migration(?!.*(test_volume_backed_live_migration))" +fi set +e #echo '4. test with Ceph for volumes and root + ephemeral disk' diff -Nru nova-17.0.10/nova/tests/unit/api/openstack/compute/admin_only_action_common.py nova-17.0.11/nova/tests/unit/api/openstack/compute/admin_only_action_common.py --- nova-17.0.10/nova/tests/unit/api/openstack/compute/admin_only_action_common.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/api/openstack/compute/admin_only_action_common.py 2019-07-10 21:45:12.000000000 +0000 @@ -30,27 +30,36 @@ self.req = fakes.HTTPRequest.blank('') self.context = self.req.environ['nova.context'] - def _stub_instance_get(self, uuid=None): - if uuid is None: + def _stub_instance_get(self, action=None, uuid=None): + if not uuid: uuid = uuidutils.generate_uuid() instance = fake_instance.fake_instance_obj(self.context, id=1, uuid=uuid, vm_state=vm_states.ACTIVE, task_state=None, launched_at=timeutils.utcnow()) + + expected_attrs = None + if action == '_migrate_live': + expected_attrs = ['numa_topology'] + self.compute_api.get( - self.context, uuid, expected_attrs=None).AndReturn(instance) + self.context, uuid, expected_attrs=expected_attrs).AndReturn( + instance) + return instance - def _stub_instance_get_failure(self, exc_info, uuid=None): - if uuid is None: - uuid = uuidutils.generate_uuid() + def _stub_instance_get_failure(self, action, exc_info, uuid): + expected_attrs = None + if action == '_migrate_live': + expected_attrs = ['numa_topology'] + self.compute_api.get( - self.context, uuid, expected_attrs=None).AndRaise(exc_info) - return uuid + self.context, uuid, expected_attrs=expected_attrs).AndRaise( + exc_info) def _test_non_existing_instance(self, action, body_map=None): uuid = uuidutils.generate_uuid() self._stub_instance_get_failure( - exception.InstanceNotFound(instance_id=uuid), uuid=uuid) + action, exception.InstanceNotFound(instance_id=uuid), uuid=uuid) self.mox.ReplayAll() controller_function = getattr(self.controller, action) @@ -68,7 +77,7 @@ method = action.replace('_', '') compute_api_args_map = compute_api_args_map or {} - instance = self._stub_instance_get() + instance = self._stub_instance_get(action) args, kwargs = compute_api_args_map.get(action, ((), {})) getattr(self.compute_api, method)(self.context, instance, *args, **kwargs) @@ -92,7 +101,7 @@ if method is None: method = action.replace('_', '') - instance = self._stub_instance_get() + instance = self._stub_instance_get(action) body = {} compute_api_args_map = {} args, kwargs = compute_api_args_map.get(action, ((), {})) @@ -120,7 +129,7 @@ if compute_api_args_map is None: compute_api_args_map = {} - instance = self._stub_instance_get() + instance = self._stub_instance_get(action) args, kwargs = compute_api_args_map.get(action, ((), {})) @@ -150,7 +159,7 @@ method = action.replace('_', '') compute_api_args_map = compute_api_args_map or {} - instance = self._stub_instance_get() + instance = self._stub_instance_get(action) args, kwargs = compute_api_args_map.get(action, ((), {})) getattr(self.compute_api, method)(self.context, instance, *args, @@ -174,7 +183,7 @@ method = action.replace('_', '') compute_api_args_map = compute_api_args_map or {} - instance = self._stub_instance_get() + instance = self._stub_instance_get(action) args, kwargs = compute_api_args_map.get(action, ((), {})) getattr(self.compute_api, method)(self.context, instance, *args, diff -Nru nova-17.0.10/nova/tests/unit/api/openstack/compute/test_migrate_server.py nova-17.0.11/nova/tests/unit/api/openstack/compute/test_migrate_server.py --- nova-17.0.10/nova/tests/unit/api/openstack/compute/test_migrate_server.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/api/openstack/compute/test_migrate_server.py 2019-07-10 21:45:12.000000000 +0000 @@ -138,7 +138,7 @@ def _test_migrate_live_succeeded(self, param): self.mox.StubOutWithMock(self.compute_api, 'live_migrate') - instance = self._stub_instance_get() + instance = self._stub_instance_get('_migrate_live') self.compute_api.live_migrate(self.context, instance, False, self.disk_over_commit, 'hostname', self.force, self.async) @@ -211,7 +211,7 @@ check_response=True): self.mox.StubOutWithMock(self.compute_api, 'live_migrate') - instance = self._stub_instance_get(uuid=uuid) + instance = self._stub_instance_get('_migrate_live', uuid=uuid) self.compute_api.live_migrate(self.context, instance, False, self.disk_over_commit, 'hostname', self.force, self.async @@ -438,7 +438,7 @@ reason="Compute host %(host)s could not be found.", host='hostname') self.mox.StubOutWithMock(self.compute_api, 'live_migrate') - instance = self._stub_instance_get() + instance = self._stub_instance_get('_migrate_live') self.compute_api.live_migrate(self.context, instance, None, self.disk_over_commit, 'hostname', self.force, self.async).AndRaise(exc) @@ -455,7 +455,7 @@ exc = exception.InvalidHypervisorType( reason="The supplied hypervisor type of is invalid.") self.mox.StubOutWithMock(self.compute_api, 'live_migrate') - instance = self._stub_instance_get() + instance = self._stub_instance_get('_migrate_live') self.compute_api.live_migrate(self.context, instance, None, self.disk_over_commit, 'hostname', self.force, self.async).AndRaise(exc) diff -Nru nova-17.0.10/nova/tests/unit/api/openstack/compute/test_volumes.py nova-17.0.11/nova/tests/unit/api/openstack/compute/test_volumes.py --- nova-17.0.10/nova/tests/unit/api/openstack/compute/test_volumes.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/api/openstack/compute/test_volumes.py 2019-07-10 21:45:12.000000000 +0000 @@ -40,6 +40,7 @@ from nova.tests.unit.api.openstack import fakes from nova.tests.unit import fake_block_device from nova.tests.unit import fake_instance +from nova.tests import uuidsentinel as uuids from nova.volume import cinder CONF = nova.conf.CONF @@ -959,6 +960,81 @@ 'shelved-offloaded instances.', six.text_type(ex)) +class SwapVolumeMultiattachTestCase(test.NoDBTestCase): + + @mock.patch('nova.api.openstack.common.get_instance') + @mock.patch('nova.volume.cinder.API.begin_detaching') + @mock.patch('nova.volume.cinder.API.roll_detaching') + def test_swap_multiattach_multiple_readonly_attachments_fails( + self, mock_roll_detaching, mock_begin_detaching, + mock_get_instance): + """Tests that trying to swap from a multiattach volume with + multiple read/write attachments will return an error. + """ + + def fake_volume_get(_context, volume_id): + if volume_id == uuids.old_vol_id: + return { + 'id': volume_id, + 'size': 1, + 'multiattach': True, + 'attachments': { + uuids.server1: { + 'attachment_id': uuids.attachment_id1, + 'mountpoint': '/dev/vdb' + }, + uuids.server2: { + 'attachment_id': uuids.attachment_id2, + 'mountpoint': '/dev/vdb' + } + } + } + if volume_id == uuids.new_vol_id: + return { + 'id': volume_id, + 'size': 1, + 'attach_status': 'detached' + } + raise exception.VolumeNotFound(volume_id=volume_id) + + def fake_attachment_get(_context, attachment_id): + return {'connection_info': {'attach_mode': 'rw'}} + + ctxt = context.get_admin_context() + instance = fake_instance.fake_instance_obj( + ctxt, uuid=uuids.server1, vm_state=vm_states.ACTIVE, + task_state=None, launched_at=datetime.datetime(2018, 6, 6)) + mock_get_instance.return_value = instance + controller = volumes_v21.VolumeAttachmentController() + with test.nested( + mock.patch.object(controller.volume_api, 'get', + side_effect=fake_volume_get), + mock.patch.object(controller.compute_api.volume_api, + 'attachment_get', + side_effect=fake_attachment_get)) as ( + mock_volume_get, mock_attachment_get + ): + req = fakes.HTTPRequest.blank( + '/servers/%s/os-volume_attachments/%s' % + (uuids.server1, uuids.old_vol_id)) + req.headers['content-type'] = 'application/json' + req.environ['nova.context'] = ctxt + body = { + 'volumeAttachment': { + 'volumeId': uuids.new_vol_id + } + } + ex = self.assertRaises( + webob.exc.HTTPBadRequest, controller.update, req, + uuids.server1, uuids.old_vol_id, body=body) + self.assertIn('Swapping multi-attach volumes with more than one ', + six.text_type(ex)) + mock_attachment_get.assert_has_calls([ + mock.call(ctxt, uuids.attachment_id1), + mock.call(ctxt, uuids.attachment_id2)], any_order=True) + mock_roll_detaching.assert_called_once_with(ctxt, uuids.old_vol_id) + + class CommonBadRequestTestCase(object): resource = None diff -Nru nova-17.0.10/nova/tests/unit/compute/test_compute_api.py nova-17.0.11/nova/tests/unit/compute/test_compute_api.py --- nova-17.0.10/nova/tests/unit/compute/test_compute_api.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/compute/test_compute_api.py 2019-07-10 21:45:12.000000000 +0000 @@ -1839,12 +1839,14 @@ def test_confirm_resize_with_migration_ref(self): self._test_confirm_resize(mig_ref_passed=True) + @mock.patch('nova.availability_zones.get_host_availability_zone', + return_value='nova') @mock.patch('nova.objects.Quotas.check_deltas') @mock.patch('nova.objects.Migration.get_by_instance_and_status') @mock.patch('nova.context.RequestContext.elevated') @mock.patch('nova.objects.RequestSpec.get_by_instance_uuid') def _test_revert_resize(self, mock_get_reqspec, mock_elevated, - mock_get_migration, mock_check): + mock_get_migration, mock_check, mock_get_host_az): params = dict(vm_state=vm_states.RESIZED) fake_inst = self._create_instance_obj(params=params) fake_inst.old_flavor = fake_inst.flavor @@ -1887,11 +1889,14 @@ def test_revert_resize(self): self._test_revert_resize() + @mock.patch('nova.availability_zones.get_host_availability_zone', + return_value='nova') @mock.patch('nova.objects.Quotas.check_deltas') @mock.patch('nova.objects.Migration.get_by_instance_and_status') @mock.patch('nova.context.RequestContext.elevated') def test_revert_resize_concurrent_fail(self, mock_elevated, - mock_get_migration, mock_check): + mock_get_migration, mock_check, + mock_get_host_az): params = dict(vm_state=vm_states.RESIZED) fake_inst = self._create_instance_obj(params=params) fake_inst.old_flavor = fake_inst.flavor @@ -2824,6 +2829,58 @@ _do_test() + def test_count_attachments_for_swap_not_found_and_readonly(self): + """Tests that attachment records that aren't found are considered + read/write by default. Also tests that read-only attachments are + not counted. + """ + ctxt = context.get_admin_context() + volume = { + 'attachments': { + uuids.server1: { + 'attachment_id': uuids.attachment1 + }, + uuids.server2: { + 'attachment_id': uuids.attachment2 + } + } + } + + def fake_attachment_get(_context, attachment_id): + if attachment_id == uuids.attachment1: + raise exception.VolumeAttachmentNotFound( + attachment_id=attachment_id) + return {'connection_info': {'attach_mode': 'ro'}} + + with mock.patch.object(self.compute_api.volume_api, 'attachment_get', + side_effect=fake_attachment_get) as mock_get: + self.assertEqual( + 1, self.compute_api._count_attachments_for_swap(ctxt, volume)) + mock_get.assert_has_calls([ + mock.call(ctxt, uuids.attachment1), + mock.call(ctxt, uuids.attachment2)], any_order=True) + + @mock.patch('nova.volume.cinder.API.attachment_get', + new_callable=mock.NonCallableMock) # asserts not called + def test_count_attachments_for_swap_no_query(self, mock_attachment_get): + """Tests that if the volume has <2 attachments, we don't query + the attachments for their attach_mode value. + """ + volume = {} + self.assertEqual( + 0, self.compute_api._count_attachments_for_swap( + mock.sentinel.context, volume)) + volume = { + 'attachments': { + uuids.server: { + 'attachment_id': uuids.attach1 + } + } + } + self.assertEqual( + 1, self.compute_api._count_attachments_for_swap( + mock.sentinel.context, volume)) + @mock.patch.object(compute_api.API, '_record_action_start') def _test_snapshot_and_backup(self, mock_record, is_snapshot=True, with_base_ref=False, min_ram=None, @@ -4409,6 +4466,9 @@ mock_br, mock_rs): fake_keypair = objects.KeyPair(name='test') + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping', + new=mock.MagicMock()) @mock.patch('nova.compute.utils.check_num_instances_quota') @mock.patch.object(self.compute_api, 'security_group_api') @mock.patch.object(self.compute_api, @@ -4441,6 +4501,8 @@ do_test() def test_provision_instances_creates_build_request(self): + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping') @mock.patch.object(objects.Instance, 'create') @mock.patch.object(self.compute_api, 'volume_api') @mock.patch('nova.compute.utils.check_num_instances_quota') @@ -4456,7 +4518,8 @@ def do_test(mock_get_min_ver, mock_get_min_ver_cells, _mock_inst_mapping_create, mock_build_req, mock_req_spec_from_components, _mock_ensure_default, - mock_check_num_inst_quota, mock_volume, mock_inst_create): + mock_check_num_inst_quota, mock_volume, mock_inst_create, + mock_create_rs_br_im): min_count = 1 max_count = 2 @@ -4519,11 +4582,14 @@ br.instance.project_id) self.assertEqual(1, br.block_device_mappings[0].id) self.assertEqual(br.instance.uuid, br.tags[0].resource_id) - br.create.assert_called_with() + mock_create_rs_br_im.assert_any_call(ctxt, rs, br, im) do_test() def test_provision_instances_creates_instance_mapping(self): + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping', + new=mock.MagicMock()) @mock.patch('nova.compute.utils.check_num_instances_quota') @mock.patch.object(objects.Instance, 'create', new=mock.MagicMock()) @mock.patch.object(self.compute_api.security_group_api, @@ -4534,8 +4600,6 @@ new=mock.MagicMock()) @mock.patch.object(objects.RequestSpec, 'from_components', mock.MagicMock()) - @mock.patch.object(objects.BuildRequest, 'create', - new=mock.MagicMock()) @mock.patch('nova.objects.InstanceMapping') def do_test(mock_inst_mapping, mock_check_num_inst_quota): inst_mapping_mock = mock.MagicMock() @@ -4614,6 +4678,8 @@ _mock_cinder_reserve_volume, _mock_cinder_check_availability_zone, _mock_cinder_get, _mock_get_min_ver_cells, _mock_get_min_ver): + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping') @mock.patch('nova.compute.utils.check_num_instances_quota') @mock.patch.object(objects, 'Instance') @mock.patch.object(self.compute_api.security_group_api, @@ -4624,7 +4690,8 @@ @mock.patch.object(objects, 'InstanceMapping') def do_test(mock_inst_mapping, mock_build_req, mock_req_spec_from_components, _mock_create_bdm, - _mock_ensure_default, mock_inst, mock_check_num_inst_quota): + _mock_ensure_default, mock_inst, mock_check_num_inst_quota, + mock_create_rs_br_im): min_count = 1 max_count = 2 @@ -4689,9 +4756,10 @@ check_server_group_quota, filter_properties, None, tags) # First instance, build_req, mapping is created and destroyed - self.assertTrue(build_req_mocks[0].create.called) + mock_create_rs_br_im.assert_called_once_with(ctxt, req_spec_mock, + build_req_mocks[0], + inst_map_mocks[0]) self.assertTrue(build_req_mocks[0].destroy.called) - self.assertTrue(inst_map_mocks[0].create.called) self.assertTrue(inst_map_mocks[0].destroy.called) # Second instance, build_req, mapping is not created nor destroyed self.assertFalse(inst_mocks[1].create.called) @@ -4716,6 +4784,8 @@ _mock_bdm, _mock_cinder_attach_create, _mock_cinder_check_availability_zone, _mock_cinder_get, _mock_get_min_ver_cells, _mock_get_min_ver): + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping') @mock.patch('nova.compute.utils.check_num_instances_quota') @mock.patch.object(objects, 'Instance') @mock.patch.object(self.compute_api.security_group_api, @@ -4726,7 +4796,8 @@ @mock.patch.object(objects, 'InstanceMapping') def do_test(mock_inst_mapping, mock_build_req, mock_req_spec_from_components, _mock_create_bdm, - _mock_ensure_default, mock_inst, mock_check_num_inst_quota): + _mock_ensure_default, mock_inst, mock_check_num_inst_quota, + mock_create_rs_br_im): min_count = 1 max_count = 2 @@ -4791,9 +4862,10 @@ check_server_group_quota, filter_properties, None, tags) # First instance, build_req, mapping is created and destroyed - self.assertTrue(build_req_mocks[0].create.called) + mock_create_rs_br_im.assert_called_once_with(ctxt, req_spec_mock, + build_req_mocks[0], + inst_map_mocks[0]) self.assertTrue(build_req_mocks[0].destroy.called) - self.assertTrue(inst_map_mocks[0].create.called) self.assertTrue(inst_map_mocks[0].destroy.called) # Second instance, build_req, mapping is not created nor destroyed self.assertFalse(inst_mocks[1].create.called) @@ -4804,6 +4876,9 @@ do_test() def test_provision_instances_creates_reqspec_with_secgroups(self): + @mock.patch.object(self.compute_api, + '_create_reqspec_buildreq_instmapping', + new=mock.MagicMock()) @mock.patch('nova.compute.utils.check_num_instances_quota') @mock.patch.object(self.compute_api, 'security_group_api') @mock.patch.object(compute_api, 'objects') diff -Nru nova-17.0.10/nova/tests/unit/compute/test_compute_mgr.py nova-17.0.11/nova/tests/unit/compute/test_compute_mgr.py --- nova-17.0.10/nova/tests/unit/compute/test_compute_mgr.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/compute/test_compute_mgr.py 2019-07-10 21:45:12.000000000 +0000 @@ -2192,11 +2192,11 @@ connection_info='{"data": {}}', volume_size=1) old_volume = { 'id': uuids.old_volume_id, 'size': 1, 'status': 'retyping', - 'multiattach': False + 'migration_status': 'migrating', 'multiattach': False } new_volume = { 'id': uuids.new_volume_id, 'size': 1, 'status': 'reserved', - 'multiattach': False + 'migration_status': 'migrating', 'multiattach': False } attachment_update.return_value = {"connection_info": {"data": {}}} get_bdm.return_value = bdm @@ -2338,12 +2338,12 @@ attachment_id=uuids.old_attachment_id, connection_info='{"data": {}}') old_volume = { - 'id': uuids.old_volume_id, 'size': 1, 'status': 'migrating', - 'multiattach': False + 'id': uuids.old_volume_id, 'size': 1, 'status': 'in-use', + 'migration_status': 'migrating', 'multiattach': False } new_volume = { 'id': uuids.new_volume_id, 'size': 1, 'status': 'reserved', - 'multiattach': False + 'migration_status': 'migrating', 'multiattach': False } get_bdm.return_value = bdm get_volume.side_effect = (old_volume, new_volume) @@ -6256,12 +6256,15 @@ expected_attrs=['metadata', 'system_metadata', 'info_cache']) self.migration = objects.Migration( context=self.context.elevated(), + id=1, uuid=mock.sentinel.uuid, instance_uuid=self.instance.uuid, new_instance_type_id=7, dest_compute=None, dest_node=None, dest_host=None, + source_compute='source_compute', + source_node='source_node', status='migrating') self.migration.save = mock.MagicMock() self.useFixture(fixtures.SpawnIsSynchronousFixture()) @@ -6615,6 +6618,8 @@ do_finish_revert_resize() def test_confirm_resize_deletes_allocations(self): + @mock.patch('nova.objects.Instance.get_by_uuid') + @mock.patch('nova.objects.Migration.get_by_id') @mock.patch.object(self.migration, 'save') @mock.patch.object(self.compute, '_notify_about_instance_usage') @mock.patch.object(self.compute, 'network_api') @@ -6625,12 +6630,17 @@ @mock.patch.object(self.instance, 'save') def do_confirm_resize(mock_save, mock_drop, mock_delete, mock_get_rt, mock_confirm, mock_nwapi, mock_notify, - mock_mig_save): - self.instance.migration_context = objects.MigrationContext() + mock_mig_save, mock_mig_get, mock_inst_get): + self.instance.migration_context = objects.MigrationContext( + new_pci_devices=None, + old_pci_devices=None) self.migration.source_compute = self.instance['host'] self.migration.source_node = self.instance['node'] - self.compute._confirm_resize(self.context, self.instance, - self.migration) + self.migration.status = 'confirming' + mock_mig_get.return_value = self.migration + mock_inst_get.return_value = self.instance + self.compute.confirm_resize(self.context, self.instance, + self.migration) mock_delete.assert_called_once_with(self.context, self.instance, self.migration, self.instance.old_flavor, @@ -6664,6 +6674,70 @@ mock_resources.return_value) do_it() + @mock.patch('nova.objects.MigrationContext.get_pci_mapping_for_migration') + @mock.patch('nova.compute.utils.add_instance_fault_from_exc') + @mock.patch('nova.objects.Migration.get_by_id') + @mock.patch('nova.objects.Instance.get_by_uuid') + @mock.patch('nova.compute.utils.notify_about_instance_usage') + @mock.patch('nova.compute.utils.notify_about_instance_action') + @mock.patch('nova.objects.Instance.save') + def test_confirm_resize_driver_confirm_migration_fails( + self, instance_save, notify_action, notify_usage, + instance_get_by_uuid, migration_get_by_id, add_fault, get_mapping): + """Tests the scenario that driver.confirm_migration raises some error + to make sure the error is properly handled, like the instance and + migration status is set to 'error'. + """ + self.migration.status = 'confirming' + migration_get_by_id.return_value = self.migration + instance_get_by_uuid.return_value = self.instance + self.instance.migration_context = objects.MigrationContext() + + def fake_delete_allocation_after_move(_context, instance, migration, + flavor, nodename): + # The migration.status must be 'confirmed' for the method to + # properly cleanup the allocation held by the migration. + self.assertEqual('confirmed', migration.status) + + error = exception.HypervisorUnavailable( + host=self.migration.source_compute) + with test.nested( + mock.patch.object(self.compute, 'network_api'), + mock.patch.object(self.compute.driver, 'confirm_migration', + side_effect=error), + mock.patch.object(self.compute, '_delete_allocation_after_move', + side_effect=fake_delete_allocation_after_move), + mock.patch.object(self.compute, + '_get_updated_nw_info_with_pci_mapping') + ) as ( + network_api, confirm_migration, delete_allocation, pci_mapping + ): + self.assertRaises(exception.HypervisorUnavailable, + self.compute.confirm_resize, + self.context, self.instance, self.migration) + # Make sure the instance is in ERROR status. + self.assertEqual(vm_states.ERROR, self.instance.vm_state) + # Make sure the migration is in error status. + self.assertEqual('error', self.migration.status) + # Instance.save is called twice, once to clear the resize metadata + # and once to set the instance to ERROR status. + self.assertEqual(2, instance_save.call_count) + # The migration.status should have been saved. + self.migration.save.assert_called_once_with() + # Allocations should always be cleaned up even if cleaning up the + # source host fails. + delete_allocation.assert_called_once_with( + self.context, self.instance, self.migration, + self.instance.old_flavor, self.migration.source_node) + # Assert other mocks we care less about. + notify_usage.assert_called_once() + notify_action.assert_called_once() + add_fault.assert_called_once() + confirm_migration.assert_called_once() + network_api.setup_networks_on_host.assert_called_once() + instance_get_by_uuid.assert_called_once() + migration_get_by_id.assert_called_once() + @mock.patch('nova.scheduler.utils.resources_from_flavor') def test_delete_allocation_after_move_confirm_by_migration(self, mock_rff): mock_rff.return_value = {} @@ -6764,6 +6838,59 @@ # No allocations by migration, legacy cleanup doit(False) + def test_confirm_resize_calls_virt_driver_with_old_pci(self): + @mock.patch.object(self.compute, '_get_resource_tracker') + @mock.patch.object(self.migration, 'save') + @mock.patch.object(self.compute, '_notify_about_instance_usage') + @mock.patch.object(self.compute, 'network_api') + @mock.patch.object(self.compute.driver, 'confirm_migration') + @mock.patch.object(self.compute, '_delete_allocation_after_move') + @mock.patch.object(self.instance, 'drop_migration_context') + @mock.patch.object(self.instance, 'save') + def do_confirm_resize(mock_save, mock_drop, mock_delete, + mock_confirm, mock_nwapi, mock_notify, + mock_mig_save, mock_rt): + # Mock virt driver confirm_resize() to save the provided + # network_info, we will check it later. + updated_nw_info = [] + + def driver_confirm_resize(*args, **kwargs): + if 'network_info' in kwargs: + nw_info = kwargs['network_info'] + else: + nw_info = args[3] + updated_nw_info.extend(nw_info) + + mock_confirm.side_effect = driver_confirm_resize + old_devs = objects.PciDeviceList( + objects=[objects.PciDevice( + address='0000:04:00.2', + request_id=uuids.pcidev1)]) + new_devs = objects.PciDeviceList( + objects=[objects.PciDevice( + address='0000:05:00.3', + request_id=uuids.pcidev1)]) + self.instance.migration_context = objects.MigrationContext( + new_pci_devices=new_devs, + old_pci_devices=old_devs) + # Create VIF with new_devs[0] PCI address. + nw_info = network_model.NetworkInfo([ + network_model.VIF( + id=uuids.port1, + vnic_type=network_model.VNIC_TYPE_DIRECT, + profile={'pci_slot': new_devs[0].address})]) + mock_nwapi.get_instance_nw_info.return_value = nw_info + self.migration.source_compute = self.instance['host'] + self.migration.source_node = self.instance['node'] + self.compute._confirm_resize(self.context, self.instance, + self.migration) + # Assert virt driver confirm_migration() was called + # with the updated nw_info object. + self.assertEqual(old_devs[0].address, + updated_nw_info[0]['profile']['pci_slot']) + + do_confirm_resize() + def test_revert_allocation(self): """New-style migration-based allocation revert.""" @@ -7206,7 +7333,8 @@ self.flags(live_migration_wait_for_vif_plug=True, group='compute') migrate_data = objects.LibvirtLiveMigrateData( wait_for_vif_plugged=True) - mock_get_bdms.return_value = objects.BlockDeviceMappingList(objects=[]) + source_bdms = objects.BlockDeviceMappingList(objects=[]) + mock_get_bdms.return_value = source_bdms mock_pre_live_mig.return_value = migrate_data self.instance.info_cache = objects.InstanceInfoCache( network_info=network_model.NetworkInfo([ @@ -7222,7 +7350,8 @@ self.instance, None, self.migration, migrate_data) self.assertEqual('error', self.migration.status) mock_rollback_live_mig.assert_called_once_with( - self.context, self.instance, 'dest-host', migrate_data) + self.context, self.instance, 'dest-host', + migrate_data=migrate_data, source_bdms=source_bdms) @mock.patch('nova.compute.rpcapi.ComputeAPI.pre_live_migration') @mock.patch('nova.compute.manager.ComputeManager._rollback_live_migration') @@ -7236,7 +7365,8 @@ self.flags(live_migration_wait_for_vif_plug=True, group='compute') migrate_data = objects.LibvirtLiveMigrateData( wait_for_vif_plugged=True) - mock_get_bdms.return_value = objects.BlockDeviceMappingList(objects=[]) + source_bdms = objects.BlockDeviceMappingList(objects=[]) + mock_get_bdms.return_value = source_bdms mock_pre_live_mig.return_value = migrate_data self.instance.info_cache = objects.InstanceInfoCache( network_info=network_model.NetworkInfo([ @@ -7253,7 +7383,8 @@ self.assertIn('Timed out waiting for events', six.text_type(ex)) self.assertEqual('error', self.migration.status) mock_rollback_live_mig.assert_called_once_with( - self.context, self.instance, 'dest-host', migrate_data) + self.context, self.instance, 'dest-host', + migrate_data=migrate_data, source_bdms=source_bdms) @mock.patch('nova.compute.rpcapi.ComputeAPI.pre_live_migration') @mock.patch('nova.compute.manager.ComputeManager._rollback_live_migration') @@ -7606,15 +7737,28 @@ migrate_data.old_vol_attachment_ids = { volume_id: orig_attachment_id} - bdm = fake_block_device.fake_bdm_object( - self.context, - {'source_type': 'volume', 'destination_type': 'volume', - 'volume_id': volume_id, 'device_name': '/dev/vdb', - 'instance_uuid': instance.uuid}) + def fake_bdm(): + bdm = fake_block_device.fake_bdm_object( + self.context, + {'source_type': 'volume', 'destination_type': 'volume', + 'volume_id': volume_id, 'device_name': '/dev/vdb', + 'instance_uuid': instance.uuid}) + bdm.save = mock.Mock() + return bdm + + # NOTE(mdbooth): Use of attachment_id as connection_info is a + # test convenience. It just needs to be a string. + source_bdm = fake_bdm() + source_bdm.attachment_id = orig_attachment_id + source_bdm.connection_info = orig_attachment_id + source_bdms = objects.BlockDeviceMappingList(objects=[source_bdm]) + + bdm = fake_bdm() bdm.attachment_id = new_attachment_id + bdm.connection_info = new_attachment_id + bdms = objects.BlockDeviceMappingList(objects=[bdm]) @mock.patch.object(compute.volume_api, 'attachment_delete') - @mock.patch.object(bdm, 'save') @mock.patch.object(compute_utils, 'notify_about_instance_action') @mock.patch.object(instance, 'save') @mock.patch.object(compute, '_notify_about_instance_usage') @@ -7622,24 +7766,24 @@ @mock.patch.object(compute, 'network_api') @mock.patch.object(objects.BlockDeviceMappingList, 'get_by_instance_uuid') - def _test(mock_get_bdms, mock_net_api, mock_remove_conn, - mock_usage, mock_instance_save, mock_action, mock_save, - mock_attach_delete): + def _test(mock_get_bdms, mock_net_api, mock_remove_conn, mock_usage, + mock_instance_save, mock_action, mock_attach_delete): # this tests that _rollback_live_migration replaces the bdm's # attachment_id with the original attachment id that is in # migrate_data. - mock_get_bdms.return_value = objects.BlockDeviceMappingList( - objects=[bdm]) + mock_get_bdms.return_value = bdms compute._rollback_live_migration(self.context, instance, None, - migrate_data) + migrate_data=migrate_data, + source_bdms=source_bdms) mock_remove_conn.assert_called_once_with(self.context, instance, bdm.volume_id, None) mock_attach_delete.called_once_with(self.context, new_attachment_id) self.assertEqual(bdm.attachment_id, orig_attachment_id) - mock_save.assert_called_once_with() + self.assertEqual(orig_attachment_id, bdm.connection_info) + bdm.save.assert_called_once_with() _test() @@ -7849,6 +7993,24 @@ doit() + def test_get_updated_nw_info_with_pci_mapping(self): + old_dev = objects.PciDevice(address='0000:04:00.2') + new_dev = objects.PciDevice(address='0000:05:00.3') + pci_mapping = {old_dev.address: new_dev} + nw_info = network_model.NetworkInfo([ + network_model.VIF( + id=uuids.port1, + vnic_type=network_model.VNIC_TYPE_NORMAL), + network_model.VIF( + id=uuids.port2, + vnic_type=network_model.VNIC_TYPE_DIRECT, + profile={'pci_slot': old_dev.address})]) + updated_nw_info = self.compute._get_updated_nw_info_with_pci_mapping( + nw_info, pci_mapping) + self.assertDictEqual(nw_info[0], updated_nw_info[0]) + self.assertEqual(new_dev.address, + updated_nw_info[1]['profile']['pci_slot']) + class ComputeManagerInstanceUsageAuditTestCase(test.TestCase): def setUp(self): diff -Nru nova-17.0.10/nova/tests/unit/compute/test_compute.py nova-17.0.11/nova/tests/unit/compute/test_compute.py --- nova-17.0.10/nova/tests/unit/compute/test_compute.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/compute/test_compute.py 2019-07-10 21:45:12.000000000 +0000 @@ -487,6 +487,58 @@ exception=expected_exception), ]) + @mock.patch.object(compute_manager.LOG, 'debug') + @mock.patch.object(compute_utils, 'EventReporter') + @mock.patch('nova.context.RequestContext.elevated') + @mock.patch('nova.compute.utils.notify_about_volume_attach_detach') + def test_attach_volume_ignore_VolumeAttachmentNotFound( + self, mock_notify, mock_elevate, mock_event, mock_debug_log): + """Tests the scenario that the DriverVolumeBlockDevice.attach flow + already deleted the volume attachment before the + ComputeManager.attach_volume flow tries to rollback the attachment + record and delete it, which raises VolumeAttachmentNotFound and is + ignored. + """ + mock_elevate.return_value = self.context + + attachment_id = uuids.attachment_id + fake_bdm = objects.BlockDeviceMapping(**self.fake_volume) + fake_bdm.attachment_id = attachment_id + instance = self._create_fake_instance_obj() + expected_exception = test.TestingException() + + def fake_attach(*args, **kwargs): + raise expected_exception + + with test.nested( + mock.patch.object(driver_block_device.DriverVolumeBlockDevice, + 'attach'), + mock.patch.object(cinder.API, 'attachment_delete'), + mock.patch.object(objects.BlockDeviceMapping, + 'destroy') + ) as (mock_attach, mock_attach_delete, mock_destroy): + mock_attach.side_effect = fake_attach + mock_attach_delete.side_effect = \ + exception.VolumeAttachmentNotFound( + attachment_id=attachment_id) + self.assertRaises( + test.TestingException, self.compute.attach_volume, + self.context, instance, fake_bdm) + mock_destroy.assert_called_once_with() + mock_notify.assert_has_calls([ + mock.call(self.context, instance, 'fake-mini', + action='volume_attach', phase='start', + volume_id=uuids.volume_id), + mock.call(self.context, instance, 'fake-mini', + action='volume_attach', phase='error', + exception=expected_exception, + volume_id=uuids.volume_id), + ]) + mock_event.assert_called_once_with( + self.context, 'compute_attach_volume', instance.uuid) + self.assertIsInstance(mock_debug_log.call_args[0][1], + exception.VolumeAttachmentNotFound) + @mock.patch.object(compute_utils, 'EventReporter') def test_detach_volume_api_raises(self, mock_event): fake_bdm = objects.BlockDeviceMapping(**self.fake_volume) @@ -5715,6 +5767,8 @@ migration_context.migration_id = migration.id migration_context.old_numa_topology = old_inst_topology migration_context.new_numa_topology = new_inst_topology + migration_context.old_pci_devices = None + migration_context.new_pci_devices = None instance.migration_context = migration_context instance.vm_state = vm_states.RESIZED @@ -5751,12 +5805,14 @@ old_pci_devices = objects.PciDeviceList( objects=[objects.PciDevice(vendor_id='1377', product_id='0047', - address='0000:0a:00.1')]) + address='0000:0a:00.1', + request_id=uuids.req1)]) new_pci_devices = objects.PciDeviceList( objects=[objects.PciDevice(vendor_id='1377', product_id='0047', - address='0000:0b:00.1')]) + address='0000:0b:00.1', + request_id=uuids.req2)]) if expected_pci_addr == old_pci_devices[0].address: expected_pci_device = old_pci_devices[0] @@ -6170,7 +6226,7 @@ @mock.patch('nova.objects.Migration.save') def test_live_migration_exception_rolls_back(self, mock_save, mock_rollback, mock_remove, - mock_get_uuid, + mock_get_bdms, mock_get_node, mock_pre, mock_get_disk): # Confirm exception when pre_live_migration fails. c = context.get_admin_context() @@ -6185,27 +6241,39 @@ dest_host = updated_instance['host'] dest_node = objects.ComputeNode(host=dest_host, uuid=uuids.dest_node) mock_get_node.return_value = dest_node - fake_bdms = objects.BlockDeviceMappingList(objects=[ + + # All the fake BDMs we've generated, in order + fake_bdms = [] + + def gen_fake_bdms(obj, instance): + # generate a unique fake connection_info every time we're called, + # simulating connection_info being mutated elsewhere. + bdms = objects.BlockDeviceMappingList(objects=[ objects.BlockDeviceMapping( **fake_block_device.FakeDbBlockDeviceDict( {'volume_id': uuids.volume_id_1, 'source_type': 'volume', + 'connection_info': + jsonutils.dumps(uuidutils.generate_uuid()), 'destination_type': 'volume'})), objects.BlockDeviceMapping( **fake_block_device.FakeDbBlockDeviceDict( {'volume_id': uuids.volume_id_2, 'source_type': 'volume', + 'connection_info': + jsonutils.dumps(uuidutils.generate_uuid()), 'destination_type': 'volume'})) - ]) + ]) + for bdm in bdms: + bdm.save = mock.Mock() + fake_bdms.append(bdms) + return bdms + migrate_data = migrate_data_obj.XenapiLiveMigrateData( block_migration=True) - block_device_info = { - 'swap': None, 'ephemerals': [], 'block_device_mapping': [], - 'root_device_name': None} - mock_get_disk.return_value = 'fake_disk' mock_pre.side_effect = test.TestingException - mock_get_uuid.return_value = fake_bdms + mock_get_bdms.side_effect = gen_fake_bdms # start test migration = objects.Migration(uuid=uuids.migration) @@ -6238,12 +6306,26 @@ self.assertEqual(vm_states.ACTIVE, instance.vm_state) self.assertIsNone(instance.task_state) self.assertEqual('error', migration.status) - mock_get_disk.assert_called_once_with(instance, - block_device_info=block_device_info) + mock_get_disk.assert_called() mock_pre.assert_called_once_with(c, - instance, True, 'fake_disk', dest_host, migrate_data) + instance, True, mock_get_disk.return_value, dest_host, + migrate_data) - mock_get_uuid.assert_called_with(c, instance.uuid) + # Assert that _rollback_live_migration puts connection_info back to + # what it was before the call to pre_live_migration. + # BlockDeviceMappingList.get_by_instance_uuid is mocked to generate + # BDMs with unique connection_info every time it's called. These are + # stored in fake_bdms in the order they were generated. We assert here + # that the last BDMs generated (in _rollback_live_migration) now have + # the same connection_info as the first BDMs generated (before calling + # pre_live_migration), and that we saved them. + self.assertGreater(len(fake_bdms), 1) + for source_bdm, final_bdm in zip(fake_bdms[0], fake_bdms[-1]): + self.assertEqual(source_bdm.connection_info, + final_bdm.connection_info) + final_bdm.save.assert_called() + + mock_get_bdms.assert_called_with(c, instance.uuid) mock_remove.assert_has_calls([ mock.call(c, instance, uuids.volume_id_1, dest_host), mock.call(c, instance, uuids.volume_id_2, dest_host)]) @@ -6603,11 +6685,13 @@ @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename') @mock.patch('nova.objects.BlockDeviceMappingList.get_by_instance_uuid') - def test_rollback_live_migration(self, mock_bdms, mock_get_node): + def test_rollback_live_migration(self, mock_bdms, mock_get_node, + migration_status=None): c = context.get_admin_context() instance = mock.MagicMock() migration = objects.Migration(uuid=uuids.migration) migrate_data = objects.LibvirtLiveMigrateData(migration=migration) + source_bdms = objects.BlockDeviceMappingList() dest_node = objects.ComputeNode(host='foo', uuid=uuids.dest_node) mock_get_node.return_value = dest_node @@ -6621,8 +6705,15 @@ @mock.patch.object(self.compute, 'network_api') def _test(mock_nw_api, mock_lmcf, mock_ra, mock_mig_save, mock_notify): mock_lmcf.return_value = False, False - self.compute._rollback_live_migration(c, instance, 'foo', - migrate_data=migrate_data) + if migration_status: + self.compute._rollback_live_migration( + c, instance, 'foo', migrate_data=migrate_data, + migration_status=migration_status, + source_bdms=source_bdms) + else: + self.compute._rollback_live_migration( + c, instance, 'foo', migrate_data=migrate_data, + source_bdms=source_bdms) mock_notify.assert_has_calls([ mock.call(c, instance, self.compute.host, action='live_migration_rollback', phase='start', @@ -6630,55 +6721,75 @@ mock.call(c, instance, self.compute.host, action='live_migration_rollback', phase='end', bdms=bdms)]) - mock_nw_api.setup_networks_on_host.assert_called_once_with( - c, instance, self.compute.host) + mock_nw_api.setup_networks_on_host.assert_has_calls([ + mock.call(c, instance, self.compute.host), + mock.call(c, instance, teardown=True) + ]) mock_ra.assert_called_once_with(mock.ANY, instance, migration) mock_mig_save.assert_called_once_with() + _test() - self.assertEqual('error', migration.status) + self.assertEqual(migration_status or 'error', migration.status) self.assertEqual(0, instance.progress) - @mock.patch('nova.objects.Migration.save') - @mock.patch.object(objects.ComputeNode, - 'get_by_host_and_nodename') - @mock.patch('nova.objects.BlockDeviceMappingList.get_by_instance_uuid') - def test_rollback_live_migration_set_migration_status(self, mock_bdms, - mock_get_node, - mock_mig_save): - c = context.get_admin_context() - instance = mock.MagicMock() - migration = objects.Migration(context=c, id=0) - migrate_data = objects.LibvirtLiveMigrateData(migration=migration) + def test_rollback_live_migration_set_migration_status(self): + self.test_rollback_live_migration(migration_status='fake') - dest_node = objects.ComputeNode(host='foo', uuid=uuids.dest_node) - mock_get_node.return_value = dest_node - bdms = objects.BlockDeviceMappingList() - mock_bdms.return_value = bdms + @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename', + return_value=objects.ComputeNode( + host='dest-host', uuid=uuids.dest_node)) + @mock.patch('nova.objects.BlockDeviceMappingList.get_by_instance_uuid', + return_value=objects.BlockDeviceMappingList()) + def test_rollback_live_migration_network_teardown_fails( + self, mock_bdms, mock_get_node): + """Tests that _rollback_live_migration calls setup_networks_on_host + directly, which raises an exception, and the migration record status + is still set to 'error' before re-raising the error. + """ + ctxt = context.get_admin_context() + instance = fake_instance.fake_instance_obj(ctxt) + migration = objects.Migration(ctxt, uuid=uuids.migration) + migrate_data = objects.LibvirtLiveMigrateData(migration=migration) + source_bdms = objects.BlockDeviceMappingList() - @mock.patch.object(self.compute, '_revert_allocation') + @mock.patch.object(self.compute, '_notify_about_instance_usage') @mock.patch('nova.compute.utils.notify_about_instance_action') - @mock.patch.object(self.compute, '_live_migration_cleanup_flags') - @mock.patch.object(self.compute, 'network_api') - def _test(mock_nw_api, mock_lmcf, mock_notify, mock_ra): - mock_lmcf.return_value = False, False - self.compute._rollback_live_migration(c, instance, 'foo', - migrate_data=migrate_data, - migration_status='fake') - mock_ra.assert_called_once_with(mock.ANY, instance, migration) - mock_notify.assert_has_calls([ - mock.call(c, instance, self.compute.host, - action='live_migration_rollback', phase='start', - bdms=bdms), - mock.call(c, instance, self.compute.host, - action='live_migration_rollback', phase='end', - bdms=bdms)]) - mock_nw_api.setup_networks_on_host.assert_called_once_with( - c, instance, self.compute.host) - _test() + @mock.patch.object(instance, 'save') + @mock.patch.object(migration, 'save') + @mock.patch.object(self.compute, '_revert_allocation') + @mock.patch.object(self.compute, '_live_migration_cleanup_flags', + return_value=(False, False)) + @mock.patch.object(self.compute.network_api, 'setup_networks_on_host', + side_effect=(None, test.TestingException)) + def _test(mock_nw_setup, _mock_lmcf, mock_ra, mock_mig_save, + mock_inst_save, _mock_notify_action, mock_notify_usage): + self.assertRaises(test.TestingException, + self.compute._rollback_live_migration, + ctxt, instance, 'dest-host', migrate_data, + migration_status='goofballs', + source_bdms=source_bdms) + # setup_networks_on_host is called twice: + # - once to re-setup networking on the source host, which for + # neutron doesn't actually do anything since the port's host + # binding didn't change since live migration failed + # - once to teardown the 'migrating_to' information in the port + # binding profile, where migrating_to points at the destination + # host (that's done in pre_live_migration on the dest host). This + # cleanup would happen in rollback_live_migration_at_destination + # except _live_migration_cleanup_flags returned False for + # 'do_cleanup'. + mock_nw_setup.assert_has_calls([ + mock.call(ctxt, instance, self.compute.host), + mock.call(ctxt, instance, teardown=True) + ]) + mock_ra.assert_called_once_with(ctxt, instance, migration) + mock_mig_save.assert_called_once_with() + # Since we failed during rollback, the migration status gets set + # to 'error' instead of 'goofballs'. + self.assertEqual('error', migration.status) - self.assertEqual('fake', migration.status) - migration.save.assert_called_once_with() + _test() @mock.patch.object(fake.FakeDriver, 'rollback_live_migration_at_destination') @@ -7947,7 +8058,10 @@ self.compute.confirm_resize(self.context, instance=instance, migration=migration) - def test_allow_confirm_resize_on_instance_in_deleting_task_state(self): + @mock.patch.object(objects.MigrationContext, + 'get_pci_mapping_for_migration') + def test_allow_confirm_resize_on_instance_in_deleting_task_state( + self, mock_pci_mapping): instance = self._create_fake_instance_obj() old_type = instance.flavor new_type = flavors.get_flavor_by_flavor_id('4') @@ -7955,6 +8069,7 @@ instance.flavor = new_type instance.old_flavor = old_type instance.new_flavor = new_type + instance.migration_context = objects.MigrationContext() fake_rt = mock.MagicMock() diff -Nru nova-17.0.10/nova/tests/unit/compute/test_resource_tracker.py nova-17.0.11/nova/tests/unit/compute/test_resource_tracker.py --- nova-17.0.10/nova/tests/unit/compute/test_resource_tracker.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/compute/test_resource_tracker.py 2019-07-10 21:45:12.000000000 +0000 @@ -568,6 +568,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -608,6 +609,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -655,6 +657,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -718,6 +721,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -780,6 +784,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -839,6 +844,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -895,6 +901,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() @mock.patch('nova.objects.InstancePCIRequests.get_by_instance', return_value=objects.InstancePCIRequests(requests=[])) @@ -963,6 +970,7 @@ actual_resources = update_mock.call_args[0][1] self.assertTrue(obj_base.obj_equal_prims(expected_resources, actual_resources)) + update_mock.assert_called_once() class TestInitComputeNode(BaseTestCase): @@ -988,7 +996,7 @@ self.assertFalse(get_mock.called) self.assertFalse(create_mock.called) self.assertTrue(pci_mock.called) - self.assertTrue(update_mock.called) + self.assertFalse(update_mock.called) @mock.patch('nova.objects.PciDeviceList.get_by_compute_node', return_value=objects.PciDeviceList()) @@ -1012,7 +1020,7 @@ get_mock.assert_called_once_with(mock.sentinel.ctx, _HOSTNAME, _NODENAME) self.assertFalse(create_mock.called) - self.assertTrue(update_mock.called) + self.assertFalse(update_mock.called) @mock.patch('nova.objects.ComputeNodeList.get_by_hypervisor') @mock.patch('nova.objects.PciDeviceList.get_by_compute_node', @@ -1177,7 +1185,7 @@ self.assertTrue(obj_base.obj_equal_prims(expected_compute, cn)) pci_tracker_mock.assert_called_once_with(mock.sentinel.ctx, 42) - self.assertTrue(update_mock.called) + self.assertFalse(update_mock.called) class TestUpdateComputeNode(BaseTestCase): @@ -1333,6 +1341,41 @@ self.assertRaises(exc.ComputeHostNotFound, self.rt.get_node_uuid, 'foo') + def test_copy_resources_no_update_allocation_ratios(self): + """Tests that a ComputeNode object's allocation ratio fields are + not set if the configured allocation ratio values are default 0.0. + """ + self._setup_rt() + compute = _COMPUTE_NODE_FIXTURES[0].obj_clone() + compute.obj_reset_changes() # make sure we start clean + self.rt._copy_resources( + compute, self.driver_mock.get_available_resource.return_value) + # Assert that the ComputeNode fields were not changed. + changes = compute.obj_get_changes() + for res in ('cpu', 'disk', 'ram'): + attr_name = '%s_allocation_ratio' % res + self.assertNotIn(attr_name, changes) + + def test_copy_resources_update_allocation_ratios_from_config(self): + """Tests that a ComputeNode object's allocation ratio fields are + set if the configured allocation ratio values are not 0.0. + """ + # Set explicit ratio config values to 1.0 (the default is 0.0). + for res in ('cpu', 'disk', 'ram'): + opt_name = '%s_allocation_ratio' % res + CONF.set_override(opt_name, 1.0) + self._setup_rt() + compute = _COMPUTE_NODE_FIXTURES[0].obj_clone() + compute.obj_reset_changes() # make sure we start clean + self.rt._copy_resources( + compute, self.driver_mock.get_available_resource.return_value) + # Assert that the ComputeNode fields were changed. + changes = compute.obj_get_changes() + for res in ('cpu', 'disk', 'ram'): + attr_name = '%s_allocation_ratio' % res + self.assertIn(attr_name, changes) + self.assertEqual(1.0, changes[attr_name]) + class TestNormalizatInventoryFromComputeNode(test.NoDBTestCase): def test_normalize_libvirt(self): diff -Nru nova-17.0.10/nova/tests/unit/conductor/tasks/test_live_migrate.py nova-17.0.11/nova/tests/unit/conductor/tasks/test_live_migrate.py --- nova-17.0.10/nova/tests/unit/conductor/tasks/test_live_migrate.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/conductor/tasks/test_live_migrate.py 2019-07-10 21:45:12.000000000 +0000 @@ -18,6 +18,7 @@ from nova.compute import rpcapi as compute_rpcapi from nova.compute import vm_states from nova.conductor.tasks import live_migrate +from nova import context as nova_context from nova import exception from nova import objects from nova.scheduler import client as scheduler_client @@ -38,7 +39,7 @@ class LiveMigrationTaskTestCase(test.NoDBTestCase): def setUp(self): super(LiveMigrationTaskTestCase, self).setUp() - self.context = "context" + self.context = nova_context.get_admin_context() self.instance_host = "host" self.instance_uuid = uuids.instance self.instance_image = "image_ref" @@ -52,6 +53,7 @@ self.instance = objects.Instance._from_db_object( self.context, objects.Instance(), db_instance) self.instance.system_metadata = {'image_hw_disk_bus': 'scsi'} + self.instance.numa_topology = None self.destination = "destination" self.block_migration = "bm" self.disk_over_commit = "doc" @@ -66,7 +68,9 @@ servicegroup.API(), scheduler_client.SchedulerClient(), self.fake_spec) - def test_execute_with_destination(self, new_mode=True): + @mock.patch('nova.availability_zones.get_host_availability_zone', + return_value='fake-az') + def test_execute_with_destination(self, mock_get_az, new_mode=True): dest_node = objects.ComputeNode(hypervisor_hostname='dest_node') with test.nested( mock.patch.object(self.task, '_check_host_is_up'), @@ -107,6 +111,8 @@ migration=self.migration, migrate_data=None) self.assertTrue(mock_save.called) + mock_get_az.assert_called_once_with(self.context, self.destination) + self.assertEqual('fake-az', self.instance.availability_zone) # make sure the source/dest fields were set on the migration object self.assertEqual(self.instance.node, self.migration.source_node) self.assertEqual(dest_node.hypervisor_hostname, @@ -123,7 +129,9 @@ def test_execute_with_destination_old_school(self): self.test_execute_with_destination(new_mode=False) - def test_execute_without_destination(self): + @mock.patch('nova.availability_zones.get_host_availability_zone', + return_value='nova') + def test_execute_without_destination(self, mock_get_az): self.destination = None self._generate_task() self.assertIsNone(self.task.destination) @@ -155,6 +163,7 @@ migration=self.migration, migrate_data=None) self.assertTrue(mock_save.called) + mock_get_az.assert_called_once_with(self.context, 'found_host') self.assertEqual('found_host', self.migration.dest_compute) self.assertEqual('found_node', self.migration.dest_node) self.assertEqual(self.instance.node, self.migration.source_node) @@ -169,6 +178,45 @@ self.assertRaises(exception.InstanceInvalidState, self.task._check_instance_is_active) + @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename') + def test_check_instance_has_no_numa_passes_no_numa(self, mock_get): + self.flags(enable_numa_live_migration=False, group='workarounds') + self.task.instance.numa_topology = None + mock_get.return_value = objects.ComputeNode( + uuid=uuids.cn1, hypervisor_type='kvm') + self.task._check_instance_has_no_numa() + + @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename') + def test_check_instance_has_no_numa_passes_non_kvm(self, mock_get): + self.flags(enable_numa_live_migration=False, group='workarounds') + self.task.instance.numa_topology = objects.InstanceNUMATopology( + cells=[objects.InstanceNUMACell(id=0, cpuset=set([0]), + memory=1024)]) + mock_get.return_value = objects.ComputeNode( + uuid=uuids.cn1, hypervisor_type='xen') + self.task._check_instance_has_no_numa() + + @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename') + def test_check_instance_has_no_numa_passes_workaround(self, mock_get): + self.flags(enable_numa_live_migration=True, group='workarounds') + self.task.instance.numa_topology = objects.InstanceNUMATopology( + cells=[objects.InstanceNUMACell(id=0, cpuset=set([0]), + memory=1024)]) + mock_get.return_value = objects.ComputeNode( + uuid=uuids.cn1, hypervisor_type='kvm') + self.task._check_instance_has_no_numa() + + @mock.patch.object(objects.ComputeNode, 'get_by_host_and_nodename') + def test_check_instance_has_no_numa_fails(self, mock_get): + self.flags(enable_numa_live_migration=False, group='workarounds') + mock_get.return_value = objects.ComputeNode( + uuid=uuids.cn1, hypervisor_type='QEMU') + self.task.instance.numa_topology = objects.InstanceNUMATopology( + cells=[objects.InstanceNUMACell(id=0, cpuset=set([0]), + memory=1024)]) + self.assertRaises(exception.MigrationPreCheckError, + self.task._check_instance_has_no_numa) + @mock.patch.object(objects.Service, 'get_by_compute_host') @mock.patch.object(servicegroup.API, 'service_is_up') def test_check_instance_host_is_up(self, mock_is_up, mock_get): diff -Nru nova-17.0.10/nova/tests/unit/conductor/test_conductor.py nova-17.0.11/nova/tests/unit/conductor/test_conductor.py --- nova-17.0.10/nova/tests/unit/conductor/test_conductor.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/conductor/test_conductor.py 2019-07-10 21:45:12.000000000 +0000 @@ -1525,7 +1525,7 @@ instance=inst_obj, **compute_args) - def test_rebuild_instance_with_request_spec(self): + def test_evacuate_instance_with_request_spec(self): inst_obj = self._create_fake_instance_obj() inst_obj.host = 'noselect' expected_host = 'thebesthost' @@ -1533,10 +1533,10 @@ expected_limits = None fake_selection = objects.Selection(service_host=expected_host, nodename=expected_node, limits=None) - fake_spec = objects.RequestSpec(ignore_hosts=[]) + fake_spec = objects.RequestSpec(ignore_hosts=[uuids.ignored_host]) rebuild_args, compute_args = self._prepare_rebuild_args( {'host': None, 'node': expected_node, 'limits': expected_limits, - 'request_spec': fake_spec}) + 'request_spec': fake_spec, 'recreate': True}) with test.nested( mock.patch.object(self.conductor_manager.compute_rpcapi, 'rebuild_instance'), @@ -1550,10 +1550,9 @@ self.conductor_manager.rebuild_instance(context=self.context, instance=inst_obj, **rebuild_args) - if rebuild_args['recreate']: - reset_fd.assert_called_once_with() - else: - reset_fd.assert_not_called() + reset_fd.assert_called_once_with() + # The RequestSpec.ignore_hosts field should be overwritten. + self.assertEqual([inst_obj.host], fake_spec.ignore_hosts) select_dest_mock.assert_called_once_with(self.context, fake_spec, [inst_obj.uuid], return_objects=True, return_alternates=False) @@ -2047,6 +2046,60 @@ self.assertTrue(mock_build.called) + @mock.patch('nova.objects.InstanceMapping.get_by_instance_uuid') + def test_cleanup_build_artifacts(self, inst_map_get): + """Simple test to ensure the order of operations in the cleanup method + is enforced. + """ + req_spec = fake_request_spec.fake_spec_obj() + build_req = fake_build_request.fake_req_obj(self.context) + instance = build_req.instance + bdms = objects.BlockDeviceMappingList(objects=[ + objects.BlockDeviceMapping(instance_uuid=instance.uuid)]) + tags = objects.TagList(objects=[objects.Tag(tag='test')]) + cell1 = self.cell_mappings['cell1'] + cell_mapping_cache = {instance.uuid: cell1} + err = exc.TooManyInstances('test') + + # We need to assert that BDMs and tags are created in the cell DB + # before the instance mapping is updated. + def fake_create_block_device_mapping(*args, **kwargs): + inst_map_get.return_value.save.assert_not_called() + + def fake_create_tags(*args, **kwargs): + inst_map_get.return_value.save.assert_not_called() + + with test.nested( + mock.patch.object(self.conductor_manager, + '_set_vm_state_and_notify'), + mock.patch.object(self.conductor_manager, + '_create_block_device_mapping', + side_effect=fake_create_block_device_mapping), + mock.patch.object(self.conductor_manager, '_create_tags', + side_effect=fake_create_tags), + mock.patch.object(build_req, 'destroy'), + mock.patch.object(req_spec, 'destroy'), + ) as ( + _set_vm_state_and_notify, _create_block_device_mapping, + _create_tags, build_req_destroy, req_spec_destroy, + ): + self.conductor_manager._cleanup_build_artifacts( + self.context, err, [instance], [build_req], [req_spec], bdms, + tags, cell_mapping_cache) + # Assert the various mock calls. + _set_vm_state_and_notify.assert_called_once_with( + test.MatchType(context.RequestContext), instance.uuid, + 'build_instances', + {'vm_state': vm_states.ERROR, 'task_state': None}, err, req_spec) + _create_block_device_mapping.assert_called_once_with( + cell1, instance.flavor, instance.uuid, bdms) + _create_tags.assert_called_once_with( + test.MatchType(context.RequestContext), instance.uuid, tags) + inst_map_get.return_value.save.assert_called_once_with() + self.assertEqual(cell1, inst_map_get.return_value.cell_mapping) + build_req_destroy.assert_called_once_with() + req_spec_destroy.assert_called_once_with() + @mock.patch('nova.objects.CellMapping.get_by_uuid') def test_bury_in_cell0_no_cell0(self, mock_cm_get): mock_cm_get.side_effect = exc.CellMappingNotFound(uuid='0') diff -Nru nova-17.0.10/nova/tests/unit/db/test_db_api.py nova-17.0.11/nova/tests/unit/db/test_db_api.py --- nova-17.0.10/nova/tests/unit/db/test_db_api.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/db/test_db_api.py 2019-07-10 21:45:12.000000000 +0000 @@ -44,6 +44,7 @@ from sqlalchemy import Integer from sqlalchemy import MetaData from sqlalchemy.orm import query +from sqlalchemy.orm import session as sqla_session from sqlalchemy import sql from sqlalchemy import Table @@ -3004,21 +3005,50 @@ test(self.ctxt) def test_instance_update_and_get_original_conflict_race(self): - # Ensure that we retry if update_on_match fails for no discernable - # reason - instance = self.create_instance_with_args() + # Ensure that we correctly process expected_task_state when retrying + # due to an unknown conflict - orig_update_on_match = update_match.update_on_match + # This requires modelling the MySQL read view, which means that if we + # have read something in the current transaction and we read it again, + # we will read the same data every time even if another committed + # transaction has since altered that data. In this test we have an + # instance whose task state was originally None, but has been set to + # SHELVING by another, concurrent transaction. Therefore the first time + # we read the data we will read None, but when we restart the + # transaction we will read the correct data. - # Reproduce the conditions of a race between fetching and updating the - # instance by making update_on_match fail for no discernable reason the - # first time it is called, but work normally the second time. - with mock.patch.object(update_match, 'update_on_match', - side_effect=[update_match.NoRowsMatched, - orig_update_on_match]): - db.instance_update_and_get_original( - self.ctxt, instance['uuid'], {'metadata': {'mk1': 'mv3'}}) - self.assertEqual(update_match.update_on_match.call_count, 2) + instance = self.create_instance_with_args( + task_state=task_states.SHELVING) + + instance_out_of_date = copy.copy(instance) + instance_out_of_date['task_state'] = None + + # NOTE(mdbooth): SQLA magic which makes this dirty object look + # like a freshly loaded one. + sqla_session.make_transient(instance_out_of_date) + sqla_session.make_transient_to_detached(instance_out_of_date) + + # update_on_match will fail first time because the actual task state + # (SHELVING) doesn't match the expected task state (None). However, + # we ensure that the first time we fetch the instance object we get + # out-of-date data. This forces us to retry the operation to find out + # what really went wrong. + with mock.patch.object(sqlalchemy_api, '_instance_get_by_uuid', + side_effect=[instance_out_of_date, instance]), \ + mock.patch.object(sqlalchemy_api, '_instance_update', + side_effect=sqlalchemy_api._instance_update): + self.assertRaises(exception.UnexpectedTaskStateError, + db.instance_update_and_get_original, + self.ctxt, instance['uuid'], + {'expected_task_state': [None]}) + sqlalchemy_api._instance_update.assert_has_calls([ + mock.call(self.ctxt, instance['uuid'], + {'expected_task_state': [None]}, None, + original=instance_out_of_date), + mock.call(self.ctxt, instance['uuid'], + {'expected_task_state': [None]}, None, + original=instance), + ]) def test_instance_update_and_get_original_conflict_race_fallthrough(self): # Ensure that is update_match continuously fails for no discernable @@ -3305,6 +3335,20 @@ self.assertEqual({}, db.instance_metadata_get(ctxt, inst_uuid)) self.assertEqual([], db.instance_tag_get_by_instance_uuid( ctxt, inst_uuid)) + + @sqlalchemy_api.pick_context_manager_reader + def _assert_instance_id_mapping(_ctxt): + # NOTE(mriedem): We can't use ec2_instance_get_by_uuid to assert + # the instance_id_mappings record is gone because it hard-codes + # read_deleted='yes' and will read the soft-deleted record. So we + # do the model_query directly here. See bug 1061166. + inst_id_mapping = sqlalchemy_api.model_query( + _ctxt, models.InstanceIdMapping).filter_by( + uuid=inst_uuid).first() + self.assertFalse(inst_id_mapping, + 'instance_id_mapping not deleted for ' + 'instance: %s' % inst_uuid) + _assert_instance_id_mapping(ctxt) ctxt.read_deleted = 'yes' self.assertEqual(values['system_metadata'], db.instance_system_metadata_get(ctxt, inst_uuid)) diff -Nru nova-17.0.10/nova/tests/unit/image/test_glance.py nova-17.0.11/nova/tests/unit/image/test_glance.py --- nova-17.0.10/nova/tests/unit/image/test_glance.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/image/test_glance.py 2019-07-10 21:45:12.000000000 +0000 @@ -408,13 +408,13 @@ self.flags(api_servers=api_servers, group='glance') def assert_retry_attempted(self, sleep_mock, client, expected_url): - client.call(self.ctx, 1, 'get', 'meow') + client.call(self.ctx, 1, 'get', args=('meow',)) sleep_mock.assert_called_once_with(1) self.assertEqual(str(client.api_server), expected_url) def assert_retry_not_attempted(self, sleep_mock, client): self.assertRaises(exception.GlanceConnectionFailed, - client.call, self.ctx, 1, 'get', 'meow') + client.call, self.ctx, 1, 'get', args=('meow',)) self.assertFalse(sleep_mock.called) @mock.patch('time.sleep') @@ -488,12 +488,12 @@ # sleep (which would be an indication of a retry) self.assertRaises(exception.GlanceConnectionFailed, - client.call, self.ctx, 1, 'get', 'meow') + client.call, self.ctx, 1, 'get', args=('meow',)) self.assertEqual(str(client.api_server), 'http://host1:9292') self.assertFalse(sleep_mock.called) self.assertRaises(exception.GlanceConnectionFailed, - client.call, self.ctx, 1, 'get', 'meow') + client.call, self.ctx, 1, 'get', args=('meow',)) self.assertEqual(str(client.api_server), 'https://host2:9293') self.assertFalse(sleep_mock.called) @@ -512,6 +512,33 @@ create_client_mock.return_value = client_mock +class TestCommonPropertyNameConflicts(test.NoDBTestCase): + + """Tests that images that have common property names like "version" don't + cause an exception to be raised from the wacky GlanceClientWrapper magic + call() method. + + :see https://bugs.launchpad.net/nova/+bug/1717547 + """ + + @mock.patch('nova.image.glance.GlanceClientWrapper._create_onetime_client') + def test_version_property_conflicts(self, mock_glance_client): + client = mock.MagicMock() + mock_glance_client.return_value = client + ctx = mock.sentinel.ctx + service = glance.GlanceImageServiceV2() + + # Simulate the process of snapshotting a server that was launched with + # an image with the properties collection containing a (very + # commonly-named) "version" property. + image_meta = { + 'id': 1, + 'version': 'blows up', + } + # This call would blow up before the fix for 1717547 + service.create(ctx, image_meta) + + class TestDownloadNoDirectUri(test.NoDBTestCase): """Tests the download method of the GlanceImageServiceV2 when the @@ -530,8 +557,8 @@ self.assertFalse(show_mock.called) self.assertFalse(open_mock.called) - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) self.assertEqual(mock.sentinel.image_chunks, res) @mock.patch.object(six.moves.builtins, 'open') @@ -546,8 +573,8 @@ self.assertFalse(show_mock.called) self.assertFalse(open_mock.called) - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) self.assertIsNone(res) data.write.assert_has_calls( [ @@ -573,8 +600,8 @@ dst_path=mock.sentinel.dst_path) self.assertFalse(show_mock.called) - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) open_mock.assert_called_once_with(mock.sentinel.dst_path, 'wb') fsync_mock.assert_called_once_with(writer) self.assertIsNone(res) @@ -604,8 +631,8 @@ self.assertFalse(show_mock.called) self.assertFalse(open_mock.called) - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) self.assertIsNone(res) data.write.assert_has_calls( [ @@ -718,8 +745,8 @@ tran_mod.download.assert_called_once_with(ctx, mock.ANY, mock.sentinel.dst_path, mock.sentinel.loc_meta) - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) fsync_mock.assert_called_once_with(writer) # NOTE(jaypipes): log messages call open() in part of the # download path, so here, we just check that the last open() @@ -767,8 +794,8 @@ mock.sentinel.image_id, include_locations=True) get_tran_mock.assert_called_once_with('file') - client.call.assert_called_once_with(ctx, 2, 'data', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'data', args=(mock.sentinel.image_id,)) fsync_mock.assert_called_once_with(writer) # NOTE(jaypipes): log messages call open() in part of the # download path, so here, we just check that the last open() @@ -1044,8 +1071,8 @@ service = glance.GlanceImageServiceV2(client) info = service.show(ctx, mock.sentinel.image_id) - client.call.assert_called_once_with(ctx, 2, 'get', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(mock.sentinel.image_id,)) is_avail_mock.assert_called_once_with(ctx, {}) trans_from_mock.assert_called_once_with({}, include_locations=False) self.assertIn('mock', info) @@ -1063,8 +1090,8 @@ with testtools.ExpectedException(exception.ImageNotFound): service.show(ctx, mock.sentinel.image_id) - client.call.assert_called_once_with(ctx, 2, 'get', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(mock.sentinel.image_id,)) is_avail_mock.assert_called_once_with(ctx, mock.sentinel.images_0) self.assertFalse(trans_from_mock.called) @@ -1082,8 +1109,8 @@ with testtools.ExpectedException(exception.ImageNotAuthorized): service.show(ctx, mock.sentinel.image_id) - client.call.assert_called_once_with(ctx, 2, 'get', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(mock.sentinel.image_id,)) self.assertFalse(is_avail_mock.called) self.assertFalse(trans_from_mock.called) reraise_mock.assert_called_once_with(mock.sentinel.image_id) @@ -1118,8 +1145,8 @@ ctx = mock.sentinel.ctx service = glance.GlanceImageServiceV2(client) image_info = service.show(ctx, glance_image.id) - client.call.assert_called_once_with(ctx, 2, 'get', - glance_image.id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(glance_image.id,)) NOVA_IMAGE_ATTRIBUTES = set(['size', 'disk_format', 'owner', 'container_format', 'status', 'id', 'name', 'created_at', 'updated_at', @@ -1143,7 +1170,8 @@ image_id = mock.sentinel.image_id info = service.show(ctx, image_id, include_locations=True) - client.call.assert_called_once_with(ctx, 2, 'get', image_id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(image_id,)) avail_mock.assert_called_once_with(ctx, mock.sentinel.image) trans_from_mock.assert_called_once_with(mock.sentinel.image, include_locations=True) @@ -1165,7 +1193,8 @@ image_id = mock.sentinel.image_id info = service.show(ctx, image_id, include_locations=True) - client.call.assert_called_once_with(ctx, 2, 'get', image_id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(image_id,)) expected = locations expected.append({'url': mock.sentinel.duri, 'metadata': {}}) self.assertIn('locations', info) @@ -1189,8 +1218,8 @@ with testtools.ExpectedException(exception.ImageNotFound): service.show(ctx, glance_image.id, show_deleted=False) - client.call.assert_called_once_with(ctx, 2, 'get', - glance_image.id) + client.call.assert_called_once_with( + ctx, 2, 'get', args=(glance_image.id,)) self.assertFalse(is_avail_mock.called) self.assertFalse(trans_from_mock.called) @@ -1214,7 +1243,7 @@ service = glance.GlanceImageServiceV2(client) images = service.detail(ctx, **params) - client.call.assert_called_once_with(ctx, 2, 'list') + client.call.assert_called_once_with(ctx, 2, 'list', kwargs={}) is_avail_mock.assert_called_once_with(ctx, mock.sentinel.images_0) trans_from_mock.assert_called_once_with(mock.sentinel.images_0) self.assertEqual([mock.sentinel.trans_from], images) @@ -1234,7 +1263,7 @@ service = glance.GlanceImageServiceV2(client) images = service.detail(ctx, **params) - client.call.assert_called_once_with(ctx, 2, 'list') + client.call.assert_called_once_with(ctx, 2, 'list', kwargs={}) is_avail_mock.assert_called_once_with(ctx, mock.sentinel.images_0) self.assertFalse(trans_from_mock.called) self.assertEqual([], images) @@ -1248,10 +1277,8 @@ service = glance.GlanceImageServiceV2(client) service.detail(ctx, page_size=5, limit=10) - client.call.assert_called_once_with(ctx, 2, 'list', - filters={}, - page_size=5, - limit=10) + client.call.assert_called_once_with( + ctx, 2, 'list', kwargs=dict(filters={}, page_size=5, limit=10)) @mock.patch('nova.image.glance._reraise_translated_exception') @mock.patch('nova.image.glance._extract_query_params_v2') @@ -1271,7 +1298,7 @@ with testtools.ExpectedException(exception.Forbidden): service.detail(ctx, **params) - client.call.assert_called_once_with(ctx, 2, 'list') + client.call.assert_called_once_with(ctx, 2, 'list', kwargs={}) self.assertFalse(is_avail_mock.called) self.assertFalse(trans_from_mock.called) reraise_mock.assert_called_once_with() @@ -1301,8 +1328,8 @@ # the call to glanceclient's update (since the image ID is # supplied as a positional arg), and that the # purge_props default is True. - client.call.assert_called_once_with(ctx, 2, 'create', - name=mock.sentinel.name) + client.call.assert_called_once_with( + ctx, 2, 'create', kwargs=dict(name=mock.sentinel.name)) trans_from_mock.assert_called_once_with({'id': '123'}) self.assertEqual(mock.sentinel.trans_from, image_meta) @@ -1338,7 +1365,7 @@ image_meta = service.create(ctx, image_mock) trans_to_mock.assert_called_once_with(image_mock) # Verify that the disk_format and container_format kwargs are passed. - create_call_kwargs = client.call.call_args_list[0][1] + create_call_kwargs = client.call.call_args_list[0][1]['kwargs'] self.assertEqual('vdi', create_call_kwargs['disk_format']) self.assertEqual('bare', create_call_kwargs['container_format']) trans_from_mock.assert_called_once_with({'id': '123'}) @@ -1395,7 +1422,7 @@ mock.sentinel.ctx) self.assertEqual(expected_disk_format, disk_format) mock_client.call.assert_called_once_with( - mock.sentinel.ctx, 2, 'get', 'image', controller='schemas') + mock.sentinel.ctx, 2, 'get', args=('image',), controller='schemas') def test_get_image_create_disk_format_default_no_schema(self): """Tests that if there is no disk_format schema we default to qcow2. @@ -1486,11 +1513,11 @@ # the call to glanceclient's update (since the image ID is # supplied as a positional arg), and that the # purge_props default is True. - client.call.assert_called_once_with(ctx, 2, 'update', - image_id=mock.sentinel.image_id, - name=mock.sentinel.name, - prop_to_keep='4', - remove_props=['prop_to_remove']) + client.call.assert_called_once_with( + ctx, 2, 'update', kwargs=dict( + image_id=mock.sentinel.image_id, name=mock.sentinel.name, + prop_to_keep='4', remove_props=['prop_to_remove'], + )) trans_from_mock.assert_called_once_with(mock.sentinel.image_meta) self.assertEqual(mock.sentinel.trans_from, image_meta) @@ -1562,11 +1589,13 @@ self.assertRaises(exception.ImageNotAuthorized, service.update, ctx, mock.sentinel.image_id, image) - client.call.assert_called_once_with(ctx, 2, 'update', - image_id=mock.sentinel.image_id, - name=mock.sentinel.name, - prop_to_keep='4', - remove_props=['prop_to_remove']) + client.call.assert_called_once_with( + ctx, 2, 'update', kwargs=dict( + image_id=mock.sentinel.image_id, + name=mock.sentinel.name, + prop_to_keep='4', + remove_props=['prop_to_remove'], + )) reraise_mock.assert_called_once_with(mock.sentinel.image_id) @@ -1580,8 +1609,8 @@ ctx = mock.sentinel.ctx service = glance.GlanceImageServiceV2(client) service.delete(ctx, mock.sentinel.image_id) - client.call.assert_called_once_with(ctx, 2, 'delete', - mock.sentinel.image_id) + client.call.assert_called_once_with( + ctx, 2, 'delete', args=(mock.sentinel.image_id,)) def test_delete_client_failure_v2(self): client = mock.MagicMock() @@ -1710,10 +1739,12 @@ 'kernel-id': 'some-id', 'updated_at': 'gte:some-date'} - client.call.assert_called_once_with(ctx, 2, 'list', - filters=expected_filters_v1, - page_size=5, - limit=10) + client.call.assert_called_once_with( + ctx, 2, 'list', kwargs=dict( + filters=expected_filters_v1, + page_size=5, + limit=10, + )) class TestTranslateToGlance(test.NoDBTestCase): diff -Nru nova-17.0.10/nova/tests/unit/network/test_neutronv2.py nova-17.0.11/nova/tests/unit/network/test_neutronv2.py --- nova-17.0.10/nova/tests/unit/network/test_neutronv2.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/network/test_neutronv2.py 2019-07-10 21:45:12.000000000 +0000 @@ -4143,56 +4143,29 @@ def test_get_pci_mapping_for_migration(self): instance = fake_instance.fake_instance_obj(self.context) instance.migration_context = objects.MigrationContext() - old_pci_devices = objects.PciDeviceList( - objects=[objects.PciDevice(vendor_id='1377', - product_id='0047', - address='0000:0a:00.1', - compute_node_id=1, - request_id='1234567890')]) - - new_pci_devices = objects.PciDeviceList( - objects=[objects.PciDevice(vendor_id='1377', - product_id='0047', - address='0000:0b:00.1', - compute_node_id=2, - request_id='1234567890')]) - - instance.migration_context.old_pci_devices = old_pci_devices - instance.migration_context.new_pci_devices = new_pci_devices - instance.pci_devices = instance.migration_context.old_pci_devices migration = {'status': 'confirmed'} - pci_mapping = self.api._get_pci_mapping_for_migration( - self.context, instance, migration) - self.assertEqual( - {old_pci_devices[0].address: new_pci_devices[0]}, pci_mapping) + with mock.patch.object(instance.migration_context, + 'get_pci_mapping_for_migration') as map_func: + self.api._get_pci_mapping_for_migration(instance, migration) + map_func.assert_called_with(False) def test_get_pci_mapping_for_migration_reverted(self): instance = fake_instance.fake_instance_obj(self.context) instance.migration_context = objects.MigrationContext() - old_pci_devices = objects.PciDeviceList( - objects=[objects.PciDevice(vendor_id='1377', - product_id='0047', - address='0000:0a:00.1', - compute_node_id=1, - request_id='1234567890')]) - - new_pci_devices = objects.PciDeviceList( - objects=[objects.PciDevice(vendor_id='1377', - product_id='0047', - address='0000:0b:00.1', - compute_node_id=2, - request_id='1234567890')]) - - instance.migration_context.old_pci_devices = old_pci_devices - instance.migration_context.new_pci_devices = new_pci_devices - instance.pci_devices = instance.migration_context.old_pci_devices migration = {'status': 'reverted'} + with mock.patch.object(instance.migration_context, + 'get_pci_mapping_for_migration') as map_func: + self.api._get_pci_mapping_for_migration(instance, migration) + map_func.assert_called_with(True) + + def test_get_pci_mapping_for_migration_no_migration_context(self): + instance = fake_instance.fake_instance_obj(self.context) + instance.migration_context = None pci_mapping = self.api._get_pci_mapping_for_migration( - self.context, instance, migration) - self.assertEqual( - {new_pci_devices[0].address: old_pci_devices[0]}, pci_mapping) + instance, None) + self.assertDictEqual({}, pci_mapping) @mock.patch.object(neutronapi, 'get_client', return_value=mock.Mock()) def test_update_port_profile_for_migration_teardown_false( diff -Nru nova-17.0.10/nova/tests/unit/objects/test_migration_context.py nova-17.0.11/nova/tests/unit/objects/test_migration_context.py --- nova-17.0.10/nova/tests/unit/objects/test_migration_context.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/objects/test_migration_context.py 2019-07-10 21:45:12.000000000 +0000 @@ -52,6 +52,23 @@ return obj +def get_fake_migration_context_with_pci_devs(ctxt=None): + obj = get_fake_migration_context_obj(ctxt) + obj.old_pci_devices = objects.PciDeviceList( + objects=[objects.PciDevice(vendor_id='1377', + product_id='0047', + address='0000:0a:00.1', + compute_node_id=1, + request_id=uuids.pcidev)]) + obj.new_pci_devices = objects.PciDeviceList( + objects=[objects.PciDevice(vendor_id='1377', + product_id='0047', + address='0000:0b:00.1', + compute_node_id=2, + request_id=uuids.pcidev)]) + return obj + + class _TestMigrationContext(object): def _test_get_by_instance_uuid(self, db_data): @@ -104,7 +121,20 @@ class TestMigrationContext(test_objects._LocalTest, _TestMigrationContext): - pass + + def test_pci_mapping_for_migration(self): + mig_ctx = get_fake_migration_context_with_pci_devs() + pci_mapping = mig_ctx.get_pci_mapping_for_migration(False) + self.assertDictEqual( + {mig_ctx.old_pci_devices[0].address: mig_ctx.new_pci_devices[0]}, + pci_mapping) + + def test_pci_mapping_for_migration_revert(self): + mig_ctx = get_fake_migration_context_with_pci_devs() + pci_mapping = mig_ctx.get_pci_mapping_for_migration(True) + self.assertDictEqual( + {mig_ctx.new_pci_devices[0].address: mig_ctx.old_pci_devices[0]}, + pci_mapping) class TestMigrationContextRemote(test_objects._RemoteTest, diff -Nru nova-17.0.10/nova/tests/unit/objects/test_request_spec.py nova-17.0.11/nova/tests/unit/objects/test_request_spec.py --- nova-17.0.10/nova/tests/unit/objects/test_request_spec.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/objects/test_request_spec.py 2019-07-10 21:45:12.000000000 +0000 @@ -504,7 +504,8 @@ fake_spec['instance_uuid']) self.assertEqual(1, req_obj.num_instances) - self.assertEqual(['host2', 'host4'], req_obj.ignore_hosts) + # ignore_hosts is not persisted + self.assertIsNone(req_obj.ignore_hosts) self.assertEqual('fake', req_obj.project_id) self.assertEqual({'hint': ['over-there']}, req_obj.scheduler_hints) self.assertEqual(['host1', 'host3'], req_obj.force_hosts) @@ -527,7 +528,7 @@ jsonutils.loads(changes['spec'])) # primitive fields - for field in ['instance_uuid', 'num_instances', 'ignore_hosts', + for field in ['instance_uuid', 'num_instances', 'project_id', 'scheduler_hints', 'force_hosts', 'availability_zone', 'force_nodes']: self.assertEqual(getattr(req_obj, field), @@ -544,6 +545,7 @@ self.assertIsNone(serialized_obj.instance_group.hosts) self.assertIsNone(serialized_obj.retry) self.assertIsNone(serialized_obj.requested_destination) + self.assertIsNone(serialized_obj.ignore_hosts) def test_create(self): req_obj = fake_request_spec.fake_spec_obj(remove_id=True) @@ -564,6 +566,39 @@ self.assertRaises(exception.ObjectActionError, req_obj.create) + def test_save_does_not_persist_requested_fields(self): + req_obj = fake_request_spec.fake_spec_obj(remove_id=True) + req_obj.create() + # change something to make sure _save_in_db is called + expected_destination = request_spec.Destination(host='sample-host') + req_obj.requested_destination = expected_destination + expected_retry = objects.SchedulerRetries( + num_attempts=2, + hosts=objects.ComputeNodeList(objects=[ + objects.ComputeNode(host='host1', hypervisor_hostname='node1'), + objects.ComputeNode(host='host2', hypervisor_hostname='node2'), + ])) + req_obj.retry = expected_retry + req_obj.ignore_hosts = [uuids.ignored_host] + + orig_save_in_db = request_spec.RequestSpec._save_in_db + with mock.patch.object(request_spec.RequestSpec, '_save_in_db') \ + as mock_save_in_db: + mock_save_in_db.side_effect = orig_save_in_db + req_obj.save() + mock_save_in_db.assert_called_once() + updates = mock_save_in_db.mock_calls[0][1][2] + # assert that the following fields are not stored in the db + # 1. ignore_hosts + data = jsonutils.loads(updates['spec'])['nova_object.data'] + self.assertIsNone(data['ignore_hosts']) + self.assertIsNotNone(data['instance_uuid']) + + # also we expect that the following fields are not reset after save + # 1. ignore_hosts + self.assertIsNotNone(req_obj.ignore_hosts) + self.assertEqual([uuids.ignored_host], req_obj.ignore_hosts) + def test_save(self): req_obj = fake_request_spec.fake_spec_obj() # Make sure the requested_destination is not persisted since it is diff -Nru nova-17.0.10/nova/tests/unit/virt/ironic/test_client_wrapper.py nova-17.0.11/nova/tests/unit/virt/ironic/test_client_wrapper.py --- nova-17.0.10/nova/tests/unit/virt/ironic/test_client_wrapper.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/ironic/test_client_wrapper.py 2019-07-10 21:45:12.000000000 +0000 @@ -81,7 +81,7 @@ # nova.utils.get_ksa_adapter().get_endpoint() self.get_ksa_adapter.assert_called_once_with( 'baremetal', ksa_auth=self.get_auth_plugin.return_value, - ksa_session='session', min_version=(1, 37), + ksa_session='session', min_version=(1, 0), max_version=(1, ksa_disc.LATEST)) expected = {'session': 'session', 'max_retries': CONF.ironic.api_max_retries, @@ -106,7 +106,7 @@ # nova.utils.get_endpoint_data self.get_ksa_adapter.assert_called_once_with( 'baremetal', ksa_auth=self.get_auth_plugin.return_value, - ksa_session='session', min_version=(1, 37), + ksa_session='session', min_version=(1, 0), max_version=(1, ksa_disc.LATEST)) # When get_endpoint_data raises any ServiceNotFound, None is returned. expected = {'session': 'session', diff -Nru nova-17.0.10/nova/tests/unit/virt/libvirt/test_driver.py nova-17.0.11/nova/tests/unit/virt/libvirt/test_driver.py --- nova-17.0.10/nova/tests/unit/virt/libvirt/test_driver.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/libvirt/test_driver.py 2019-07-10 21:45:12.000000000 +0000 @@ -6954,6 +6954,66 @@ _set_cache_mode.assert_called_once_with(config) self.assertEqual(config_guest_disk.to_xml(), config.to_xml()) + @mock.patch.object(libvirt_driver.LibvirtDriver, '_get_volume_driver') + @mock.patch.object(libvirt_driver.LibvirtDriver, '_attach_encryptor') + def test_connect_volume_encryption_success( + self, mock_attach_encryptor, mock_get_volume_driver): + + drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) + mock_volume_driver = mock.MagicMock( + spec=volume_drivers.LibvirtBaseVolumeDriver) + mock_get_volume_driver.return_value = mock_volume_driver + + connection_info = {'driver_volume_type': 'fake', + 'data': {'device_path': '/fake', + 'access_mode': 'rw', + 'volume_id': uuids.volume_id}} + encryption = {'provider': encryptors.LUKS, + 'encryption_key_id': uuids.encryption_key_id} + instance = mock.sentinel.instance + + drvr._connect_volume(self.context, connection_info, instance, + encryption=encryption) + + mock_get_volume_driver.assert_called_once_with(connection_info) + mock_volume_driver.connect_volume.assert_called_once_with( + connection_info, instance) + mock_attach_encryptor.assert_called_once_with( + self.context, connection_info, encryption, True) + mock_volume_driver.disconnect_volume.assert_not_called() + + @mock.patch.object(libvirt_driver.LibvirtDriver, '_get_volume_driver') + @mock.patch.object(libvirt_driver.LibvirtDriver, '_attach_encryptor') + def test_connect_volume_encryption_fail( + self, mock_attach_encryptor, mock_get_volume_driver): + + drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) + mock_volume_driver = mock.MagicMock( + spec=volume_drivers.LibvirtBaseVolumeDriver) + mock_get_volume_driver.return_value = mock_volume_driver + + connection_info = {'driver_volume_type': 'fake', + 'data': {'device_path': '/fake', + 'access_mode': 'rw', + 'volume_id': uuids.volume_id}} + encryption = {'provider': encryptors.LUKS, + 'encryption_key_id': uuids.encryption_key_id} + instance = mock.sentinel.instance + mock_attach_encryptor.side_effect = processutils.ProcessExecutionError + + self.assertRaises(processutils.ProcessExecutionError, + drvr._connect_volume, + self.context, connection_info, instance, + encryption=encryption) + + mock_get_volume_driver.assert_called_once_with(connection_info) + mock_volume_driver.connect_volume.assert_called_once_with( + connection_info, instance) + mock_attach_encryptor.assert_called_once_with( + self.context, connection_info, encryption, True) + mock_volume_driver.disconnect_volume.assert_called_once_with( + connection_info, instance) + @mock.patch.object(key_manager, 'API') @mock.patch.object(libvirt_driver.LibvirtDriver, '_get_volume_encryption') @mock.patch.object(libvirt_driver.LibvirtDriver, '_use_native_luks') @@ -7221,11 +7281,13 @@ mock.patch.object(drvr._host, 'get_guest'), mock.patch('nova.virt.libvirt.driver.LOG'), mock.patch.object(drvr, '_connect_volume'), + mock.patch.object(drvr, '_disconnect_volume'), mock.patch.object(drvr, '_get_volume_config'), mock.patch.object(drvr, '_check_discard_for_attach_volume'), mock.patch.object(drvr, '_build_device_metadata'), ) as (mock_get_guest, mock_log, mock_connect_volume, - mock_get_volume_config, mock_check_discard, mock_build_metadata): + mock_disconnect_volume, mock_get_volume_config, + mock_check_discard, mock_build_metadata): mock_conf = mock.MagicMock() mock_guest = mock.MagicMock() @@ -7239,6 +7301,50 @@ self.context, connection_info, instance, "/dev/vdb", disk_bus=bdm['disk_bus'], device_type=bdm['device_type']) mock_log.warning.assert_called_once() + mock_disconnect_volume.assert_called_once() + + @mock.patch('nova.virt.libvirt.blockinfo.get_info_from_bdm') + def test_attach_volume_with_libvirt_exception(self, mock_get_info): + drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) + instance = objects.Instance(**self.test_instance) + connection_info = {"driver_volume_type": "fake", + "data": {"device_path": "/fake", + "access_mode": "rw"}} + bdm = {'device_name': 'vdb', + 'disk_bus': 'fake-bus', + 'device_type': 'fake-type'} + disk_info = {'bus': bdm['disk_bus'], 'type': bdm['device_type'], + 'dev': 'vdb'} + libvirt_exc = fakelibvirt.make_libvirtError(fakelibvirt.libvirtError, + "Target vdb already exists', device is busy", + error_code=fakelibvirt.VIR_ERR_INTERNAL_ERROR) + + with test.nested( + mock.patch.object(drvr._host, 'get_guest'), + mock.patch('nova.virt.libvirt.driver.LOG'), + mock.patch.object(drvr, '_connect_volume'), + mock.patch.object(drvr, '_disconnect_volume'), + mock.patch.object(drvr, '_get_volume_config'), + mock.patch.object(drvr, '_check_discard_for_attach_volume'), + mock.patch.object(drvr, '_build_device_metadata'), + ) as (mock_get_guest, mock_log, mock_connect_volume, + mock_disconnect_volume, mock_get_volume_config, + mock_check_discard, mock_build_metadata): + + mock_conf = mock.MagicMock() + mock_guest = mock.MagicMock() + mock_guest.attach_device.side_effect = libvirt_exc + mock_get_volume_config.return_value = mock_conf + mock_get_guest.return_value = mock_guest + mock_get_info.return_value = disk_info + mock_build_metadata.return_value = objects.InstanceDeviceMetadata() + + self.assertRaises(fakelibvirt.libvirtError, drvr.attach_volume, + self.context, connection_info, instance, "/dev/vdb", + disk_bus=bdm['disk_bus'], device_type=bdm['device_type']) + mock_log.exception.assert_called_once_with(u'Failed to attach ' + 'volume at mountpoint: %s', '/dev/vdb', instance=instance) + mock_disconnect_volume.assert_called_once() @mock.patch('nova.utils.get_image_from_system_metadata') @mock.patch('nova.virt.libvirt.blockinfo.get_info_from_bdm') @@ -7724,8 +7830,9 @@ @mock.patch('os_brick.encryptors.get_encryption_metadata') @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._get_volume_encryptor') + @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._use_native_luks') def test_detach_encryptor_encrypted_volume_meta_missing(self, - mock_get_encryptor, mock_get_metadata): + mock_use_native_luks, mock_get_encryptor, mock_get_metadata): """Assert that if missing the encryption metadata of an encrypted volume is fetched and then used to detach the encryptor for the volume. """ @@ -7735,6 +7842,7 @@ encryption = {'provider': 'luks', 'control_location': 'front-end'} mock_get_metadata.return_value = encryption connection_info = {'data': {'volume_id': uuids.volume_id}} + mock_use_native_luks.return_value = False drvr._detach_encryptor(self.context, connection_info, None) @@ -7746,8 +7854,9 @@ @mock.patch('os_brick.encryptors.get_encryption_metadata') @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._get_volume_encryptor') + @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._use_native_luks') def test_detach_encryptor_encrypted_volume_meta_provided(self, - mock_get_encryptor, mock_get_metadata): + mock_use_native_luks, mock_get_encryptor, mock_get_metadata): """Assert that when provided there are no further attempts to fetch the encryption metadata for the volume and that the provided metadata is then used to detach the volume. @@ -7757,6 +7866,7 @@ mock_get_encryptor.return_value = mock_encryptor encryption = {'provider': 'luks', 'control_location': 'front-end'} connection_info = {'data': {'volume_id': uuids.volume_id}} + mock_use_native_luks.return_value = False drvr._detach_encryptor(self.context, connection_info, encryption) @@ -7765,6 +7875,27 @@ encryption) mock_encryptor.detach_volume.assert_called_once_with(**encryption) + @mock.patch('nova.virt.libvirt.host.Host.find_secret') + @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._use_native_luks') + @mock.patch('nova.virt.libvirt.driver.LibvirtDriver._get_volume_encryptor') + def test_detach_encryptor_native_luks_device_path_secret_missing(self, + mock_get_encryptor, mock_use_native_luks, mock_find_secret): + """Assert that the encryptor is not built when native LUKS is + available, the associated volume secret is missing and device_path is + also missing from the connection_info. + """ + drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) + encryption = {'provider': 'luks', 'control_location': 'front-end', + 'encryption_key_id': uuids.encryption_key_id} + connection_info = {'data': {'volume_id': uuids.volume_id}} + mock_find_secret.return_value = False + mock_use_native_luks.return_value = True + + drvr._detach_encryptor(self.context, connection_info, encryption) + + mock_find_secret.assert_called_once_with('volume', uuids.volume_id) + mock_get_encryptor.assert_not_called() + @mock.patch.object(host.Host, "has_min_version") def test_use_native_luks(self, mock_has_min_version): drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) @@ -10886,23 +11017,38 @@ instance = objects.Instance(**self.test_instance) backing_file = imagecache.get_cache_fname(instance.image_ref) + backfile_path = os.path.join(base_dir, backing_file) + disk_size = 10747904 + virt_disk_size = 25165824 disk_info = [ {u'backing_file': backing_file, - u'disk_size': 10747904, + u'disk_size': disk_size, u'path': u'disk_path', u'type': u'qcow2', - u'virt_disk_size': 25165824}] + u'virt_disk_size': virt_disk_size}] + def fake_copy_image(src, dest, **kwargs): + # backing file should be present and have a smaller size + # than instance root disk in order to assert resize_image() + if dest == backfile_path: + # dest is created under TempDir() fixture, + # it will go away after test cleanup + with open(dest, 'a'): + pass with test.nested( - mock.patch.object(libvirt_driver.libvirt_utils, 'copy_image'), + mock.patch.object(libvirt_driver.libvirt_utils, 'copy_image', + side_effect=fake_copy_image), mock.patch.object(libvirt_driver.libvirt_utils, 'fetch_image', side_effect=exception.ImageNotFound( image_id=uuids.fake_id)), - ) as (copy_image_mock, fetch_image_mock): + mock.patch.object(imagebackend.Qcow2, 'resize_image'), + mock.patch.object(imagebackend.Image, 'get_disk_size', + return_value=disk_size), + ) as (copy_image_mock, fetch_image_mock, resize_image_mock, + get_disk_size_mock): conn._create_images_and_backing(self.context, instance, "/fake/instance/dir", disk_info, fallback_from_host="fake_host") - backfile_path = os.path.join(base_dir, backing_file) kernel_path = os.path.join(CONF.instances_path, self.test_instance['uuid'], 'kernel') @@ -10924,6 +11070,7 @@ mock.call(self.context, kernel_path, instance.kernel_id), mock.call(self.context, ramdisk_path, instance.ramdisk_id) ]) + resize_image_mock.assert_called_once_with(virt_disk_size) mock_utime.assert_called() @@ -14104,8 +14251,9 @@ @mock.patch('nova.objects.InstanceList.get_by_filters', return_value=objects.InstanceList(objects=[ objects.Instance(uuid=uuids.instance, + vm_state=vm_states.ACTIVE, task_state=task_states.DELETING)])) - def test_disk_over_committed_size_total_disk_not_found_ignore( + def test_disk_over_committed_size_total_disk_not_found_ignore_task_state( self, mock_get, mock_bdms, mock_get_disk_info, mock_list_domains): """Tests that we handle DiskNotFound gracefully for an instance that is undergoing a task_state transition. @@ -14125,7 +14273,32 @@ return_value=objects.BlockDeviceMappingList()) @mock.patch('nova.objects.InstanceList.get_by_filters', return_value=objects.InstanceList(objects=[ - objects.Instance(uuid=uuids.instance, task_state=None)])) + objects.Instance(uuid=uuids.instance, + task_state=None, + vm_state=vm_states.RESIZED)])) + def test_disk_over_committed_size_total_disk_not_found_ignore_vmstate( + self, mock_get, mock_bdms, mock_get_disk_info, mock_list_domains): + """Tests that we handle DiskNotFound gracefully for an instance that + is resized but resize is not confirmed yet. + """ + mock_dom = mock.Mock() + mock_dom.XMLDesc.return_value = "" + mock_dom.UUIDString.return_value = uuids.instance + mock_list_domains.return_value = [mock_dom] + drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False) + self.assertEqual(0, drvr._get_disk_over_committed_size_total()) + + @mock.patch('nova.virt.libvirt.host.Host.list_instance_domains') + @mock.patch('nova.virt.libvirt.driver.LibvirtDriver.' + '_get_instance_disk_info_from_config', + side_effect=exception.DiskNotFound(location='/opt/stack/foo')) + @mock.patch('nova.objects.BlockDeviceMappingList.bdms_by_instance_uuid', + return_value=objects.BlockDeviceMappingList()) + @mock.patch('nova.objects.InstanceList.get_by_filters', + return_value=objects.InstanceList(objects=[ + objects.Instance(uuid=uuids.instance, + vm_state=vm_states.ACTIVE, + task_state=None)])) def test_disk_over_committed_size_total_disk_not_found_reraise( self, mock_get, mock_bdms, mock_get_disk_info, mock_list_domains): """Tests that we handle DiskNotFound gracefully for an instance that @@ -14999,6 +15172,10 @@ + + + + """ @@ -15091,6 +15268,9 @@ tx_errors=0, tx_octets=0, tx_packets=0) + + expected.add_nic(mac_address='54:56:00:a6:40:40') + self.assertDiagnosticsEqual(expected, actual) @mock.patch.object(host.Host, "list_instance_domains") diff -Nru nova-17.0.10/nova/tests/unit/virt/libvirt/test_guest.py nova-17.0.11/nova/tests/unit/virt/libvirt/test_guest.py --- nova-17.0.10/nova/tests/unit/virt/libvirt/test_guest.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/libvirt/test_guest.py 2019-07-10 21:45:12.000000000 +0000 @@ -324,6 +324,28 @@ # Some time later, we can do the wait/retry to ensure detach self.assertRaises(exception.DeviceNotFound, retry_detach) + @mock.patch.object(libvirt_guest.Guest, "detach_device") + def test_detach_device_with_retry_operation_internal(self, mock_detach): + # This simulates a retry of the transient/live domain detach + # failing because the device is not found + conf = mock.Mock(spec=vconfig.LibvirtConfigGuestDevice) + conf.to_xml.return_value = "" + self.domain.isPersistent.return_value = True + + get_config = mock.Mock(return_value=conf) + fake_device = "vdb" + fake_exc = fakelibvirt.make_libvirtError( + fakelibvirt.libvirtError, "", + error_message="operation failed: disk vdb not found", + error_code=fakelibvirt.VIR_ERR_INTERNAL_ERROR, + error_domain=fakelibvirt.VIR_FROM_DOMAIN) + mock_detach.side_effect = [None, fake_exc] + retry_detach = self.guest.detach_device_with_retry( + get_config, fake_device, live=True, + inc_sleep_time=.01, max_retry_count=3) + # Some time later, we can do the wait/retry to ensure detach + self.assertRaises(exception.DeviceNotFound, retry_detach) + def test_detach_device_with_retry_invalid_argument(self): # This simulates a persistent domain detach failing because # the device is not found diff -Nru nova-17.0.10/nova/tests/unit/virt/libvirt/test_imagebackend.py nova-17.0.11/nova/tests/unit/virt/libvirt/test_imagebackend.py --- nova-17.0.10/nova/tests/unit/virt/libvirt/test_imagebackend.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/libvirt/test_imagebackend.py 2019-07-10 21:45:12.000000000 +0000 @@ -21,6 +21,7 @@ import tempfile from castellan import key_manager +import ddt import fixtures import mock from oslo_concurrency import lockutils @@ -57,6 +58,7 @@ return FakeSecret() +@ddt.ddt class _ImageTestCase(object): def mock_create_image(self, image): @@ -203,6 +205,24 @@ self.assertEqual(2361393152, image.get_disk_size(image.path)) get_disk_size.assert_called_once_with(image.path) + def _test_libvirt_info_scsi_with_unit(self, disk_unit): + # The address should be set if bus is scsi and unit is set. + # Otherwise, it should not be set at all. + image = self.image_class(self.INSTANCE, self.NAME) + disk = image.libvirt_info(disk_bus='scsi', disk_dev='/dev/sda', + device_type='disk', cache_mode='none', + extra_specs={}, hypervisor_version=4004001, + disk_unit=disk_unit) + if disk_unit: + self.assertEqual(0, disk.device_addr.controller) + self.assertEqual(disk_unit, disk.device_addr.unit) + else: + self.assertIsNone(disk.device_addr) + + @ddt.data(5, None) + def test_libvirt_info_scsi_with_unit(self, disk_unit): + self._test_libvirt_info_scsi_with_unit(disk_unit) + class FlatTestCase(_ImageTestCase, test.NoDBTestCase): @@ -1276,6 +1296,7 @@ model) +@ddt.ddt class RbdTestCase(_ImageTestCase, test.NoDBTestCase): FSID = "FakeFsID" POOL = "FakePool" @@ -1500,6 +1521,17 @@ super(RbdTestCase, self).test_libvirt_info() + @ddt.data(5, None) + @mock.patch.object(rbd_utils.RBDDriver, "get_mon_addrs") + def test_libvirt_info_scsi_with_unit(self, disk_unit, mock_mon_addrs): + def get_mon_addrs(): + hosts = ["server1", "server2"] + ports = ["1899", "1920"] + return hosts, ports + mock_mon_addrs.side_effect = get_mon_addrs + + super(RbdTestCase, self)._test_libvirt_info_scsi_with_unit(disk_unit) + @mock.patch.object(rbd_utils.RBDDriver, "get_mon_addrs") def test_get_model(self, mock_mon_addrs): pool = "FakePool" diff -Nru nova-17.0.10/nova/tests/unit/virt/libvirt/volume/test_volume.py nova-17.0.11/nova/tests/unit/virt/libvirt/volume/test_volume.py --- nova-17.0.10/nova/tests/unit/virt/libvirt/volume/test_volume.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/libvirt/volume/test_volume.py 2019-07-10 21:45:12.000000000 +0000 @@ -13,6 +13,7 @@ # License for the specific language governing permissions and limitations # under the License. +import ddt import mock from nova import exception @@ -130,6 +131,7 @@ return ret +@ddt.ddt class LibvirtVolumeTestCase(LibvirtISCSIVolumeBaseTestCase): def _assertDiskInfoEquals(self, tree, disk_info): @@ -373,3 +375,21 @@ conf = libvirt_driver.get_config(connection_info, self.disk_info) tree = conf.format_dom() self.assertIsNone(tree.find("encryption")) + + @ddt.data(5, None) + def test_libvirt_volume_driver_address_tag_scsi_unit(self, disk_unit): + # The address tag should be set if bus is scsi and unit is set. + # Otherwise, it should not be set at all. + libvirt_driver = volume.LibvirtVolumeDriver(self.fake_host) + connection_info = {'data': {'device_path': '/foo'}} + disk_info = {'bus': 'scsi', 'dev': 'sda', 'type': 'disk'} + if disk_unit: + disk_info['unit'] = disk_unit + conf = libvirt_driver.get_config(connection_info, disk_info) + tree = conf.format_dom() + address = tree.find('address') + if disk_unit: + self.assertEqual('0', address.attrib['controller']) + self.assertEqual(str(disk_unit), address.attrib['unit']) + else: + self.assertIsNone(address) diff -Nru nova-17.0.10/nova/tests/unit/virt/test_hardware.py nova-17.0.11/nova/tests/unit/virt/test_hardware.py --- nova-17.0.10/nova/tests/unit/virt/test_hardware.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/test_hardware.py 2019-07-10 21:45:12.000000000 +0000 @@ -2398,6 +2398,26 @@ got_pinning = {x: x for x in range(0, 4)} self.assertEqual(got_pinning, inst_pin.cpu_pinning) + def test_get_pinning_host_siblings_fails(self): + """Validate that pinning fails if there are no free host CPUs. + + This can happen due to raciness caused by nova-scheduler not claiming + pinned CPU resources, meaning multiple instances can race, landing on + the same host and attempting to claim the same resources. + """ + # This host has 8 cores but 2 of them are disabled via 'vcpu_pin_set', + # meaning they have no thread siblings and aren't included in + # 'siblings' below. This can break thread-aware checks, which assume a + # sum(siblings) == sum(cpus). + host_pin = objects.NUMACell(id=0, cpuset=set([0, 1, 2, 3, 4, 6]), + memory=4096, memory_usage=0, + siblings=[set([0, 1]), set([2, 3])], + mempages=[], pinned_cpus=set([0, 1, 2, 3])) + inst_pin = objects.InstanceNUMACell(cpuset=set([0, 1]), + memory=2048) + inst_pin = hw._numa_fit_instance_cell_with_pinning(host_pin, inst_pin) + self.assertIsNone(inst_pin) + def test_get_pinning_require_policy_no_siblings(self): host_pin = objects.NUMACell( id=0, diff -Nru nova-17.0.10/nova/tests/unit/virt/xenapi/test_agent.py nova-17.0.11/nova/tests/unit/virt/xenapi/test_agent.py --- nova-17.0.10/nova/tests/unit/virt/xenapi/test_agent.py 2019-03-24 23:12:23.000000000 +0000 +++ nova-17.0.11/nova/tests/unit/virt/xenapi/test_agent.py 2019-07-10 21:45:04.000000000 +0000 @@ -19,6 +19,7 @@ import mock from os_xenapi.client import host_agent from os_xenapi.client import XenAPI +from oslo_concurrency import processutils from oslo_utils import uuidutils from nova import exception @@ -311,6 +312,19 @@ mock_add_fault.assert_called_once_with(error, mock.ANY) + @mock.patch('oslo_concurrency.processutils.execute') + def test_run_ssl_successful(self, mock_execute): + mock_execute.return_value = ('0', + '*** WARNING : deprecated key derivation used.' + 'Using -iter or -pbkdf2 would be better.') + agent.SimpleDH()._run_ssl('foo') + + @mock.patch('oslo_concurrency.processutils.execute', + side_effect=processutils.ProcessExecutionError( + exit_code=1, stderr=('ERROR: Something bad happened'))) + def test_run_ssl_failure(self, mock_execute): + self.assertRaises(RuntimeError, agent.SimpleDH()._run_ssl, 'foo') + class UpgradeRequiredTestCase(test.NoDBTestCase): def test_less_than(self): diff -Nru nova-17.0.10/nova/virt/fake.py nova-17.0.11/nova/virt/fake.py --- nova-17.0.10/nova/virt/fake.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/fake.py 2019-07-10 21:45:12.000000000 +0000 @@ -544,7 +544,11 @@ return def confirm_migration(self, context, migration, instance, network_info): - return + # Confirm migration cleans up the guest from the source host so just + # destroy the guest to remove it from the list of tracked instances + # unless it is a same-host resize. + if migration.source_compute != migration.dest_compute: + self.destroy(context, instance, network_info) def pre_live_migration(self, context, instance, block_device_info, network_info, disk_info, migrate_data): diff -Nru nova-17.0.10/nova/virt/hardware.py nova-17.0.11/nova/virt/hardware.py --- nova-17.0.10/nova/virt/hardware.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/hardware.py 2019-07-10 21:45:12.000000000 +0000 @@ -693,6 +693,17 @@ for sib in available_siblings: for threads_no in range(1, len(sib) + 1): sibling_sets[threads_no].append(sib) + + # Because we don't claim pinned CPUs in the scheduler, it is possible for + # multiple instances to race and land on the same host. When this happens, + # the CPUs that we thought were free may not longer be free. We should fail + # fast in this scenario. + if not sibling_sets: + LOG.debug('No available siblings. This is likely due to a race ' + 'caused by multiple instances attempting to claim the same ' + 'resources') + return + LOG.debug('Built sibling_sets: %(siblings)s', {'siblings': sibling_sets}) pinning = None @@ -871,6 +882,7 @@ if (instance_cell.cpu_thread_policy != fields.CPUThreadAllocationPolicy.REQUIRE and not pinning): + sibling_set = sibling_sets[min(sibling_sets)] pinning = list(zip(sorted(instance_cell.cpuset), itertools.chain(*sibling_set))) diff -Nru nova-17.0.10/nova/virt/ironic/client_wrapper.py nova-17.0.11/nova/virt/ironic/client_wrapper.py --- nova-17.0.10/nova/virt/ironic/client_wrapper.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/ironic/client_wrapper.py 2019-07-10 21:45:12.000000000 +0000 @@ -99,14 +99,20 @@ ksa_adap = utils.get_ksa_adapter( nova.conf.ironic.DEFAULT_SERVICE_TYPE, ksa_auth=auth_plugin, ksa_session=sess, - min_version=IRONIC_API_VERSION, + min_version=(IRONIC_API_VERSION[0], 0), max_version=(IRONIC_API_VERSION[0], ks_disc.LATEST)) ironic_url = ksa_adap.get_endpoint() + ironic_url_none_reason = 'returned None' except exception.ServiceNotFound: # NOTE(efried): No reason to believe service catalog lookup # won't also fail in ironic client init, but this way will # yield the expected exception/behavior. ironic_url = None + ironic_url_none_reason = 'raised ServiceNotFound' + + if ironic_url is None: + LOG.warning("Could not discover ironic_url via keystoneauth1: " + "Adapter.get_endpoint %s", ironic_url_none_reason) try: cli = ironic.client.get_client(IRONIC_API_VERSION[0], diff -Nru nova-17.0.10/nova/virt/libvirt/driver.py nova-17.0.11/nova/virt/libvirt/driver.py --- nova-17.0.10/nova/virt/libvirt/driver.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/libvirt/driver.py 2019-07-10 21:45:12.000000000 +0000 @@ -73,6 +73,7 @@ from nova.compute import power_state from nova.compute import task_states from nova.compute import utils as compute_utils +from nova.compute import vm_states import nova.conf from nova.console import serial as serial_console from nova.console import type as ctype @@ -1251,8 +1252,15 @@ encryption=None, allow_native_luks=True): vol_driver = self._get_volume_driver(connection_info) vol_driver.connect_volume(connection_info, instance) - self._attach_encryptor(context, connection_info, encryption, - allow_native_luks) + try: + self._attach_encryptor( + context, connection_info, encryption, allow_native_luks) + except Exception: + # Encryption failed so rollback the volume connection. + with excutils.save_and_reraise_exception(logger=LOG): + LOG.exception("Failure attaching encryptor; rolling back " + "volume connection", instance=instance) + vol_driver.disconnect_volume(connection_info, instance) def _should_disconnect_target(self, context, connection_info, instance): connection_count = 0 @@ -1404,6 +1412,14 @@ return self._host.delete_secret('volume', volume_id) if encryption is None: encryption = self._get_volume_encryption(context, connection_info) + # NOTE(lyarwood): Handle bug #1821696 where volume secrets have been + # removed manually by returning if native LUKS decryption is available + # and device_path is not present in the connection_info. This avoids + # VolumeEncryptionNotSupported being thrown when we incorrectly build + # the encryptor below due to the secrets not being present above. + if (encryption and self._use_native_luks(encryption) and + not connection_info['data'].get('device_path')): + return if encryption: encryptor = self._get_volume_encryptor(connection_info, encryption) @@ -1483,11 +1499,17 @@ # distributions provide Libvirt 3.3.0 or earlier with # https://libvirt.org/git/?p=libvirt.git;a=commit;h=7189099 applied. except libvirt.libvirtError as ex: - if 'Incorrect number of padding bytes' in six.text_type(ex): - LOG.warning(_('Failed to attach encrypted volume due to a ' - 'known Libvirt issue, see the following bug for details: ' - 'https://bugzilla.redhat.com/show_bug.cgi?id=1447297')) - raise + with excutils.save_and_reraise_exception(): + if 'Incorrect number of padding bytes' in six.text_type(ex): + LOG.warning(_('Failed to attach encrypted volume due to a ' + 'known Libvirt issue, see the following bug ' + 'for details: ' + 'https://bugzilla.redhat.com/1447297')) + else: + LOG.exception(_('Failed to attach volume at mountpoint: ' + '%s'), mountpoint, instance=instance) + self._disconnect_volume(context, connection_info, instance, + encryption=encryption) except Exception: LOG.exception(_('Failed to attach volume at mountpoint: %s'), mountpoint, instance=instance) @@ -7655,7 +7677,7 @@ dest=target, host=fallback_from_host, receive=True) - image.cache(fetch_func=copy_from_host, + image.cache(fetch_func=copy_from_host, size=size, filename=filename) def _create_images_and_backing(self, context, instance, instance_dir, @@ -7997,14 +8019,19 @@ # should ignore this instance and move on. if guest.uuid in local_instances: inst = local_instances[guest.uuid] - if inst.task_state is not None: + # bug 1774249 indicated when instance is in RESIZED + # state it might also can't find back disk + if (inst.task_state is not None or + inst.vm_state == vm_states.RESIZED): LOG.info('Periodic task is updating the host ' 'stats; it is trying to get disk info ' 'for %(i_name)s, but the backing disk ' 'was removed by a concurrent operation ' - '(task_state=%(task_state)s)', + '(task_state=%(task_state)s) and ' + '(vm_state=%(vm_state)s)', {'i_name': guest.name, - 'task_state': inst.task_state}, + 'task_state': inst.task_state, + 'vm_state': inst.vm_state}, instance=inst) err_ctxt.reraise = False @@ -8503,28 +8530,37 @@ errors_count=stats[4]) except libvirt.libvirtError: pass - for interface in dom_io["ifaces"]: + + for interface in xml_doc.findall('./devices/interface'): + mac_address = interface.find('mac').get('address') + target = interface.find('./target') + + # add nic that has no target (therefore no stats) + if target is None: + diags.add_nic(mac_address=mac_address) + continue + + # add nic with stats + dev = target.get('dev') try: - # interfaceStats might launch an exception if the method - # is not supported by the underlying hypervisor being - # used by libvirt - stats = domain.interfaceStats(interface) - diags.add_nic(rx_octets=stats[0], - rx_errors=stats[2], - rx_drop=stats[3], - rx_packets=stats[1], - tx_octets=stats[4], - tx_errors=stats[6], - tx_drop=stats[7], - tx_packets=stats[5]) + if dev: + # interfaceStats might launch an exception if the + # method is not supported by the underlying hypervisor + # being used by libvirt + stats = domain.interfaceStats(dev) + diags.add_nic(mac_address=mac_address, + rx_octets=stats[0], + rx_errors=stats[2], + rx_drop=stats[3], + rx_packets=stats[1], + tx_octets=stats[4], + tx_errors=stats[6], + tx_drop=stats[7], + tx_packets=stats[5]) + except libvirt.libvirtError: pass - # Update mac addresses of interface if stats have been reported - if diags.nic_details: - nodes = xml_doc.findall('./devices/interface/mac') - for index, node in enumerate(nodes): - diags.nic_details[index].mac_address = node.get('address') return diags @staticmethod diff -Nru nova-17.0.10/nova/virt/libvirt/guest.py nova-17.0.11/nova/virt/libvirt/guest.py --- nova-17.0.10/nova/virt/libvirt/guest.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/libvirt/guest.py 2019-07-10 21:45:12.000000000 +0000 @@ -404,7 +404,8 @@ except libvirt.libvirtError as ex: with excutils.save_and_reraise_exception(): errcode = ex.get_error_code() - if errcode == libvirt.VIR_ERR_OPERATION_FAILED: + if errcode in (libvirt.VIR_ERR_OPERATION_FAILED, + libvirt.VIR_ERR_INTERNAL_ERROR): errmsg = ex.get_error_message() if 'not found' in errmsg: # This will be raised if the live domain diff -Nru nova-17.0.10/nova/virt/libvirt/imagebackend.py nova-17.0.11/nova/virt/libvirt/imagebackend.py --- nova-17.0.10/nova/virt/libvirt/imagebackend.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/libvirt/imagebackend.py 2019-07-10 21:45:12.000000000 +0000 @@ -180,11 +180,17 @@ return info def disk_scsi(self, info, disk_unit): - # The driver is responsible to create the SCSI controller - # at index 0. - info.device_addr = vconfig.LibvirtConfigGuestDeviceAddressDrive() - info.device_addr.controller = 0 + # NOTE(melwitt): We set the device address unit number manually in the + # case of the virtio-scsi controller, in order to allow attachment of + # up to 256 devices. So, we should only be setting the address tag + # if we intend to set the unit number. Otherwise, we will let libvirt + # handle autogeneration of the address tag. + # See https://bugs.launchpad.net/nova/+bug/1792077 for details. if disk_unit is not None: + # The driver is responsible to create the SCSI controller + # at index 0. + info.device_addr = vconfig.LibvirtConfigGuestDeviceAddressDrive() + info.device_addr.controller = 0 # In order to allow up to 256 disks handled by one # virtio-scsi controller, the device addr should be # specified. diff -Nru nova-17.0.10/nova/virt/libvirt/volume/volume.py nova-17.0.11/nova/virt/libvirt/volume/volume.py --- nova-17.0.10/nova/virt/libvirt/volume/volume.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/libvirt/volume/volume.py 2019-07-10 21:45:12.000000000 +0000 @@ -94,16 +94,21 @@ if data.get('discard', False) is True: conf.driver_discard = 'unmap' - if disk_info['bus'] == 'scsi': + # NOTE(melwitt): We set the device address unit number manually in the + # case of the virtio-scsi controller, in order to allow attachment of + # up to 256 devices. So, we should only be setting the address tag + # if we intend to set the unit number. Otherwise, we will let libvirt + # handle autogeneration of the address tag. + # See https://bugs.launchpad.net/nova/+bug/1792077 for details. + if disk_info['bus'] == 'scsi' and 'unit' in disk_info: # The driver is responsible to create the SCSI controller # at index 0. conf.device_addr = vconfig.LibvirtConfigGuestDeviceAddressDrive() conf.device_addr.controller = 0 - if 'unit' in disk_info: - # In order to allow up to 256 disks handled by one - # virtio-scsi controller, the device addr should be - # specified. - conf.device_addr.unit = disk_info['unit'] + # In order to allow up to 256 disks handled by one + # virtio-scsi controller, the device addr should be + # specified. + conf.device_addr.unit = disk_info['unit'] if connection_info.get('multiattach', False): # Note that driver_cache should be disabled (none) when using diff -Nru nova-17.0.10/nova/virt/xenapi/agent.py nova-17.0.11/nova/virt/xenapi/agent.py --- nova-17.0.10/nova/virt/xenapi/agent.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/virt/xenapi/agent.py 2019-07-10 21:45:12.000000000 +0000 @@ -21,6 +21,7 @@ from os_xenapi.client import host_agent from os_xenapi.client import XenAPI +from oslo_concurrency import processutils from oslo_log import log as logging from oslo_serialization import base64 from oslo_serialization import jsonutils @@ -422,11 +423,18 @@ 'pass:%s' % self._shared, '-nosalt'] if decrypt: cmd.append('-d') - out, err = utils.execute(*cmd, - process_input=encodeutils.safe_encode(text)) - if err: - raise RuntimeError(_('OpenSSL error: %s') % err) - return out + try: + out, err = utils.execute( + *cmd, + process_input=encodeutils.safe_encode(text), + check_exit_code=True) + if err: + LOG.warning("OpenSSL stderr: %s", err) + return out + except processutils.ProcessExecutionError as e: + raise RuntimeError( + _('OpenSSL errored with exit code %(exit_code)d: %(stderr)s') % + {'exit_code': e.exit_code, 'stderr': e.stderr}) def encrypt(self, text): return self._run_ssl(text).strip('\n') diff -Nru nova-17.0.10/nova/volume/cinder.py nova-17.0.11/nova/volume/cinder.py --- nova-17.0.10/nova/volume/cinder.py 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/nova/volume/cinder.py 2019-07-10 21:45:12.000000000 +0000 @@ -321,6 +321,9 @@ d['shared_targets'] = vol.shared_targets d['service_uuid'] = vol.service_uuid + if hasattr(vol, 'migration_status'): + d['migration_status'] = vol.migration_status + return d diff -Nru nova-17.0.10/nova.egg-info/pbr.json nova-17.0.11/nova.egg-info/pbr.json --- nova-17.0.10/nova.egg-info/pbr.json 2019-03-24 23:14:28.000000000 +0000 +++ nova-17.0.11/nova.egg-info/pbr.json 2019-07-10 21:47:09.000000000 +0000 @@ -1 +1 @@ -{"git_version": "946c26e077", "is_release": true} \ No newline at end of file +{"git_version": "de7a8f9e44", "is_release": true} \ No newline at end of file diff -Nru nova-17.0.10/nova.egg-info/PKG-INFO nova-17.0.11/nova.egg-info/PKG-INFO --- nova-17.0.10/nova.egg-info/PKG-INFO 2019-03-24 23:14:28.000000000 +0000 +++ nova-17.0.11/nova.egg-info/PKG-INFO 2019-07-10 21:47:09.000000000 +0000 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: nova -Version: 17.0.10 +Version: 17.0.11 Summary: Cloud computing fabric controller Home-page: https://docs.openstack.org/nova/latest/ Author: OpenStack diff -Nru nova-17.0.10/nova.egg-info/SOURCES.txt nova-17.0.11/nova.egg-info/SOURCES.txt --- nova-17.0.10/nova.egg-info/SOURCES.txt 2019-03-24 23:14:28.000000000 +0000 +++ nova-17.0.11/nova.egg-info/SOURCES.txt 2019-07-10 21:47:10.000000000 +0000 @@ -822,6 +822,7 @@ doc/source/cli/nova-spicehtml5proxy.rst doc/source/cli/nova-status.rst doc/source/cli/nova-xvpvncproxy.rst +doc/source/common/numa-live-migration-warning.txt doc/source/configuration/config.rst doc/source/configuration/index.rst doc/source/configuration/policy.rst @@ -1840,6 +1841,7 @@ nova/tests/functional/api_samples_test_base.py nova/tests/functional/integrated_helpers.py nova/tests/functional/test_aggregates.py +nova/tests/functional/test_availability_zones.py nova/tests/functional/test_compute_mgr.py nova/tests/functional/test_images.py nova/tests/functional/test_instance_actions.py @@ -2476,6 +2478,7 @@ nova/tests/functional/db/test_archive.py nova/tests/functional/db/test_build_request.py nova/tests/functional/db/test_cell_mapping.py +nova/tests/functional/db/test_compute_api.py nova/tests/functional/db/test_compute_node.py nova/tests/functional/db/test_connection_switch.py nova/tests/functional/db/test_console_auth_token.py @@ -2522,6 +2525,7 @@ nova/tests/functional/regressions/test_bug_1595962.py nova/tests/functional/regressions/test_bug_1620248.py nova/tests/functional/regressions/test_bug_1627838.py +nova/tests/functional/regressions/test_bug_1669054.py nova/tests/functional/regressions/test_bug_1670627.py nova/tests/functional/regressions/test_bug_1671648.py nova/tests/functional/regressions/test_bug_1675570.py @@ -2547,6 +2551,7 @@ nova/tests/functional/regressions/test_bug_1797580.py nova/tests/functional/regressions/test_bug_1806064.py nova/tests/functional/regressions/test_bug_1806515.py +nova/tests/functional/regressions/test_bug_1830747.py nova/tests/functional/wsgi/__init__.py nova/tests/functional/wsgi/test_flavor_manage.py nova/tests/functional/wsgi/test_interfaces.py @@ -3318,6 +3323,8 @@ placement-api-ref/source/usages.inc playbooks/legacy/nova-cells-v1/post.yaml playbooks/legacy/nova-cells-v1/run.yaml +playbooks/legacy/nova-grenade-live-migration/post.yaml +playbooks/legacy/nova-grenade-live-migration/run.yaml playbooks/legacy/nova-live-migration/post.yaml playbooks/legacy/nova-live-migration/run.yaml playbooks/legacy/nova-lvm/post.yaml @@ -3458,6 +3465,7 @@ releasenotes/notes/bug-1753550-image-ref-url-notifications-42df5911a46b7de7.yaml releasenotes/notes/bug-1759316-nova-status-api-version-check-183fac0525bfd68c.yaml releasenotes/notes/bug-1763183-service-delete-with-instances-d7c5c47e4ce31239.yaml +releasenotes/notes/bug-1775418-754fc50261f5d7c3.yaml releasenotes/notes/bug-1778044-f498ee2f2cfb35ea.yaml releasenotes/notes/bug-1801702-c8203d3d55007deb.yaml releasenotes/notes/bug-hyperv-1629040-e1eb35a7b31d9af8.yaml @@ -3538,6 +3546,7 @@ releasenotes/notes/deprecate_xenapi_torrent_downloader-ebcbb3d5f929d893.yaml releasenotes/notes/deprecates-multinic-floatingipaction-osvirtualinterface-api-73b24e5304635e9d.yaml releasenotes/notes/deprecates-proxy-apis-5e11d7c4ae5227d2.yaml +releasenotes/notes/disable-live-migration-with-numa-bc710a1bcde25957.yaml releasenotes/notes/disable_ec2_api_by_default-0ec0946433fc7119.yaml releasenotes/notes/disco_volume_libvirt_driver-916428b8bd852732.yaml releasenotes/notes/discover-hosts-by-service-06ee20365b895127.yaml diff -Nru nova-17.0.10/PKG-INFO nova-17.0.11/PKG-INFO --- nova-17.0.10/PKG-INFO 2019-03-24 23:14:30.000000000 +0000 +++ nova-17.0.11/PKG-INFO 2019-07-10 21:47:11.000000000 +0000 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: nova -Version: 17.0.10 +Version: 17.0.11 Summary: Cloud computing fabric controller Home-page: https://docs.openstack.org/nova/latest/ Author: OpenStack diff -Nru nova-17.0.10/playbooks/legacy/nova-cells-v1/run.yaml nova-17.0.11/playbooks/legacy/nova-cells-v1/run.yaml --- nova-17.0.10/playbooks/legacy/nova-cells-v1/run.yaml 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-cells-v1/run.yaml 2019-07-10 21:45:12.000000000 +0000 @@ -13,12 +13,12 @@ set -x cat > clonemap.yaml << EOF clonemap: - - name: openstack-infra/devstack-gate + - name: openstack/devstack-gate dest: devstack-gate EOF /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ - git://git.openstack.org \ - openstack-infra/devstack-gate + https://opendev.org \ + openstack/devstack-gate executable: /bin/bash chdir: '{{ ansible_user_dir }}/workspace' environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/playbooks/legacy/nova-grenade-live-migration/post.yaml nova-17.0.11/playbooks/legacy/nova-grenade-live-migration/post.yaml --- nova-17.0.10/playbooks/legacy/nova-grenade-live-migration/post.yaml 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-grenade-live-migration/post.yaml 2019-07-10 21:45:04.000000000 +0000 @@ -0,0 +1,15 @@ +- hosts: primary + tasks: + + - name: Copy files from {{ ansible_user_dir }}/workspace/ on node + synchronize: + src: '{{ ansible_user_dir }}/workspace/' + dest: '{{ zuul.executor.log_root }}' + mode: pull + copy_links: true + verify_host: true + rsync_opts: + - --include=/logs/** + - --include=*/ + - --exclude=* + - --prune-empty-dirs diff -Nru nova-17.0.10/playbooks/legacy/nova-grenade-live-migration/run.yaml nova-17.0.11/playbooks/legacy/nova-grenade-live-migration/run.yaml --- nova-17.0.10/playbooks/legacy/nova-grenade-live-migration/run.yaml 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-grenade-live-migration/run.yaml 2019-07-10 21:45:04.000000000 +0000 @@ -0,0 +1,58 @@ +- hosts: primary + name: nova-grenade-live-migration + tasks: + + - name: Ensure legacy workspace directory + file: + path: '{{ ansible_user_dir }}/workspace' + state: directory + + - shell: + cmd: | + set -e + set -x + cat > clonemap.yaml << EOF + clonemap: + - name: openstack/devstack-gate + dest: devstack-gate + EOF + /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ + https://opendev.org \ + openstack/devstack-gate + executable: /bin/bash + chdir: '{{ ansible_user_dir }}/workspace' + environment: '{{ zuul | zuul_legacy_vars }}' + + - shell: + cmd: | + set -e + set -x + export PROJECTS="openstack/grenade $PROJECTS" + export PYTHONUNBUFFERED=true + export DEVSTACK_GATE_CONFIGDRIVE=0 + export DEVSTACK_GATE_NEUTRON=1 + export DEVSTACK_GATE_TEMPEST_NOTESTS=1 + export DEVSTACK_GATE_GRENADE=pullup + # By default grenade runs only smoke tests so we need to set + # RUN_SMOKE to False in order to run live migration tests using + # grenade + export DEVSTACK_LOCAL_CONFIG="RUN_SMOKE=False" + # LIVE_MIGRATE_BACK_AND_FORTH will tell Tempest to run a live + # migration of the same instance to one compute node and then back + # to the other, which is mostly only interesting for grenade since + # we have mixed level computes. + export DEVSTACK_LOCAL_CONFIG+=$'\n'"LIVE_MIGRATE_BACK_AND_FORTH=True" + export BRANCH_OVERRIDE=default + export DEVSTACK_GATE_TOPOLOGY="multinode" + if [ "$BRANCH_OVERRIDE" != "default" ] ; then + export OVERRIDE_ZUUL_BRANCH=$BRANCH_OVERRIDE + fi + function post_test_hook { + /opt/stack/new/nova/nova/tests/live_migration/hooks/run_tests.sh + } + export -f post_test_hook + cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh + ./safe-devstack-vm-gate-wrap.sh + executable: /bin/bash + chdir: '{{ ansible_user_dir }}/workspace' + environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/playbooks/legacy/nova-live-migration/run.yaml nova-17.0.11/playbooks/legacy/nova-live-migration/run.yaml --- nova-17.0.10/playbooks/legacy/nova-live-migration/run.yaml 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-live-migration/run.yaml 2019-07-10 21:45:12.000000000 +0000 @@ -13,12 +13,12 @@ set -x cat > clonemap.yaml << EOF clonemap: - - name: openstack-infra/devstack-gate + - name: openstack/devstack-gate dest: devstack-gate EOF /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ - git://git.openstack.org \ - openstack-infra/devstack-gate + https://opendev.org \ + openstack/devstack-gate executable: /bin/bash chdir: '{{ ansible_user_dir }}/workspace' environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/playbooks/legacy/nova-lvm/run.yaml nova-17.0.11/playbooks/legacy/nova-lvm/run.yaml --- nova-17.0.10/playbooks/legacy/nova-lvm/run.yaml 2019-03-24 23:12:23.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-lvm/run.yaml 2019-07-10 21:45:04.000000000 +0000 @@ -13,12 +13,12 @@ set -x cat > clonemap.yaml << EOF clonemap: - - name: openstack-infra/devstack-gate + - name: openstack/devstack-gate dest: devstack-gate EOF /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ - git://git.openstack.org \ - openstack-infra/devstack-gate + https://opendev.org \ + openstack/devstack-gate executable: /bin/bash chdir: '{{ ansible_user_dir }}/workspace' environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/playbooks/legacy/nova-multiattach/run.yaml nova-17.0.11/playbooks/legacy/nova-multiattach/run.yaml --- nova-17.0.10/playbooks/legacy/nova-multiattach/run.yaml 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-multiattach/run.yaml 2019-07-10 21:45:12.000000000 +0000 @@ -13,12 +13,12 @@ set -x cat > clonemap.yaml << EOF clonemap: - - name: openstack-infra/devstack-gate + - name: openstack/devstack-gate dest: devstack-gate EOF /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ - git://git.openstack.org \ - openstack-infra/devstack-gate + https://opendev.org \ + openstack/devstack-gate executable: /bin/bash chdir: '{{ ansible_user_dir }}/workspace' environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/playbooks/legacy/nova-next/run.yaml nova-17.0.11/playbooks/legacy/nova-next/run.yaml --- nova-17.0.10/playbooks/legacy/nova-next/run.yaml 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/playbooks/legacy/nova-next/run.yaml 2019-07-10 21:45:12.000000000 +0000 @@ -13,12 +13,12 @@ set -x cat > clonemap.yaml << EOF clonemap: - - name: openstack-infra/devstack-gate + - name: openstack/devstack-gate dest: devstack-gate EOF /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \ - git://git.openstack.org \ - openstack-infra/devstack-gate + https://opendev.org \ + openstack/devstack-gate executable: /bin/bash chdir: '{{ ansible_user_dir }}/workspace' environment: '{{ zuul | zuul_legacy_vars }}' diff -Nru nova-17.0.10/releasenotes/notes/bug-1775418-754fc50261f5d7c3.yaml nova-17.0.11/releasenotes/notes/bug-1775418-754fc50261f5d7c3.yaml --- nova-17.0.10/releasenotes/notes/bug-1775418-754fc50261f5d7c3.yaml 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/releasenotes/notes/bug-1775418-754fc50261f5d7c3.yaml 2019-07-10 21:45:04.000000000 +0000 @@ -0,0 +1,10 @@ +--- +fixes: + - | + The `os-volume_attachments`_ update API, commonly referred to as the swap + volume API will now return a ``400`` (BadRequest) error when attempting to + swap from a multi attached volume with more than one active read/write + attachment resolving `bug #1775418`_. + + .. _os-volume_attachments: https://developer.openstack.org/api-ref/compute/?expanded=update-a-volume-attachment-detail + .. _bug #1775418: https://launchpad.net/bugs/1775418 diff -Nru nova-17.0.10/releasenotes/notes/disable-live-migration-with-numa-bc710a1bcde25957.yaml nova-17.0.11/releasenotes/notes/disable-live-migration-with-numa-bc710a1bcde25957.yaml --- nova-17.0.10/releasenotes/notes/disable-live-migration-with-numa-bc710a1bcde25957.yaml 1970-01-01 00:00:00.000000000 +0000 +++ nova-17.0.11/releasenotes/notes/disable-live-migration-with-numa-bc710a1bcde25957.yaml 2019-07-10 21:45:04.000000000 +0000 @@ -0,0 +1,25 @@ +--- +upgrade: + - | + Live migration of instances with NUMA topologies is now disabled by default + when using the libvirt driver. This includes live migration of instances + with CPU pinning or hugepages. CPU pinning and huge page information for + such instances is not currently re-calculated, as noted in `bug #1289064`_. + This means that if instances were already present on the destination host, + the migrated instance could be placed on the same dedicated cores as these + instances or use hugepages allocated for another instance. Alternately, if + the host platforms were not homogeneous, the instance could be assigned to + non-existent cores or be inadvertently split across host NUMA nodes. + + The `long term solution`_ to these issues is to recalculate the XML on the + destination node. When this work is completed, the restriction on live + migration with NUMA topologies will be lifted. + + For operators that are aware of the issues and are able to manually work + around them, the ``[workarounds] enable_numa_live_migration`` option can + be used to allow the broken behavior. + + For more information, refer to `bug #1289064`_. + + .. _bug #1289064: https://bugs.launchpad.net/nova/+bug/1289064 + .. _long term solution: https://blueprints.launchpad.net/nova/+spec/numa-aware-live-migration diff -Nru nova-17.0.10/.zuul.yaml nova-17.0.11/.zuul.yaml --- nova-17.0.10/.zuul.yaml 2019-03-24 23:12:31.000000000 +0000 +++ nova-17.0.11/.zuul.yaml 2019-07-10 21:45:12.000000000 +0000 @@ -8,7 +8,7 @@ Contains common configuration. timeout: 10800 required-projects: - - openstack-infra/devstack-gate + - openstack/devstack-gate - openstack/nova - openstack/tempest irrelevant-files: @@ -35,7 +35,7 @@ each other. timeout: 10800 required-projects: - - openstack-infra/devstack-gate + - openstack/devstack-gate - openstack/nova - openstack/tempest irrelevant-files: @@ -154,6 +154,38 @@ devstack_localrc: TEMPEST_COMPUTE_TYPE: compute_legacy +- job: + name: nova-grenade-live-migration + parent: nova-dsvm-multinode-base + description: | + Multi-node grenade job which runs nova/tests/live_migration/hooks tests. + In other words, this tests live migration with mixed-version compute + services which is important for things like rolling upgrade support. + The former name for this job was + "legacy-grenade-dsvm-neutron-multinode-live-migration". + required-projects: + - openstack/grenade + run: playbooks/legacy/nova-grenade-live-migration/run.yaml + post-run: playbooks/legacy/nova-grenade-live-migration/post.yaml + irrelevant-files: + - ^(placement-)?api-.*$ + - ^(test-|)requirements.txt$ + - ^.*\.rst$ + - ^.git.*$ + - ^api-.*$ + - ^doc/.*$ + - ^nova/hacking/.*$ + - ^nova/locale/.*$ + - ^nova/tests/.*\.py$ + - ^nova/tests/functional/.*$ + - ^nova/tests/unit/.*$ + - ^releasenotes/.*$ + - ^setup.cfg$ + - ^tests-py3.txt$ + - ^tools/.*$ + - ^tox.ini$ + voting: false + - project: # Please try to keep the list of job names sorted alphabetically. templates: @@ -188,6 +220,7 @@ - ^tools/.*$ - ^tox.ini$ - nova-cells-v1 + - nova-grenade-live-migration - nova-live-migration - nova-multiattach - nova-next @@ -223,28 +256,25 @@ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - legacy-grenade-dsvm-neutron-multinode-live-migration: + - devstack-plugin-ceph-tempest: voting: false irrelevant-files: - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ - ^.*\.rst$ - ^.git.*$ - - ^api-.*$ - ^doc/.*$ - ^nova/hacking/.*$ - ^nova/locale/.*$ - - ^nova/tests/.*\.py$ - - ^nova/tests/functional/.*$ - - ^nova/tests/unit/.*$ + - ^nova/tests/.*$ - ^releasenotes/.*$ - ^setup.cfg$ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - devstack-plugin-ceph-tempest: - voting: false + - neutron-tempest-linuxbridge: irrelevant-files: + - ^(?!nova/network/.*)(?!nova/virt/libvirt/vif.py).*$ - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ - ^.*\.rst$ @@ -258,9 +288,9 @@ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - neutron-tempest-linuxbridge: + - tempest-multinode-full: + voting: false irrelevant-files: - - ^(?!nova/network/.*)(?!nova/virt/libvirt/vif.py).*$ - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ - ^.*\.rst$ @@ -274,8 +304,7 @@ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - tempest-multinode-full: - voting: false + - tempest-full: irrelevant-files: - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ @@ -290,7 +319,7 @@ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - tempest-full: + - neutron-grenade: irrelevant-files: - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ @@ -305,7 +334,7 @@ - ^tests-py3.txt$ - ^tools/.*$ - ^tox.ini$ - - neutron-grenade: + - tempest-full-py3: irrelevant-files: - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ @@ -362,6 +391,21 @@ irrelevant-files: - ^(placement-)?api-.*$ - ^(test-|)requirements.txt$ + - ^.*\.rst$ + - ^.git.*$ + - ^doc/.*$ + - ^nova/hacking/.*$ + - ^nova/locale/.*$ + - ^nova/tests/.*$ + - ^releasenotes/.*$ + - ^setup.cfg$ + - ^tests-py3.txt$ + - ^tools/.*$ + - ^tox.ini$ + - tempest-full-py3: + irrelevant-files: + - ^(placement-)?api-.*$ + - ^(test-|)requirements.txt$ - ^.*\.rst$ - ^.git.*$ - ^doc/.*$