OpenStack Compute (nova)

bug blocks DB migration that changes column type

Bug #1482416 reported by Chris Friesen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
Undecided
Unassigned

Bug Description

I'm trying to make the following change as a DB migration

+ # Table instances, modify field 'vcpus_used' to Float
+ compute_nodes = Table('compute_nodes', meta, autoload=True)
+ vcpus_used = getattr(compute_nodes.c, 'vcpus_used')
+ vcpus_used.alter(type=Float)

This works at runtime (using PostgreSQL) but when running the unit tests (using sqlite) I get the following:

nova.tests.unit.db.test_migrations.TestNovaMigrationsSQLite.test_models_sync
----------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/home/cfriesen/devel/wrlinux-x/addons/wr-cgcs/layers/cgcs/git/nova/.tox/py27/lib/python2.7/site-packages/oslo_db/sqlalchemy/test_migrations.py", line 588, in test_models_sync
        "Models and migration scripts aren't in sync:\n%s" % msg)
      File "/home/cfriesen/devel/wrlinux-x/addons/wr-cgcs/layers/cgcs/git/nova/.tox/py27/lib/python2.7/site-packages/unittest2/case.py", line 690, in fail
        raise self.failureException(msg)
    AssertionError: Models and migration scripts aren't in sync:
    [ ( 'add_constraint',
        UniqueConstraint(Column('host', String(length=255), table=<compute_nodes>), Column('hypervisor_hostname', String(length=255), table=<compute_nodes>), Column('deleted', Integer(), table=<compute_nodes>, default=ColumnDefault(0))))]

This appears to be an interaction between two things, the change I made to alter the vcpus_used column, and a previous change (commit 2db4a1ac, migration version 279) to add the uniq_compute_nodes0host0hypervisor_hostname constraint. sqlite doesn't support altering columns, so there's a workaround to fake it, and this workaround seems to be running into problems with the constraint.

I suspect that this means that anyone wanting to change the type of a column in the "compute_nodes" table (or possibly any table with a similar constraint) will run into a similar problem.

See original description
Tags: compute db
Revision history for this message
Chris Friesen (cbf123) wrote :

alaski pointed out that the constraint doesn't even touch the column that I'm trying to alter, so there must be something else going on. One possibility is that we're actually recreating the entire table, not just the column being altered.

description: updated
description: updated
Revision history for this message
Mike Bayer (zzzeek) wrote :

column alters aren't supported in Nova going forward due to non-scripted online schema migrations in any case. Consult nova devs here but I think you'd need to add a new column to store the new data and ensure the database models synchronize values between the two columns.

Revision history for this message
Mike Bayer (zzzeek) wrote :

also yes, sqlalchemy-migrate recreates the whole table in order to do a migration. nova should not be testing migrations on SQlite at all IMO it's pointless.

Revision history for this message
Matt Riedemann (mriedem) wrote :

To clarify, nova is not supporting drops on tables or columns:

https://review.openstack.org/#/c/197349/

sqlite doesn't support alter:

http://www.sqlite.org/lang_altertable.html

So you have to drop and recreate things with sqlite to fake it out, that's what sqlalchemy-migrate does with foreign key constraints and unique constraints, e.g. https://review.openstack.org/#/c/157878/.

So if you needed to support a column alter in nova and the failure is with sqlite, then fix sqlalchemy-migrate.

Chris Friesen (cbf123)
Changed in nova:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.