euca-delete-volume cannot delete volumes in error state

There are also nova-manage commands for dealing with messed up volumes.
Check nova-manage volume.
On Feb 10, 2011 9:20 PM, "Wayne A. Walls" <email address hidden> wrote:
> Public bug reported:
>
> If all the appropriate services are not running for nova-volume, a
> volume will still try to be created, but will be stuck in 'error' state.
> These cannot be deleted via euca-delete-volumes. You can, however,
> delete all the invalid volumes from nova.volumes in the DB which will
> remove the rogue entries.
>
> ##########
> root@osdemo05:~# euca-describe-volumes
> VOLUME vol-00000001 1 nova creating (wayne, None, None, None) 2011-02-10
15:19:41
> VOLUME vol-00000002 1 nova error (wayne, osdemo03, None, None) 2011-02-10
23:42:41
> ...
> ...
> VOLUME vol-00000008 2 nova error (wayne, osdemo03, None, None) 2011-02-11
00:07:13
> VOLUME vol-00000009 2 nova available (wayne, osdemo03, None, None)
2011-02-11 00:07:37
>
> root@osdemo05:~# euca-delete-volume vol-00000008
> Unknown: Unknown: Volume status must be available
> ##########
>
> ** Affects: nova
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/716847
>
> Title:
> euca-delete-volume cannot delete volumes in error state
>
> Status in OpenStack Compute (Nova):
> New
>
> Bug description:
> If all the appropriate services are not running for nova-volume, a
> volume will still try to be created, but will be stuck in 'error'
> state. These cannot be deleted via euca-delete-volumes. You can,
> however, delete all the invalid volumes from nova.volumes in the DB
> which will remove the rogue entries.
>
> ##########
> root@osdemo05:~# euca-describe-volumes
> VOLUME vol-00000001 1 nova creating (wayne, None, None, None) 2011-02-10
15:19:41
> VOLUME vol-00000002 1 nova error (wayne, osdemo03, None, None) 2011-02-10
23:42:41
> ...
> ...
> VOLUME vol-00000008 2 nova error (wayne, osdemo03, None, None) 2011-02-11
00:07:13
> VOLUME vol-00000009 2 nova available (wayne, osdemo03, None, None)
2011-02-11 00:07:37
>
> root@osdemo05:~# euca-delete-volume vol-00000008
> Unknown: Unknown: Volume status must be available
> ##########
>
>

Greetings, Vish! That may have worked on the 'error' state ones, not sure now that I don't have any, but this one stuck in 'creating' is not going anywhere.

#####
root@osdemo05:~# euca-describe-volumes
VOLUME vol-00000001 1 nova creating (wayne, None, None, None) 2011-02-10 15:19:41
VOLUME vol-0000000b 7 nova available (wayne, osdemo03, None, None) 2011-02-11 06:58:46
VOLUME vol-00000009 2 nova available (wayne, osdemo03, None, None) 2011-02-11 00:07:37

root@osdemo05:~# nova-manage volume delete vol-00000001

root@osdemo05:~# euca-describe-volumes
VOLUME vol-00000001 1 nova creating (wayne, None, None, None) 2011-02-10 15:19:41
VOLUME vol-0000000b 7 nova available (wayne, osdemo03, None, None) 2011-02-11 06:58:46
VOLUME vol-00000009 2 nova available (wayne, osdemo03, None, None) 2011-02-11 00:07:37
#####

The issue is probably that volumes.host is empty for a volume in 'creating'
state. So the delete message is being cast to an invalid topic. This could
be fixed by deleting host-less volumes directly from nova-manage. I did a
quick fix for this, but launchpad is under maintenance atm so I'll push it
up tomorrow.

In general, db.queue_get_for should probably raise an exception if it is
sent a blank host as a parameter.

Anthony

On Thu, Feb 10, 2011 at 11:13 PM, Wayne A. Walls
<email address hidden>wrote:

> Greetings, Vish! That may have worked on the 'error' state ones, not
> sure now that I don't have any, but this one stuck in 'creating' is not
> going anywhere.
>
> #####
> root@osdemo05:~# euca-describe-volumes
> VOLUME vol-00000001 1 nova creating (wayne, None,
> None, None) 2011-02-10 15:19:41
> VOLUME vol-0000000b 7 nova available (wayne, osdemo03,
> None, None) 2011-02-11 06:58:46
> VOLUME vol-00000009 2 nova available (wayne, osdemo03,
> None, None) 2011-02-11 00:07:37
>
> root@osdemo05:~# nova-manage volume delete vol-00000001
>
> root@osdemo05:~# euca-describe-volumes
> VOLUME vol-00000001 1 nova creating (wayne, None,
> None, None) 2011-02-10 15:19:41
> VOLUME vol-0000000b 7 nova available (wayne, osdemo03,
> None, None) 2011-02-11 06:58:46
> VOLUME vol-00000009 2 nova available (wayne, osdemo03,
> None, None) 2011-02-11 00:07:37
> #####
>
> --
> You received this bug notification because you are subscribed to
> OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/716847
>
> Title:
> euca-delete-volume cannot delete volumes in error state
>
> Status in OpenStack Compute (Nova):
> New
>
> Bug description:
> If all the appropriate services are not running for nova-volume, a
> volume will still try to be created, but will be stuck in 'error'
> state. These cannot be deleted via euca-delete-volumes. You can,
> however, delete all the invalid volumes from nova.volumes in the DB
> which will remove the rogue entries.
>
> ##########
> root@osdemo05:~# euca-describe-volumes
> VOLUME vol-00000001 1 nova creating (wayne, None,
> None, None) 2011-02-10 15:19:41
> VOLUME vol-00000002 1 nova error (wayne, osdemo03,
> None, None) 2011-02-10 23:42:41
> ...
> ...
> VOLUME vol-00000008 2 nova error (wayne, osdemo03,
> None, None) 2011-02-11 00:07:13
> VOLUME vol-00000009 2 nova available (wayne,
> osdemo03, None, None) 2011-02-11 00:07:37
>
> root@osdemo05:~# euca-delete-volume vol-00000008
> Unknown: Unknown: Volume status must be available
> ##########
>
>
>

Sorry - this bug (as opened) is still present in trunk, when using the EC2 API. Is our position that an admin must intervene here?

root@openstack:~/openstack# euca-describe-volumes
VOLUME vol-00000001 1 nova error (justinsb, openstack, None, None) 2011-03-24T20:28:17Z
VOLUME vol-00000002 1 nova available (justinsb, openstack, None, None) 2011-03-24T20:30:23Z
root@openstack:~/openstack# euca-delete-volume vol-00000001
ApiError: ApiError: Volume status must be available

woops

The thinking going in was that volumes in error (or 'creating') states should require admin interaction in order to delete them, to make sure that the cause for the error could be investigated by the administrator. So rather than allowing users to resolve issues like this, the 'fix' for this bug was to provide nova-manage commands to resolve the errors.

Admin intervention (at least for Cactus until we better understand the
problem) is fine with me. We should probably re-close this one as "Won't
Fix" if this is acceptable to all.

Closing as wontfix (or rather, fixed another way)