image list displays a maximum of 25 images

Bug #1443089 reported by Everett Toews
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
python-openstackclient
Fix Released
High
Unassigned
python-openstackclient (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

1. Set env vars for HP Helion Public Cloud [1]

export OS_TENANT_NAME=xxx
export OS_USERNAME=xxx
export OS_PASSWORD=xxx
export OS_AUTH_URL=https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0/
export OS_REGION_NAME=region-b.geo-1

2. List images

$ openstack image list
...
[25 images displayed]
...

If you go to the Images page [2], you see that there are 103 images available.

[1] https://community.hpcloud.com/article/command-line-interface-cli-tool-installation-instructions
[2] https://horizon.hpcloud.com/project/images/

Revision history for this message
Steve Martinelli (stevemar) wrote :

whats the output of # glance image-list?

Revision history for this message
Steve Martinelli (stevemar) wrote :

looks like glanceclient has an option for page size:
https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/images.py#L206

page_size = kwargs.get('page_size', DEFAULT_PAGE_SIZE) where DEFAULT_PAGE_SIZE is 20.

I wonder how horizon and glance CLI handle this, or HP public cloud for that matter.

Revision history for this message
Steve Martinelli (stevemar) wrote :

horizon has a setting and glanceclient CLI has a page-size option:

glance CLI:
https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/shell.py#L56
@utils.arg('--page-size', metavar='<SIZE>', default=None, type=int,
           help='Number of images to request in each paginated request.')

horizon:
https://github.com/openstack/horizon/blob/8e223c950ed69e80034fc8ef7d0861ecd8aeb07d/openstack_dashboard/api/glance.py#L66-L69
def image_list_detailed(request, marker=None, sort_dir='desc',
                        sort_key='created_at', filters=None, paginate=False):
    limit = getattr(settings, 'API_RESULT_LIMIT', 1000)
    page_size = utils.get_page_size(request)

Revision history for this message
Steve Martinelli (stevemar) wrote :

Also... looks like OSC had this at one point?

https://github.com/openstack/python-openstackclient/blob/master/openstackclient/image/v1/image.py#L351-L357

@Everett, want to try using `openstack image list --page-size 120` ?

Revision history for this message
Everett Toews (everett-toews) wrote :

Just tried `openstack image list --page-size 120` and got the same 25 images only result. Note that `openstack help image list` doesn't mention anything about --page-size.

`glance image-list` returns 103 images.

Revision history for this message
Steve Martinelli (stevemar) wrote :

Cool. I wasn't expecting --page-size to show up in the help output, that option is suppressed for some reason.

Looking at this a bit further... apparently this option never worked. Glance is using pagination (and filtering?) so this is getting ugly really fast.

Changed in python-openstackclient:
status: New → Confirmed
importance: Undecided → High
milestone: none → m9
Revision history for this message
Steve Martinelli (stevemar) wrote :

Added a 'limit' option, and that seems to solve things. Don't know if it agrees with dtroyers vision of handling paging going forward thought.

steve:python-openstackclient$ openstack image list | wc -l
29
steve:python-openstackclient$ openstack image list --limit 100 | wc -l
33

Tested with >25 images created.

Not sure how to glanceclient is warping this to get everything by default. Also not sure if that's something we want to encourage.

Revision history for this message
Steve Martinelli (stevemar) wrote :

FYI - the limit option does not exist in the current OSC code / release. Just something I'm playing around with in my env.

Revision history for this message
Steve Martinelli (stevemar) wrote :

@Everett, is this using v1 or v2 of the images API?

Revision history for this message
Everett Toews (everett-toews) wrote :

Looks like HP is using v1 API [1].

$ openstack --debug image list
...
DEBUG: openstackclient.image.client Instantiating image api: <class 'openstackclient.api.image_v1.APIv1'>
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://region-b.geo-1.images.hpcloudsvc.com:443/v1.0/images -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: {SHA1}xxx"

Hmmmmm...tried this on Rackspace too expecting the v2 API [2]. It returns the same 25 only images but I get this wackiness.

$ openstack --debug image list
...
DEBUG: openstackclient.image.client Instantiating image api: <class 'openstackclient.api.image_v1.APIv1'>
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://iad.images.api.rackspacecloud.com/v2/v1/images -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: {SHA1}xxx"

I'm not too sure what to make of that. https://iad.images.api.rackspacecloud.com/v2/v1/images isn't a valid URL. Maybe openstack --debug isn't logging what's really going on?

[1] https://github.com/openstack/os-client-config/blob/master/os_client_config/vendors.py#L23
[2] https://github.com/openstack/os-client-config/blob/master/os_client_config/vendors.py#L32

Revision history for this message
Dean Troyer (dtroyer) wrote :
Changed in python-openstackclient:
milestone: m9 → m10
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-openstackclient (master)

Change abandoned by Steve Martinelli (<email address hidden>) on branch: master
Review: https://review.openstack.org/172763
Reason: i'll revisit this after the summit

Dean Troyer (dtroyer)
Changed in python-openstackclient:
status: Fix Committed → Fix Released
Revision history for this message
Frode Nordahl (fnordahl) wrote :

Correction: The current releases only contains the code for v1 API and not for v2. Is there any reason for that? From a user perspective it is a clear regression over the use of the old glance CLI tool.

Revision history for this message
Frode Nordahl (fnordahl) wrote :
Revision history for this message
Frode Nordahl (fnordahl) wrote :

This fix should be backported to the Ubuntu packages.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-openstackclient - 3.4.1-0ubuntu2

---------------
python-openstackclient (3.4.1-0ubuntu2) zesty; urgency=medium

  * debian/patches/max-images.patch: Allow more than maximum of 25 images to
    be listed. (LP: #1443089)

 -- Chuck Short <email address hidden> Mon, 12 Dec 2016 08:33:47 -0500

Changed in python-openstackclient (Ubuntu):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.