OpenStack Identity (keystone)

Fetching all mappings may become too slow

Bug #1775207 reported by Pavlo Shchelokovskyy on 2018-06-05

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Identity (keystone)	Fix Released	Undecided	Pavlo Shchelokovskyy	OpenStack Identity (keystone) rocky-3

Bug Description

While fixing bug 1582585 the change I2c266e91f2f05be760f8a3eea3738868243cc9c6 started fetching all mappings from SQL and performing in-memory joins.

However, with many users and groups mappings such fetching may become too slow - for instance, with ~125k users CLI command "openstack domain list" takes up to 7 seconds and login to the Horizon - up to 20 seconds.

Additionally filtering the corresponding query in get_domain_mapping_list method by entity_type speeds up things somewhat.

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-05: Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/572446

Changed in keystone:
assignee:	nobody → Pavlo Shchelokovskyy (pshchelo)
status:	New → In Progress

Lance Bragstad (lbragstad) on 2018-06-25

tags:

added: office-hours

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-07-17: Fix merged to keystone (master)

Reviewed: https://review.openstack.org/572446
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=4abd9926ab7fb79bbe86a22657a2474790a8fcb8
Submitter: Zuul
Branch: master

commit 4abd9926ab7fb79bbe86a22657a2474790a8fcb8
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Wed May 30 21:27:58 2018 +0300

Filter by entity_type in get_domain_mapping_list

with many users and groups in a domain fetching all mappings (for both
users and groups) may become inefficient.

    In an environment with approx 125k users and 150 groups in the mapping
    table and SAML2+LDAP auth/backend, this patch reduced the time
    for first (uncached) 'openstack token issue' command from 12 to 3 seconds.
    Similar improvements were seen with time to login to Horizon as well.

Change-Id: Iccbef534ff7e723f8b1461bb1169e2da66cc1dea
Closes-Bug: #1775207

Changed in keystone:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-08-09: Fix included in openstack/keystone 14.0.0.0rc1

This issue was fixed in the openstack/keystone 14.0.0.0rc1 release candidate.

Lance Bragstad (lbragstad) on 2019-01-28

Changed in keystone:
milestone:	none → rocky-3

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-11-10: Fix proposed to keystone (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/762207

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-11-20: Fix merged to keystone (stable/queens)

Reviewed: https://review.opendev.org/762207
Committed: https://opendev.org/openstack/keystone/commit/0153ab781d387cc0fdb3c93fbd7be5c897bcf933
Submitter: Zuul
Branch: stable/queens

commit 0153ab781d387cc0fdb3c93fbd7be5c897bcf933
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Wed May 30 21:27:58 2018 +0300

Filter by entity_type in get_domain_mapping_list

with many users and groups in a domain fetching all mappings (for both
users and groups) may become inefficient.

    Change-Id: Iccbef534ff7e723f8b1461bb1169e2da66cc1dea
    Closes-Bug: #1775207
    (cherry picked from commit 4abd9926ab7fb79bbe86a22657a2474790a8fcb8)