getting quotas from database has severe performance implications

Fix proposed to branch: master
Review: https://review.openstack.org/15747

Reviewed: https://review.openstack.org/15747
Committed: http://github.com/openstack/quantum/commit/513307fa916c5461bb9a144b74b562b9d34a1c65
Submitter: Jenkins
Branch: master

commit 513307fa916c5461bb9a144b74b562b9d34a1c65
Author: Juergen Brendel <email address hidden>
Date: Thu Nov 8 15:00:11 2012 +1300

    Use DB count to get resource counts.

    Fixes bug 1075369.

    During quota check we used to simply retrieve the entire collection
    of resources from the database, then counting them in Python. This
    patch introduces a specialized _get_collection_count() method, which
    instead take advantage of the DB's built-in count capabilities.

    In order to take advantage of this, plugins can now implement
    get_*_count() methods for their resources. This is used (if present)
    by the quota checking function.

    Patch incorporates review feedback from Dan W, Alex Xu, Zhongyue Luo,
    Edgar Magana, Akihiro Motoki and gongysh.

    Change-Id: I87e2d0294e116e8147fed2ee90c9eb0cf1a54362

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16386

here is ML post showing how bad things get at large scales: http://lists.openstack.org/pipermail/openstack-dev/2012-November/002375.html

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16952

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/17000

Reviewed: https://review.openstack.org/16952
Committed: http://github.com/openstack/quantum/commit/ca7fc10dac532fdea5398b394ac3e58b1c3d8791
Submitter: Jenkins
Branch: stable/folsom

commit ca7fc10dac532fdea5398b394ac3e58b1c3d8791
Author: Dan Wendlandt <email address hidden>
Date: Tue Nov 27 13:14:54 2012 -0800

    Use DB count to get resource counts.

    Fixes bug 1075369, and includes bugs 1081259, 1083696 which fixed
    regression in the original version of this patch and bug 1083180
    which improves the documentation of the original patch.

    During quota check we used to simply retrieve the entire collection
    of resources from the database, then counting them in Python. This
    patch introduces a specialized _get_collection_count() method, which
    instead take advantage of the DB's built-in count capabilities.

    In order to take advantage of this, plugins can now implement
    get_*_count() methods for their resources. This is used (if present)
    by the quota checking function.

    Change-Id: Idc347cfacc363f2cac47356296514fe735625f29

Hello Edgar, or anyone else affected,

Accepted quantum into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/quantum/2012.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

This bug was fixed in the package quantum - 2012.2.1-0ubuntu1

---------------
quantum (2012.2.1-0ubuntu1) quantal-proposed; urgency=low

  * Resynchronize with stable/folsom (1e774867) (LP: #1085255):
    - [aeabb42] There are routing problems when the dnsmasq port does not come
      first in the routing table (LP: #1083238)
    - [04aab72] Quantum linux bridge not optimized with libvirt (LP: #1078210)
    - [ca7fc10] getting quotas from database has severe performance implications
      (LP: #1075369)
    - [66605e8] failed to update an external network into non external network
      (LP: #1083387)
    - [c60051a] Quantum test suite leaks memory like a sieve (LP: #1065276)
    - [3179dfc] clear_db() does incomplete db teardown (LP: #1080988)
    - [c1e19d7] Unauthorized command: cat /proc/None/cmdline (LP: #1077651)
    - [af9e076] At times a instance will not receive an IP address from the DHCP
      agent (LP: #1081664)
    - [e0d1a7d] allow multiple floating-ip on single port if they use different
      fixed ips and/or external nets (LP: #1057844)
    - [8471d79] Delete port fails to gateway ip (LP: #1079980)
    - [aca8b4a] fixed_ip allocation which is not included within
      allocation_pools makes error when delete port or re-create port
      (LP: #1077292)
    - [eacc9d3] Mapping same bridge to different phyiscal networks succeed
      (LP: #1067669)
    - [51b4c82] python-quantum: not region aware (LP: #1080793)
    - [6f0a486] delete floatingip should be in one transaction to delete port
      (LP: #1080516)
    - [db6cda7] Remove qpid configuration variables no longer supported
    - [a112840] Allow NVP plugin to use per-tenant quota extension
    - [82b1a55] Quantum service does not restart after reboot (LP: #1073999)
    - [c01a839] There are some cases that L3 API with an invalid parameter
      returns 500. (LP: #1064765)
    - [26b383f] external network can be plugged also as internal network for one
      router (LP: #1053633)
    - [49f649c] There is a lot of cases that API with an invalid parameter
      returns 500. (LP: #1062046)
    - [4546a18] When create subnet, you con set up the value as cidr (the value
      isn't cidr form). (LP: #1067959)
    - [9ba453a] killfilter should handle updated/deleted executables
      (LP: #1073768)
    - [7c8a55c] a port which is not able to delete is made when floatingip
      create fails. (LP: #1064748)
    - [c9b84cf] Linux bridge port update causes exception (LP: #1072713)
    - [cb57932] I can't add interface to router, if there is another port in
      non-shared network of other tenant (LP: #1057558)
    - [574e278] Ryu plugin does not support Security Groups (LP: #1059393)
    - [607f486] tap device added to integration bridge without tag
      (LP: #1064070)
    - [21a0fdf] L3 agent external network flag (LP: #1056720)
    - [5cbaff4] router create with external_gateway_info fails with 500 always.
      (LP: #1064235)
    - [63b81f6] l3 db operations failed in multiple transactions (LP: #1070335)
    - [bff17fb] Ensure that the SqlSoup import is still supported.
    - [e091a29] l3_nat_agent was renamed to l3_agent
    - [9030969] remove default value of 'local_ip' of 10...