kuryr-kubernetes

Usage of IPDB leaks processes

Bug #1728996 reported by Michal Dulko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kuryr-kubernetes
Fix Released
High
Michal Dulko

Bug Description

IPDB module spawns thread that will listen for changes on the netns it's watching. If that thread isn't stopped (using IPDB.release() method), process that used that IPDB object will hang forever until it's killed manually.

We need to switch all of our IPDB usages to context managers to avoid that.

Antoni Segura Puimedon (celebdor)
Changed in kuryr-kubernetes:
assignee: nobody → Michal Dulko (michal-dulko-f)
importance: Undecided → High
Revision history for this message
Michal Dulko (michal-dulko-f) wrote :

I don't know why my patch wasn't caught automatically, but here it is: https://review.openstack.org/#/c/517406/

Changed in kuryr-kubernetes:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kuryr-kubernetes (master)

Reviewed: https://review.openstack.org/517406
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=1c2320e11e77e3c996b4a6ba753097ec5f048ced
Submitter: Zuul
Branch: master

commit 1c2320e11e77e3c996b4a6ba753097ec5f048ced
Author: Michał Dulko <email address hidden>
Date: Thu Nov 2 19:30:51 2017 +0100

    Prevent pyroute2.IPDB threads leaking

    pyroute2.IPDB is an interesting entity. It's much more than a simple
    interface to `ip` command, as it is more similar to a database process.
    When created IPDB object spawns a thread that will be responsible for
    updating the object with changes to the underlying OS. Thread stays up
    until the user will call `release()` method.

    Turns out code in kuryr_kubernetes.cni.binding wasn't taking that into
    account and was slowly leaking threads (and possibly processes). This
    became apparent when running Kuryr with CNI daemon enabled.

    This commit fixes the problem by switching all IPDB usages to context
    managers, so `release()` method is called automatically. Also the IPDB
    objects cache is removed as already released IPDB objects cannot be
    reused. kuryr_kubernetes.cni.binding modules were missing unit tests,
    this commit adds them as well.

    Change-Id: I82afda3f217dac56228677bb66703c3d80e5d751
    Closes-Bug: 1728996

Changed in kuryr-kubernetes:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kuryr-kubernetes 0.3.0

This issue was fixed in the openstack/kuryr-kubernetes 0.3.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.