swiftclient.service upload leaks filehandles for large files

Bug #1559079 reported by Chris Stratford
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
python-swiftclient
Fix Committed
Undecided
Kazufumi Noto
python-swiftclient (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

When uploading large files (i.e. multi-segment ones) using the swiftclient.service functions, if you pass a file path instead of a file object it seems to leave the files open after upload.

For example:
tmpfile = "/tmp/wibble"
obj = SwiftUploadObject(tmpfile, object_name=tmpfile)
swift.upload(container="mycontainer", objects=[obj])

After this, lsof shows /tmp/wibble is still open (multiple times)

This approach works around the problem:
with open(tmpfile, "r") as f:
    obj = SwiftUploadObject(f, object_name=tmpfile)
    swift.upload(container=container, objects=[obj]):

While I've seen this on the wily packaged version, it also appears to be a problem in the upstream (3.0) python module.

$ lsb_release -rd
Description: Ubuntu 15.10
Release: 15.10

$ apt-cache policy python-swiftclient
python-swiftclient:
  Installed: 1:2.6.0-1ubuntu1
  Candidate: 1:2.6.0-1ubuntu1
  Version table:
 *** 1:2.6.0-1ubuntu1 0
        500 http://gb.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages
        100 /var/lib/dpkg/status

Revision history for this message
Tim Burke (1-tim-z) wrote :
Changed in python-swiftclient:
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-swiftclient (Ubuntu):
status: New → Confirmed
Kazufumi Noto (knoto)
Changed in python-swiftclient:
assignee: nobody → Kazufumi Noto (knoto)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-swiftclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/446527

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-swiftclient (master)

Reviewed: https://review.openstack.org/446527
Committed: https://git.openstack.org/cgit/openstack/python-swiftclient/commit/?id=809e4cf98fceed1ddef033c86199cca48708d710
Submitter: Jenkins
Branch: master

commit 809e4cf98fceed1ddef033c86199cca48708d710
Author: Kazufumi Noto <email address hidden>
Date: Wed Mar 15 20:47:33 2017 +0000

    Close file handle after upload job

    The opened file for upload is not closed.
    This fix prevents possible file handle leak.

    Closes-Bug: #1559079
    Change-Id: Ibc58667789e8f54c74ae2bbd32717a45f7b30550

Changed in python-swiftclient:
status: In Progress → Fix Released
Revision history for this message
Tim Burke (1-tim-z) wrote :
Changed in python-swiftclient:
status: Fix Released → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-swiftclient 3.4.0

This issue was fixed in the openstack/python-swiftclient 3.4.0 release.

Changed in python-swiftclient (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.