Failing to stream object in time causes silent truncation

Bug #1389972 reported by Stuart Bishop
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
python-swiftclient
New
Undecided
Unassigned

Bug Description

If there is a delay between starting a chunked download using get_object and actually reading in the chunks, the downloaded object will be silently truncated. For example, this 700MB file ends up truncated to just over 1MB, and nobody would notice unless they checked the length or md5 after the download. Using python-swiftclient 2.0.3

>>> c = get_swift_con()
>>> headers, obj_gen = c.get_object('librarian_2272', '1136356164', resp_chunk_size=1024*1024)
>>> pprint(headers)
{'accept-ranges': 'bytes',
 'content-length': '734397484',
 'content-type': 'application/octet-stream',
 'date': 'Thu, 06 Nov 2014 06:50:38 GMT',
 'etag': '0d52f3bf8e7834c02d4135b3664408e7',
 'last-modified': 'Wed, 05 Nov 2014 19:16:39 GMT',
 'x-timestamp': '1415214998.16562',
 'x-trans-id': 'tx9d83eba118f34e55bfad4-00545b1a3e'}
>>> time.sleep(300)
>>> for chunk in obj_gen:
... print len(chunk)
...
1048576
159600

Revision history for this message
Tim Burke (1-tim-z) wrote :

I think this should have been addressed in https://review.openstack.org/#/c/159208/ (commit 4af623bcf171a63240849b84b9359a4f74471455). Mind checking out the latest release and verifying that it works for you?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.