Failing to stream object in time causes silent truncation
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
python-swiftclient |
New
|
Undecided
|
Unassigned |
Bug Description
If there is a delay between starting a chunked download using get_object and actually reading in the chunks, the downloaded object will be silently truncated. For example, this 700MB file ends up truncated to just over 1MB, and nobody would notice unless they checked the length or md5 after the download. Using python-swiftclient 2.0.3
>>> c = get_swift_con()
>>> headers, obj_gen = c.get_object(
>>> pprint(headers)
{'accept-ranges': 'bytes',
'content-length': '734397484',
'content-type': 'application/
'date': 'Thu, 06 Nov 2014 06:50:38 GMT',
'etag': '0d52f3bf8e7834
'last-modified': 'Wed, 05 Nov 2014 19:16:39 GMT',
'x-timestamp': '1415214998.16562',
'x-trans-id': 'tx9d83eba118f3
>>> time.sleep(300)
>>> for chunk in obj_gen:
... print len(chunk)
...
1048576
159600
I think this should have been addressed in https:/ /review. openstack. org/#/c/ 159208/ (commit 4af623bcf171a63 240849b84b9359a 4f74471455) . Mind checking out the latest release and verifying that it works for you?