SLO upload fails where object name has URL encoded characters
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Fix Released
|
Medium
|
Unassigned | ||
python-swiftclient |
Confirmed
|
Medium
|
Unassigned |
Bug Description
When uploading static large objects where the object name has an encoded character, the Swift server returns a 400 Bad Request when uploading the manifest file as it can't find the segments that have been uploaded, even though the paths in the error response match those in the manifest.
A reproducible test that can be run against a fresh Swift Devstack install can be found here: https:/
The long and short of it is:
>> PUT http://
<< 201 Created
>> PUT http://
<< 201 Created
>> PUT http://
>> "[{"path"
<< HTTP/1.1 400 Bad Request
<< "{"Errors": [["slo-
Note that the paths in the JSON response from the 400 bad request are identical to the original paths that were PUT against.
The following python-swiftclient command will reproduce this:
fallocate -l 38547913 large.file
swift --debug \
--os-auth-url http://
--os-
--os-username demo \
--os-password secretadmin \
upload --use-slo --segment-size 33554432 \
--object-name 'Files/
slo-test large.file
You don't show us the contents of your manifest file, but at a guess, you have put the URL-encoded version of the object name in the manifest - not the actual object name.
To clarify:
- Object names must be UTF8 strings --- that the name used internally in Swift, and appears in container listing
- To use an object name in the path of a HTTP request (e.g., when used in the PUT to upload the object), you must URL-encode the path. This is pretty much standard HTTP. Nowever, when it arrives on the proxy, Swift will un-encode the path to get the actual name.
I ntice, picking one of your examples that
AUTH_ d92af3a8d7f44a1 091fe38f863ebb6 9e/slo- test/Files/ OpenOffice. org%203. 3%20%2528en- GB%2529% 20Installation% 20Files/ openofficeorg1. cab/slo/ 1490950292. 028000/ 38547913/ 0/00000001
decodes as
Files/ OpenOffice. org 3.3 %28en-GB%29 Installation Files/openoffic eorg1.cab/ slo/1490950292. 028000/ 38547913/ 0/00000002
-- see the "%29" -- just check you have not double-endoded. As-is, Swift will think that "%28" is part of the name, not the "(" character.