euca-upload-bundle fails when connecting to Eucalyptus and a path in bucket is specified

I'm surprised it works in S3 at all. AWS's documentation [1] doesn't allow '/' characters in bucket names and further recommends naming buckets such that they are valid as part of DNS names.

[1] http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?BucketRestrictions.html

The command specified in the bug report doesn't work with AWS S3 unless the user has actually created the bucket somehow. I just tested this.

For instance, if you use the option "-b mybucket/abcd/efgh" S3 will correctly interpret this request (PUT to mybucket/abcd/efgh) as an empty put to the bucket "mybucket" with the keyname "abcd/efgh." Which is correct behavior.

The user must have already have a bucket, "mybucket." Otherwise the key "abcd/efgh" cannot be created because the bucket does not exist. Before the key can be created, the bucket needs to be created with a PUT to "mybucket"

"/"s are not allowed in the bucket name. The "/" is interpreted as part of the key. The bucket must exist for that key to be created.

So the bug report is partly right and partly wrong.

The real bug is that Walrus should be returning a 404 and not a 400 when someone tries to access "mybucket/abcd/efgh" if the bucket does not exist.

The command will not work on S3 or against Walrus if the bucket (the part of the string before the first "/") was never created, but the right error to send back is a 404 (no such bucket) and not a 400 (bad request).

This is fixed in revno 1257 in lp:eucalyptus.

Hope all this makes sense,
neil

Neil,

This means that I need to create the required directory structure (f14-basic/fedora/14/1.0/x86_64) in my bucket (goldmann-test) first and after this upload the image?

--Marek

Marek, it means that the bucket ("goldmann-test") needs to be created first. The "directory" structure is part of the key.

Hope that helps.
neil

Neil,

I created the bucket, but I still get the same result:

$ ~/s3cmd-0.9.8.3$ ./s3cmd ls
2011-02-01 19:43 s3://goldmann-test

$ euca-upload-bundle -U http://URL:8773/services/Walrus -b goldmann-test/f14-basic/fedora/14/1.0/x86_64 -m build/appliances/x86_64/fedora/14/f14-basic/s3-plugin/ami/f14-basic.ec2.manifest.xml -a KEY -s SECRET_KEY
Checking bucket: goldmann-test/f14-basic/fedora/14/1.0/x86_64
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error
S3ResponseError: 400 Bad Request
Failure: 400 Bad Request
Could not determine operation name for /services/Walrus/goldmann-test/f14-basic/fedora/14/1.0/x86_64/

Using: euca2ools-1.3.1-6.fc14.noarch and partnercloud.

Can you confirm that my call shouldn't fail?

It shouldn't fail as of revno 1257 as noted above :)

Did you try that revno?

I'm not sure what's running on partner cloud :)

Definitely not that :)

This is fixed in the source but not released in package form yet.

This would be nice to have it in 2.0.4.

after a chat on #eucalyptus-devel, this is not present (the mis-handling of /) in eucal3 (marking invalid for -devel). It is targeted for 2.0.4, but the IRC chat seems to imply that more work may be needed to understand the right behavior here.