juju does not work with Walrus when s3-uri has a suffix

I am facing the same problem.

version: 0.5+bzr440-1ju
platform: eucalyptus
Instance: oneiric

Can you add some insight as to how the S3 URI is used? Is there a bucket that has been created that has the configuration file for the juju charm? Also, what s3 tool is being used to communicate with Walrus?

Regards,

Harold Spencer, Jr.
Technical Support Engineer
Eucalyptus Systems, Inc.

This problem appears in the first steps of juju installation. I don't know well juju's architecture to explain how it uses Walrus, maybe someone of juju dev team could help.

Checking out the txaws-list-buckets command, I was able to narrow down where the issue may lie. It seems that txaws is expecting some additional information for the S3 URL. I used the python debugger to step through txaws-list-buckets and I was able to rule out the "/" issue.

Also, because of this error, nothing is ever sent to the wire. The communication is dead before hitting the Walrus component.

Version:

# txaws-list-buckets --version
txAWS 0.0.1

Test Command:

# txaws-list-buckets -a <access-key> -s <secret-key> -U 192.168.2.75:8773/services/Walrus
> /usr/bin/txaws-list-buckets(35)<module>()
-> region = AWSServiceRegion(
(Pdb) next
> /usr/bin/txaws-list-buckets(36)<module>()
-> creds=creds, region=options.region, s3_endpoint=options.url)
(Pdb) print options.url
192.168.2.75:8773/services/Walrus
(Pdb) next
TypeError: "__init__() got an unexpected keyword argument 's3_endpoint'"
> /usr/bin/txaws-list-buckets(36)<module>()
-> creds=creds, region=options.region, s3_endpoint=options.url)

## Without "/"

# txaws-list-buckets -a <access-key> -s <secret-key> -U 192.168.2.75:8773
> /usr/bin/txaws-list-buckets(35)<module>()
-> region = AWSServiceRegion(
(Pdb) next
> /usr/bin/txaws-list-buckets(36)<module>()
-> creds=creds, region=options.region, s3_endpoint=options.url)
(Pdb) print options.url
192.168.2.75:8773
(Pdb) next
TypeError: "__init__() got an unexpected keyword argument 's3_endpoint'"
> /usr/bin/txaws-list-buckets(36)<module>()
-> creds=creds, region=options.region, s3_endpoint=options.url)

Hope this helps.

Harold Spencer Jr.
Support Engineer
Eucalyptus Systems, Inc.

Sorry, I forgot to add my instance and cloud information:

## Instance ##

root@ubuntu:~# lsb_release --release
Release: 10.04

root@ubuntu:~# python --version
Python 2.6.5

## Eucalyptus Cloud Setup ##

RHEL 5.5, Eucalyptus Open Source 2.0.3, Network mode: Managed

Regards,

Harold Spencer Jr.
Support Engineer
Eucalyptus Systems, Inc.

The problem in this case is that Walrus is breaking the convention established by the real S3 of owning the full path namespace of the TCP address that hosts the service.

So, while in S3 you'd access <addr>/bucket-name/key, in Walrus one has to use something like <addr>/path/for/walrus/bucket-name/key

It's not a big deal, but it's unconventional, and libraries have to be adapted to support it. IIRC, even boto itself, which is used in Eucalytpus tools, had to be patched by them to support that.

Perhaps as a workaround until walrus can be run without a prefix, a user could run an apache server using mod_proxy to send these queries to walrus.. Should be as simple as:

ProxyPass / http://real.walrus.server/walrus/path

Its not clear we have the resource availability to fix this for 12.04.. i'd really like to make it work though, but properly its a bug in txaws for eucalyptus s3 support.

This patch honors a suffix in an s3 uri as used by Walrus. It also passes the IpProtocol arg when creating a security group which works around a bug in Euca 3.x.

This patch causes juju to use the 'latest' api version rather than 1.0 when determining the instance id via a meta service call

This issue is now being tracked upstream at http://eucalyptus.atlassian.net/browse/EUCA-2788

Please watch that issue for further updates.

Can somebody please comment on the status? It seems to be dead upstream. I'm thinking this really is just a bug in Euca and we won't be addressing it in Euca unless users request a workaround.

Hi Clint,

I actually worked around this in the txaws library quite easily, but then discovered other issues that prevented JuJu from working that were over my head, so I punted it to engineering. I'll check in on the status tomorrow and update this bug with relevant info when I have it.

Thanks!