bulk extract-archive doesn't seem to work for non-admin user

Sorry, this is using swift 1.8.0

https://review.openstack.org/#/c/38381/

here's the patch for this bug

David,

   I'm testing this out and all seems to work fairly well (still seeing
some issues with X-Container-Meta-Quota-Count adherence, but I think
that's a function of the container quota code.

   One question. What does the 'yield_frequency' value actually control
here? Would it have anything to do with timing for the eventual
consistency errors that can occur with the quota code?

Thanks,

Steve Mayer
<email address hidden>

On 23 Jul 2013, at 15:04, David Goetz wrote:

> https://review.openstack.org/#/c/38381/
>
> here's the patch for this bug
>
> --
> You received this bug notification because you are subscribed to the
> bug
> report.
> https://bugs.launchpad.net/bugs/1203182
>
> Title:
> bulk extract-archive doesn't seem to work for non-admin user
>
> Status in OpenStack Object Storage (Swift):
> New
>
> Bug description:
> When performing the bulk extract-archive operation, it appears that
> the action only functions properly if the user is an admin user.
> Attempting to perform this action with a user that has the ReadWrite
> role ACL privileges fails with a 403 Forbidden on each file in the
> archive. NOTE: This same user can write/delete normal objects into
> the container successfully.
>
> Here are details from a question I asked on this issue:
>
> I'm trying to figure out if the bulk.py extract-archive feature is
> supposed to work for non-admin accounts that have the X-Container-
> Write privilege? I realize that if the extract-archive action would
> attempt to create a container, this would require that the action be
> performed by an admin account, but what if the extraction is to occur
> into an existing container with the proper X-Container-Write ACL set?
>
> Anyone have any information to this effect?
>
> Here's what I'm seeing:
>
> 1. If I authenticate as a service admin user, all bulk operations
> succeed (and as an admin, I appear to be immune to container quota
> values. Not sure if this is correct, but it makes sense).
>
> 2. If I authenticate as a user that is part of a ReadWrite role that
> is assigned to the X-Container-Write ACL, I get 403 Forbidden errors
> when attempting to auto-extract the archive either into an existing
> container or by trying to create the container as part of the
> extraction (I would expect this scenario to fail during the container
> creation, so that is okay).
>
> What I've verified is that the keystoneauth authorize function in a
> normal upload is hit twice. Once by the proxy.server (where there are
> no role definitions returned by the getattr(req, 'acl', None) call),
> and then again by the proxy.controller.obj class (where the getattr
> call does return the proper ReadWrite role value and the file upload
> succeeds.
>
> When doing the auto-extract though, for each file that is attempted to
> be extracted, only the first call by the proxy.server class seems to
> hit the keystone.authorize() function. Since there is no role
> information associated with this call, the denied_response is
> returned.
>
> I'm at a loss as to this difference in behavior other than the fact
> that bulk.py is making subrequests using Request.blank(). Is something
> getting lost in the mix?
>
> The proxy-se...

The quota count isn't going to work perfectly because the container size info is cached for 60 seconds (i think you can lower that if you wanted). All the yield_frequency does is that's how often the server will spit out an ' ' to keep the connection alive while its processing the request.

I can't seem to reproduce this with current trunk (2013-08-09), this seems to works fine I can do bulk delete which has a ACL write access to that container.

This is the paste:

http://pastie.org/pastes/8221526/text?key=dneozmycn7ratsduz2lrw

using devstack keystone latest.

Thanks for the info. I'll take a look into this further from our side.

Have there been any changes in this area between Grizzly and trunk that
might account for the difference in behavior?

Did you try to extract-archive method (which is what the bug was
originally filed for)?

Thanks,

Steve Mayer
<email address hidden>

On 9 Aug 2013, at 4:43, Chmouel Boudjnah wrote:

> I can't seem to reproduce this with current trunk (2013-08-09), this
> seems to works fine I can do bulk delete which has a ACL write access
> to
> that container.
>
> This is the paste:
>
> http://pastie.org/pastes/8221526/text?key=dneozmycn7ratsduz2lrw
>
> using devstack keystone latest.
>
> --
> You received this bug notification because you are subscribed to the
> bug
> report.
> https://bugs.launchpad.net/bugs/1203182
>
> Title:
> bulk extract-archive doesn't seem to work for non-admin user
>
> Status in OpenStack Object Storage (Swift):
> New
>
> Bug description:
> When performing the bulk extract-archive operation, it appears that
> the action only functions properly if the user is an admin user.
> Attempting to perform this action with a user that has the ReadWrite
> role ACL privileges fails with a 403 Forbidden on each file in the
> archive. NOTE: This same user can write/delete normal objects into
> the container successfully.
>
> Here are details from a question I asked on this issue:
>
> I'm trying to figure out if the bulk.py extract-archive feature is
> supposed to work for non-admin accounts that have the X-Container-
> Write privilege? I realize that if the extract-archive action would
> attempt to create a container, this would require that the action be
> performed by an admin account, but what if the extraction is to occur
> into an existing container with the proper X-Container-Write ACL set?
>
> Anyone have any information to this effect?
>
> Here's what I'm seeing:
>
> 1. If I authenticate as a service admin user, all bulk operations
> succeed (and as an admin, I appear to be immune to container quota
> values. Not sure if this is correct, but it makes sense).
>
> 2. If I authenticate as a user that is part of a ReadWrite role that
> is assigned to the X-Container-Write ACL, I get 403 Forbidden errors
> when attempting to auto-extract the archive either into an existing
> container or by trying to create the container as part of the
> extraction (I would expect this scenario to fail during the container
> creation, so that is okay).
>
> What I've verified is that the keystoneauth authorize function in a
> normal upload is hit twice. Once by the proxy.server (where there are
> no role definitions returned by the getattr(req, 'acl', None) call),
> and then again by the proxy.controller.obj class (where the getattr
> call does return the proper ReadWrite role value and the file upload
> succeeds.
>
> When doing the auto-extract though, for each file that is attempted to
> be extracted, only the first call by the proxy.server class seems to
> hit the keystone.authorize() function. Since there is no role
> information associated with this call, the denied_response is
> returned.
>
> I'm at a loss as to this difference in behavior other than the fact
> that...