Cloudfiles backend does not handle container ACLs

I have written a patch for the Pyrax backend to fix this bug, which uses get_container() initially, and only uses create_container() if necessary. It handles both completely unprivileged users, and read-only privileged users.
I can provide a diff in a separate comment if necessary?

Please do. Thanks!

On Thu, Sep 7, 2017 at 9:15 AM, Matthew S <email address hidden>
wrote:

> I have written a patch for the Pyrax backend to fix this bug, which uses
> get_container() initially, and only uses create_container() if necessary.
> It handles both completely unprivileged users, and read-only privileged
> users.
> I can provide a diff in a separate comment if necessary?
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1715650
>
> Title:
> Cloudfiles backend does not handle container ACLs
>
> Status in Duplicity:
> New
>
> Bug description:
> duplicity --version - duplicity 0.7.14
> python2 - Python 2.7.6
> Ubuntu 14.04.5
> Using a Rackspace Cloudfiles container (with ACLs enabled)
> Accessed via the Pyrax backend (also tested with Cloudfiles)
>
> With Duplicity, both the old Cloudfiles and newer Pyrax backend (both
> connect to Rackspace Cloudfiles-the-service) do not work with
> container ACLs(1). The backends try to create a container, which with
> a normal privileged user either creates a container, or returns an
> existing container of the same name. However, to take advantage of
> Rackspace container ACLs means using an unprivileged user, which only
> has read/write access to a single container, and no other privileges.
> Consequently a Duplicity backup results in a 403 Forbidden permissions
> error, even when the user being used is able to read/write just fine.
>
> 1) https://developer.rackspace.com/docs/cloud-files/v1/use-cases
> /additional-container-services-information/#container-access-control-
> lists
>
> Traceback of Pyrax error:
>
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1532, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1526, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1364, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
> File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py",
> line 1116, in ProcessCommandLine
> backup, local_pathname = set_backend(args[0], args[1])
> File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py",
> line 1005, in set_backend
> globals.backend = backend.get_backend(bend)
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 223, in get_backend
> obj = get_backend_object(url_string)
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 209, in get_backend_object
> return factory(pu)
> File "/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py",
> line 72, in __init__
> self.container = pyrax.cloudfiles.create_container(container)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line
> 103, in create
> return self._manager.create(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/object_storage.py",
> line 833, in create
> resp, resp_body = self.api.method_put(uri, headers=headers)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line
> 260, in method_put
> return self...

The following patch has been tested in our work environment, and the code is provided “as is” without any guarantee or warranty of any kind, either expressed or implied. It is licensed GPLv2, to comply with the license of this project (http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/COPYING).

/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py

--- _cf_pyrax.py.orig 2017-09-06 16:35:26.528312165 +0100
+++ _cf_pyrax.py 2017-09-07 11:28:22.729342589 +0100
@@ -70,7 +70,25 @@

         self.client_exc = pyrax.exceptions.ClientException
         self.nso_exc = pyrax.exceptions.NoSuchObject
- self.container = pyrax.cloudfiles.create_container(container)
+
+ #query rackspace for the specified container name
+ try:
+ self.container = pyrax.cloudfiles.get_container(container)
+ except pyrax.exceptions.Forbidden as e:
+ log.FatalError("%s : %s \n" % (e.__class__.__name__, util.uexc(e))+
+ "Container may exist, but access was denied.\n"+
+ "If this container exists, please check its X-Container-Read/Write headers.\n"+
+ "Otherwise, please check your credentials and permissions.",
+ log.ErrorCode.backend_permission_denied)
+ except pyrax.exceptions.NoSuchContainer as e:
+ try:
+ self.container = pyrax.cloudfiles.create_container(container)
+ except pyrax.exceptions.Forbidden as e:
+ log.FatalError("%s : %s \n" % (e.__class__.__name__, util.uexc(e))+
+ "Container does not exist, but creation was denied.\n"+
+ "You may be using a read-only user that can view but not create containers.\n"+
+ "Please check your credentials and permissions.",
+ log.ErrorCode.backend_permission_denied)

     def _error_code(self, operation, e):
         if isinstance(e, self.nso_exc):

Please note, I have observed that this patch will not display its warnings if working credentials are used for the first backup, and then changed to non-working credentials for a second backup.
Duplicity only displays "Attempt x failed. Forbidden: <html><h1>Forbidden</h1><p>Access was denied to this resource.</p></html> (HTTP 403)" at that point, with no further context. Unfortunately I do not have the time to debug this further.

It appears my patch post has been malformed when posted as a comment, as the whitespace has been taken out. Do you have any advice on a better place to submit this?

Apologies, I have found the relevant button. :-) Patch is attached.