Duplicity 0.7.00

2014-10-23 Kenneth Loafman <email address hidden>

    * Prep for 0.7.0
    * Merged in lp:~ed.so/duplicity/manpage.blocksize
      - add --max_blocksize doc
      - reorder 'a note on filename prefixes' into alphabetical order

2014-10-18 Kenneth Loafman <email address hidden>

    * Merged in lp:~johnleach/duplicity/1315437-swift-container-create
      - Check to see if the swift container exists before trying to create it,
        in case we don't have permissions to create containers. Fixes #1315437
    * Merged in lp:~moritzm/duplicity/duplicity
      - Use lftp for both FTP and FTPS
    * Adjust unit tests to expect single FTP backend

2014-10-16 Kenneth Loafman <email address hidden>

    * Merged in lp:~ed.so/duplicity/0.7-dpbx.importfix
      - fix this showstopper with the dropbox backend
        "NameError: global name 'rest' is not defined"

2014-10-12 Kenneth Loafman <email address hidden>

    * Merged in lp:~hooloovoo/duplicity/updated-README-REPO
      - Changes to README-REPO to reflect the restructuring of the directories.
    * Merged in lp:~jflaker/duplicity/BugFix1325215
      - The reference to "--progress_rate" in the man page as a parameter is
        incorrect. Should be "--progress-rate".

2014-09-30 Kenneth Loafman <email address hidden>

    * Fixed bug 1375304 with patch supplied by Aleksandar Ivanovic

2014-08-17 Kenneth Loafman <email address hidden>

    * Added sxbacked.py, Skylable backend. Waiting on man page updates.
    * Merged in lp:~ed.so/duplicity/manpage.verify
      - Clarify verify's functionality as wished for by a user surprised with a big
        bandwidth bill from rackspace.
    * Merged in lp:~jeffreydavidrogers/duplicity/duplicity
      - This change fixes two small typos in the duplicity man page.

2014-06-28 Kenneth Loafman <email address hidden>

    * Merged in lp:~3v1n0/duplicity/copy.com-backend
      - I've added a backend for Copy.com cloud storage, this supports all the
        required operations and works as it should from my tests.
      - You can use it by calling duplicity with something like:
        copy://<email address hidden>:<email address hidden>/duplicity
      - The only thing I've concerns with is the optimized support for _delete_list
        which can't be enabled here because the test_delete_list tries also to
        delete a not-existing files, and it requires the backend not to raise an
        exception in that case (is this somewhat wanted or could we do the same as
        for _delete or _query?)
    * Merged in lp:~ed.so/duplicity/webdav200fix-0.7
      - webdav backend fix "BackendException: Bad status code 200 reason OK. " when
        restarting an interrupted backup and overwriting partially uploaded volumes.
    * Merged in lp:~mterry/duplicity/webdav-fixes
      - This branch fixes two issues I saw when testing the webdav backend:
      - 1) Errors like the following: "Attempt 1 failed. BackendException: File
        /tmp/duplicity-LQ1a0i-tempdir/mktemp-u2aiyX-2 not found locally after get
        from backend". These were caused by the _get() method not calling setdata()
        on the local path object, so the rest of the code thought it didn't exist.
      - 2) Some odd issues from stale responses/data. We have a couple places in
        webdavbackend.py where we close the connection before making a request
        because of this problem. But I've changed it to do it every time, more
        reliably, by putting a _close() call inside the request() method.
      - With this, the webdav backend seems fine to me.
    * Merged in lp:~antmak/duplicity/0.7-par2-fix
      - Useful fix for verbatim par2cmdline options (like "-t" in par2-tbb version)
    * Fixed bug 1327550: OverflowError: signed integer is greater than maximum
      - Major and minor device numbers are supposed to be one byte each. Someone
        has crafted a special system image using OpenVZ where the major and minor
        device numbers are much larger (ploop devices). We treat them as (0,0).

2014-05-11 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/py2.6.0
      - Support python 2.6.0.
      - Without this branch, we only support python >= 2.6.5 because that's when
        python's urlparse.py module became its more modern incarnation. (I won't
        get into the wisdom of them making such a change in the middle of the
        2.6 lifecycle.)
      - Also, the version of lockfile that I have (0.8) doesn't work with python
        2.6.0 or 2.6.1 due to their implementation of
        threading.current_thread().ident returning None unexpectedly. So this
        branch tells lockfile not to worry about adding the current thread's
        identifier to the lock filename (we don't need a separate lock per thread,
        since our locking is per process).
      - I've tested with 2.6.0 and 2.7.6 (both extremes of our current support).
    * Update shebang line to python2 instead of python to avoid confusion.

2014-05-07 Kenneth Loafman <email address hidden>

    * Applied expat fix from edso. See answer #12 in
      https://answers.launchpad.net/duplicity/+question/248020

2014-04-30 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/encode-exceptions
      - Because exceptions often contain file paths, they have the same problem
        with Python 2.x's implicit decoding using the 'ascii' encoding that we've
        experienced before. So I added a new util.uexc() method that uses the
        util.ufn() method to convert an exception to a unicode string and used it
        around the place.
      - Bugs fixed: 1289288, 1311176, 1313966

2014-04-29 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/backend-unification
      - Reorganize and simplify backend code. Specifically:
        - Formalize the expected API between backends and duplicity. See the new
          file duplicity/backends/README for the instructions I've given authors.
        - Add some tests for our backend wrapper class as well as some tests for
          individual backends. For several backends that have some commands do all
          the heavy lifting (hsi, tahoe, ftp), I've added fake little mock commands
          so that we can test them locally. This doesn't truly test our integration
          with those commands, but at least lets us test the backend glue code.
        - Removed a lot of duplicate and unused code which backends were using (or
          not using). This branch drops 700 lines of code (~20%)
          in duplicity/backends!
        - Simplified expectations of backends. Our wrapper code now does all the
          retrying, and all the exception handling. Backends can 'fire and forget'
          trusting our wrappers to give the user a reasonable error message.
          Obviously, backends can also add more details and make nicer error
          messages. But they don't *have* to.
        - Separate out the backend classes from our wrapper class. Now there is no
          possibility of namespace collision. All our API methods use one
          underscore. Anything else (zero or two underscores) are for the backend
          class's use.
        - Added the concept of a 'backend prefix' which is used by par2 and gio
          backends to provide generic support for "schema+" in urls -- like par2+
          or gio+. I've since marked the '--gio' flag as deprecated, in favor of
          'gio+'. Now you can even nest such backends like
          par2+gio+file://blah/blah.
        - The switch to control which cloudfiles backend had a typo. I fixed this,
          but I'm not sure I should have? If we haven't had complaints, maybe we
          can just drop the old backend.
        - I manually tested all the backends we have (except hsi and tahoe -- but
          those are simple wrappers around commands and I did test those via mocks
          per above). I also added a bunch more manual backend tests to
          ./testing/manual/backendtest.py, which can now be run like the above to
          test all the files you have configured in config.py or you can pass it a
          URL which it will use for testing (useful for backend authors).

2014-04-26 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/py3-map-filter
      - In py3, map and filter return iterable objects, not lists. So in each case
        we use them, I've either imported the future version or switched to a list
        comprehension if we really wanted a list.

2014-04-25 Kenneth Loafman <email address hidden>

    * Fixed bug #1312328 WebDAV backend can't understand 200 OK response to DELETE
      - Allow both 200 and 204 as valid response to delete

2014-04-20 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/more-test-reorg
      - Here's another test reorganization / modernization branch. It does the
        following things:
        - Drop duplicity/misc.py. It is confusing to have both misc.py and util.py,
          and most of the code in misc.py was no longer used. I moved the one
          function that was still used into util.py.
        - Consolidated the various ways to run tests into just one. I made tox runs
          go through ./setup.py test, rather than nosetests. And I made the
          ./testing/run-tests scripts just call tox. Now we no longer need nosetests
          as a test dependency (although you can still use it if you want).
        - Added two more code quality automated tests: a pep8 one and a pylint one.
          I disabled almost all checks in each program that gave a warning. These
          tests just establish a baseline for future improvement.
        - Moved the test helper code into TestCase subclasses that all tests can
          use. And used more code sharing and setUp/tearDown cleverness to remove
          duplicated code.
        - Reorganized the tests in ./testing/tests into ./testing/functional and
          ./testing/unit -- for whether they drive duplicity as a subprocess or
          whether they import and test code directly. Each dir can have specialized
          TestCase subclasses now.
        - Renamed the files in ./testing/unit to more clearly indicate which file
          in ./duplicity they are unit testing.
        - Added some helper methods for tests to set environment and globals.*
          parameters more safely (i.e. without affecting other tests) by
          automatically cleaning up any such changes during test tearDown.
        - Removed test_unicode.py, since it is kind of dumb. It used to be more
          useful, but now with py2.6, we are just testing that one line of code
          in it is actually there.

2014-04-19 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/2.6isms
      - Here's a whole stack of minor syntax modernizations that will become
        necessary in python3. They all work in python2.6.
      - I've added a new test to keep us honest and prevent backsliding on these
        modernizations. It runs 2to3 and will fail the test if 2to3 finds anything
        that needs fixing (with a specific set of exceptions carved out).
      - This branch has most of the easy 2to3 fixes, the ones with obvious and
        safe syntax changes.
      - We could just let 2to3 do them for us, but ideally we use 2to3 as little
        as possible, since it doesn't always know how to solve a given problem.
        I will propose a branch later that actually does use 2to3 to generate
        python3 versions of duplicity if they are requested. But this is a first
        step to clean up the code base.
    * Merged in lp:~mterry/duplicity/drop-static
      - Drop static.py.
      - This is some of the oldest code in duplicity! A bzr blame says it is
        unmodified (except for whitespace / comment changes) since revision 1.
      - But it's not needed anymore. Not really even since we updated to python2.4,
        which introduced the @staticmethod decorator. So this branch drops it and
        its test file.
    * Merged in lp:~mterry/duplicity/encode-for-print
      - Encode translated strings before passing them to 'print'.
      - The print command can only apparently handle bytes. So when we pass it
        unicode, it freaks out. There were only four instances I saw where we used
        print, so I figured it was easiest to just convert them to use the log
        framework too.
      - That way all user-visible strings go through that framework and are subject
        to the same encoding rules.

2014-04-17 Kenneth Loafman <email address hidden>

    * Merged in lp:~fredrik-loch/duplicity/duplicity-S3-SSE
      - Adds support for server side encryption as requested in Bug #996660
    * Merged in lp:~mterry/duplicity/consolidate-tests
      - Consolidate all the duplicity-running code in the test framework.
      - This takes the four test files [1] that run duplicity itself (i.e. the
        high-level functional test files) and consolidates their code. Before, it
        was madness. Each one had its own run_duplicity method, with its own
        slightly tweaked features. This should reduce a lot of duplication.
      - [1] cleanuptest.py, finaltest.py, restarttest.py, and badupload.py
    * Merged in lp:~mterry/duplicity/modern-testing
      - Enable/use more modern testing tools like nosetests and tox as well as more
        common setup.py hooks like test and sdist.
      - Specifically:
        * move setup.py to toplevel where most tools and users expect it
        * Move and adjust test files to work when running "nosetests" in toplevel
          directory. Specifically, do a lot more of the test setup in
          tests/__init__.py rather than the run-tests scripts
        * Add small tox.ini file for using tox, which is a wrapper for using
          virtualenv. Only enable for py26 and py27 right now, since modern
          setuptools dropped support for <2.6 (and tox 1.7 recently dropped <2.6)
        * Add setup.py hooks for test and sdist which are both standard targets
          (sdist just outsources to dist/makedist right now)
    * Merged in lp:~mterry/duplicity/require-2.6
      - Require at least Python 2.6.
      - Our code base already requires 2.6, because 2.6-isms have crept in. Usually
        because we or a contributor didn't think to test with 2.4. And frankly,
        I'm not even sure how to test with 2.4 on a modern system.
    * Merged in lp:~mterry/duplicity/drop-u1
      - Ubuntu One is closing shop. So no need to support a u1 backend anymore.
    * Merged in lp:~mterry/duplicity/fix-drop-u1
      - Looks like when the drop-u1 branch got merged, its conflict got resolved
        badly. Here is the right version of backend.py to use (and also drops
        u1backend.py from POTFILES).
    * Merged in lp:~mterry/duplicity/drop-pexpect
      - Drop our local copy of pexpect in favor of a system version.
      - It's only used by the pexpect ssh backend (and if you're opting into that,
        you probably can expect that you will need pexpect) and the tests.
      - I've done a quick smoketest (backed up and restored using
        --ssh-backend=pexpect) and it seemed to work fine with a modern version
        of pexpect.

2014-03-10 Kenneth Loafman <email address hidden>

     * Merged in lp:~germer/duplicity/par2
      - This branch adds Par2 recovery files to duplicity. It is a wrapper backend
        which will create the recovery files and upload them all together with the
        wrapped backend. Corrupt archives will be detected and repaired (if
        possible) on the fly during restore.
      - It can be used with url-string par2+webdavs://USER@HOST/PATH
      - Fixes https://bugs.launchpad.net/duplicity/+bug/426282

2014-03-06 Kenneth Loafman <email address hidden>

    * Merged in lp:~ed.so/duplicity/fix.dpbx
      - Fix dpbx backend "NameError: global name 'rest' is not defined"
    * Merged in lp:~prateek/duplicity/botoimportfix
      - Switches the boto backend back to using lazy imports so there are no
        complaints during the importing of backends.

2014-02-26 Kenneth Loafman <email address hidden>

    * Merged in lp:~prateek/duplicity/s3-glacier
      - Fixes https://bugs.launchpad.net/duplicity/+bug/1039511
        - Adds support to detect when a file is on Glacier and initiates a restore
          to S3. Also merges overlapping code in the boto backends
      - Fixes https://bugs.launchpad.net/duplicity/+bug/1243246
        - Adds a --s3_multipart_max_timeout input option to limit the max execution
          time of a chunked upload to S3. Also adds debug message to calculate
          upload speed.

2014-02-24 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/pexpect-fix
      - duplicity has its own copy of pexpect. Use that instead of requiring one
        from the system.

2014-02-05 Kenneth Loafman <email address hidden>

    * Merged in lp:~mterry/duplicity/gpg-encode
      - getpass.getpass(prompt) eventually calls str(prompt). Which is a no go,
        if the prompt contains unicode. Here's a patch to always pass getpass() a
        byte string.
      - Our tests didn't catch this because they always set PASSPHRASE. I've added
        a test that passes the passphrase via stdin.

2014-01-31 Kenneth Loafman <email address hidden>

    * Applied two patches from mailing list message at:
      https://lists.nongnu.org/archive/html/duplicity-talk/2014-01/msg00030.html
      "Added command line options to use different prefixes for manifest/sig/archive files"
      This resolves https://bugs.launchpad.net/duplicity/+bug/1170161 and provides
      a workaround for https://bugs.launchpad.net/duplicity/+bug/1170113