Change log for diffoscope package in Ubuntu

diffoscope (190) unstable; urgency=medium

  [ Chris Lamb ]
  * Don't raise a traceback if we cannot de-marshal Python bytecode to support
    Python 3.7 loading newer .pyc files.
    (Closes: reproducible-builds/diffoscope#284)
  * Fix Python tests under Python 3.7 with file 5.39+.

  [ Vagrant Cascadian ]
  * Skip Python bytecode testing when "file" is older than 5.39.

  [ Roland Clobus ]
  * Detect whether the GNU_BUILD_ID field has been modified.

 -- Chris Lamb <email address hidden>  Fri, 05 Nov 2021 08:47:27 +0000

Available diffs

• diff from 189 to 190 (2.4 KiB)

diffoscope (189) unstable; urgency=medium

  [ Chris Lamb ]
  * Try some alternative suffixes (eg. ".py") to support distributions that
    strip or retain them. (Closes: reproducible-builds/diffoscope#283)
  * Skip Python bytecode testing where we do not have an expected diff.
    (Closes: reproducible-builds/diffoscope#284)
  * Refactor the find_executable utility into an explicit method.
  * Split out a custom call to assert_diff to support a .startswith equivalent.
  * Use skipif instead of manual conditionals in some tests.

  [ Vagrant Cascadian ]
  * Add an external tool reference for Guix to support ppudump and dumppdf.

  [ Sergei Trofimovich ]
  * Update uImage test output for file(1) version 5.41.

  [ Jelle van der Waa ]
  * Add Arch Linux as CI test target.
  * Add external tools on Arch Linux for ffmpeg, openssl and ocalobjinfo.

 -- Chris Lamb <email address hidden>  Fri, 29 Oct 2021 09:47:04 +0100

Available diffs

• diff from 188 to 189 (3.1 KiB)

diffoscope (188) unstable; urgency=medium

  * Add support for Python Sphinx inventory files, usually named objects.inv.
  * Fix Python bytecode decompilation tests with Python 3.10+.
    (Closes: reproducible-builds/diffoscope#278)

 -- Chris Lamb <email address hidden>  Fri, 22 Oct 2021 09:08:55 +0100

Available diffs

• diff from 187 to 188 (4.7 KiB)

diffoscope (187) unstable; urgency=medium

  * Add support for comparing .pyc files. Thanks to Sergei Trofimovich.
    (Closes: reproducible-builds/diffoscope#278)

 -- Chris Lamb <email address hidden>  Fri, 08 Oct 2021 09:01:52 +0100

diffoscope (180) unstable; urgency=medium

  * Don't include specific ".debug"-like lines in the output, as it invariably
    a duplicate of the debug ID that exists in a better form in the readelf(1)
    differences for this file.
  * Also ignore include short "GCC" lines that differs on a single prefix byte
    too. These are distracting, not very useful and are simply the strings(1)
    command's idea of the build ID, which, again, is displayed nearby in the
    file's diff.
  * Update the invocation arguments and tests for the latest version of
    odt2txt.

 -- Chris Lamb <email address hidden>  Fri, 06 Aug 2021 13:57:31 +0100

Available diffs

• diff from 179 to 180 (2.2 KiB)

diffoscope (179) unstable; urgency=medium

  * Ensure that various LLVM tools are installed, even when testing whether
    a MacOS binary has zero differences when compared to itself.
    (Closes: reproducible-builds/diffoscope#270)

 -- Chris Lamb <email address hidden>  Fri, 30 Jul 2021 09:05:01 +0100

Available diffs

• diff from 177 to 179 (13.7 KiB)
• diff from 178 to 179 (1.6 KiB)

diffoscope (178) unstable; urgency=medium

  [ Chris Lamb ]
  * Don't traceback on an broken symlink in a directory.
    (Closes: reproducible-builds/diffoscope#269)
  * Rewrite the calculation of a file's "fuzzy hash" to make the control
    flow cleaner.

  [ Balint Reczey ]
  * Support .deb package members compressed with the Zstandard algorithm.
    (LP: #1923845)

  [ Jean-Romain Garnier ]
  * Overhaul the Mach-O executable file comparator.
  * Implement tests for the Mach-O comparator.
  * Switch to new argument format for the LLVM compiler.
  * Fix test_libmix_differences in testsuite for the ELF format.
  * Improve macOS compatibility for the Mach-O comparator.
  * Add llvm-readobj and llvm-objdump to the internal EXTERNAL_TOOLS data
    structure.

  [ Mattia Rizzolo ]
  * Invoke gzip(1) with the short option variants to support Busybox's gzip.

 -- Chris Lamb <email address hidden>  Fri, 16 Jul 2021 15:37:59 +0100

Available diffs

• diff from 177 to 178 (12.9 KiB)

diffoscope (177) unstable; urgency=medium

  [ Keith Smiley ]
  * Improve support for Apple "provisioning profiles".
  * Fix ignoring objdump tests on MacOS.

 -- Chris Lamb <email address hidden>  Fri, 04 Jun 2021 10:03:04 +0100

Available diffs

• diff from 176 to 177 (1.4 KiB)

diffoscope (176) unstable; urgency=medium

  * Update ffmpeg tests to work with ffmpeg 4.4.
    (Closes: reproducible-builds/diffoscope#258)

 -- Chris Lamb <email address hidden>  Fri, 28 May 2021 10:52:25 +0100

Available diffs

• diff from 175 to 176 (1.2 KiB)

diffoscope (175) unstable; urgency=medium

  * Use the actual filesystem path name (instead of diffoscope's concept of the
    source name) to correct APK filename filtering when an APK file is in
    another container -- we need to filter the auto-generated "1.apk" instead
    of "original-filename.apk". (Closes: reproducible-builds/diffoscope#255)
  * Don't call os.path.basename twice.
  * Correct grammar in a fsimage.py debug message.
  * Add a comment about stripping filenames.

 -- Chris Lamb <email address hidden>  Fri, 14 May 2021 10:43:10 +0100

Available diffs

• diff from 174 to 175 (1.4 KiB)

diffoscope (174) unstable; urgency=medium

  [ Chris Lamb ]
  * Check that we are parsing an actual Debian .buildinfo file, not just
    a file with that extension.
    (Closes: #987994, reproducible-builds/diffoscope#254)
  * Support signed .buildinfo files again -- file(1) reports them as
    "PGP signed message".

  [ Mattia Rizzolo ]
  * Make the testsuite pass with file(1) version 5.40.
  * Embed some short test fixtures in the test code itself.
  * Fix recognition of compressed .xz files with file(1) 5.40.

 -- Chris Lamb <email address hidden>  Fri, 07 May 2021 10:48:54 +0100

Available diffs

• diff from 173 to 174 (2.3 KiB)

diffoscope (173) unstable; urgency=medium

  [ Chris Lamb ]
  * Add support for showing annotations in PDF files.
    (Closes: reproducible-builds/diffoscope#249)
  * Move to assert_diff in test_pdf.py.

  [ Zachary T Welch ]
  * Difference.__init__: Demote unified_diff argument to a Python "kwarg".

 -- Chris Lamb <email address hidden>  Fri, 30 Apr 2021 08:51:21 +0100

Available diffs

• diff from 171 to 173 (5.2 KiB)
• diff from 172 to 173 (4.2 KiB)

diffoscope (172) unstable; urgency=medium

  * If zipinfo(1) shows a difference but we cannot uncover a difference within
    the underlying .zip or .apk file, add a comment and show the binary
    comparison. (Closes: reproducible-builds/diffoscope#246)
  * Make "error extracting X, falling back to binary comparison E" error
    message nicer.

 -- Chris Lamb <email address hidden>  Fri, 02 Apr 2021 08:49:24 +0100

diffoscope (171) unstable; urgency=medium

  [ Mattia Rizzolo ]
  * Do not list as a "skipping reason" tools that do exist.
  * Drop the "compose" tool from the list of required tools for these tests,
    since it doesn't seem to be required.

 -- Chris Lamb <email address hidden>  Fri, 26 Mar 2021 09:35:37 +0000

Available diffs

• diff from 170 to 171 (1.1 KiB)

diffoscope (170) unstable; urgency=medium

  [ Chris Lamb ]
  * Avoid frequent long lines in RPM header outputs that cause very very slow
    HTML outputs. (Closes: reproducible-builds/diffoscope#245)
  * Fix test_libmix_differences on openSUSE Tumbleweed.
    (Closes: #reproducible-builds/diffoscope#244)
  * Move test_rpm to use the assert_diff utility helper.

  [ Hans-Christoph Steiner ]
  * Add a diffoscope.tools.get_tools() method to support programmatically
    fetching Diffoscope's config.

  [ Roland Clobus ]
  * Become tolerant of malformed Debian .changes files.

 -- Chris Lamb <email address hidden>  Fri, 19 Mar 2021 10:30:53 +0000

Available diffs

• diff from 169 to 170 (4.1 KiB)

diffoscope (169) unstable; urgency=medium

  [ Chris Lamb ]
  * Optimisations:
    - Use larger buffer/block sizes when extracting files from libarchive-
      based archives.
    - Use a much-shorter CSS class (instead of "diffponct") to dramatically
      reduce uncompressed HTML output.

  * Logging improvements:
    - Don't emit "Unable to stat file" warning/debug messages; we have
      entirely-artificial directory entries such as ELF sections which, of
      course, never exist as filesystem files.
    - Don't emit a "Returning a FooContainer" logging message - we already emit
      "Instantiating a FooContainer" one and are unlikely to fail in the
      middle.
    - Format the report size logging messages when generating HTML reports.
    - Add the target directory when logging which directory we are extracting
      containers to.

  * Miscellaneous:
    - Ignore "--debug" and similar arguments when creating a (hopefully useful)
      temporary directory.
    - Ensure all internal temporary directories have useful names.
    - Clarify a comment regarding diffoscope not extracting excluded files.

  [ Vagrant Cascadian ]
  * Skip a DEX-related test if the "procyon" tool is unavailable.

 -- Chris Lamb <email address hidden>  Fri, 12 Mar 2021 15:33:54 +0000

Available diffs

• diff from 168 to 169 (3.4 KiB)

diffoscope (168) unstable; urgency=medium

  * Don't call difflib.Differ.compare with very large inputs; it is at least
    O(n^2) and makes diffoscope appear to hang.
    (Closes: reproducible-builds/diffoscope#240)
  * Don't use "Inheriting PATH of X" in debug log message; use "PATH is X".
  * Correct the capitalisation of jQuery.

 -- Chris Lamb <email address hidden>  Sat, 27 Feb 2021 09:34:37 +0000

Available diffs

• diff from 167 to 168 (1.1 KiB)

diffoscope (167) unstable; urgency=medium

  * Temporary directory handling:
    - Ensure we cleanup our temporary directory by avoiding confusion between
      the TemporaryDirectory instance and the underlying directory.
      (Closes: #981123)
    - Use a potentially-useful suffix to our temporary directory based on the
      command-line passed to diffoscope.
    - Fix some tempfile/weakref interaction in Python 3.7 (ie. Debian buster).
      (Closes: reproducible-builds/diffoscope#239)
    - If our temporary directory does not exist anymore (eg. it has been
      cleaned up in tests, signal handling or reference counting),  make sure
      we recreate it.

  * Bug fixes:
    - Don't rely on magic.Magic(...) to have an identical API between file's
      magic.py and PyPI's "python-magic" library.
      (Closes: reproducible-builds/diffoscope#238)
    - Don't rely on dumpimage returning an appropriate exit code; check that
      the file actually exists after we call it.

  * Codebase changes:
    - Set a default Config.extended_filesystem_attributes.
    - Drop unused Config.acl and Config.xattr attributes.
    - Tidy imports in diffoscope/comparators/fit.py.

  * Tests:
    - Add u-boot-tools to test dependencies so that salsa.debian.org pipelines
      actually test the new FIT comparator.
    - Strip newlines when determining Black version to avoid "requires black
      >= 20.8b1 (18.9b0\n detected)" in test output (NB. embedded newline).
    - Gnumeric is back in testing so re-add to test dependencies.
    - Use assert_diff (over get_data, etc.) in the FIT and APK comparators.
    - Mark test_apk.py::test_android_manifest as being allowed to fail for now.
    - Fix the FIT tests in buster and unstable.

 -- Chris Lamb <email address hidden>  Fri, 19 Feb 2021 09:06:42 +0000

Available diffs

• diff from 163 to 167 (14.8 KiB)
• diff from 166 to 167 (4.6 KiB)

diffoscope (166) unstable; urgency=medium

  [ Chris Lamb ]
  * New features and bugfixes:
    - Explicitly remove our top-level temporary directory.
      (Closes: #981123, reproducible-builds/diffoscope#234)
    - Increase fuzzy matching threshold to 130 ensure that we show more
      differences. (Closes: reproducible-builds/diffoscope#232)
    - Save our sys.argv in our top-level temporary directory in case it
      helps debug current/errant temporary directories.
    - Prefer to use "magic.Magic" over the "magic.open" compatibility
      interface. (Closes: reproducible-builds/diffoscope#236)
    - Reduce fuzzy threshold to 110 to prevent some test failures.
      (Closes: reproducible-builds/diffoscope#233)

  * Output improvements:
    - Show fuzzyness amount in percentage terms, not out of the
      rather-arbitrary "400".
    - Improve the logging of fuzzy matching.
    - Print the free space in our temporary directory when we create it, not
      from within diffoscope.main.

  * Codebase improvements:
    - Tidy the diffoscopecomparators.utils.fuzzy module.
    - Update my copyright years.
    - Clarify the grammar of a comment.
    - Clarify in a comment that __del__ is not always called, so temporary
      directories are not neccessary removed the *moment* they go out of scope.

  [ Conrad Ratschan ]
  * Fix U-Boot Flattened Image Tree ("FIT") image detection for larger image
    files. (MR: reproducible-builds/diffoscope!75)

 -- Chris Lamb <email address hidden>  Sat, 30 Jan 2021 12:31:22 +0000

Available diffs

• diff from 165 to 166 (4.0 KiB)

diffoscope (165) unstable; urgency=medium

  [ Dimitrios Apostolou ]
  * Introduce the --no-acl and --no-xattr arguments [later collapsed to
    --extended-filesystem-attributes] to improve performance.
  * Avoid calling the external stat command.

  [ Chris Lamb ]
  * Collapse --acl and --xattr into --extended-filesystem-attributes to cover
    all of these extended attributes, defaulting the new option to false (ie.
    to not check these very expensive external calls).

  [ Mattia Rizzolo ]
  * Override several lintian warnings regarding prebuilt binaries in the
  * source.
  * Add a pytest.ini file to explicitly use Junit's xunit2 format.
  * Ignore the Python DeprecationWarning message regarding the `imp` module
    deprecation as it comes from a third-party library.
  * debian/rules: filter the content of the d/*.substvars files

 -- Chris Lamb <email address hidden>  Sat, 23 Jan 2021 08:57:13 +0000

Available diffs

• diff from 164 to 165 (2.8 KiB)

diffoscope (164) unstable; urgency=medium

  [ Chris Lamb ]
  * Truncate jsondiff differences at 512 bytes lest they consume the entire page.
  * Wrap our external call to cmp(1) with a profile (to match the internal
    profiling).
  * Add a note regarding the specific ordering of the new
    all_tools_are_listed test.

  [ Dimitrios Apostolou ]
  * Performance improvements:
    - Improve speed of has_same_content by spawning cmp(1) less frequently.
    - Log whenever the external cmp(1) command is spawn.ed
    - Avoid invoking external diff for identical, short outputs.
  * Rework handling of temporary files:
    - Clean up temporary directories as we go along, instead of at the end.
    - Delete FIFO files when the FIFO feeder's context manager exits.

  [ Mattia Rizzolo ]
  * Fix a number of potential crashes in --list-debian-substvars, including
    explicitly listing lipo and otool as external tools.
   - Remove redundant code and let object destructors clean up after themselves.

  [ Conrad Ratschan ]
  * Add a comparator for Flattened Image Trees (FIT) files, a boot image format
    used by U-Boot.

 -- Chris Lamb <email address hidden>  Fri, 08 Jan 2021 13:00:55 +0000

Available diffs

• diff from 163 to 164 (8.6 KiB)

diffoscope (163) unstable; urgency=medium

  [ Chris Lamb ]
  * Bug fixes & new features:
    - Normalise "ret" to "retq" in objdump output to support multiple versions
      of obdump(1). (Closes: #976760, reproducible-builds/diffoscope#227)
    - Don't show progress indicators when running zstd(1).
      (Closes: reproducible-builds/diffoscope#226)
    - Move the slightly-confusing behaviour of loading an "existing diff" if a
      single file is passed to diffoscope to the new --load-existing-diff
      command.
  * Output improvements:
    - Use pprint.pformat in the JSON comparator to serialise the differences
      from jsondiff to make the output render better.
    - Correct tense in --debug log output.
  * Code quality:
    * Don't use an old-style "super" call.
    - Rewrite the filter routine for post-processing output from readelf(1).
    - Update my copyright years.
    - Remove unncessary PEP 263 encoding lines (replaced via PEP 3120).
    - Use "minimal" instead of "basic" as a variable name to match the
      underlying package name.
    - Add comment regarding Java tests for diffoscope contributors who are not
      using Debian. (Re: reproducible-builds/diffoscope!58)
  * Debian packaging:
    - Update debian/copyright to match copyright notices in source-tree.
      (Closes: reproducible-builds/diffoscope#224)
    - Ensure the new "diffoscope-minimal" package has a different short
      description from the main "diffoscope" one.

  [ Jean-Romain Garnier ]
  * Add tests for OpenJDK 14.

  [ Conrad Ratschan ]
  * Add comparator for "legacy" uboot uImage files.
    (MR: reproducible-builds/diffoscope!69)

 -- Chris Lamb <email address hidden>  Fri, 11 Dec 2020 10:07:28 +0000

Available diffs

• diff from 162ubuntu1 (in Ubuntu) to 163 (18.8 KiB)

diffoscope (162ubuntu1) hirsute; urgency=medium

  * Skip three tests with binutils 2.35.50. See Debian #976760.

 -- Matthias Klose <email address hidden>  Tue, 08 Dec 2020 11:13:51 +0100

Available diffs

• diff from 162 (in Debian) to 162ubuntu1 (4.8 KiB)

diffoscope (162) unstable; urgency=medium

  [ Chris Lamb ]
  * Don't depends on radare2 in the Debian autopkgtests as it will not be in
    bullseye due to security considerations (#950372). (Closes: #975313)
  * Avoid "Command `s p a c e d o u t` failed" messages when creating an
    artificial CalledProcessError instance in our generic from_operation
    feeder creator.
  * Overhaul long and short descriptions.
  * Use the operation's full name so that "command failed" messages include
    its arguments.
  * Add a missing comma in a comment.

  [ Jelmer Vernooij ]
  * Add missing space to the error message when only one argument is passed to
    diffoscope.

  [ Holger Levsen ]
  * Update Standards-Version to 4.5.1.

  [ Mattia Rizzolo ]
  * Split the diffoscope package into a diffoscope-minimal package that
    excludes the larger packages from Recommends. (Closes: #975261)
  * Drop support for Python 3.6.

 -- Chris Lamb <email address hidden>  Fri, 27 Nov 2020 09:41:32 +0000

Available diffs

• diff from 161ubuntu2 (in Ubuntu) to 162 (4.2 KiB)

diffoscope (161ubuntu2) hirsute; urgency=medium

  * Don't depends on radare2 in the Debian autopkgtests. Closes: #975313.

 -- Matthias Klose <email address hidden>  Wed, 25 Nov 2020 11:54:06 +0100

Available diffs

• diff from 161 (in Debian) to 161ubuntu2 (1.4 KiB)
• diff from 161ubuntu1 to 161ubuntu2 (612 bytes)

diffoscope (161ubuntu1) hirsute; urgency=medium

  * Don't depends on radare2 in the Debian autopkgtests. Closes: #975313.

 -- Matthias Klose <email address hidden>  Tue, 24 Nov 2020 20:21:25 +0100

Available diffs

• diff from 161 (in Debian) to 161ubuntu1 (1.1 KiB)

diffoscope (161) unstable; urgency=medium

  [ Chris Lamb ]
  * Fix failing testsuite: (Closes: #972518)
    - Update testsuite to support OCaml 4.11.1. (Closes: #972518)
    - Reapply Black and bump minimum version to 20.8b1.
  * Move the OCaml tests to the assert_diff helper.

  [ Jean-Romain Garnier ]
  * Add support for radare2 as a disassembler.

  [ Paul Spooren ]
  * Automatically deploy Docker images in the continuous integration pipeline.

 -- Chris Lamb <email address hidden>  Tue, 20 Oct 2020 15:52:28 +0100

Available diffs

• diff from 160 to 161 (9.0 KiB)

diffoscope (160) unstable; urgency=medium

  * Check that pgpdump is actually installed before attempting to run it.
    Thanks to Gianfranco Costamagna (locutusofborg). (Closes: #969753)
  * Add some documentation for the EXTERNAL_TOOLS dictionary.
  * Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump
    against all files that are recognised by file(1) as "data".

 -- Chris Lamb <email address hidden>  Fri, 11 Sep 2020 10:08:38 +0100

Available diffs

• diff from 159ubuntu3 (in Ubuntu) to 160 (2.2 KiB)

diffoscope (159ubuntu3) groovy; urgency=medium

  * Skip some tests if pgpdump is not installed (See: #969753)

 -- Gianfranco Costamagna <email address hidden>  Mon, 07 Sep 2020 20:46:23 +0200

Available diffs

• diff from 156 (in Debian) to 159ubuntu3 (17.3 KiB)
• diff from 159ubuntu2 to 159ubuntu3 (420 bytes)

diffoscope (159ubuntu2) groovy; urgency=medium

  * Skip some tests if pgpdump is not installed (See: #969753)

 -- Gianfranco Costamagna <email address hidden>  Mon, 07 Sep 2020 20:46:23 +0200

Available diffs

• diff from 159ubuntu1 to 159ubuntu2 (654 bytes)

diffoscope (159ubuntu1) groovy; urgency=medium

  * Skip some tests if pgpdump is not installed

 -- Gianfranco Costamagna <email address hidden>  Mon, 07 Sep 2020 20:46:23 +0200

Available diffs

• diff from 159 (in Debian) to 159ubuntu1 (1.2 KiB)

diffoscope (159) unstable; urgency=medium

  [ Chris Lamb ]
  * Show "ordering differences only" in strings(1) output.
    (Closes: reproducible-builds/diffoscope#216)
  * Don't alias output from "os.path.splitext" to variables that we do not end
    up using.
  * Don't raise exceptions when cleaning up after a guestfs cleanup failure.

  [ Jean-Romain Garnier ]
  * Make "Command" subclass a new generic Operation class.

 -- Chris Lamb <email address hidden>  Fri, 04 Sep 2020 11:12:52 +0100

Available diffs

• diff from 158 to 159 (9.1 KiB)

diffoscope (158) unstable; urgency=medium

  * Improve PGP support:
    - Support extracting of files within PGP signed data.
      (Closes: reproducible-builds/diffoscope#214)
    - pgpdump(1) can successfully parse some unrelated, non-PGP binary files,
      so check that the parsed output contains something remotely sensible
      before identifying it as a PGP file.
  * Don't use Python's repr(...)-style output in "Calling external command"
    logging output.
  * Correct a typo of "output" in an internal comment.

 -- Chris Lamb <email address hidden>  Fri, 28 Aug 2020 11:53:10 +0100

Available diffs

• diff from 157 to 158 (5.7 KiB)

diffoscope (157) unstable; urgency=medium

  [ Chris Lamb ]

  * Try obsensibly "data" files named .pgp against pgpdump to determine whether
    they are PGP files. (Closes: reproducible-builds/diffoscope#211)
  * Don't raise an exception when we encounter XML files with "<!ENTITY>"
    declarations inside the DTD, or when a DTD or entity references an external
    resource. (Closes: reproducible-builds/diffoscope#212)
  * Temporarily drop gnumeric from Build-Depends as it has been removed from
    testing due to Python 2.x deprecation. (Closes: #968742)
  * Codebase changes:
    - Add support for multiple file extension matching; we previously supported
      only a single extension to match.
    - Move generation of debian/tests/control.tmp to an external script.
    - Move to our assert_diff helper entirely in the PGP tests.
    - Drop some unnecessary control flow, unnecessary dictionary comprehensions
      and some unused imports found via pylint.
  * Include the filename in the "... not identified by any comparator"
    logging message.

 -- Chris Lamb <email address hidden>  Fri, 21 Aug 2020 12:24:25 +0100

Available diffs

• diff from 156 to 157 (7.4 KiB)

diffoscope (156) unstable; urgency=medium

  [ Chris Lamb ]
  * Update PPU tests for compatibility with Free Pascal versions 3.2.0 or
    greater. (Closes: #968124)
  * Emit a debug-level logging message when our ppudump(1) version does not
    match file header.
  * Add and use an assert_diff helper that loads and compares a fixture output
    to avoid a bunch of test boilerplate.

  [ Frazer Clews ]
  * Apply some pylint suggestions to the codebase.

 -- Chris Lamb <email address hidden>  Fri, 14 Aug 2020 10:04:26 +0100

Available diffs

• diff from 154 to 156 (7.5 KiB)
• diff from 155 to 156 (5.8 KiB)

diffoscope (155) unstable; urgency=medium

  [ Chris Lamb ]
  * Bump Python requirement from 3.6 to 3.7 - most distributions are either
    shipping3.5 or 3.7, so supporting 3.6 is not somewhat unnecessary and also
    more difficult to test locally.
  * Improvements to setup.py:
    - Apply the Black source code reformatter.
    - Add some URLs for the site of PyPI.org.
    - Update "author" and author email.
  * Explicitly support Python 3.8.

  [ Frazer Clews ]
  * Move away from the deprecated logger.warn method logger.warning.

  [ Mattia Rizzolo ]
  * Document ("classify") on PyPI that this project works with Python 3.8.

 -- Chris Lamb <email address hidden>  Fri, 07 Aug 2020 12:09:50 +0100

Available diffs

• diff from 154 to 155 (2.3 KiB)

diffoscope (154) unstable; urgency=medium

  [ Chris Lamb ]

  * Add support for F2FS filesystems.
    (Closes: reproducible-builds/diffoscope#207)
  * Allow "--profile" as a synonym for "--profile=-".
  * Add an add_comment helper method so don't mess with our _comments list
    directly.
  * Add missing bullet point in a previous changelog entry.
  * Use "human-readable" over unhyphenated version.
  * Add a bit more debugging around launching guestfs.
  * Profile the launch of guestfs filesystems.
  * Correct adding a comment when we cannot extract a filesystem due to missing
    guestfs module.

 -- Chris Lamb <email address hidden>  Fri, 31 Jul 2020 10:26:15 +0100

Available diffs

• diff from 153 to 154 (2.2 KiB)

diffoscope (153) unstable; urgency=medium

  [ Chris Lamb ]

  * Drop some legacy argument styles; --exclude-directory-metadata and
    --no-exclude-directory-metadata have been replaced with
    --exclude-directory-metadata={yes,no}.

  * Code improvements:

    - Make it easier to navigate the main.py entry point.
    - Use a relative import for get_temporary_directory in diffoscope.diff.
    - Rename bail_if_non_existing to exit_if_paths_do_not_exist.
    - Rewrite exit_if_paths_do_not_exist to not check files multiple times.

  * Documentation improvements:

    - CONTRIBUTING.md:

      - Add a quick note about adding/suggesting new options.
      - Update and expand the release process documentation.
      - Add a reminder to regenerate debian/tests/control.

    - README.rst:

      - Correct URL to build job on Jenkins.
      - Clarify and correct contributing info to point to salsa.debian.org.

 -- Chris Lamb <email address hidden>  Fri, 24 Jul 2020 12:03:51 +0100

Available diffs

• diff from 150 to 153 (8.4 KiB)
• diff from 152 to 153 (3.7 KiB)

diffoscope (152) unstable; urgency=medium

  [ Chris Lamb ]

  * Bug fixes:

    - Don't require zipnote(1) to determine differences in a .zip file as we
      can use libarchive directly.

  Reporting improvements:

    - Don't emit "javap not found in path" if it is available in the path but
      it did not result in any actual difference.
    - Fix "... not available in path" messages when looking for Java
      decompilers; we were using the Python class name (eg. "<class
      'diffoscope.comparators.java.Javap'>") over the actual command we looked
      for (eg. "javap").

  * Code improvements:

    - Replace some simple usages of str.format with f-strings.
    - Tidy inline imports in diffoscope.logging.
    - In the RData comparator, always explicitly return a None value in the
      failure cases as we return a non-None value in the "success" one.

  [ Jean-Romain Garnier ]
  * Improve output of side-by-side diffs, detecting added lines better.
    (MR: reproducible-builds/diffoscope!64)
  * Allow passing file with list of arguments to ArgumentParser (eg.
    "diffoscope @args.txt"). (MR: reproducible-builds/diffoscope!62)

 -- Chris Lamb <email address hidden>  Sat, 18 Jul 2020 11:00:13 +0100

Available diffs

• diff from 151 to 152 (4.1 KiB)

diffoscope (151) unstable; urgency=medium

  [ Chris Lamb]

  * Improvements and bug fixes:

    - Pass the absolute path when extracting members from SquashFS images as we
      run the command with our working directory set to the temporary
      directory. (Closes: #964365, reproducible-builds/diffoscope#189)
    - Increase the minimum length of the output from strings(1) to 8 characters
      to avoid unnecessary diff noise. (Re. reproducible-builds/diffoscope#148)

  * Logging improvements:

    - Fix the compare_files message when the file does not have a literal name.
    - Reduce potential log noise by truncating the has_some_content messages.

  * Codebase changes:

    - Clarify use of a "null" diff in order to remember an exit code.
    - Don't alias a variable when don't end up it; use "_" instead.
    - Use a  "NullChanges" file to represent missing data in the Debian package
      comparator.
    - Update some miscellaneous terms.

 -- Chris Lamb <email address hidden>  Fri, 10 Jul 2020 10:52:56 +0100

Available diffs

• diff from 150 to 151 (2.1 KiB)

diffoscope (150) unstable; urgency=medium

  [ Chris Lamb ]
  * Don't crash when listing entries in archives if they don't have a listed
    size (such as hardlinks in .ISO files).
    (Closes: reproducible-builds/diffoscope#188)
  * Dump PE32+ executables (including EFI applications) using objdump.
    (Closes: reproducible-builds/diffoscope#181)
  * Tidy detection of JSON files due to missing call to File.recognizes that
    checks against the output of file(1) which was also causing us to attempt
    to parse almost every file using json.loads. (Whoops.)
  * Drop accidentally-duplicated copy of the new --diff-mask tests.
  * Logging improvements:
    - Split out formatting of class names into a common method.
    - Clarify that we are generating presenter formats in the opening logs.

  [ Jean-Romain Garnier ]
  * Remove objdjump(1) offsets before instructions to reduce diff noise.
    (Closes: reproducible-builds/diffoscope!57)

 -- Chris Lamb <email address hidden>  Fri, 03 Jul 2020 11:04:46 +0100

Available diffs

• diff from 149 to 150 (5.3 KiB)

diffoscope (149) unstable; urgency=medium

  [ Chris Lamb ]
  * Update tests for file 5.39. (Closes: reproducible-builds/diffoscope#179)
  * Downgrade the tlsh warning message to an "info" level warning.
    (Closes: #888237, reproducible-builds/diffoscope#29)
  * Use the CSS "word-break" property over manually adding U+200B zero-width
    spaces that make copy-pasting cumbersome.
    (Closes: reproducible-builds/diffoscope!53)

  * Codebase improvements:
    - Drop some unused imports from the previous commit.
    - Prevent an unnecessary .format() when rendering difference comments.
    - Use a semantic "AbstractMissingType" type instead of remembering to check
      for both "missing" files and missing containers.

  [ Jean-Romain Garnier ]
  * Allow user to mask/filter reader output via --diff-mask=REGEX.
    (MR: reproducible-builds/diffoscope!51)
  * Make --html-dir child pages open in new window to accommodate new web
    browser content security policies.
  * Fix the --new-file option when comparing directories by merging
    DirectoryContainer.compare and Container.compare.
    (Closes: reproducible-builds/diffoscope#180)
  * Fix zsh completion for --max-page-diff-block-lines.

  [ Mattia Rizzolo ]
  * Do not warn about missing tlsh during tests.

 -- Chris Lamb <email address hidden>  Fri, 26 Jun 2020 15:57:41 +0100

Available diffs

• diff from 148 to 149 (10.1 KiB)

diffoscope (148) unstable; urgency=medium

  [ Daniel Fullmer ]
  * Fix a regression in the CBFS comparator due to changes in our_check_output.

  [ Chris Lamb ]
  * Add a remark in the deb822 handling re. potential security issue in the
    .changes, .dsc, .buildinfo comparator.

 -- Chris Lamb <email address hidden>  Fri, 19 Jun 2020 11:38:20 +0100

Available diffs

• diff from 146 to 148 (11.9 KiB)
• diff from 147 to 148 (1.1 KiB)

diffoscope (147) unstable; urgency=medium

  * New features:

    - Add output from strings(1) to ELF binaries. It is intended this will
      expose expose build paths that are hidden somewhere within the objdump(1)
      output. (Closes: reproducible-builds/diffoscope#148)
    - Add basic zsh shell tab-completion support.
      (Closes: reproducible-builds/diffoscope#158)

  * Bug fixes:

    - Prevent a traceback when comparing a PDF document that does not contain
      any metadata, ie. it is missing a PDF "/Info" stanza.
      (Closes: reproducible-builds/diffoscope#150)
    - Fix compatibility with jsondiff 1.2.0 which was causing a traceback and
      log the version of jsondiff we are using to aid debugging in the future.
      (Closes: reproducible-builds/diffoscope#159
    - Fix an issue in GnuPG keybox handling that left filenames in the diff.
    - Don't mask an existing test name; ie. ensure it is actually run.

  * Reporting:

    - Log all calls to subprocess.check_output by using our own wrapper utility.
      (Closes: reproducible-builds/diffoscope#151)

  * Code improvements:

    - Replace references to "WF" with "Wagner-Fischer" for clarity.
    - Drop a large number of unused imports (list_libarchive,
      ContainerExtractionError, etc.)
    - Don't assign exception to a variable that we do not use.
    - Compare string values with the equality operator, not via "is" identity.
    - Don't alias an open file to a variable when we don't use it.
    - Don't alias "filter" builtin.
    - Refactor many small parts of the HTML generation, dropping explicit
      u"unicode" strings, tidying the generation of the "Offset X, Y lines
      modified" messages, moving to PEP 498 f-strings where appropriate, etc.
    - Inline a number of single-used utility methods.

 -- Chris Lamb <email address hidden>  Thu, 11 Jun 2020 12:50:34 +0100

Available diffs

• diff from 146 to 147 (11.6 KiB)

diffoscope (146) unstable; urgency=medium

  [ Chris Lamb ]
  * Refactor .changes and .buildinfo handling to show all details (including
    the GPG header and footer components), even when referenced files are not
    present. (Closes: reproducible-builds/diffoscope#122)
  * Normalise filesystem stat(2) "birth times" (ie. st_birthtime) in the same
    way we do with stat(1)'s "Access:" and "Change:" times to fix a
    nondetermistic build failure on GNU Guix.
    (Closes: reproducible-builds/diffoscope#74)
  * Drop the (default) subprocess.Popen(shell=False) keyword argument so that
    the more unsafe shell=True is more obvious.
  * Ignore lower vs. upper-case when ordering our file format descriptions.
  * Don't skip string normalisation in Black.

  [ Mattia Rizzolo ]
  * Add a "py3dist" override for the rpm-python module (Closes: #949598)
  * Bump the debhelper compat level to 13 and use the new
    execute_after_*/execture_before_* style rules.
  * Fix a spelling error in changelog.

  [ Daniel Fullmer ]
  * Mount GuestFS filesystem images readonly.

  [ Jean-Romain Garnier ]
  * Prevent an issue where (for example) LibarchiveMember's has_same_content
    method is called regardless of the actual type of file.

 -- Chris Lamb <email address hidden>  Sat, 30 May 2020 12:36:51 +0100

Available diffs

• diff from 145 to 146 (93.6 KiB)

diffoscope (145) unstable; urgency=medium

  [ Chris Lamb ]

  * Improvements:

    - Add support for Apple Xcode mobile provisioning .mobilepovision files.
      (Closes: reproducible-builds/diffoscope#113)
    - Add support for printing the signatures via apksigner(1).
      (Closes: reproducible-builds/diffoscope#121)
    - Use SHA256 over MD5 when generating page names for the HTML directory
      presenter, validate checksums for files referenced in .changes files
      using SHA256 too, and move to using SHA256 in "Too much input for diff"
      output too. (Closes: reproducible-builds/diffoscope#124)
    - Don't leak the full path of the temporary directory in "Command [..]
      exited with 1".  (Closes: reproducible-builds/diffoscope#126)
    - Identify "iOS App Zip archive data" files as .zip files.
      (Closes: reproducible-builds/diffoscope#116)

  * Bug fixes:

    - Correct "differences" typo in the ApkFile handler.
      (Closes: reproducible-builds/diffoscope#127)

  * Reporting/output improvements:

    - Never emit the same id="foo" TML anchor reference twice, otherwise
      identically-named parts will not be able to linked to via "#foo".
      (Closes: reproducible-builds/diffoscope#120)
    - Never emit HTML with empty "id" anchor lements as it is not possible to
      link to "#" (vs "#foo"). We use "#top" as a fallback value so it will
      work for the top-level parent container.
    - Clarify the message when we cannot find the "debian" Python module.
    - Clarify "Command [..] failed with exit code" to remove duplicate "exited
      with exit" but also to note that diffoscope is intepreting this as an
      error.
    - Add descriptions for the 'fallback' Debian module file types.
    - Rename the --debugger command-line argument to --pdb.

  * Testsuite improvements:

    - Prevent CI (and runtime) apksigner test failures due to lack of
      binfmt_misc on Salsa CI and elsewhere.

  * Codebase improvements:

    - Initially add a pair of comments to tidy up a slightly abstraction level
      violating code in diffoscope.comparators.mising_file and the
      .dsc/.buildinfo file handling, but replace this later by by inlining
      MissingFile's special handling of deb822 to prevent leaking through
      abstraction layers in the first place.
    - Use a BuildinfoFile (etc.) regardless of whether the associated files
      such as the orig.tar.gz and the .deb are present, but don't treat them as
      actual containers. (Re: reproducible-builds/diffoscope#122)
    - Rename the "Openssl" command class to "OpenSSLPKCS7" to accommodate other
      commands with this prefix.
    - Wrap a docstring across multiple lines, drop an inline pprint import and
      comment the HTMLPrintContext class, etc.

  [ Emanuel Bronshtein ]
  * Avoid build-cache in building the released Docker image.
    (Closes: reproducible-builds/diffoscope#123)

  [ Holger Levsen ]
  * Wrap long lines in older changelog entries.

 -- Chris Lamb <email address hidden>  Sat, 23 May 2020 09:31:26 +0100

Available diffs

• diff from 144 to 145 (9.9 KiB)

diffoscope (144) unstable; urgency=medium

  [ Chris Lamb ]

  * Improvements:

    - Print the amount of free space that we have available in our temporary
      directory as a debugging message.
    - Remove (broken) fuzzy matching of JSON files as file 5.35 (in buster,
      released 2018-10-18) supports recognising JSON data.
      (Closes: reproducible-builds/diffoscope#106)
    - Don't pretty-print the JSON output by default as it will usually be so
      complicated to be unreadable by the human eye and it can be easily
      replaced by piping to "| jq".
    - Don't print a traceback if we pass a single, missing argument to
      diffoscope (eg. a JSON diff to re-load).

  * Reporting/output improvements:

    - Reduce the default number of maximum standard error lines printed from 50
      to 25; usually the error is obvious by that point.
    - Clarify the message when we truncate the number of lines to standard error.
    - Clarify when an external command emits for both files, otherwise it can
      look like diffoscope is repeating itself when it is actually being run
      twice.
    - Don't repeat "stderr from {}" if both commands emit the same thing.

  * Dockerfile improvements:

    - Use ARG instead of ENV for DEBIAN_FRONTEND so we do not set this
      environment variable at runtime.
      (Closes: reproducible-builds/diffoscope#103)
    - Run diffoscope as a non-root user in the runtime container.
      (Closes: reproducible-builds/diffoscope#102)
    - Add a .dockerignore file to whitelist files we need in our container.
      Thanks to Emanuel Bronshtein for the original idea.
      (Closes: reproducible-builds/diffoscope#105)
    - Install/remove the build-essential package during build so we can install
      the recommended packages from Git.

  * Testsuite improvements:

    - Include the Black output in the assertion failure too.
    - Update the Black self-test; we don't care about the length of the black
      output, rather whether it has some or, preferably, not.

  * Codebase improvements:

    - Bump the officially required version of Python from 3.5 to 3.6.
      (Closes: reproducible-builds/diffoscope#117)
    - Drop an unused shlex import.
    - Instruct linters to pass over a bare try-except when cleaning up
      temporary files used to extract archives.
    - Format diffoscope/comparators/utils/command.py according to Black 19.10b0-3.
    - Drop entries from Uploaders that have not uploaded in over three years
      with esteemed thanks for their previous work.
    - Drop .travis.yml; we are using Salsa now and likely would not give
      support for running on Travis CI at this point.

  [ Jelle van der Waa ]
  * Update LLVM diff for LLVM version 10.

  [ Vagrant Cascadian ]
  * Add external tool reference on openssl for GNU Guix.

 -- Chris Lamb <email address hidden>  Thu, 14 May 2020 16:16:06 +0100

Available diffs

• diff from 137 to 144 (22.4 KiB)
• diff from 143 to 144 (7.3 KiB)

diffoscope (143) unstable; urgency=medium

  * Add support for .p7c and .p7b certificates.
    (Closes: reproducible-builds/diffoscope#94)
  * Add "pdftotext" as a requirement to run the PDF test_metadata text. Special
    thanks to Chocimier/Piotr for the debugging work.
    (Closes: reproducible-builds/diffoscope#99)
  * Improve the documentation of FALLBACK_FILE_TYPE_HEADER_PREFIX and
    FILE_TYPE_HEADER_PREFIX to note that only the first 16 bytes are used.

 -- Chris Lamb <email address hidden>  Mon, 27 Apr 2020 13:14:07 +0100

diffoscope (137) unstable; urgency=medium

  * Also extract classes2.dex, classes3.dex etc. from .apk files.
    (Closes: reproducible-builds/diffoscope#88)
  * Add generalised support for "ignoring" returncodes and move special-casing
    of returncodes in the zip comparator to this.
  * Accommodate sng returning with a UNIX exit code of 1 even if there minor
    errors in the file (discovered via #950806).

 -- Chris Lamb <email address hidden>  Tue, 18 Feb 2020 17:15:59 +0000

Available diffs

• diff from 136 to 137 (1.7 KiB)

diffoscope (136) unstable; urgency=medium

  [ Chris Lamb ]
  * Improvements:
    - Support external build tools. (Closes: reproducible-builds/diffoscope#87)
    - Fallback to the regular .zip container format for .apk files if apktool
      is not available.
    - Clarify that "install X" in difference comment messages actually refer to
      a system/distribution package.
    - Drop the --max-report-size-child and --max-diff-block-lines-parent
      options; both deprecated and scheduled for removal in January 2018.

  * Bug fixes:
    - No longer raise a KeyError exception if we request an invalid member from a
      directory container.

  * Logging improvements:
    - Log a debug-level message if we cannot open a file as container due to a
      missing tool in order to assist diagnosing issues.
    - If we don't know the HTML output name, don't emit an enigmatic "html
      output for" message.
    - Add the current PATH environment variable to the "Normalising locale..."
      debug-level message.
    - Print the "Starting diffoscope $VERSION" line as the first line.
    - Correct a debug-level message for compare_meta calls to quote the
      arguments correctly.

  * Refactoring:
    - Add support for alternative container types for a file, allowing for
      runtime (vs import time) control of fallbacks such as adding comments.
      and append a comment to a difference if we fallback to an inferior
      container format due to missing a tool.
    - Factor-out the generation of "foo not available in path" difference
      comment messages as a helper method in the exception that represents
      them.

  * Code improvements:
    - Tidy diffoscope.main's configure method, factoring out the set of the
      Config() global out of the run_diffoscope method and inlining the
      functionality of maybe_set_limit, etc.
    - Rename diffoscope.locale module to diffoscope.environ as we are modifying
      things beyond just the locale (eg. calling tzset(), etc.)
    - Drop unused "Difference" import from the APK comparator.
    - Drop an assertion that is guaranteed by parallel "if" conditional.
    - Add a "noqa" line to avoid a false-positive flake8 "unused import" warning.
    - Turn down the "volume" for a recommendation in a comment.

  * Release/source-code management:
    - Add a .git-blame-ignore-revs file to improve the output of git-blame(1) by
      ignoring large changes when introducing the Black source code reformatter
      and update the CONTRIBUTING.md guide on how to optionally use it locally.
    - Convert CONTRIBUTING.rst to CONTRIBUTING.md and include it in the
      PyPI.org release.

  * Test improvements
    - Refresh and update the fixtures for the .ico tests to match the latest
      version of Imagemagick in Debian unstable.

  [ Holger Levsen ]
  * Bump Standards Version to 4.5.0, no changes needed.

  [ Marc Herbert ]
  * Search for expected keywords in the output of cbfstool tests and not a
    specific output. (Closes: reproducible-builds/diffoscope!42)

 -- Chris Lamb <email address hidden>  Fri, 24 Jan 2020 15:38:57 +0000

Available diffs

• diff from 135 to 136 (13.7 KiB)

diffoscope (135) unstable; urgency=medium

  [ Chris Lamb ]
  * Extract resources.arsc files (as well as classes.dex) from Android .apk
    files to ensure that we at least show differences there. (Closes: #916359)
  * Always pass a filename with a ".zip" extension to zipnote otherwise it
    will return with an exit code of 9 and revert to a binary diff for the
    entire file.
  * Don't exhaustively output the entire HTML report when testing the
    regression for #875281; parsing the JSON and pruning the tree should be
    enough.
  * Factor out running all of our zipinfo variants into a new method and use
    this routine when comparing .apk files, thus now also displaying any
    differences exposed by bsdtar.
  * Testsuite improvements:
    - Always allow fixtures called "output*".
    - Actually test the output of the test_html_regression_875281 test.
  * Add a note to the "Contributing" page to suggest enable concurrency when
    running the tests locally.

  [ Marc Herbert ]
  * Fix a misplaced debug "Loading diff from stdin" logging message.
  * Add new "Testing" section to CONTRIBUTING.rst.

 -- Chris Lamb <email address hidden>  Tue, 14 Jan 2020 14:26:07 +0000

Available diffs

• diff from 134 to 135 (3.3 KiB)

diffoscope (134) unstable; urgency=medium

  [ Chris Lamb ]
  * Ensure that autopkgtests are run with our pyproject.toml present for the
    correct "black" source code formatter settings. (Closes: #945993)
  * Tidy some unnecessary boolean logic in the ISO9660 tests.
  * Rename the "text_option_with_stdiout" to "text_option_with_stdout".
  * Include the libarchive file listing for ISO images to ensure that
    timestamps (not just dates) are visible in any difference.
    (Closes: reproducible-builds/diffoscope#81)

  [ Eli Schwartz ]
  * Fix an exception in the progressbar handler.

  [ Vagrant Cascadian ]
  * Add an external tool reference for zstd on GNU Guix.

 -- Chris Lamb <email address hidden>  Fri, 27 Dec 2019 19:17:16 +0000

Available diffs

• diff from 133 to 134 (2.2 KiB)

diffoscope (133) unstable; urgency=medium

  * Correct the substitution/filtering of paths in ELF output to avoid
    unnecessary differences depending on the path name provided on the
    commandline. (Closes: #945572)
  * Silence/correct a SyntaxWarning message due to incorrectly comparing an
    integer by identity (is) over equality (==). (Closes: #945531)

 -- Chris Lamb <email address hidden>  Thu, 28 Nov 2019 10:03:33 +0000

Available diffs

• diff from 132 to 133 (1.1 KiB)

diffoscope (132) unstable; urgency=medium

  * Allow all possible .zip variations to return with non-zero exit codes, not
    just known types we can explicitly identify (eg. Java .jmod and .jar
    files). (Closes: reproducible-builds/diffoscope#78)
  * Also permit UTF-8 encoded .dsc and .changes files.
    (Re: reproducible-builds/diffoscope#78)
  * Rework a long string of "or" statements into a loop with a "break".

 -- Chris Lamb <email address hidden>  Tue, 26 Nov 2019 10:31:44 +0000

Available diffs

• diff from 131 to 132 (1.9 KiB)

diffoscope (131) unstable; urgency=medium

  * Clarify in the HTML and text outputs that the limits are per output format,
    not global. (Closes: #944882, reproducible-builds/diffoscope#76)
    - Bump the previous "max_page_size" limit from 400 kB to 4 MB.
  * Update an Android manifest test to reflect that parsed XML attributes are
    returned in a sorted or otherwise novel manner under Python 3.8.
  * Update code to reflect version 19.10b0 of the black source code
    reformatter and don't run our self-test for versions earlier than this as
    it will generate different results and thus fail.
  * Limit .dsc and .buildinfo matching; don't attempt to compare them as
    ELF sections or similar. (Closes: reproducible-builds/diffoscope#77)
  * Add a comment that the text_ascii{1,2} test fixture files are used in
    multiple places so are not trivial to generate on the fly.

 -- Chris Lamb <email address hidden>  Sat, 23 Nov 2019 15:22:51 -0500

Available diffs

• diff from 125 to 131 (229.7 KiB)
• diff from 130 to 131 (4.4 KiB)

diffoscope (130) unstable; urgency=medium

  [ Chris Lamb ]
  * debian/tests/basic-command-line:
    Move from deprecated ADTTMP to AUTOPKGTEST_TMP.
  * Correct the matching of R .rds files by also detecting newer versions
    of this file format.
  * Drop unused BASE_DIR global in the tests.
  * Try and ensure that new test data files are generated dynamically, ie.
    at least no new ones are added without "good" reasons.
  * Truncate the tcpdump expected diff to 8KB (from ~600KB).
  * Refresh OCaml test fixtures to support OCaml >= 4.08.1.  Closes: #944709
  * Correct reference to the ".rdx" extension in a comment.
  * Update XML test for Python 3.8+.
  * Don't use line-base dbuffering when communucating with subprocesses
    in binary mode. (Closes: reproducible-builds/diffoscope#75)

  [ Jelle van der Waa ]
  * Add support for comparing .zst files are created by zstd.
    (Closes: reproducible-builds/diffoscope!34)

 -- Mattia Rizzolo <email address hidden>  Thu, 14 Nov 2019 11:22:20 +0100

Available diffs

• diff from 129 to 130 (217.6 KiB)

diffoscope (129) unstable; urgency=medium

  * Call R's "deparse" function to ensure that we do not error out and revert
    to a binary diff when processing .rdb files with internal "vector" types as
    they do not automatically coerce to strings.
  * Add the ability to pass Python bytestrings to external commands and
    pass our long script to parse R .rdb files using this new method over a
    long command-line argument
  * Use Rscript's --vanilla option over --no-environ as this also enables
    --no-save, --no-restore, --no-site-file and --no-init-file.
  * Improve command-line error messages:
    - Split out formatting into a separate utility function.
    - Truncate very long lines when displaying them as an external source
      of data.
    - When printing an error from a command, format the command for the user.
  * Use "exit code" over "return code" when referring to UNIX error codes in
    displayed differences.

 -- Chris Lamb <email address hidden>  Mon, 28 Oct 2019 11:15:18 +0000

Available diffs

• diff from 128 to 129 (2.7 KiB)

diffoscope (128) unstable; urgency=medium

  * Query the container for the full path of the parallel R .rdx file for a
    .rdb file as well as looking in the same directory. This ensures that
    comparing two .deb/.changes files shows a varying path introduced in
    version 127.

 -- Chris Lamb <email address hidden>  Fri, 25 Oct 2019 09:21:40 +0100

Available diffs

• diff from 126 to 128 (4.7 KiB)

diffoscope (126) unstable; urgency=medium

  [ Chris Lamb ]
  * Track and report on missing Python modules.
    (Closes: reproducible-builds/diffoscope#72)
  * Drop some unnecessary control flow.
  * Drop explicit inheriting from 'object' class; unnecessary in Python 3.
  * Bump Standards-Version to 4.4.1.

  [ Mattia Rizzolo ]
  * Exit with 2 instead of 1 in case of no disk space.

 -- Chris Lamb <email address hidden>  Mon, 14 Oct 2019 12:29:47 -0700

Available diffs

• diff from 125 to 126 (6.6 KiB)

diffoscope (125) unstable; urgency=medium

  * The test_libmix_differences ELF test requires xxd. (Closes: #940645)

 -- Chris Lamb <email address hidden>  Wed, 18 Sep 2019 11:44:57 +0200

Available diffs

• diff from 123ubuntu1 (in Ubuntu) to 125 (4.9 KiB)
• diff from 124ubuntu1 (in Ubuntu) to 125 (584 bytes)

diffoscope (124ubuntu1) eoan; urgency=medium

  * Cherry-pick commit 7ddebfdc67a7f8ec5eee0b045c90c5b8ae6230d9 from Debian
    git See: 940645

 -- Gianfranco Costamagna <email address hidden>  Wed, 18 Sep 2019 14:58:24 +0200

Available diffs

• diff from 124 (in Debian) to 124ubuntu1 (533 bytes)

diffoscope (124) unstable; urgency=medium

  [ Chris Lamb ]
  * Also conditionally skip the identification and "no differences" tests as we
    require the Ocaml compiler to be present when building the test files
    themselves. (Closes: #940471)

  [ Mattia Rizzolo ]
  * Permit all sorts of version suffixes when checking the Debian package
    version matches setup.py, not just for backports. (Closes: #939387)

  [ Vagrant Cascadian ]
  * Add external tools on GNU Guix for odt2txt, sng and pppdump

  [ Marc Herbert ]
  * Remove StaticLibFile in the ELF comparator -- ArFile.compare_details() is
    superior. (Closes: reproducible-builds/diffoscope#64)

 -- Chris Lamb <email address hidden>  Tue, 17 Sep 2019 12:19:07 +0200

Available diffs

• diff from 123ubuntu1 (in Ubuntu) to 124 (4.8 KiB)

diffoscope (123ubuntu1) eoan; urgency=medium

  * Add ocaml-nox to pytest

 -- Gianfranco Costamagna <email address hidden>  Mon, 16 Sep 2019 16:33:43 +0200

Available diffs

• diff from 122 (in Debian) to 123ubuntu1 (5.4 KiB)
• diff from 123 (in Debian) to 123ubuntu1 (1.0 KiB)

diffoscope (123) unstable; urgency=medium

  [ Chris Lamb ]
  * Build OCaml test input files on-demand rather than shipping them with the
    package in order to prevent test failures with OCaml 4.08.
    (Closes: #939386)
  * Update test skipping messages:
    - When skipping tests due to the lack of installed tool, print the package
      that may provide it.
    - Update "requires foo module" messages to clarify that they are regarding
      Python modules, not packages.
  * Rebuild the test squashfs images to exclude the character device as they
    requires root or fakeroot to extract. (reproducible-builds/diffoscope#65)
  * Remove accidentally-committed test fixture generation code from tests.
  * Set long_description_content_type to 'text/x-rst' in setup.py to appease
    the PyPI.org linter.

  [ Mattia Rizzolo ]
  * Don't crash when the progress is requested but the Python module is
    missing. (Closes: #939085)

  [ Vagrant Cascadian ]
  * Add external tools for gifbuild, javap and kbxutil in GNU Guix.

  [ Marc Herbert ]
  * In the cbfs tests, add a "Comp" column test to support Coreboot
    utils > 4.6.

 -- Chris Lamb <email address hidden>  Sat, 07 Sep 2019 14:16:31 +0100

Available diffs

• diff from 122build1 (in Ubuntu) to 123 (4.7 KiB)

diffoscope (122build1) eoan; urgency=medium

  * Rebuild against new GHC abi.

 -- Gianfranco Costamagna <email address hidden>  Sat, 07 Sep 2019 17:39:19 +0200

Available diffs

• diff from 122 (in Debian) to 122build1 (310 bytes)

diffoscope (122) unstable; urgency=medium

  [ Chris Lamb ]
  * Apply patch from László Böszörményi to update the squashfs test output and
    bump the required version for the test itself. (Closes: #935684)
  * Skip calls to unsquashfs when we are not root or fakeroot.
    (Re: reproducible-builds/diffoscope#63)
  * Include either standard error or standard output (and not just the latter)
    if/when an external command fails.
  * Fix a few unicode/bytes issues:
    - Avoid a possible traceback caused by a str/bytes confusion when handling
      the output of failing external commands.
    - Ensure that all of our artificially-created subprocess.CalledProcessError
      instances have `output` instances that are bytes objects, not str.
  * Improve debugging output:
    * Add the containing module name to the (eg.) "Using StaticLibFile for ..."
    * Improve and condense output when creating our Comparator object types.
  * Correct a reference to `parser.diff` as `diff` in this context is a Python
    function in the module, not the actual output returned from diff(1).
  * Add the "wabt" Debian package to the test dependencies so that we run the
    wasm tests.

  [ Mattia Rizzolo ]
  * Now that we test-require wabt, expect that its tools to be available during
    autopkgtests.

 -- Chris Lamb <email address hidden>  Fri, 30 Aug 2019 08:52:25 +0100

Available diffs

• diff from 120 to 122 (7.2 KiB)
• diff from 121 to 122 (4.4 KiB)

diffoscope (121) unstable; urgency=medium

  * Don't fallback to a (useless) raw hexdump when readelf(1) reports an minor
    issue in a section in an ELF binary. For example, when the "frames" section
    is of the "NOBITS" type, its contents are apparently "unreliable" and thus
    readelf(1) exits with a returncode of 1. (Closes: #849407, #931962)
  * Add support to Difference.from_command_exc and friends to optionally ignore
    specified returncodes from the called program and treat them as "no"
    difference.
  * Simplify the parsing of the optional "command_args" argument to the
    from_command and from_command_exc methods in the Difference class.

 -- Chris Lamb <email address hidden>  Fri, 16 Aug 2019 09:49:07 -0700

Available diffs

• diff from 120 to 121 (3.6 KiB)

diffoscope (120) unstable; urgency=medium

  * No-change sourceful after accidentally uploading binaries in order to
    migration to testing.

 -- Chris Lamb <email address hidden>  Mon, 29 Jul 2019 17:17:46 -0300

Available diffs

• diff from 119 to 120 (477 bytes)

diffoscope (119) unstable; urgency=medium

  [ Chris Lamb ]
  * If a command fails to execute but does not print anything to standard
    error, try and include the first line of standard output in the message we
    include in the diff. This was motivated by readelf(1) returning its error
    messages on stdout. (Closes: #931963)
  * Add support for Java ".jmod" modules. Not all versions of file(1) support
    detection of Jmod files yet, so we perform a manual comparison instead.
    (Closes: #933308)
  * Re-add "return code" noun to "Command `foo` exited with X" error messages.
  * Factor out the ability to ignore the exit codes of "zipinfo" and
    "zipinfo -v" in the presence of non-standard headers.
  * Only override the exit code from our special-cased calls to zipinfo(1) if
    they are 1 or 2 to avoid potentially masking real errors.
  * Also add missing textual description entries for ZipFile and
    MozillaZipFile file types.
  * Skip extra newline in "Output:\n<none>".
  * Cease ignoring test failures in stable-backports.
  * Bump debhelper compat level to 12.

  [ Marc Herbert ]
  * Catch failures to disassemble and rescue all other differences.
    (Closes: reproducible-builds/diffoscope!29)

 -- Chris Lamb <email address hidden>  Mon, 29 Jul 2019 15:22:24 -0300

Available diffs

• diff from 118 to 119 (4.1 KiB)

diffoscope (118) unstable; urgency=medium

  * Don't fail in autopkgtests when, for example, we do not have sufficiently
    newer or older version of file. (Closes: #931881)
  * Also include python3-tlsh in test dependencies.
  * Tidy handling of DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS:
    - Merge two previously-overlapping environment variables.
    - Use repr(..)-style output when printing test status.
    - Add some explicit return values to appease pylint.

 -- Chris Lamb <email address hidden>  Sat, 13 Jul 2019 10:23:29 -0300

Available diffs

• diff from 113ubuntu1 (in Ubuntu) to 118 (14.4 KiB)
• diff from 117ubuntu3 (in Ubuntu) to 118 (4.1 KiB)

diffoscope (117ubuntu3) eoan; urgency=medium

  * Merge from Debian unstable. Remaining changes:
    - add python3-tlsh to pytest (already committed on git master branch)
    - tweak rules to allow different changelog and version
    - revert file 5.37 change until debian bug 931881 is sorted out

 -- Gianfranco Costamagna <email address hidden>  Fri, 12 Jul 2019 09:25:35 +0200

Available diffs

• diff from 117ubuntu2 to 117ubuntu3 (1.1 KiB)

diffoscope (117ubuntu2) eoan; urgency=medium

  * Revert: "Add support for file 5.37."

 -- Gianfranco Costamagna <email address hidden>  Thu, 11 Jul 2019 15:16:15 +0200

Available diffs

• diff from 117ubuntu1 to 117ubuntu2 (2.3 KiB)

diffoscope (117ubuntu1) eoan; urgency=medium

  * Upload to Ubuntu, adding some test to testsuite since now
    we fail on missing tools:
    - python3-tlsh to pytest.
    - tweak rules to allow different changelog and version

Available diffs

• diff from 116ubuntu1 to 117ubuntu1 (2.2 KiB)

diffoscope (116ubuntu1) eoan; urgency=medium

  * Upload to Ubuntu, adding some test to testsuite since now
    we fail on missing tools:
    - python3-tlsh to pytest.
    - add apktool, zipinfo, pedump, oggDump, ppudump to
      DIFFOSCOPE_TESTS_MISSING_TOOLS, because they are not available
      everywhere
    - tweak rules to allow different changelog and version
  * Drop changes, not needed anymore
    - Temporarily disable the fsimage tests during autopkgtest.

Available diffs

• diff from 116~ubuntu1 to 116ubuntu1 (1.3 KiB)

diffoscope (116~ubuntu1) eoan; urgency=medium

  * Upload to Ubuntu, adding some test to testsuite since now
    we fail on missing tools:
    - python3-tlsh to pytest.
    NOTE: this testsuite might fail on architectures where fpc is not
    available, in this case exporting DIFFOSCOPE_TESTS_FAIL_ON_MISSING_TOOLS=0
    in testsuite might be the solution, for now the special fpc case
    (unavailable everywhere) is handled with:
    apktool, zipinfo and ppudump added to DIFFOSCOPE_TESTS_MISSING_TOOLS

 -- Gianfranco Costamagna <email address hidden>  Tue, 09 Jul 2019 08:46:17 +0200

Available diffs

• diff from 116~build1 to 116~ubuntu1 (1.3 KiB)