-
diffoscope (160) unstable; urgency=medium
* Check that pgpdump is actually installed before attempting to run it.
Thanks to Gianfranco Costamagna (locutusofborg). (Closes: #969753)
* Add some documentation for the EXTERNAL_TOOLS dictionary.
* Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump
against all files that are recognised by file(1) as "data".
-- Chris Lamb <email address hidden> Fri, 11 Sep 2020 10:08:38 +0100
-
diffoscope (159ubuntu3) groovy; urgency=medium
* Skip some tests if pgpdump is not installed (See: #969753)
-- Gianfranco Costamagna <email address hidden> Mon, 07 Sep 2020 20:46:23 +0200
-
diffoscope (159ubuntu2) groovy; urgency=medium
* Skip some tests if pgpdump is not installed (See: #969753)
-- Gianfranco Costamagna <email address hidden> Mon, 07 Sep 2020 20:46:23 +0200
-
diffoscope (159ubuntu1) groovy; urgency=medium
* Skip some tests if pgpdump is not installed
-- Gianfranco Costamagna <email address hidden> Mon, 07 Sep 2020 20:46:23 +0200
-
diffoscope (159) unstable; urgency=medium
[ Chris Lamb ]
* Show "ordering differences only" in strings(1) output.
(Closes: reproducible-builds/diffoscope#216)
* Don't alias output from "os.path.splitext" to variables that we do not end
up using.
* Don't raise exceptions when cleaning up after a guestfs cleanup failure.
[ Jean-Romain Garnier ]
* Make "Command" subclass a new generic Operation class.
-- Chris Lamb <email address hidden> Fri, 04 Sep 2020 11:12:52 +0100
-
diffoscope (158) unstable; urgency=medium
* Improve PGP support:
- Support extracting of files within PGP signed data.
(Closes: reproducible-builds/diffoscope#214)
- pgpdump(1) can successfully parse some unrelated, non-PGP binary files,
so check that the parsed output contains something remotely sensible
before identifying it as a PGP file.
* Don't use Python's repr(...)-style output in "Calling external command"
logging output.
* Correct a typo of "output" in an internal comment.
-- Chris Lamb <email address hidden> Fri, 28 Aug 2020 11:53:10 +0100
-
diffoscope (157) unstable; urgency=medium
[ Chris Lamb ]
* Try obsensibly "data" files named .pgp against pgpdump to determine whether
they are PGP files. (Closes: reproducible-builds/diffoscope#211)
* Don't raise an exception when we encounter XML files with "<!ENTITY>"
declarations inside the DTD, or when a DTD or entity references an external
resource. (Closes: reproducible-builds/diffoscope#212)
* Temporarily drop gnumeric from Build-Depends as it has been removed from
testing due to Python 2.x deprecation. (Closes: #968742)
* Codebase changes:
- Add support for multiple file extension matching; we previously supported
only a single extension to match.
- Move generation of debian/tests/control.tmp to an external script.
- Move to our assert_diff helper entirely in the PGP tests.
- Drop some unnecessary control flow, unnecessary dictionary comprehensions
and some unused imports found via pylint.
* Include the filename in the "... not identified by any comparator"
logging message.
-- Chris Lamb <email address hidden> Fri, 21 Aug 2020 12:24:25 +0100
-
diffoscope (156) unstable; urgency=medium
[ Chris Lamb ]
* Update PPU tests for compatibility with Free Pascal versions 3.2.0 or
greater. (Closes: #968124)
* Emit a debug-level logging message when our ppudump(1) version does not
match file header.
* Add and use an assert_diff helper that loads and compares a fixture output
to avoid a bunch of test boilerplate.
[ Frazer Clews ]
* Apply some pylint suggestions to the codebase.
-- Chris Lamb <email address hidden> Fri, 14 Aug 2020 10:04:26 +0100
-
diffoscope (155) unstable; urgency=medium
[ Chris Lamb ]
* Bump Python requirement from 3.6 to 3.7 - most distributions are either
shipping3.5 or 3.7, so supporting 3.6 is not somewhat unnecessary and also
more difficult to test locally.
* Improvements to setup.py:
- Apply the Black source code reformatter.
- Add some URLs for the site of PyPI.org.
- Update "author" and author email.
* Explicitly support Python 3.8.
[ Frazer Clews ]
* Move away from the deprecated logger.warn method logger.warning.
[ Mattia Rizzolo ]
* Document ("classify") on PyPI that this project works with Python 3.8.
-- Chris Lamb <email address hidden> Fri, 07 Aug 2020 12:09:50 +0100
-
diffoscope (154) unstable; urgency=medium
[ Chris Lamb ]
* Add support for F2FS filesystems.
(Closes: reproducible-builds/diffoscope#207)
* Allow "--profile" as a synonym for "--profile=-".
* Add an add_comment helper method so don't mess with our _comments list
directly.
* Add missing bullet point in a previous changelog entry.
* Use "human-readable" over unhyphenated version.
* Add a bit more debugging around launching guestfs.
* Profile the launch of guestfs filesystems.
* Correct adding a comment when we cannot extract a filesystem due to missing
guestfs module.
-- Chris Lamb <email address hidden> Fri, 31 Jul 2020 10:26:15 +0100
-
diffoscope (153) unstable; urgency=medium
[ Chris Lamb ]
* Drop some legacy argument styles; --exclude-directory-metadata and
--no-exclude-directory-metadata have been replaced with
--exclude-directory-metadata={yes,no}.
* Code improvements:
- Make it easier to navigate the main.py entry point.
- Use a relative import for get_temporary_directory in diffoscope.diff.
- Rename bail_if_non_existing to exit_if_paths_do_not_exist.
- Rewrite exit_if_paths_do_not_exist to not check files multiple times.
* Documentation improvements:
- CONTRIBUTING.md:
- Add a quick note about adding/suggesting new options.
- Update and expand the release process documentation.
- Add a reminder to regenerate debian/tests/control.
- README.rst:
- Correct URL to build job on Jenkins.
- Clarify and correct contributing info to point to salsa.debian.org.
-- Chris Lamb <email address hidden> Fri, 24 Jul 2020 12:03:51 +0100
-
diffoscope (152) unstable; urgency=medium
[ Chris Lamb ]
* Bug fixes:
- Don't require zipnote(1) to determine differences in a .zip file as we
can use libarchive directly.
Reporting improvements:
- Don't emit "javap not found in path" if it is available in the path but
it did not result in any actual difference.
- Fix "... not available in path" messages when looking for Java
decompilers; we were using the Python class name (eg. "<class
'diffoscope.comparators.java.Javap'>") over the actual command we looked
for (eg. "javap").
* Code improvements:
- Replace some simple usages of str.format with f-strings.
- Tidy inline imports in diffoscope.logging.
- In the RData comparator, always explicitly return a None value in the
failure cases as we return a non-None value in the "success" one.
[ Jean-Romain Garnier ]
* Improve output of side-by-side diffs, detecting added lines better.
(MR: reproducible-builds/diffoscope!64)
* Allow passing file with list of arguments to ArgumentParser (eg.
"diffoscope @args.txt"). (MR: reproducible-builds/diffoscope!62)
-- Chris Lamb <email address hidden> Sat, 18 Jul 2020 11:00:13 +0100
-
diffoscope (151) unstable; urgency=medium
[ Chris Lamb]
* Improvements and bug fixes:
- Pass the absolute path when extracting members from SquashFS images as we
run the command with our working directory set to the temporary
directory. (Closes: #964365, reproducible-builds/diffoscope#189)
- Increase the minimum length of the output from strings(1) to 8 characters
to avoid unnecessary diff noise. (Re. reproducible-builds/diffoscope#148)
* Logging improvements:
- Fix the compare_files message when the file does not have a literal name.
- Reduce potential log noise by truncating the has_some_content messages.
* Codebase changes:
- Clarify use of a "null" diff in order to remember an exit code.
- Don't alias a variable when don't end up it; use "_" instead.
- Use a "NullChanges" file to represent missing data in the Debian package
comparator.
- Update some miscellaneous terms.
-- Chris Lamb <email address hidden> Fri, 10 Jul 2020 10:52:56 +0100
-
diffoscope (150) unstable; urgency=medium
[ Chris Lamb ]
* Don't crash when listing entries in archives if they don't have a listed
size (such as hardlinks in .ISO files).
(Closes: reproducible-builds/diffoscope#188)
* Dump PE32+ executables (including EFI applications) using objdump.
(Closes: reproducible-builds/diffoscope#181)
* Tidy detection of JSON files due to missing call to File.recognizes that
checks against the output of file(1) which was also causing us to attempt
to parse almost every file using json.loads. (Whoops.)
* Drop accidentally-duplicated copy of the new --diff-mask tests.
* Logging improvements:
- Split out formatting of class names into a common method.
- Clarify that we are generating presenter formats in the opening logs.
[ Jean-Romain Garnier ]
* Remove objdjump(1) offsets before instructions to reduce diff noise.
(Closes: reproducible-builds/diffoscope!57)
-- Chris Lamb <email address hidden> Fri, 03 Jul 2020 11:04:46 +0100
-
diffoscope (149) unstable; urgency=medium
[ Chris Lamb ]
* Update tests for file 5.39. (Closes: reproducible-builds/diffoscope#179)
* Downgrade the tlsh warning message to an "info" level warning.
(Closes: #888237, reproducible-builds/diffoscope#29)
* Use the CSS "word-break" property over manually adding U+200B zero-width
spaces that make copy-pasting cumbersome.
(Closes: reproducible-builds/diffoscope!53)
* Codebase improvements:
- Drop some unused imports from the previous commit.
- Prevent an unnecessary .format() when rendering difference comments.
- Use a semantic "AbstractMissingType" type instead of remembering to check
for both "missing" files and missing containers.
[ Jean-Romain Garnier ]
* Allow user to mask/filter reader output via --diff-mask=REGEX.
(MR: reproducible-builds/diffoscope!51)
* Make --html-dir child pages open in new window to accommodate new web
browser content security policies.
* Fix the --new-file option when comparing directories by merging
DirectoryContainer.compare and Container.compare.
(Closes: reproducible-builds/diffoscope#180)
* Fix zsh completion for --max-page-diff-block-lines.
[ Mattia Rizzolo ]
* Do not warn about missing tlsh during tests.
-- Chris Lamb <email address hidden> Fri, 26 Jun 2020 15:57:41 +0100
-
diffoscope (148) unstable; urgency=medium
[ Daniel Fullmer ]
* Fix a regression in the CBFS comparator due to changes in our_check_output.
[ Chris Lamb ]
* Add a remark in the deb822 handling re. potential security issue in the
.changes, .dsc, .buildinfo comparator.
-- Chris Lamb <email address hidden> Fri, 19 Jun 2020 11:38:20 +0100
-
diffoscope (147) unstable; urgency=medium
* New features:
- Add output from strings(1) to ELF binaries. It is intended this will
expose expose build paths that are hidden somewhere within the objdump(1)
output. (Closes: reproducible-builds/diffoscope#148)
- Add basic zsh shell tab-completion support.
(Closes: reproducible-builds/diffoscope#158)
* Bug fixes:
- Prevent a traceback when comparing a PDF document that does not contain
any metadata, ie. it is missing a PDF "/Info" stanza.
(Closes: reproducible-builds/diffoscope#150)
- Fix compatibility with jsondiff 1.2.0 which was causing a traceback and
log the version of jsondiff we are using to aid debugging in the future.
(Closes: reproducible-builds/diffoscope#159
- Fix an issue in GnuPG keybox handling that left filenames in the diff.
- Don't mask an existing test name; ie. ensure it is actually run.
* Reporting:
- Log all calls to subprocess.check_output by using our own wrapper utility.
(Closes: reproducible-builds/diffoscope#151)
* Code improvements:
- Replace references to "WF" with "Wagner-Fischer" for clarity.
- Drop a large number of unused imports (list_libarchive,
ContainerExtractionError, etc.)
- Don't assign exception to a variable that we do not use.
- Compare string values with the equality operator, not via "is" identity.
- Don't alias an open file to a variable when we don't use it.
- Don't alias "filter" builtin.
- Refactor many small parts of the HTML generation, dropping explicit
u"unicode" strings, tidying the generation of the "Offset X, Y lines
modified" messages, moving to PEP 498 f-strings where appropriate, etc.
- Inline a number of single-used utility methods.
-- Chris Lamb <email address hidden> Thu, 11 Jun 2020 12:50:34 +0100
-
diffoscope (146) unstable; urgency=medium
[ Chris Lamb ]
* Refactor .changes and .buildinfo handling to show all details (including
the GPG header and footer components), even when referenced files are not
present. (Closes: reproducible-builds/diffoscope#122)
* Normalise filesystem stat(2) "birth times" (ie. st_birthtime) in the same
way we do with stat(1)'s "Access:" and "Change:" times to fix a
nondetermistic build failure on GNU Guix.
(Closes: reproducible-builds/diffoscope#74)
* Drop the (default) subprocess.Popen(shell=False) keyword argument so that
the more unsafe shell=True is more obvious.
* Ignore lower vs. upper-case when ordering our file format descriptions.
* Don't skip string normalisation in Black.
[ Mattia Rizzolo ]
* Add a "py3dist" override for the rpm-python module (Closes: #949598)
* Bump the debhelper compat level to 13 and use the new
execute_after_*/execture_before_* style rules.
* Fix a spelling error in changelog.
[ Daniel Fullmer ]
* Mount GuestFS filesystem images readonly.
[ Jean-Romain Garnier ]
* Prevent an issue where (for example) LibarchiveMember's has_same_content
method is called regardless of the actual type of file.
-- Chris Lamb <email address hidden> Sat, 30 May 2020 12:36:51 +0100
-
diffoscope (145) unstable; urgency=medium
[ Chris Lamb ]
* Improvements:
- Add support for Apple Xcode mobile provisioning .mobilepovision files.
(Closes: reproducible-builds/diffoscope#113)
- Add support for printing the signatures via apksigner(1).
(Closes: reproducible-builds/diffoscope#121)
- Use SHA256 over MD5 when generating page names for the HTML directory
presenter, validate checksums for files referenced in .changes files
using SHA256 too, and move to using SHA256 in "Too much input for diff"
output too. (Closes: reproducible-builds/diffoscope#124)
- Don't leak the full path of the temporary directory in "Command [..]
exited with 1". (Closes: reproducible-builds/diffoscope#126)
- Identify "iOS App Zip archive data" files as .zip files.
(Closes: reproducible-builds/diffoscope#116)
* Bug fixes:
- Correct "differences" typo in the ApkFile handler.
(Closes: reproducible-builds/diffoscope#127)
* Reporting/output improvements:
- Never emit the same id="foo" TML anchor reference twice, otherwise
identically-named parts will not be able to linked to via "#foo".
(Closes: reproducible-builds/diffoscope#120)
- Never emit HTML with empty "id" anchor lements as it is not possible to
link to "#" (vs "#foo"). We use "#top" as a fallback value so it will
work for the top-level parent container.
- Clarify the message when we cannot find the "debian" Python module.
- Clarify "Command [..] failed with exit code" to remove duplicate "exited
with exit" but also to note that diffoscope is intepreting this as an
error.
- Add descriptions for the 'fallback' Debian module file types.
- Rename the --debugger command-line argument to --pdb.
* Testsuite improvements:
- Prevent CI (and runtime) apksigner test failures due to lack of
binfmt_misc on Salsa CI and elsewhere.
* Codebase improvements:
- Initially add a pair of comments to tidy up a slightly abstraction level
violating code in diffoscope.comparators.mising_file and the
.dsc/.buildinfo file handling, but replace this later by by inlining
MissingFile's special handling of deb822 to prevent leaking through
abstraction layers in the first place.
- Use a BuildinfoFile (etc.) regardless of whether the associated files
such as the orig.tar.gz and the .deb are present, but don't treat them as
actual containers. (Re: reproducible-builds/diffoscope#122)
- Rename the "Openssl" command class to "OpenSSLPKCS7" to accommodate other
commands with this prefix.
- Wrap a docstring across multiple lines, drop an inline pprint import and
comment the HTMLPrintContext class, etc.
[ Emanuel Bronshtein ]
* Avoid build-cache in building the released Docker image.
(Closes: reproducible-builds/diffoscope#123)
[ Holger Levsen ]
* Wrap long lines in older changelog entries.
-- Chris Lamb <email address hidden> Sat, 23 May 2020 09:31:26 +0100
-
diffoscope (144) unstable; urgency=medium
[ Chris Lamb ]
* Improvements:
- Print the amount of free space that we have available in our temporary
directory as a debugging message.
- Remove (broken) fuzzy matching of JSON files as file 5.35 (in buster,
released 2018-10-18) supports recognising JSON data.
(Closes: reproducible-builds/diffoscope#106)
- Don't pretty-print the JSON output by default as it will usually be so
complicated to be unreadable by the human eye and it can be easily
replaced by piping to "| jq".
- Don't print a traceback if we pass a single, missing argument to
diffoscope (eg. a JSON diff to re-load).
* Reporting/output improvements:
- Reduce the default number of maximum standard error lines printed from 50
to 25; usually the error is obvious by that point.
- Clarify the message when we truncate the number of lines to standard error.
- Clarify when an external command emits for both files, otherwise it can
look like diffoscope is repeating itself when it is actually being run
twice.
- Don't repeat "stderr from {}" if both commands emit the same thing.
* Dockerfile improvements:
- Use ARG instead of ENV for DEBIAN_FRONTEND so we do not set this
environment variable at runtime.
(Closes: reproducible-builds/diffoscope#103)
- Run diffoscope as a non-root user in the runtime container.
(Closes: reproducible-builds/diffoscope#102)
- Add a .dockerignore file to whitelist files we need in our container.
Thanks to Emanuel Bronshtein for the original idea.
(Closes: reproducible-builds/diffoscope#105)
- Install/remove the build-essential package during build so we can install
the recommended packages from Git.
* Testsuite improvements:
- Include the Black output in the assertion failure too.
- Update the Black self-test; we don't care about the length of the black
output, rather whether it has some or, preferably, not.
* Codebase improvements:
- Bump the officially required version of Python from 3.5 to 3.6.
(Closes: reproducible-builds/diffoscope#117)
- Drop an unused shlex import.
- Instruct linters to pass over a bare try-except when cleaning up
temporary files used to extract archives.
- Format diffoscope/comparators/utils/command.py according to Black 19.10b0-3.
- Drop entries from Uploaders that have not uploaded in over three years
with esteemed thanks for their previous work.
- Drop .travis.yml; we are using Salsa now and likely would not give
support for running on Travis CI at this point.
[ Jelle van der Waa ]
* Update LLVM diff for LLVM version 10.
[ Vagrant Cascadian ]
* Add external tool reference on openssl for GNU Guix.
-- Chris Lamb <email address hidden> Thu, 14 May 2020 16:16:06 +0100
-
diffoscope (143) unstable; urgency=medium
* Add support for .p7c and .p7b certificates.
(Closes: reproducible-builds/diffoscope#94)
* Add "pdftotext" as a requirement to run the PDF test_metadata text. Special
thanks to Chocimier/Piotr for the debugging work.
(Closes: reproducible-builds/diffoscope#99)
* Improve the documentation of FALLBACK_FILE_TYPE_HEADER_PREFIX and
FILE_TYPE_HEADER_PREFIX to note that only the first 16 bytes are used.
-- Chris Lamb <email address hidden> Mon, 27 Apr 2020 13:14:07 +0100
-
diffoscope (137) unstable; urgency=medium
* Also extract classes2.dex, classes3.dex etc. from .apk files.
(Closes: reproducible-builds/diffoscope#88)
* Add generalised support for "ignoring" returncodes and move special-casing
of returncodes in the zip comparator to this.
* Accommodate sng returning with a UNIX exit code of 1 even if there minor
errors in the file (discovered via #950806).
-- Chris Lamb <email address hidden> Tue, 18 Feb 2020 17:15:59 +0000
