libgcrypt < 1.10.2 returns wrong sha3 hashes for inputs > 4 GiB

Bug #2044852 reported by Tobias Heider
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libgcrypt20 (Ubuntu)
Status tracked in Noble
Jammy
Fix Committed
Undecided
Unassigned
Lunar
Fix Released
Undecided
Unassigned
Mantic
Fix Released
Undecided
Unassigned
Noble
Fix Released
Undecided
Unassigned

Bug Description

[ Impact ]

SHA3 produces wrong results for inputs bigger than 4 GiB

[ Test Plan ]

Calculate sha3 hash of a big input file and compare with output of
another implementation like OpenSSL.

Expected behavior: same output
Actual behavior: different output

Run reproducer attached below (if your machine can afford to allocate 5G RAM
at once) and see that the patch fixes the assertion error.

[ Where problems could occur ]

People relying on the broken hash might be surprised by the new fixed
result. The impact is hopefully low since SHA3 from libgcrypt is not
too widely used, especially not with this input size.

[ Other Info ]

From upstream bug report:

The SHA3 functions give wrong results for inputs larger than 4GB, because the originally size_t argument handled as unsigned int in keccak_write and leads to integer overflows. This does not happen if the data is fed into the md_write by smaller chunks. More information and reproducers are available from Clemens in the attached bug.

The fix that should solve the problem (use of the size_t) is available now at gitlab: https://gitlab.com/redhat-crypto/libgcrypt/libgcrypt-mirror/-/merge_requests/6 Comments welcomed.

I was considering updating the some of the hash tests to capture this issue, but did not find a simple way to do that yet so I will keep it on you to decide if you believe some regression test is needed here.

Upstream Bug: https://dev.gnupg.org/T6217
Upstream Fix: https://dev.gnupg.org/rC9c828129b2058c3f36e07634637929a54e8377ee

[ WARNING ]

!!! Warning !!!

hashtest.c reproducer allocates 5GB of RAM, do not run on 32-bit architectures.

Do not run if you don't have that much RAM free, as it will likely trigger OOM and may kill your machine.

!!! Warning !!!

Revision history for this message
Tobias Heider (tobhe) wrote :

Attached is a reproducer adapted from the regression test included with newer libgcrypt versions. Compiled with `cc -o hashtest hashtest.c -lgcrypt` it will die with an assertion in the error case.

Bad:
$ ./hashtest
Ohhhh jeeee: Assertion `count < bsize' failed (../../cipher/keccak.c:864:keccak_write)
Aborted (core dumped)
$

Good:
$ ./hashtest
$

Revision history for this message
Tobias Heider (tobhe) wrote :

Fix for jammy attached

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

!!! Warning !!!

hashtest.c allocates 5GB of RAM, do not run on 32-bit architectures.

Do not run if you don't have that much RAM free, as it will likely trigger OOM kill of your machine.

!!! Warning !!!

description: updated
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "libgcrypt.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Tobias Heider (tobhe)
description: updated
Revision history for this message
Lena Voytek (lvoytek) wrote :

Thanks for the patch Tobias! I created a PPA to test it here: https://launchpad.net/~lvoytek/+archive/ubuntu/libgcrypt-sha3-test

I confirmed it works by running the following:

# lxc launch ubuntu:jammy test-libgcrypt-jammy
# lxc shell test-libgcrypt-jammy

# add-apt-repository ppa:lvoytek/libgcrypt-sha3-test
# apt update && apt dist-upgrade -y
# apt install wget gcc libgcrypt20-dev -y
# wget https://bugs.launchpad.net/ubuntu/+source/libgcrypt20/+bug/2044852/+attachment/5723975/+files/hashtest.c
# cc -o hashtest hashtest.c -lgcrypt
# ./hashtest

Overall the patch looks good, I'd just recommend using a DEP-3 header to show the origin of the change, etc for easy access later

Revision history for this message
Tobias Heider (tobhe) wrote :

Thanks for testing Lena! Updated diff including Origin tag attached

Revision history for this message
Simon Chopin (schopin) wrote :

Hi Tobias,

Thanks for the debdiff, it looks good to be uploaded. However, SRU policy states that we fix later releases first: https://wiki.ubuntu.com/StableReleaseUpdates#General_Requirements

Could you clarify the status of all supported releases as well as devel, and if relevant, provide the relevant debdiffs?

TIA :)

Revision history for this message
Simon Chopin (schopin) wrote :

Answering to myself: According to bug title, this is fixed in Noble and Mantic, but not in Lunar and Jammy.

I'm unsubscribing ubuntu-sponsors, please resubscribe once you address the Lunar question :)

Changed in libgcrypt20 (Ubuntu Mantic):
status: New → Fix Released
Changed in libgcrypt20 (Ubuntu Noble):
status: New → Fix Released
Revision history for this message
Tobias Heider (tobhe) wrote :

It turns out that lunar is also already fixed since Debian backported the fix to 1.10.1-3, see d/p/35-keccak-Use-size_t-to-avoid-integer-overflow.patch

So this really only needs to be fixed in jammy. Resubscribing sponsors.

Changed in libgcrypt20 (Ubuntu Lunar):
status: New → Fix Released
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading libgcrypt20_1.9.4-3ubuntu3.1.dsc: done.
  Uploading libgcrypt20_1.9.4-3ubuntu3.1.debian.tar.xz: done.
  Uploading libgcrypt20_1.9.4-3ubuntu3.1_source.buildinfo: done.
  Uploading libgcrypt20_1.9.4-3ubuntu3.1_source.changes: done.
Successfully uploaded packages.

Thank you

(inline used correct sru version number, and added bug reference)

Changed in libgcrypt20 (Ubuntu Jammy):
status: New → In Progress
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello Tobias, or anyone else affected,

Accepted libgcrypt20 into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libgcrypt20/1.9.4-3ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in libgcrypt20 (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (libgcrypt20/1.9.4-3ubuntu3.1)

All autopkgtests for the newly accepted libgcrypt20 (1.9.4-3ubuntu3.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

systemd/249.11-0ubuntu3.11 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#libgcrypt20

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.