"Content-Range: */<file size>" on non-416 responses considered invalid

Bug #1657567 reported by Julian Andres Klode
40
This bug affects 7 people
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
Fix Released
Low
Julian Andres Klode
Xenial
Fix Released
Low
Unassigned
Yakkety
Fix Released
Low
Unassigned

Bug Description

APT only allows Content-Range: */<filesize> to be specified on a 416 response. Sourceforge sometimes replies with that in a 302 redirect.

We should probably just accept and silently ignore that content-range field for other values.

[Impact]
Issue breaks and other services that respond with a Content-Range header in a 302 redirect, or any non 416 one.

[Test Case]
Run

/usr/lib/apt/apt-helper download-file -o debug::acquire::http=1 http://www.jak-software.de/lp1657567 ubuntu.iso

Before:

E: Failed to fetch http://www.jak-software.de/lp1657567 The HTTP server sent an invalid Content-Range header Bad header data

After:

Redirect is followed successfully and a zesty live image is being downloaded.

[Regression Potential]
None

Changed in apt (Ubuntu):
status: New → Triaged
importance: Undecided → Low
Changed in apt (Ubuntu):
assignee: nobody → Julian Andres Klode (juliank)
Revision history for this message
Julian Andres Klode (juliank) wrote :

Fix committed:

https://anonscm.debian.org/cgit/apt/apt.git/commit/?id=4759a70

commit 4759a702081297bde66982efed8b2b7fd39ca27c
Author: Julian Andres Klode <email address hidden>
Date: Wed Jan 18 20:39:27 2017 +0100

    basehttp: Only read Content-Range on 416 and 206 responses

    This fixes issues with sourceforge where the redirector includes
    such a Content-Range in a 302 redirect. Since we do not really know
    what file is meant in a redirect, let's just ignore it for all
    responses other than 416 and 206.

    Maybe we should also get rid of the other errors, and just ignore
    the field in those cases as well?

    LP: #1657567

Changed in apt (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.4~beta4ubuntu1

---------------
apt (1.4~beta4ubuntu1) zesty; urgency=medium

  * basehttp: Only read Content-Range on 416 and 206 responses (LP: #1657567)
  * Only merge acquire items with the same meta key (Closes: #838441)
  * Workaround debian/copyright symlink

 -- Julian Andres Klode <email address hidden> Wed, 25 Jan 2017 12:07:50 +0100

Changed in apt (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
rpr nospam (rpr-nospam) wrote :

Please, backport this to xenial.

Revision history for this message
Julian Andres Klode (juliank) wrote :

It will come to (yakkety and) xenial eventually. But it's not urgent enough to warrant a prioritised upload on its own IMO. There's some other stuff I want to figure out first - like merging new translations, and some other bug fixes from the 1.4 series.

Revision history for this message
Travisgevans (travisgevans) wrote :

I thought it might be helpful to anyone still having this problem (where the fix hasn't been backported yet) to mention that the workaround is described in Bug #1607535 (essentially, delete the affected partial download files in /var/lib/update-notifier/package-data-downloads/partial/ and then try reinstalling again).

Revision history for this message
Julian Andres Klode (juliank) wrote :
Changed in apt (Ubuntu Xenial):
status: New → Triaged
Changed in apt (Ubuntu Yakkety):
status: New → Triaged
Changed in apt (Ubuntu Xenial):
status: Triaged → In Progress
Changed in apt (Ubuntu Yakkety):
status: Triaged → In Progress
Changed in apt (Ubuntu Xenial):
importance: Undecided → Low
Changed in apt (Ubuntu Yakkety):
importance: Undecided → Low
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Since this (and a few other) bug is mentioned in the SRU changelog, please update the description to include the SRU template. There seems to be a master bug for the SRU, but each bug should *at least* have a clearly written test-case. Thanks!

Revision history for this message
Julian Andres Klode (juliank) wrote :

Oh, sorry, did not notice that it did not have one.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Seems like sourceforge is fixed now, so we need to come up with a new test.

description: updated
description: updated
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Julian, or anyone else affected,

Accepted apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.2.20 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Julian Andres Klode (juliank) wrote :

Verified in 1.2.20.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Chris J Arges (arges) wrote :

Hello Julian, or anyone else affected,

Accepted apt into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.3.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt (Ubuntu Yakkety):
status: In Progress → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Jon Grimm (jgrimm)
tags: added: verification-done-xenial verification-needed-yakkety
removed: verification-needed
Revision history for this message
Julian Andres Klode (juliank) wrote :

Verified broken in 1.3.4, and fixed in 1.3.5.

tags: added: verification-done-yakkety
removed: verification-needed-yakkety
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.3.5

---------------
apt (1.3.5) yakkety; urgency=medium

  * Microrelease covering important fixes of 1.4~rc2 (LP: #1668280)

  [ David Kalnischkies ]
  * don't install new deps of candidates for kept back pkgs
  * keep Release.gpg on untrusted to trusted IMS-Hit (Closes: 838779)
    (LP: #1657440)
  * reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivileges (Closes: 842877)
  * add TMP/TEMP/TEMPDIR to the TMPDIR DropPrivileges dance
  * react to trig-pend only if we have nothing else to do
  * correct cross & disappear progress detection
  * improve arch-unqualified dpkg-progress parsing
  * don't perform implicit crossgrades involving M-A:same
  * do not configure unconfigured to be removed packages
  * skip unconfigure for unconfigured to-be removed pkgs
  * get pdiff files from the same mirror as the index
  * let {dsc,tar,diff}-only implicitly enable download-only
  * ensure generation of valid EDSP error stanzas
  * fix minimum pkgs option for dpkg --recursive usage
  * don't show update stats if cache generation is disabled
  * don't lock dpkg in 'apt-get clean'
  * don't lock dpkg in update commands
  * avoid validate/delete/load race in cache generation
  * fix 'install --no-download' mode
  * remove 'old' FAILED files in the next acquire call (Closes: 846476)
  * stop rred from leaking debug messages on recovered errors (Closes: #850759)

  [ Edgar Fuß ]
  * http: clear content before reporting the failure (Closes: #465572)

  [ Paul Wise ]
  * show output as documented for APT::Periodic::Verbose 2 (Closes: 845599)

  [ John R. Lenton ]
  * bash-completion: Only complete understood file paths for install
    (LP: #1645815)

  [ Lukasz Kawczynski ]
  * Honour Acquire::ForceIPv4/6 in the https transport

  [ Julian Andres Klode ]
  * basehttp: Only read Content-Range on 416 and 206 responses (LP: #1657567)
  * Only merge acquire items with the same meta key (Closes: #838441)
  * Do not package names representing .dsc/.deb/... files (Closes: #854794)
  * Don't use -1 fd and AT_SYMLINK_NOFOLLOW for faccessat()
    Thanks to James Clarke for debugging these issues
  * CMake: Install statvfs.h to include/sys, not just include/

 -- Julian Andres Klode <email address hidden> Mon, 27 Feb 2017 15:02:40 +0100

Changed in apt (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.2.20

---------------
apt (1.2.20) xenial; urgency=medium

  * Microrelease covering fixes of 1.4~rc2 (LP: #1668285)

  [ David Kalnischkies ]
  * don't install new deps of candidates for kept back pkgs
  * keep Release.gpg on untrusted to trusted IMS-Hit (Closes: 838779)
    (LP: #1657440)
  * reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivileges (Closes: 842877)
  * add TMP/TEMP/TEMPDIR to the TMPDIR DropPrivileges dance
  * let {dsc,tar,diff}-only implicitly enable download-only
  * don't show update stats if cache generation is disabled
  * don't lock dpkg in 'apt-get clean'
  * don't lock dpkg in update commands
  * avoid validate/delete/load race in cache generation
  * remove 'old' FAILED files in the next acquire call (Closes: 846476)
  * stop rred from leaking debug messages on recovered errors (Closes: #850759)

  [ Paul Wise ]
  * show output as documented for APT::Periodic::Verbose 2 (Closes: 845599)

  [ John R. Lenton ]
  * bash-completion: Only complete understood file paths for install
    (LP: #1645815)

  [ Lukasz Kawczynski ]
  * Honour Acquire::ForceIPv4/6 in the https transport

  [ Julian Andres Klode ]
  * basehttp: Only read Content-Range on 416 and 206 responses (LP: #1657567)
  * Only merge acquire items with the same meta key (Closes: #838441)
  * Do not package names representing .dsc/.deb/... files (Closes: #854794)
  * Don't use -1 fd and AT_SYMLINK_NOFOLLOW for faccessat()
    Thanks to James Clarke for debugging these issues

 -- Julian Andres Klode <email address hidden> Mon, 27 Feb 2017 15:29:18 +0100

Changed in apt (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.