(xenial+) apt-cache fails to run if a single sources.list.d entry is not readable

Status changed to 'Confirmed' because the bug affects multiple users.

I'm not yet sure about how to deal with this bug, but non-world-readable files are generally considered unsupported by the apt team, they cause errors everywhere in the package management stack.

So apt does not support being used on a system where a given sources.list entry might be marked root-only to conceal HTTP username/password?

Not really, we told everyone years ago to use the netrc for that.

On Wed, Jul 05, 2017 at 04:05:43PM -0000, Julian Andres Klode wrote:
> Not really, we told everyone years ago to use the netrc for that.

Sorry, what is 'netrc'? I've never heard of this, and while I'm not closely
involved in apt development, I don't find any reference to 'netrc' in the
apt.conf or sources.list manpages. I manage to find
<https://blueprints.launchpad.net/ubuntu/+spec/foundations-lucid-apt-netrc-mechanism>,
but Dir::Etc::netrc is not documented in apt-config(8) nor do I find
documentation for format. This seems like a good enhancement to apt, but it
seems to be underdocumented and there are lots of downstream integration
points with sources.list which have never been updated to know about this?

Regarding the bug itself: I wouldn't exactly call it a regression, but it wasn't a super-intended change either. If I see it right I "broke" it in 2015 by fixing a compiler warning, which indicated that a check which should have been since ever never applied. So, that it worked before was just as well a bug… long story short I guess we can make that a warning.

Why an error/warning? Having a different view on what packages are available depending on if you are root or not is a cause for confusion by users and tools alike as you confirm the non-root view, but the root view is applied, which might include additional packages, different versions or even (additional) unauthenticated packages you haven't approved… (but the tools think you have). Similar things happen for non-root preferences/configs and are hence discouraged.

For netrc/auth.conf documentation we have e.g. Debian bug #811181 tracking it. It is just that nobody has written documentation yet. I assume it is waiting for someone who actually wants that feature to write a few sentences about it.

Fixed in 1.5~beta2, will need a manual sync once it's available in LP.

This bug was fixed in the package apt - 1.5~rc1~ubuntu1

---------------
apt (1.5~rc1~ubuntu1) artful; urgency=medium

  * Redefine APT_CONST to mean APT_PURE
  * Replace APT_CONST with APT_PURE everywhere
  * Make test-bug-818628-unreadable-source work on !amd64

 -- Julian Andres Klode <email address hidden> Thu, 24 Aug 2017 17:56:28 +0200

Correct changelog:

apt (1.5~beta2) unstable; urgency=medium

  [ M. Willis Monroe ]
  * Minor grammar fix

  [ Zhou Mo ]
  * zh_CN.po: update Simplified Chinese programs translation

  [ David Kalnischkies ]
  * don't expect more downloads from failed transactions
  * remove reference to a-t-debtorrent in description
  * ignore SIGPIPE in dump solver if forwarding
  * support compressed extended_states file for bug triage
  * don't move failed pdiff indexes out of partial (Closes: 869425)
  * don't try to parse all fields starting with HTTP as status-line
  * send weak-only hashes to methods
  * fail earlier if server answers with too much data
  * fail early in http if server answer is too small as well
  * use FileFd to parse all apt configuration files
  * show warnings instead of errors if files are unreadable (LP: #1701852)
  * reimplement and document auth.conf (Closes: #811181)
  * lookup login info for proxies in auth.conf
  * allow the auth.conf to be root:root owned
  * update URI scheme descriptions in sources.list(5)
  * show a warning for Debian shutting down FTP services
  * suggest using auth.conf for sources with passwords
  * ftparchive: sort discovered filenames before writing indexes.
    Thanks to Chris Lamb for initial patch & Stefan Lippers-Hollmann for testing
    (Closes: 869557)
  * don't keep configuration files open needlessly
  * don't hang if multiple sources use unavailable method (Closes: 870675)

  [ Beatrice Torracca ]
  * Italian manpage translation update (Closes: 858877)

  [ Apollon Oikonomopoulos ]
  * Handle supported components with slashes in sources.list (Closes: #868127)

  [ Julian Andres Klode ]
  * Drop cacheiterators.h include
  * methods/aptmethod.h: Add missing fileutl.h include
  * Reformat and sort all includes with clang-format
  * cacheiterators: Warn about direct include and don't include pkgcache.h
  * Update gitignore with new files
  * Use C++11 threading support instead of pthread
  * Always warn if --force-yes is validly specified, not just if used
  * Work around float rounding change in gcc 7 on i386
  * Handle GCC 7 std::string operator ABI break (Closes: #871275)
  * debian: Update symbols for libapt-pkg5.0

  [ Paul Wise ]
  * Support zero delay for the various APT::Periodic activities
  * Support seconds, minutes, hours and days for APT::Periodic intervals
  * Switch from /org to /srv in example apt-ftparchive configuration

  [ Balint Reczey ]
  * Gracefully terminate process when stopping apt-daily-upgrade (LP: #1690980)

  [ Dominik ]
  * doc: Add '--allow-unauthenticated' to '--force-yes'

 -- Julian Andres Klode <email address hidden> Thu, 17 Aug 2017 19:28:00 +0200

(sorry, I did not generate the .changes file against 1.5~beta1...).