apport doesn't submit date information from crash reports

This was actually on purpose, in version 0.111 in intrepid:

  * apport/crashdb_impl/launchpad.py: Do not write the "Date:" field on
    upload(), and fetch it from the bug metadata in download().

This version removed a lot of redundant information from reports, to make them easier to read and focus triagers on the important bits. Since a bug already records its reported date, this seemed superfluous.

Do you think this should be added back for some reason?

Being able to correlate when a crash occurs with other event in logfiles is usually pretty important. In the bug I linked to, the person attempting to diagnose the issue had a hypothesis that the bug had occurred during an upgrade and not after the reboot; however, the only way to verify when the crash actually occurred was to look at the date stamp on the crashfile itself to confirm or deny that hypothesis.

I share your concern with not overwhelming triagers with all the information available when a bug is reported; perhaps the dropped information could be combined into a separate attachment (e.g. CrashDetails) so that it doesn't clutter the bug's description but is available should it be necessary for diagnosing the bug?

On Wed, Apr 22, 2009 at 05:40:28PM -0000, Steve Beattie wrote:
> Being able to correlate when a crash occurs with other event in logfiles
> is usually pretty important. In the bug I linked to, the person
> attempting to diagnose the issue had a hypothesis that the bug had
> occurred during an upgrade and not after the reboot; however, the only
> way to verify when the crash actually occurred was to look at the date
> stamp on the crashfile itself to confirm or deny that hypothesis.
>
> I share your concern with not overwhelming triagers with all the
> information available when a bug is reported; perhaps the dropped
> information could be combined into a separate attachment (e.g.
> CrashDetails) so that it doesn't clutter the bug's description but is
> available should it be necessary for diagnosing the bug?

I assume Martin subscribed me because the suggestion to remove it came from
me (I don't quite recall). FWIW, I have no objection restoring it if it's
useful.

--
 - mdz

On Wed, Apr 22, 2009 at 06:01:55PM -0000, Matt Zimmerman wrote:
> On Wed, Apr 22, 2009 at 05:40:28PM -0000, Steve Beattie wrote:
> > Being able to correlate when a crash occurs with other event in logfiles
> > is usually pretty important. In the bug I linked to, the person
> > attempting to diagnose the issue had a hypothesis that the bug had
> > occurred during an upgrade and not after the reboot; however, the only
> > way to verify when the crash actually occurred was to look at the date
> > stamp on the crashfile itself to confirm or deny that hypothesis.
> >
> > I share your concern with not overwhelming triagers with all the
> > information available when a bug is reported; perhaps the dropped
> > information could be combined into a separate attachment (e.g.
> > CrashDetails) so that it doesn't clutter the bug's description but is
> > available should it be necessary for diagnosing the bug?
>
> I assume Martin subscribed me because the suggestion to remove it came from
> me (I don't quite recall). FWIW, I have no objection restoring it if it's
> useful.

A lot of the information that apport collects isn't generally useful...
except when it is. As a specific example, consider the contents
of /proc/PID/attr/current -- for 99% of bugs, this is going to be
uninteresting information, but if it actually contains an apparmor
policy name or an selinux context, you need to take that into account
when diagnosing the bug report as it may be the security policy that is
causing the issue. (Though I have it on my todo list to add an apport
collection item that looks for relevant audit events to attach to make
it easier to identify such issues.)

Until apport grows an expert system, the information that it should
collect that is important to identifying the problem at hand is going
to be a rough heuristic at best. So I really don't like the idea of
discarding information that we've bothered to collect.

I *do* think it makes sense to consider the presentation of the collected
and submitted information separately from the data collection itself;
I'm all for trying to limit the information that's included in the
description to that which is commonly useful, but I think it makes
sense to have the information that is uncommonly useful still available.
This is why I suggested a separate attachment that contains such things;
they would not clutter the main report (except for the attachment itself),
but would still be available for diagnosis if needed.

--
Steve Beattie
<email address hidden>
http://NxNW.org/~steve/

Alright, will add it back.

On Wed, Apr 22, 2009 at 07:04:57PM -0000, Steve Beattie wrote:
> A lot of the information that apport collects isn't generally useful...
> except when it is. As a specific example, consider the contents
> of /proc/PID/attr/current -- for 99% of bugs, this is going to be
> uninteresting information, but if it actually contains an apparmor
> policy name or an selinux context, you need to take that into account
> when diagnosing the bug report as it may be the security policy that is
> causing the issue. (Though I have it on my todo list to add an apport
> collection item that looks for relevant audit events to attach to make
> it easier to identify such issues.)

In cases like these, it's a good idea to attach the information *only* if
it's interesting, since we can determine that from the system. If
/proc/pid/attr/current says only "unconfined", this doesn't carry any new
information, and we should just omit it.

Similarly, we can only attach configuration files which differ from the
default. There's a hookutils function for this already, which I hope to see
enabled by default for Ubuntu.

This also helps to make the information stand out if it *is* interesting.

--
 - mdz

Added back in trunk r1414.

I also made ProcAttrCurrent only appear if it is not "unconfined" in trunk r1415, thanks for the suggestion.

On Thu, Apr 23, 2009 at 08:43:07AM -0000, Matt Zimmerman wrote:
> In cases like these, it's a good idea to attach the information *only* if
> it's interesting, since we can determine that from the system. If
> /proc/pid/attr/current says only "unconfined", this doesn't carry any new
> information, and we should just omit it.

Except where something was expected to have an apparmor policy applied
to it but does not -- it can be detected by the absence of the entry but
detection by omission is generally a hard thing for people to do. That
said, I'll concede the point.

> Similarly, we can only attach configuration files which differ from the
> default. There's a hookutils function for this already, which I hope to see
> enabled by default for Ubuntu.

I recall using it to pull out the gconf settings for the
gnome-power-manager apport hook I wrote. Two issues that I see:

  1) In the devel cycle, defaults can change. Given our overwhelming
     number of bugs, the current defaults that a triager sees for a
     package may differ from the defaults as they were when the bug
     was reported.
  2) Our defaults may differ from an upstream's defaults and that may
     not be obvious to an upstream developer looking at our bug reports.

But I'll stop bike-shedding and shut up now (or at least can continue
the discussion elsewhere, if desired).

--
Steve Beattie
<email address hidden>
http://NxNW.org/~steve/

This bug was fixed in the package apport - 1.1-0ubuntu1

---------------
apport (1.1-0ubuntu1) karmic; urgency=low

  * New upstream release:
    - Drop some remaining distro specific pieces of code from non-backends.
    - Add hookutils methods for attaching relevant packages, greatly improve
      attach_alsa() for sound problem debugging.
    - Move launchpad crash database implementation from ever-breaking
      python-launchpad-bugs (screenscraping) to launchpadlib (official and
      stable Launchpad API). (LP: #353879)
    - Add new field Report.pid which gets set on add_proc_info() and can be
      used by hooks.
    - setup.py: Properly clean up all generated files, install missing
      mimetypes/text-x-apport.svg icon symlink.
    - Add README file.
    - Add translations from Launchpad.
    - Remove preloadlib/*; it's undermaintained, and not really useful any
      more these days.
    - Various bug fixes; most visible being the misnamed
      etc/default/apport.default file (which should just be
      etc/default/apport).
  * Merge some bug fixes from trunk:
    - launchpad.py: Send and read Date: field again, reverting r1128; it is
      useful after all. (LP: #349139)
    - report.py, add_proc_info(): Only add ProcAttrCurrent if it is not
      "unconfined".
    - ui.py: Detect invalid PIDs (such as for kernel processes) and give a
      friendly error message. (LP: #360608)
    - report.py, add_hooks_info(): Always run common hooks, and run source
      package hooks if we do not have a binary package name. (LP: #350131)
    - launchpad.py: Consider socket errors when connecting as transient, so
      that crash-digger doesn't stop completely on them.
  * Drop debian/apport.README.Debian, superseded by upstream README.
  * Drop debian/apport.links, done by upstream setup.py now.
  * debian/rules, debian/apport.preinst: Drop upgrade fix for misnamed default
    file again, was only necessary for intra-Jaunty upgrades.
  * debian/control: python-launchpad-bugs → python-launchpadlib dependencies.
  * debian/local/apport-collect: Drop launchpadlib login code, just use the
    CrashDatabase implementation from apport/crashdb_impl/launchpad.py.
  * Make package backportable to hardy and intrepid:
    - debian/control: Relax python-central buil-dependency to 0.5.6.
    - debian/rules: Determine DH_PYCENTRAL value ("include-links" vs.
      "nomove") based on the installed pycentral version.
    - debian/rules: Only supply --install-layout=deb when Python version is
      2.6.
  * apport/hookutils.py: Add docstring for attach_hardware, thanks Matt
    Zimmerman! (Merged from lp:~mdz/apport/hookutils)
  * apport/crashdb_impl/launchpad.py: Support older wadllib API
    where bug.date_created was a string instead of a datetime object.
    (Cherrypicked from trunk).
  * debian/control: Drop apport dependency to python-xdg, it's not required.
    (LP: #354172)
  * debian/control: Drop gdb from Depends: to Recommends:. (LP: #354172)
  * debian/local/apport-collect: Print a friendly error message instead of
    crashing if the bug number is not an integer. (LP: #351050)
  * debian/local/apport-collect: Change incomplete tasks back to "New" ...