<module> in Python crash signatures is too generic

Bug #920403 reported by Martin Pitt
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Daisy
Won't Fix
Medium
Unassigned
apport (Ubuntu)
Fix Released
High
Martin Pitt

Bug Description

Bug 440889 got a lot of duplicates assigned which are not really. The original crash was

Traceback (most recent call last):
  File "/usr/bin/software-center", line 71, in <module>
    from softwarecenter.app import SoftwareCenterApp
  File "/usr/share/software-center/softwarecenter/app.py", line 42, in <module>
    from view.installedpane import InstalledPane
  File "/usr/share/software-center/softwarecenter/view/installedpane.py", line 36, in <module>
    from softwarepane import SoftwarePane, wait_for_apt_cache_ready
  File "/usr/share/software-center/softwarecenter/view/softwarepane.py", line 33, in <module>
    from appdetailsview import AppDetailsView
  File "/usr/share/software-center/softwarecenter/view/appdetailsview.py", line 49, in <module>
    from widgets.wkwidget import WebkitWidget
  File "/usr/share/software-center/softwarecenter/view/widgets/wkwidget.py", line 27, in <module>
    import webkit
ImportError: libicui18n.so.40: cannot open shared object file: No such file or directory

while the invalid duplicate is

Traceback (most recent call last):
  File "/usr/bin/software-center", line 131, in <module>
    from softwarecenter.ui.gtk3.app import SoftwareCenterAppGtk3
  File "/usr/share/software-center/softwarecenter/ui/gtk3/app.py", line 82, in <module>
    from softwarecenter.ui.gtk3.panes.installedpane import InstalledPane
  File "/usr/share/software-center/softwarecenter/ui/gtk3/panes/installedpane.py", line 36, in <module>
    from softwarecenter.ui.gtk3.models.appstore2 import (
  File "/usr/share/software-center/softwarecenter/ui/gtk3/models/appstore2.py", line 33, in <module>
    from softwarecenter.backend.reviews import get_review_loader
  File "/usr/share/software-center/softwarecenter/backend/reviews/__init__.py", line 32, in <module>
    from bsddb import db as bdb
  File "/usr/lib/python2.7/bsddb/__init__.py", line 67, in <module>
    import _bsddb
ImportError: No module named _bsddb

the pattern for both is

sqlite> select * from crashes where crash_id = 440889;
/usr/share/software-center/software-center:ImportError:<module>:<module>:<module>:<module>:<module>:<module>|440889||2009-10-02 20:40:05

Martin Pitt (pitti)
Changed in apport (Ubuntu):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Martin Pitt (pitti)
Martin Pitt (pitti)
summary: - different python crashes misidentified as duplicates
+ crash signature for ImportErrors is too generic
Martin Pitt (pitti)
summary: - crash signature for ImportErrors is too generic
+ Do not put <module> into Python crash signatures
summary: - Do not put <module> into Python crash signatures
+ <module> in Python crash signatures is too generic
Revision history for this message
Martin Pitt (pitti) wrote :

Fixed in trunk r2186.

Changed in apport (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.5 KiB)

This bug was fixed in the package apport - 1.94-0ubuntu1

---------------
apport (1.94-0ubuntu1) precise; urgency=low

  [ Martin Pitt ]
  * New upstream release:
    - apport: Set the group of written reports to "whoopsie" if that group
      exists.
    - Fix tests to run properly against the system-installed modules and
      binaries.
    - test/run: Run under LC_MESSAGES=C to avoid test failures due to
      translated strings.
    - general-hooks/generic.py: Also attach xsession-errors for programs that
      link to libgtk-3.
    - launchpad.py: Properly handle "Expired" status, to avoid marking new
      bugs as duplicates of expired ones. (LP: #941854)
    - apport: Fix crash if the "whoopsie" group does not exist. (LP: #942326)
    - report.py, crash_signature(): Do not put "<module>" frames into Python
      crash signatures that happen outside of function/method calls. Fall back
      to the file/line number as a frame description instead. This will do a
      much better job at disambiguating e. g. different ImportError crashes.
      (LP: #920403)
    - Make "binary changed since the time of the crash" error message more
      comprehensible, thanks Paolo Rotolo. (LP: #942830)
    - crashdb.py, check_duplicate(): It can happen that a bug gets identified
      as being a duplicate of bug S by symbolic signatures and a duplicate of
      bug A by address signatures. Empirical evidence shows that this is due
      to the unavoidable jitter in stack traces (A and S not being identified
      as duplicates as their signatures differ slightly) and not a logic
      error. So instead of erroring out, duplicate all three bugs and keep the
      lowest number as the master ID. (LP: #943117)
    - Revert the usage of multiple nested threads during data collection, and
      switch back to only using one UI thread. The UI implementations can, and
      now do, decide between showing a spinner and showing a progress dialog
      in the ui_*_info_collection_progress() methods. This fixes libX11
      crashes when multiple UI threads do changes concurrently (LP: #901675),
      and also avoids multi-thread induced crashes in Pango (LP: #943661). The
      removal of the collect() method also fixes the new crashes in it.
      (LP: #942098, #939803)
    - ui.py, get_desktop_entry(): Fix crash on uninstalled package.
      (LP: #940984)
    - data/unkillable_shutdown: Fix crash on race condition when PID goes away
      while the report is created. (LP: #546369)
    - apport/hookutils.py, pci_devices(): Fix crash on unexpected lines from
      lspci. (LP: #904489)
    - Drop hardcoded "Ubuntu" words again which crept in with the whoopsie
      support merge. Use the DistroRelease: field.
    - apport-kde: Fix Home page URL in KApplication metadata.
    - apport-gtk: Fix resizability and size after hiding details.
      (LP: #405418)
    - test/run: Drop "local" argument. This now tests against the source tree
      when run in the source tree root, and against the system
      libraries/programs when run from anywhere else.
    - test/run: Consider command line arguments as test names and only run
      those when given. Also support just runni...

Read more...

Changed in apport (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

This change in apport modified the way that crash signatures are calculated for python crashes and subsequently we have some buckets that have use the old crash signature. For example:

https://errors.ubuntu.com/bucket/?id=/usr/share/unattended-upgrades/unattended-upgrade-shutdown%3AValueError%3A%3Cmodule%3E

I download an instance in this bucket:

PYTHONPATH=/home/bdmurray/source-trees/daisy/daisy tools/create_apport_report.py 658f7a70-4957-11e2-872a-e4115b0f8a4a unattended.crash

and resubmitted the crash and it appeared in a different bucket:

https://errors.ubuntu.com/bucket/?id=/usr/share/unattended-upgrades/unattended-upgrade-shutdown:ValueError:/usr/share/unattended-upgrades/unattended-upgrade-shutdown@34

This bucket is more specific than the previous one. This could also be confirmed via the following:

import apport
report = apport.Report()
crash_sig = report.load(open('/home/bdmurray/unattended.crash', 'rb'))
report.crash_signature()
u'/usr/share/unattended-upgrades/unattended-upgrade-shutdown:ValueError:/usr/share/unattended-upgrades/unattended-upgrade-shutdown@34'

It may be a good idea to look for oopses with a crash signature that has the string '<module>' in them and recreate the crash_signature and then rebucket those oopses.

Changed in daisy:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Brian Murray (brian-murray) wrote :

This was fixed during the development cycle for Precise so SRU'ing it is not necessary.

Revision history for this message
Brian Murray (brian-murray) wrote :

Given that this was fixed in 12.04 it doesn't really seem worth going back and cleaning up this data any more.

Changed in daisy:
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.