inconsistencies in the way to detect a missing attribute

Bug #142410 reported by Luca Olivetti
2
Affects Status Importance Assigned to Milestone
Zope 2
Fix Released
Medium
Unassigned

Bug Description

When catalogging/metadataing objects the catalog and plugin indexes try to detect a missing attribute in order to avoid indexing or storing metadata for a non existing attribute.
The problem is that each plugin index and the metadata extraction use different methods:

1) metadata (see recordify in Catalog.py) uses MV from Missing to indicate a missing attribute

2) UnIndex (and all indexes derived from it) uses an private attribute _marker defined as []

3) KeywordIndex uses () as a missing attribute and just catalogs everything

The problem is that if the attribute is provided by (e.g.) a python script, it's impossible for the python script to indicate a missing attribute, or it has to be done in several different methods:

a) for metadata is possible to return MV from Missing to indicate a missing attribute

b) for indexes deriving from UnIndex is impossible to indicate a missing attribute (since _marker is private)

c) for keyword indexes one has to return ()

d) even if it was possible to import the private _merker, it's not the same _marker for each index.

I think the solution would be to use the same value everywhere to detect a missing attribute, and this value should be useable from external method, like, e.g., MV from Missing, or a non private attribute imported from the same module.

Tags: bugday catalog
Revision history for this message
Toby Dickenson (htrd) wrote :

Status: Pending => Accepted

 Supporters added: htrd

> 1) metadata (see recordify in Catalog.py) uses MV from Missing to
> indicate a missing attribute
>
> 2) UnIndex (and all indexes derived from it) uses an private attribute
> _marker defined as []

For both of these two you are describing implementation details that you dont need to know to be able to achieve what you want. Just raise an AttributeError and it *should* just work.

It looks like metadata (see recordify in Catalog.py) wont handle an AttributeError inside an accessor method. I think *this* is a bug.

> 3) KeywordIndex uses () as a missing attribute and just catalogs everything

Yes. KeywordIndex has switched this behaviour several times in its history, and I agree that the current behaviour is wrong. I believe an AttributeError 'should' cause the object to not appear in the index, rather than be indexed with no properties.

Revision history for this message
Casey Duncan (caseman) wrote :

I agree. So this means the behavior should be that:

* If an attribute error is raised by a callable attribute during indexing, the object will be
  discarded and the error trapped (Perhaps with a blather log entry). This should hold true for
  indexes and metadata.

* Objects that are missing an attribute should not be in indexes for that attribute. Doing so is
  a waste of resources not to mention inconsistent.

These rules should be documented in the ZCatalog and PlugInIndex interfaces.

Revision history for this message
Toby Dickenson (htrd) wrote :

Status: Accepted => Pending

 Supporters removed: htrd

Unassigning myself.... Im going to have to leave this for someone else.

Revision history for this message
Tres Seaver (tseaver) wrote :

I'm attaching a patch which updates the plugin index interface docstring, and changes
the KeywordIndex implementation to avoid storing any value for items with either
AttributeErrors or empty keyword lists. All tests pass with the patch applied.

Tres Seaver (tseaver)
tags: added: bugday
removed: bug
Changed in zope2:
status: New → Triaged
Changed in zope2:
assignee: nobody → Jens Vagelpohl (jens-dataflake)
Revision history for this message
Jens Vagelpohl (dataflake-deactivatedaccount-deactivatedaccount) wrote :

I have applied and slightly extended the patch, and added a few tests.

Zope 2.12 branch: http://svn.zope.org/?view=rev&rev=112542

Zope trunk: http://svn.zope.org/?view=rev&rev=112543

All Zope tests and all CMF tests pass with these changes.

Changed in zope2:
status: Triaged → Fix Committed
Changed in zope2:
milestone: none → 2.12.7
Changed in zope2:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.