Zeitgeist Framework

Error inserting a event with a byte array payload into zeitgeist

Bug #557708 reported by Randal Barlow on 2010-04-07

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Zeitgeist Framework	Fix Released	Medium	Siegfried Gevatter	Zeitgeist Framework 0.3.3

Bug Description

When trying to insert a event with a utf-8 payload Zeitgeist throws an error.

ERROR:zeitgeist.engine:error while inserting 'Event([dbus.Array([dbus.String(u''), dbus.String(u'1270578721000'), dbus.String(u'http://zeitgeist-project.com/schema/1.0/core#VisitEvent'), dbus.String(u'http://zeitgeist-project.com/schema/1.0/core#UserActivity'), dbus.String(u'application://firefox.desktop'), dbus.String(u'')], signature=dbus.Signature('s')), [Subject([dbus.String(u'http://zeitgeist-project.com'), dbus.String(u'http://zeitgeist-project.com/schema/1.0/core#UnknownInterpretation'), dbus.String(u'http://zeitgeist-project.com/schema/1.0/core#UnknownManifestation'), dbus.String(u''), dbus.String(u'text/html'), dbus.String(u'Sample ZeitgeistSharp Test Event'), dbus.String(u'')])], dbus.Array([dbus.Byte(60), dbus.Byte(67), dbus.Byte(111), dbus.Byte(110), dbus.Byte(116), dbus.Byte(101), dbus.Byte(110), dbus.Byte(116), dbus.Byte(32), dbus.Byte(110), dbus.Byte(97), dbus.Byte(109), dbus.Byte(101), dbus.Byte(61), dbus.Byte(39), dbus.Byte(87), dbus.Byte(101), dbus.Byte(98), dbus.Byte(32), dbus.Byte(104), dbus.Byte(105), dbus.Byte(115), dbus.Byte(116), dbus.Byte(111), dbus.Byte(114), dbus.Byte(121), dbus.Byte(39), dbus.Byte(62), dbus.Byte(60), dbus.Byte(116), dbus.Byte(104), dbus.Byte(117), dbus.Byte(109), dbus.Byte(98), dbus.Byte(110), dbus.Byte(97), dbus.Byte(105), dbus.Byte(108), dbus.Byte(32), dbus.Byte(117), dbus.Byte(114), dbus.Byte(105), dbus.Byte(61), dbus.Byte(39), dbus.Byte(102), dbus.Byte(105), dbus.Byte(108), dbus.Byte(101), dbus.Byte(58), dbus.Byte(47), dbus.Byte(47), dbus.Byte(47), dbus.Byte(104), dbus.Byte(111), dbus.Byte(109), dbus.Byte(101), dbus.Byte(47), dbus.Byte(116), dbus.Byte(101), dbus.Byte(104), dbus.Byte(107), dbus.Byte(47), dbus.Byte(46), dbus.Byte(99), dbus.Byte(97), dbus.Byte(99), dbus.Byte(104), dbus.Byte(101), dbus.Byte(47), dbus.Byte(115), dbus.Byte(111), dbus.Byte(109), dbus.Byte(101), dbus.Byte(116), dbus.Byte(104), dbus.Byte(117), dbus.Byte(109), dbus.Byte(98), dbus.Byte(46), dbus.Byte(112), dbus.Byte(110), dbus.Byte(103), dbus.Byte(39), dbus.Byte(47), dbus.Byte(62), dbus.Byte(60), dbus.Byte(108), dbus.Byte(97), dbus.Byte(117), dbus.Byte(110), dbus.Byte(99), dbus.Byte(104), dbus.Byte(101), dbus.Byte(114), dbus.Byte(32), dbus.Byte(99), dbus.Byte(111), dbus.Byte(109), dbus.Byte(109), dbus.Byte(97), dbus.Byte(110), dbus.Byte(100), dbus.Byte(61), dbus.Byte(39), dbus.Byte(36), dbus.Byte(123), dbus.Byte(97), dbus.Byte(112), dbus.Byte(112), dbus.Byte(108), dbus.Byte(105), dbus.Byte(99), dbus.Byte(97), dbus.Byte(116), dbus.Byte(105), dbus.Byte(111), dbus.Byte(110), dbus.Byte(125), dbus.Byte(32), dbus.Byte(36), dbus.Byte(123), dbus.Byte(115), dbus.Byte(117), dbus.Byte(98), dbus.Byte(106), dbus.Byte(101), dbus.Byte(99), dbus.Byte(116), dbus.Byte(95), dbus.Byte(117), dbus.Byte(114), dbus.Byte(105), dbus.Byte(125), dbus.Byte(39), dbus.Byte(47), dbus.Byte(62), dbus.Byte(60), dbus.Byte(47), dbus.Byte(67), dbus.Byte(111), dbus.Byte(110), dbus.Byte(116), dbus.Byte(101), dbus.Byte(110), dbus.Byte(116), dbus.Byte(62)], signature=dbus.Signature('y'))])'
Traceback (most recent call last):
  File "/usr/local/share/zeitgeist/_zeitgeist/engine/main.py", line 383, in _insert_event_without_error
    return self._insert_event(event, sender)
  File "/usr/local/share/zeitgeist/_zeitgeist/engine/main.py", line 412, in _insert_event
    "INSERT INTO payload (value) VALUES (?)", event.payload)
  File "/usr/local/share/zeitgeist/_zeitgeist/engine/sql.py", line 42, in execute
    return super(UnicodeCursor, self).execute(statement, parameters)
ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 146 supplied.
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000756s
DEBUG:zeitgeist.engine:Got 1 events in 0.000157s

Related branches

lp:zeitgeist/0.1

lp:~kamstrup/zeitgeist/payload-fix

Merged into lp:zeitgeist/0.1

Zeitgeist Framework Team: Pending requested 2010-04-12

lp:ubuntu/lucid/zeitgeist

Revision history for this message

Randal Barlow (email-tehk) wrote on 2010-04-07:

This script will insert an event which triggers this error in the daemon Edit (892 bytes, text/x-python)

Revision history for this message

Mikkel Kamstrup Erlandsen (kamstrup) wrote on 2010-04-08:

Thanks for the test case! I'll integrate it into our test suite. And let me state for the record that I Love You! This is exactly how I imagined event.payload would be used :-) (I can hint to you that some of the other ZG developers had their doubts about event.payload :-))

Changed in zeitgeist:
status:	New → Confirmed
importance:	Undecided → Medium
assignee:	nobody → Mikkel Kamstrup Erlandsen (kamstrup)

Revision history for this message

Randal Barlow (email-tehk) wrote on 2010-04-08: Re: [Bug 557708] Re: Error inserting a event with a byte array payload into zeitgeist

Thanks, the payload really should work perfectly for Journal. I also want to
note that the byte array will fail on ascii encoding too

On Apr 8, 2010 1:55 AM, "Mikkel Kamstrup Erlandsen" <
<email address hidden>> wrote:

Thanks for the test case! I'll integrate it into our test suite. And let
me state for the record that I Love You! This is exactly how I imagined
event.payload would be used :-) (I can hint to you that some of the
other ZG developers had their doubts about event.payload :-))

** Changed in: zeitgeist
Status: New => Confirmed

** Changed in: zeitgeist
Importance: Undecided => Medium

** Changed in: zeitgeist
Assignee: (unassigned) => Mikkel Kamstrup Erlandsen (kamstrup)

--
Error inserting a event with a byte array payload into zeitgeist
https://bugs.launchpad.net/bug...
Status in Zeitgeist Framework: Confirmed

Bug description:
When trying to insert a event with a utf-8 payload Zeitgeist throws an
error.

ERR...

Siegfried Gevatter (rainct) on 2010-04-08

Changed in zeitgeist:
status:	Confirmed → Fix Released
milestone:	none → 0.3.3
assignee:	Mikkel Kamstrup Erlandsen (kamstrup) → Siegfried Gevatter (rainct)

Revision history for this message

Mikkel Kamstrup Erlandsen (kamstrup) wrote on 2010-04-08:

As discussed on IRC this bug is not completely dead yet.

We to do something like:

a = dbus.Array ( [dbus.Byte(ord("z")), dbus.Byte(ord("g")) ])
b = bytearray()
for q in a : b.append(q)

buf = sqlite3.Binary(b)
and then insert that buf in the sql call. Biggest problem is that 'bytearray' is Python 2.6+ only

- or maybe I am rambling...

Changed in zeitgeist:
status:	Fix Released → Confirmed

Revision history for this message

Siegfried Gevatter (rainct) wrote on 2010-04-12:

Anything new on this?

Revision history for this message

Mikkel Kamstrup Erlandsen (kamstrup) wrote on 2010-04-12:

Nope sorry... I am spending the little spare time I have on the ontology branch

Revision history for this message

Mikkel Kamstrup Erlandsen (kamstrup) wrote on 2010-04-12:

Looked a bit more on this... Handling binary data in Python <= 2.5 is just a mess... But it looks like the 'array' module should be able to help us out...

Consider a table defined as:

CREATE TABLE tab (s VARCHAR, b BLOB)

Then the following works, but is Python >= 2.6 only:

  data = bytearray([dbus.Byte(1),dbus.Byte(2)])
  b = sqlite3.Binary(data)
  cursor.execute ("INSERT INTO tab VALUES ('teststring', ?)", (b,))

It looks like we could do equivalently in Python 2.5 with:

  # Create binary array
  data = array.array ("B", [dbus.Byte(1), dbus.Byte(2)])
  b = sqlite3.Binary(a)
  cursor.execute ("INSERT INTO tab VALUES ('teststring', ?)", (b,))

But the very last execute statement fails with:

  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ValueError: could not convert BLOB to buffer

-- quite unhelpful backtrace that is :-S Any ideas much appreciated...

Mikkel Kamstrup Erlandsen (kamstrup) on 2010-04-13