Unclassified videos missing/library rescan fail

Bug #415018 reported by Paul van Tilburg
34
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Moovida
Fix Released
Critical
Florian Boucault
elisa (Ubuntu)
Invalid
Undecided
Unassigned
moovida (Ubuntu)
Fix Released
Undecided
Andrew Starr-Bochicchio

Bug Description

Since Moovida 1.0.5/1.0.6 a whole lot of files that are normally in my Unclassified Videos are simply gone
after a reindex (I messed stuff up and moved it around and ran a rescan). They are not visible in TV Shows nor in Movies, while I assumed that at least files turning out to be videos but not findable in themoviedb or thetvdb would end up in Unclassified but this seems not to be the case.

When running Moovida, I get the following error hundreds of times:
WARN MainThread gst_metadata_slave_process_protocol Aug 17 21:36:48 Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/elisa/plugins/gstreamer/decodebin2_pipeline.py", line 317, in _bus_message_state_changed_cb

  File "build/bdist.linux-i686/egg/elisa/plugins/gstreamer/decodebin2_pipeline.py", line 784, in _done

  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 243, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 312, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 328, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/usr/lib/python2.5/site-packages/twisted/protocols/amp.py", line 937, in makeResponseFor
    originalFailure)
twisted.protocols.amp.BadLocalReturn: <bound method GstMetadataSlaveProtocol.get_metadata of <GstMetadataSlaveProtocol None at 0x12aaed0>> returned {'uri': 'file:///home/media/Videos/(Hidden).avi', 'metadata': [{'value': (<twisted.protocols.amp.String object at 0x1302450>, 'video'), 'key': 'file_type'}, {'value': (<twisted.protocols.amp.String object at 0x1302510>, 'video/x-divx'), 'key': 'mime_type'}, {'value': (<twisted.protocols.amp.String object at 0x1302550>, u'UT lol'), 'key': 'artist'}, {'value': (<twisted.protocols.amp.String object at 0x1302590>, 'file:///home/media/Videos/TV Shows/(Hidden).avi'), 'key': 'uri'}]} and <class 'elisa.plugins.gstreamer.amp_protocol.GetMetadata'> could not serialize it Traceback: <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0x8c in position 18: ordinal not in range(128)
build/bdist.linux-i686/egg/elisa/plugins/gstreamer/decodebin2_pipeline.py:784:_done
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:243:callback
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:312:_startRunCallbacks
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:328:_runCallbacks
--- <exception caught here> ---
/usr/lib/python2.5/site-packages/twisted/protocols/amp.py:928:makeResponseFor
/usr/lib/python2.5/site-packages/twisted/protocols/amp.py:1421:makeResponse
/usr/lib/python2.5/site-packages/twisted/protocols/amp.py:2227:_objectsToStrings
/usr/lib/python2.5/site-packages/twisted/protocols/amp.py:1133:toBox
/usr/lib/python2.5/site-packages/twisted/protocols/amp.py:1306:toStringProto

(Slave-14 stderr) (elisa/plugins/amp/master.py:71)
WARN MainThread gst_metadata_master Aug 17 21:36:48 no slave running in _next_request (elisa/plugins/gstreamer/amp_master.py:220)
WARN MainThread gst_metadata_master Aug 17 21:36:48 restarting slave (elisa/plugins/gstreamer/amp_master.py:297)
WARN MainThread gst_metadata_slave_process_protocol Aug 17 21:36:48 Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/elisa/plugins/gstreamer/decodebin2_pipeline.py", line 317, in _bus_message_state_changed_cb

  File "build/bdist.linux-i686/egg/elisa/plugins/gstreamer/decodebin2_pipeline.py", line 784, in _done

  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 243, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 312, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 328, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/usr/lib/python2.5/site-packages/twisted/protocols/amp.py", line 826, in _safeEmit
    aBox._sendTo(self.boxSender)
  File "/usr/lib/python2.5/site-packages/twisted/protocols/amp.py", line 481, in _sendTo
    proto.sendBox(self)
  File "/usr/lib/python2.5/site-packages/twisted/protocols/amp.py", line 1844, in sendBox
    self.transport.write(box.serialize())
  File "/usr/lib/python2.5/site-packages/twisted/internet/tcp.py", line 420, in write
    abstract.FileDescriptor.write(self, bytes)
  File "/usr/lib/python2.5/site-packages/twisted/internet/abstract.py", line 170, in write
    raise TypeError("Data must not be unicode")
exceptions.TypeError: Data must not be unicode
(Slave-16 stderr) (elisa/plugins/amp/master.py:71)
WARN MainThread gst_metadata_master Aug 17 21:36:48 restarting slave (elisa/plugins/gstreamer/amp_master.py:297)

Revision history for this message
Philippe Normand (philn) wrote :

What Twisted version do you use? On what distro? I can't reproduce this issue on Jaunty, running python 2.6 and Twisted 8.2.

Revision history for this message
Philippe Normand (philn) wrote :

Also, are you using the latest version of all your moovida plugins? Updating only some plugins is not an option and you probably know that already ;)

Revision history for this message
Paul van Tilburg (paulvt) wrote :

Using Moovida on Debian Exp/Sid... python-twisted* 8.2.0-2.
Well, the plugin updater says everything is fine.

Revision history for this message
Olivier Tilloy (osomon) wrote :

The problem is that the protocol used for communication between Moovida and the metadata slaves (based on AMP) currently doesn't support unicode. AMP itself supports unicode so it's a matter of implementing it correctly in Moovida (non trivial as there are several hardcoded encoding conversions happening in the code).

Changed in elisa:
importance: Undecided → High
status: New → Confirmed
assignee: nobody → Guillaume Emont (guijemont)
Revision history for this message
Alexander Hunziker (alex-hunziker) wrote :

I have the same problem. It doesn't seem to affect videos, which are added to the library just fine. But none of the audio tracks are imported successfully.

Revision history for this message
Philippe Normand (philn) wrote :

Got same issue on my eeepc running a fresh jaunty install

Revision history for this message
Florian Boucault (fboucault) wrote :

I am experiencing the same issue on a fresh Karmic installation. None of the music is indexed and the scanning process is painfully slow.

Revision history for this message
Florian Boucault (fboucault) wrote :

Bumping to critical as it breaks a major system (media scanning) and could affect a majority of users.

Changed in elisa:
importance: High → Critical
Revision history for this message
Florian Boucault (fboucault) wrote :

A fix is available and a merge request was sent:

https://www.moovida.com/quality/review/request/%<email address hidden>%3E

Changed in elisa:
assignee: Guillaume Emont (guijemont) → Kaleo (fboucault)
milestone: none → 1.0.7
status: Confirmed → In Progress
Revision history for this message
Kẏra (thekyriarchy) wrote :

I have the same problem but with my entire audio collection

Olivier Tilloy (osomon)
Changed in elisa:
status: In Progress → Fix Committed
Changed in elisa:
status: Fix Committed → Fix Released
Changed in elisa (Ubuntu):
status: New → Invalid
Changed in moovida (Ubuntu):
status: New → In Progress
assignee: nobody → Andrew Starr-Bochicchio (andrewsomething)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package moovida - 1.0.7-1ubuntu1

---------------
moovida (1.0.7-1ubuntu1) karmic; urgency=low

  * Merge with Debian Experimental (LP: #415018), remaining changes:
   - debian/rules:
    + Don't use site-packages
    + Move DEB_PYTHON_INSTALL_ARGS_ALL after the includes.

moovida (1.0.7-1) experimental; urgency=low

  * New upstream release.
  * First fully synced upload to experimental (closes: #537128).
  * debian/control:
    - Added transitional package elisa now that Moovida approximates
      the functionality of Elisa, courtesy of Andrew Starr-Bochicchio
      (closes: #541853).
    - Bumped the standards version to 3.8.3; no changes required.
    - Change the depend on python-setuptools to
      python-pkg-resources | python-setuptools; only python-pkg-resources
      is needed as runtime dependency.
    - Reorder the depends based on the alphabet for clarity.
  * debian/rules:
    - Change the CDBS hooks to register cleanup of upstream files.
      Fix courtesy of Stefano Zacchiroli.

 -- Andrew Starr-Bochicchio <email address hidden> Tue, 15 Sep 2009 18:43:10 -0400

Changed in moovida (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Olivier Tilloy (osomon) wrote :

@Paul, Alexander, Danny: is this bug fixed for you in the latest release (Moovida 1.0.7)?

Revision history for this message
Olivier Tilloy (osomon) wrote :

If not please subscribe to bug #438032 and add detailed information about your configuration.

Revision history for this message
Alexander Hunziker (alex-hunziker) wrote :

Fixed for me

Revision history for this message
Paul van Tilburg (paulvt) wrote :

Sorry, yes, it is!

Revision history for this message
Kẏra (thekyriarchy) wrote :

it isn't for me, i'll subscribe to that

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.