CI fails most tests on UITK trunk

the MIR transition broke your ui-toolkit tests. haven't looked into your merge proposal, but given that we dont have 100% pass rate here: http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/mako/96:20131014:20131014/4715/, I think its likely that you are not in able to use upstream merger effectively again.

Have you checked why you have those issues?

also,

14:46 < asac> t1mp: in short. best way to go is to submit a merge proposal that fixes your tests

If fixes are identified, please ping #ubuntu-ci-eng if manual intervention is needed (fginther, lool, vila).

for example, a standard button click as in https://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-maguro/2418/testReport/junit/ubuntuuitoolkit.tests.gallery.test_gallery/ButtonsTestCase/test_buttons_standard_button_/ fails with:

test-log: {{{
13:13:00.735 INFO globals:49 - ************************************************************
13:13:00.736 INFO globals:50 - Starting test ubuntuuitoolkit.tests.gallery.test_gallery.ButtonsTestCase.test_buttons (standard button)
13:13:00.737 WARNING testcase:118 - No tracing available - install the python-autopilot-trace package!
13:13:00.743 WARNING testcase:155 - Process manager backend unavailable, application snapshot support disabled.
13:13:00.971 INFO __init__:156 - Launching process: ['/usr/lib/arm-linux-gnueabihf/qt5/bin/qmlscene', '-testability', '-I/usr/lib/modules', '/usr/lib/ubuntu-ui-toolkit/examples/ubuntu-ui-toolkit-gallery/ubuntu-ui-toolkit-gallery.qml', '--desktop_file_hint=/usr/lib/ubuntu-ui-toolkit/examples/ubuntu-ui-toolkit-gallery/ubuntu-ui-toolkit-gallery.desktop']
13:13:13.188 INFO testcase:552 - waiting for process to exit.
13:13:13.190 INFO testcase:554 - Killing process 2748
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntuuitoolkit/tests/gallery/test_gallery.py", line 39, in setUp
    super(GalleryTestCase, self).setUp()
  File "/usr/lib/python2.7/dist-packages/ubuntuuitoolkit/tests/__init__.py", line 117, in setUp
    self.launch_application()
  File "/usr/lib/python2.7/dist-packages/ubuntuuitoolkit/tests/__init__.py", line 127, in launch_application
    app_type='qt')
  File "/usr/lib/python2.7/dist-packages/autopilot/testcase.py", line 284, in launch_test_application
    dbus_bus
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 187, in get_autopilot_proxy_object_for_process
    dbus_bus=dbus_bus,
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 271, in get_proxy_object_for_existing_process
    raise ProcessSearchError("Search criteria returned no results")
ProcessSearchError: Search criteria returned no results

So it seems to fail in opening the application from tests/__init__.py in:

    def launch_application(self):
        desktop_file_path = self._get_desktop_file_path()
        self.app = self.launch_test_application(
            base.get_qmlscene_launch_command(),
            "-I" + _get_module_include_path(),
            self.test_qml_file_path,
            '--desktop_file_hint={0}'.format(desktop_file_path),
            emulator_base=emulators.UbuntuUIToolkitEmulatorBase,
            app_type='qt')

the current theory is that Mir restricted the location of the desktop files to /usr/share/applications or ~/.local/share/applications/ and we were using --desktop_file_hint=/usr/lib/ubuntu-ui-toolkit/examples/ubuntu-ui-toolkit-gallery/ubuntu-ui-toolkit-gallery.desktop'. Let's see if fixing the path solves our issues.

the theory above does not seem to hold. Paths were fixed but still the tests fail.

I installed the latest image and ubuntu-ui-toolkit-autopilot, and executed "autopilot run -v -o /tmp/ubuntuuitoolkit.log -f xml ubuntuuitoolkit 2>&1 | tee /tmp/autopilot.log" as user phablet on the device.

Results:
ubuntuuitoolkit.log: https://pastebin.canonical.com/99082/
autopilot.log: https://pastebin.canonical.com/99083/

syslog: https://pastebin.canonical.com/99084/
tests were first executed some time after 20:00 phone time.

executing this:

qmlscene /usr/lib/ubuntu-ui-toolkit/examples/ubuntu-ui-toolkit-gallery/ubuntu-ui-toolkit-gallery.qml

gives me:

__pthread_gettid -2
Segmentation fault (core dumped)

running 'qmlscene' with no parameters also segfaults for me.

phablet@ubuntu-phablet:~$ cat test.qml
import QtQuick 2.0
Rectangle {
  color: "green"
  width: 400
  height: 400
}
phablet@ubuntu-phablet:~$ qmlscene test.qml
__pthread_gettid -2
Segmentation fault (core dumped)
phablet@ubuntu-phablet:~$

phablet@ubuntu-phablet:~$ qmlscene --desktop_file_hint=/usr/share/applications/ubuntu-ui-toolkit-gallery.desktop test.qml
__pthread_gettid -2
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

callbacks 0x42adfd8d 0x42adfd39
creating surface at (0, 58) with size (720, 1222) with title 'qmlscene'Segmentation fault (core dumped)
phablet@ubuntu-phablet:~$

If I repeatedly run the exact same test 10 times it changes from OK to ProcessSearchError.
autopilot run ubuntuuitoolkit.tests.gallery.test_gallery.GenericTests.test_progress_and_activity

Although I cannot verify easily I think processes pile up somewhere, I saw a ton of thumbnails in the dash which I tried to remove, they no longer appear but I can see the slowness of the device consistent with running many apps still.

Related unity issue: https://code.launchpad.net/~gerboland/unity-mir/fix-leaks/+merge/191449

It all works now :D