nexus4 window mode mouse motion bounded incorrectly with external monitor

Might be bug 1313610, or closely related.

Yeah pretty sure it's the same as bug 1313610. We've never updated our cursor bounds correctly on rotation...

unduping for the moment, after some discussion with anpok, while this is similar we're not totally convinced it's the same

@duflu also, your are welcome please try this - specifically notice that if you move the mouse completely horizontal, the mouse will stick in a position that is inconsistent (e.g. sometimes sticks at a position further left or shorter left from the right edge than the previous time)
also, when the mouse "sticks" to a position while moving completely horizontal to the right, keep moving horizontal to the right, then move completely horizontal back to the left....the distance you travel back to the left is ~ the same before the mouse is "picked up" and "unstuck" giving it a feeling the position is changing for the mouse, however not rendering correctly.

note this is silo 0 where unity8 is doing the render - using the mir relative mouse events
to install
flash nexus4 with --channel=ubuntu-touch/rc-proposed/ubuntu
then install silo 0, unfortunately the silo is dirty holding back unity8 (atm i think) - so even if you install using ctrain tools you'll need to dist-upgrade
so....citrain device-upgrade 0 <device pin/password> , then apt-get update, apt-get dist-upgrade

more information i should add - this is not seen on nexus7, which in windowed mode is always in landscape and doesnt rotate
on nexus4, if a second monitor is not attached, this is not seen, even in windowed mode (which rotates to landscape, e.g. just connect a bt mouse will put into windowed mode)
but once a monitor is connected to the nexus 4, this can be seen

Yeah there may be some common things to fix with bug 1313610 but possibly some things different here.

OK, I can see one problem already. While Unity8 does its own screen rotation without notifying Mir, Mir cannot do anything to help it update the cursor bounds. Because Unity8 is not telling Mir that the display has rotated, Mir will keep the old cursor bounds...

Portrait:

$ mirout /var/run/mir_socket
Connected to server: /var/run/mir_socket
Card 0: Max 2 simultaneous outputs
Output 0: Card 0, LVDS, connected, 768x1280+0+0, used, on, 61mm x 102mm (4.7"), normal
     768x1280 60.00*+
Output 1: Card 0, DisplayPort, disconnected

Landscape (incorrectly shows orientation still "normal"):

$ mirout /var/run/mir_socket
Connected to server: /var/run/mir_socket
Card 0: Max 2 simultaneous outputs
Output 0: Card 0, LVDS, connected, 768x1280+0+0, used, on, 61mm x 102mm (4.7"), normal
     768x1280 60.00*+
Output 1: Card 0, DisplayPort, disconnected

So if Unity8 is going to keep doing rotation independently of Mir, it either needs to find a way to update the Mir display config without making Mir actually apply that display config, or Unity8 do mouse movement entirely within itself using relative motion.

Oh... is there another Mir bug too? Is Mir failing to provide continuous relative motion data without hitting invisible bounds?

Aha! Confirmed on arale with a USB mouse and mir-demos.

Relative motion events usually work continuously, and come in even after the cursor hits the screen boundary. However Mir stops sending relative motion events if you've rotated the screen (touch+volume key in mir_proving_server) and the cursor happens to be outside of the original non-rotated dimensions. That's a Mir bug.

My mistake. Not a Mir bug. I was looking at MIR_CLIENT_INPUT_RECEIVER_REPORT=log and expecting the client to receive motion events after the cursor had left the window, which was wrong.

Using a more correct test where the gesture starts in the app window (click but don't release), then you can see the client continues to receive relative motion data even after hitting the edge of the screen:

[1440559624.200713] <DEBUG> input-receiver: Received event:pointer_event(when=1440559624199932000 (0.685269 ms ago), from=6, motion, button_state=1, x=1919, y=1151, dx=1, dy=1, vscroll=0, hscroll=0, modifiers=1)

[1440559624.208718] <DEBUG> input-receiver: Received event:pointer_event(when=1440559624207936000 (0.687115 ms ago), from=6, motion, button_state=1, x=1919, y=1151, dx=0, dy=1, vscroll=0, hscroll=0, modifiers=1)

[1440559624.377396] <DEBUG> input-receiver: Received event:pointer_event(when=1440559624376062000 (1.160346 ms ago), from=6, motion, button_state=1, x=1919, y=1151, dx=0, dy=1, vscroll=0, hscroll=0, modifiers=1)

So Mir seems to be working properly. It even keeps sending relative motion events after the cursor has hit the edge.

Please try this:

   restart unity8 MIR_CLIENT_INPUT_RECEIVER_REPORT=log
   tail -f ~/.cache/upstart/unity8.log

and see if Unity8 actually is still receiving input events with relative data (dx, dy) when the problem occurs.

Please try comment #10 as that should help us to identify if the problem is on the Mir side or Unity8.

I will investigate the claim that qtmir is still receiving relative mouse movement even when you see the cursor hitting an invisible boundary

Mir keeps sending pointer events after it hits a boundary only if you're using the primary display. If you connect and external monitor, it doesn't work like that anymore.

Steps to reproduce:
- Flash an N7 and install silo0 in it.
- Add "initctl set-env --global QT_LOGGING_RULES=qtmir.mir.input.debug=true" to /usr/share/upstart/sessions/unity8.conf
- sudo restart lightdm
- in a phablet terminal: tail -f ~/.cache/upstart/unity8.log
- connect a bluetooth mouse and check the output, you should see a lot of entries like "qtmir.mir.input: Received MirPointerEvent(x=971,y=573,relative_x=-1,relative_y=0)"
- see that the mouse goes through the entire screen and you always get mouse events even when the pointer reaches a boundary
- now conenct an external monitor and see what happens

expected outcome:
mouse events keep coming like before

actual outcome:
you have invisible boundaries since mouse events stop coming when the pointer reaches some width and height boundary.

Note that the cursor you see on the screen is drawn by unity8 and it's driven solely by the relative movement in MirPointerEvents.

To get the debug output you've to make sure your qtmir is at least lp:~unity-team/qtmir/silo0 revision 369 ("QtEventFeeder: log the pointer events it gets from Mir")

Can you double check using the report in comment #10? It's still theoretically a small possibility that the bug lies in unity8 till that's confirmed.

Dropped back to High (https://wiki.ubuntu.com/Bugs/Importance)

@duflu

There you go:
http://paste.ubuntu.com/12213056/

When you hit the invisible boundary on the external monitor nothing comes up in the log. Neither from mir nor qtmir. Happy now? :)

Now we know what the problem is, and it will be gone once qtmir+unity8 start using one surface per display instead of cloning displays as it's currently happening.

yep, like daniel said we'll lower priority on this since qtmir multimonitor branch is getting cleaned up and landed.

Comment #18 kind of makes sense.

In trying to replicate the bug using just Mir demos I found one way to do it: Have a client (which in our case is Unity8 itself) that fails to resize to cover all displays. As it fails to resize to cover all displays, the client (Unity8 shell) will fail to get motion events that are outside of its window.

So this problem can be solved in Unity8 by resizing the Unity8 Mir surface to cover all displays, or it can be solved more reliably with a Mir enhancement (which we plan on already) --> implement pointer grabs so that the cursor location does not ever stop the owning surface from receiving pointer events.

Either way, seems like Mir is working correctly. In the absence of pointer grab support, clients (including Unity8) stop getting pointer events when the cursor leaves their window.

Actually there might still be a related bug in Mir. Seems unlikely, but not really sure yet.

I'm fairly sure this bug is invalid for Mir per comment #20. Although pointer grabs in Mir would help to solve the problem, that's really an enhancement for later. In comment #21 I was just being cautious in case anyone can correct me.

Status changed to 'Confirmed' because the bug affects multiple users.

I've found the same kind of bug, but no external monitor required --> bug 1502805.

this is fixed in silo 22 which should land soon

Actually input still gets in a pretty sorry state when an external monitor is connected.
But waiting for silo 22 to land before raising the issue to mir guys. Will make for easier debugging and communication.

seems corrected in latest image
channel: ubuntu-touch/rc-proposed/ubuntu-pd
version version: 54
version ubuntu: 20151026
version device: 20150911
version custom: 20150929-2-vivid