Can't change virtual terminal on login screen or when auto-login is enabled

Hmm, no idea right now.

Try turning on debugging in /etc/gdm3/custom.conf and see if anything is outputted in the journal when you try this.

If this happens in disco, an upstream bug would be good too if you don't mind. Cheers!

https://gitlab.gnome.org/GNOME/gdm/issues/466

# gdb attach <CHVT_PID>

(gdb) bt F
#0 0x00007fd01e9c65d7 in ioctl () at ../sysdeps/unix/syscall-template.S:78
No locals.
#1 0x000055fbea90bce1 in main (argc=2, argv=<optimized out>) at chvt.c:45
        fd = 3
        num = <optimized out>

$ cat /proc/5222/stack
[<0>] __vt_event_wait.isra.3.part.4+0x40/0x90
[<0>] vt_waitactive+0x80/0xd0
[<0>] vt_ioctl+0xd34/0x1150
[<0>] tty_ioctl+0xf6/0x8c0
[<0>] do_vfs_ioctl+0xa8/0x630
[<0>] ksys_ioctl+0x75/0x80
[<0>] __x64_sys_ioctl+0x1a/0x20
[<0>] do_syscall_64+0x5a/0x120
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[<0>] 0xffffffffffffffff

# drivers/tty/vt/vt_ioctl.c
/**
* vt_waitactive - active console wait
* @event: event code
* @n: new console
*
* Helper for event waits. Used to implement the legacy
* event waiting ioctls in terms of events
*/

int vt_waitactive(int n)
{
struct vt_event_wait vw;
do {
vw.event.event = VT_EVENT_SWITCH;
__vt_event_queue(&vw);
if (n == fg_console + 1) {
__vt_event_dequeue(&vw);
break;
}
__vt_event_wait(&vw);
__vt_event_dequeue(&vw);
if (vw.done == 0)
return -EINTR;
} while (vw.event.newev != n);
return 0;
}

# drivers/tty/vt/vt_ioctl.c
static void __vt_event_wait(struct vt_event_wait *vw)
{
/* Wait for it to pass */
wait_event_interruptible(vt_event_waitqueue, vw->done);
}

# include/linux/wait.h
/**
* wait_event_interruptible - sleep until a condition gets true
* @wq_head: the waitqueue to wait on
* @condition: a C expression for the event to wait for
*
* The process is put to sleep (TASK_INTERRUPTIBLE) until the
* @condition evaluates to true or a signal is received.
* The @condition is checked each time the waitqueue @wq_head is woken up.
*
* wake_up() has to be called after changing any variable that could
* change the result of the wait condition.
*
* The function will return -ERESTARTSYS if it was interrupted by a
* signal and 0 if @condition evaluated to true.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1817738

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

I was able to reproduce the bug, and this upstream fix[1] look promising.

On my test machine, using a test package containing this patch on top of current gdm3 bionic, chvt doesn't hang anymore, and work as expected.

I'd would like more feedback before I proceed with an SRU.

[1] - 39fb4ff6 manager: don't run autologin display on tty1

- Eric

I have a laptop that exhibits this behavior. The switching of the ttys will hang about 95% of the time (1 in 20). With this test package, it now works 100% of the time as long as I don't switch to tty1.

I tested it 5 times in a row with a reboot between each.

I can Ctrl+Alt+F# without any issue.

If I run "sudo chvt 1", it will not switch out of that tty again until I reboot my laptop again. Using 2 through 12 all work and switch the display to that tty.

Thanks,
Chris Newcomer

Note: Desktop team kindly offered to update the vcs and do the upload in bionic on my behalf.

Thanks !

Uploaded in bionic by desktop team. Thanks.

Hello Eric, or anyone else affected,

Accepted gdm3 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gdm3/3.28.3-0ubuntu18.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

By installing the test packages from PPA https://launchpad.net/~slashd/+archive/ubuntu/lp1817738, login process stops at Ubuntu greeting screen and gnome-shell fails to start up normally. In syslog expecting:

  gnome-shell[1330]: GNOME Shell started at Fri Mar 08 2019 03:09:04 GMT-0500 (EST)

got:

  gnome-shell[1348]: driver/gl/cogl-framebuffer-gl.c:311: GL error (1282): Invalid operation
  gnome-shell[1348]: cogl-sampler-cache.c:200: GL error (1280): Invalid enumeration value
  gnome-shell[1348]: clutter-offscreen-effect.c:205: Unable to create an Offscreen buffer
  gnome-shell[1348]: message repeated 23 times: [ clutter-offscreen-effect.c:205: Unable to create an Offscreen buffer]

Still investigating.

Tested both testppa and proposed:

[UPGRADE] gdm3:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.4+testpkgb1
[UPGRADE] gir1.2-gdm-1.0:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.4+testpkgb1
[UPGRADE] libgdm1:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.4+testpkgb1

and

[UPGRADE] gdm3:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.5
[UPGRADE] gir1.2-gdm-1.0:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.5
[UPGRADE] libgdm1:amd64 3.28.3-0ubuntu18.04.4 -> 3.28.3-0ubuntu18.04.5

gnome-shell locks up with the same symptoms.

# Bionic
| |4.15|4.18|4.19
|-----------------|----|----|----
|nouveau |[1] |[2] |[1]
|-----------------|----|----|----
|nvidia |[1] |[1] |[1]

* kernel:
   * 4.15.0-46-generic (bionic)
   * 4.18.0-16-generic (bionic)
   * 4.19.0-13-generic (disco)
* nvidia-driver:
   * 390.116-0ubuntu0.18.04.1

1. hang in splash
2. reproduce steps for bug 1817738 works, but hangs at switching to VT-1 from gnome-shell

@vicamo,

Seems like you are experiencing the same problem as I experience when being on VT-1.
Which was part of the reason why we SRU'd this change to avoid gmd3 to use VT-1 by default.
The problem can't be reproduce using other VT.

Seems like something acquires VT-1 (systemd ? gdm3 ?) and prevent the VT switch for VT-1.

The root cause of this problem is plymouth, indeed.

The 'chvt N' command blocks if the VT/tty is in VT_AUTO + KD_GRAPHICS state.

In this state the kernel bails out early in the ioctl(VT_ACTIVATE) syscall
and does not post the VT_EVENT_SWITCH event the ioctl(VT_WAITACTIVE) syscall
will be waiting for -- causing chvt to block.

The function path is:

  vt_ioctl(tty0, VT_ACTIVATE, ...)
  -> set_console()
     -> if (... VT_AUTO && KD_GRAPHICS ...) return -EINVAL; <<-- bails out.
     -> schedule_console_callback(); return 0; <<-- continue to send event.
        -> console_callback()
           -> change_console()
              -> complete_change_console()
                 -> vt_event_post(VT_EVENT_SWITCH, ...)

  vt_ioctl(tty0, VT_WAITACTIVE, ...)
  -> vt_waitactive()
     -> __vt_event_wait(VT_EVENT_SWITCH) <<-- blocks/wait to receive event.

gdm properly sets the VT out of VT_AUTO mode (which causes chvt not to block)
after it tells plymouth to deactivate.

BUT plymouth can set it back to VT_AUTO mode afterward, regardless, while it
handles the udev event of the DRM/DRI graphics card addition, as that causes
the VT/tty to be reconfigured.

This can be verified with plymouth debugging, e.g., kernel boot option
'plymouth.debug=file:/run/plymouth.debug', plus source code inspection:

1) gdm calls 'plymouth deactivate', which calls
   ply_terminal_close()
    -> ply_terminal_stop_watching_for_vt_changes()
       -> if (terminal->is_watching_for_vt_changes == true) ioctl(VT_SETMODE, VT_AUTO)
       -> terminal->is_watching_for_vt_changes = false

 [ply-boot-server.c:LINE] print_connection_process_identity:connection is from pid PID (/bin/plymouth deactivate) with parent pid PID (/usr/sbin/gdm)
...
 [ply-terminal.c:LINE] ply_terminal_close:restoring color palette
 [ply-terminal.c:LINE] ply_terminal_close:stop watching tty fd
...

2) plymouth udev event timeout expires, it notices the DRM/DRI devices,
   and re-enables the VT watching while processing those; in the calls:

   ply_terminal_open()
   -> ply_terminal_watch_for_vt_changes()
      -> terminal->is_watching_for_vt_changes = true;

 [ply-device-manager.c:LINE] create_devices_from_udev:Timeout elapsed, looking for devices from udev
 ...
 [ply-device-manager.c:LINE] create_devices_for_terminal_and_renderer_type:creating devices for /dev/dri/card0 (renderer type: 1) (terminal: /dev/tty1)
 ...
 [./plugin.c:LINE] load_driver:Opening '/dev/dri/card0'
 [ply-terminal.c:LINE] ply_terminal_open:trying to open terminal '/dev/tty1'

3) init calls 'plymouth quit --retain-splash' which goes into
   ply_terminal_close() again, and since watching is true, it
   sets the VT into VT_AUTO again (see calls in #1 above) ...
   (*after* gdm had already set the VT up out of VT_AUTO).

 [ply-boot-server.c:LINE] print_connection_process_identity:connection is from pid PID (/bin/plymouth quit --retain-splash) with parent pid PID (/sbin/init splash)
...
 [ply-terminal.c:LINE] ply_terminal_close:restoring color palette
 [ply...

Sponsor note:

Before uploading plymouth, I would prefer we ask SRU team to drop the gdm3 workaround in bionic-proposed (which failed anyway due to regression in gnome-shell).

Once gdm3 is dropped, I'm fine with sts-sponsor to proceed with the upload of plymouth.

- Eric

Thanks for checking and fixing up this, Eric and Dan. :)

https://launchpad.net/ubuntu/+source/gdm3/+publishinghistory
Deleted Bionic proposed main gnome 3.28.3-0ubuntu18.04.5

Hello Eric, or anyone else affected,

Accepted plymouth into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/plymouth/0.9.3-1ubuntu7.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Verification successful with bionic-proposed.

Tested in login screen and auto-login session;
repeated 5 times with consistent results.

bionic-updates
--------------

$ dpkg -s plymouth | grep ^Version:
Version: 0.9.3-1ubuntu7.18.04.1

$ grep -wo splash /proc/cmdline
splash

$ dmesg | grep console
[ 0.004000] console [tty0] enabled

$ sudo fgconsole
1

$ sudo chvt 4
^C # blocked.

$ sudo strace -e ioctl chvt 4
<...>
ioctl(3, VT_ACTIVATE, 0x4) = 0
ioctl(3, VT_WAITACTIVE, 0x4

<blocks until...> ^C) = -1 EINTR (Interrupted system call)
strace: Process 1518 detached

bionic-proposed
---------------

$ dpkg -s plymouth | grep ^Version:
Version: 0.9.3-1ubuntu7.18.04.2

$ grep -wo splash /proc/cmdline
splash

$ dmesg | grep console
[ 0.004000] console [tty0] enabled

$ sudo fgconsole
1

$ sudo chvt 4
$ # finished.

This bug was fixed in the package plymouth - 0.9.3-1ubuntu7.18.04.2

---------------
plymouth (0.9.3-1ubuntu7.18.04.2) bionic; urgency=medium

  [ Mauricio Faria de Oliveira ]
  * d/p/0013-device-manager-don-t-watch-for-udev-events-when-deac_bionic.patch:
    - Don't process udev events after we've been deactivated (e.g., by gdm).
      These can cause the VT/tty to be set back to VT_AUTO mode after it's
      set to VT_PROCESS mode (by gdm), which blocks 'chvt' indefinitely on
      that VT (login screen or auto-login session in tty1). (LP: #1817738)

  [ Brian Murray ]
  * debian/patches/git_helper_renderer.patch,
    debian/patches/git_desactivate_renderer.patch:
    - 'device-manager: free keyboards when deactivating a renderer',
      (LP: #1794292)

 -- Dan Streetman <email address hidden> Thu, 04 Apr 2019 11:41:18 -0400

The verification of the Stable Release Update for plymouth has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

I'm seeing this same issue (hung syscall on VT) from systemd-sleep on a suspend operation with Xubuntu (and therefore lightdm). I see a similar stack-trace too. 18.04 with 4.15.0-56-lowlatency.

https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1744370

apt list plymouth
Listing... Done
plymouth/bionic-updates,now 0.9.3-1ubuntu7.18.04.2 amd64 [installed,automatic]

$ sudo cat /proc/$(pidof systemd-sleep)/stack

[<0>] __vt_event_wait.isra.2.part.3+0x40/0x90
[<0>] vt_waitactive+0x7b/0xd0
[<0>] vt_move_to_console+0x46/0xc0
[<0>] pm_prepare_console+0x23/0x40
[<0>] pm_suspend+0x19c/0x3b0
[<0>] state_store+0x85/0xf0
[<0>] kobj_attr_store+0x12/0x20
[<0>] sysfs_kf_write+0x3c/0x50
[<0>] kernfs_fop_write+0x125/0x1a0
[<0>] __vfs_write+0x1b/0x40
[<0>] vfs_write+0xb1/0x1a0
[<0>] SyS_write+0x55/0xc0
[<0>] do_syscall_64+0x73/0x130
[<0>] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[<0>] 0xffffffffffffffff