ubiquity doesn't preselect the right default keyboard layout

I'll look into it.

I can confirm the same bug in Ubuntu 18.04 and Xubuntu 18.04 (daily builds of today). I'll update the bug description accordingly.

Maybe helpful extra information: for Dutch, the right default entries in /etc/default/keyboard should be:

XKBMODEL="pc105"
XKBLAYOUT="us"
XKBVARIANT="intl"

Subscribing the Desktop Team, maybe they have some information that will lead to an easy fix.

If I install Ubuntu in Swedish, I expect the installer to propose the basic se keyboard layout, which it also did previously. So I'm a bit surprised to hear that you would like it to suggest us+intl in the case of a Dutch installation; I would have thought you'd prefer it to pick the basic nl keyboard layout.

Anyway, the pre-selection of keyboard layout seems to be broken. Whichever language you pick, the basic English (US) layout is suggested.

Previously the window for setting keyboard layout showed up after you had selected the time zone, but is now the first window you see when clicking the install button. It may have something to do with it, or not.

"Try Ubuntu" is broken in this respect as well - I don't want the English (US) layout when trying in Swedish!

I subscribed Mathieu Trudel-Lapierre and Jean-Baptiste Lallement, who both have made changes to ubiquity lately. Also raised the importance to "High" - I think this is an important issue; bad first impression for most non-US users.

@gunnarhj: well, the Dutch keyboard has unfortunately disappeared completely, somewhere at the end of the previous century.... So since 20 years, all keyboards in The Netherlands are US keyboards which need the "intl" xkeybvariant (with dead keys for the accents). :-)

Thanks for helping to bring this matter to the attention of the right people!

@Pjotr12345: Thanks for the lesson on Dutch keyboards. :)

I can confirm that it's a regression compared to 16.04. On 16.04, if the selected language matches the time zone location, a reasonable guess is pre-set in the window for setting keyboard layout.

So if I select Nederlands and pretend to live in Haag, the installer indeed suggests the us+intl layout.

This behavior was broken in 17.10, and I have a feeling that it's related to the switch from unity-settings-daemon to gnome-settings-daemon.

Compare:

http://bazaar.launchpad.net/~unity-settings-daemon-team/unity-settings-daemon/trunk/view/head:/plugins/keyboard/gsd-keyboard-manager.c

with:

https://gitlab.gnome.org/GNOME/gnome-settings-daemon/blob/master/plugins/keyboard/gsd-keyboard-manager.c

The fix of bug #1719938 resulted in this patch:

https://bazaar.launchpad.net/~ubuntu-desktop/gnome-settings-daemon/ubuntu/view/head:/debian/patches/ubuntu_ibus_configs.patch

which handles it for some locales which typically need input methods. But for all other locales it simply defaults to the basic English (US) layout.

As already mentioned, this may make the live session for many (most?) languages disappointing. Typing is an essential part of the desktop experience.

OP said that this also affects Xubuntu and Lubuntu so its not likely only gnome-settings-daemon involved here, since neither of them use it.

Thank you for your bug report, could you give some details
- where do you pick the language? on the syslinux first text based boot? or in the installer?
- do you use the install only mode or the live session?
- is that specific to that language or do you see similar issues in other locales?

I tried on a daily bionic iso
- boot the iso
- touch a key to see the syslinux menu, select nederlands
- select "ubuntu installieren"
-> it boots to ubiquity only with netherlands select
- do next
-> "Engels (VS) with "dode toesten" is selected

that seems about right?

Sebastien Bacher: what happened on your system is the right result, but that's not what happens on my machine.... Still not, when I use the latest daily Ubuntu iso (timestamped 20180-04-03 07:03).

It's a test machine with UEFI running in full UEFI mode, which might account for the difference with the result from your test. My approach is straightforward:

- I boot my laptop from the iso.
- it shows only the normal Grub-like "terminal" lines, no GUI, with "Try Ubuntu without installing" as first line of the four. Which is normal for UEFI.
- I select the first line
- after the desktop appears, I establish Wifi connection
- I start Ubiquity
- In the welcome screen of Ubiquity, I select "Nederlands"
- I click the button "Verder" (Continue)
- I'm being presented with the wrong preselected "Engels (VS)" keyboard layout, like I described in the opening message in this thread.

Just tested some other languages: exactly the same error happens when I select Deutsch, Francais, Espanol or Italiano. They all present English (US) as default preselected keyboard layout.

@Sebastien Bacher: have you already tested it with a UEFI machine running in full UEFI mode, on the bare metal (not in VirtualBox)?

For the sake of completeness, I repeated the test with the latest daily build of Xubuntu Bionic (timestamped: 2018-04-04 06:05). Again: the Xubuntu iso shows precisely the same results as with the Ubuntu iso.

This seems to rule out gnome-settings-daemon as culprit...

the scenario you describe is totally different, you don't select the language on the "grub like menu" and you boot to the live session mode and not to the "ubiquity only" (which is the third entry), there seems to be an issue with ubiquity in any case because if you go back and pick another language and do next the keyboard is not updated

I'm able to reproduce Sebastien's success by following the steps in comment #9.

At the same time I'm able to reproduce the failure as reported by both Pjotr12345 and myself via these steps:

- boot the iso
- patiently wait, without pressing any key, until Ubiquity boots
  [yes Seb, I can be patient sometimes]
-> Ubiquity is booted with English preselected (see attached image)
- Change to Nederlands
-> The display language changes instantly
- Click the "ubuntu installieren" button
-> The basic English (US) layout is selected

So it seems like the language must be selected before Ubiquity boots to get a reasonably guessed preselected layout.

But how would the user know that pressing a key initially makes a difference when they are not told so? The official installation instruction here:

https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-desktop

does not mention the syslinux menu either.

The desired behavior is reasonably that when you change the language on Ubiquity's welcome screen, the preselected layout should be changed to match the selected language (just as the display language is).

Btw, with the press-a-key approach, "Try Ubuntu" enters a session with the correct layout in /etc/default/keyboard. So yes, this is probably a pure ubiquity bug, not affecting gnome-settings-daemon.

jibel is looking at this one but it looks like a console-setup issue at this point

From the logs it seems that even if the locale is set to a non-english language, the config variable xkb-keymap introduced in 17.10 is set to 'us' and Ubiquity uses this value to update the UI.

Apr 17 08:55:11 debconf (filter): <-- GET debian-installer/locale
debconf (developer): <-- GET debian-installer/locale
debconf (developer): --> 1 fr_FR.UTF-8
Apr 17 08:55:11 debconf (filter): <-- GET localechooser/languagelist
debconf (developer): <-- GET localechooser/languagelist
debconf (developer): --> 1 fr
Apr 17 08:55:11 debconf (filter): <-- GET debian-installer/country
debconf (developer): <-- GET debian-installer/country
debconf (developer): --> 1 FR
Apr 17 08:55:11 debconf (filter): <-- GET keyboard-configuration/xkb-keymap
debconf (developer): <-- GET keyboard-configuration/xkb-keymap
debconf (developer): --> 1 us
Apr 17 08:55:11 debconf (filter): <-- GET keyboard-configuration/modelcode
debconf (developer): <-- GET keyboard-configuration/modelcode
debconf (developer): --> 1
Apr 17 08:55:11 debconf (filter): <-- GET keyboard-configuration/layoutcode
debconf (developer): <-- GET keyboard-configuration/layoutcode
debconf (developer): --> 1
Apr 17 08:55:11 debconf (filter): <-- GET keyboard-configuration/variantcode
debconf (developer): <-- GET keyboard-configuration/variantcode
debconf (developer): --> 1
Apr 17 08:55:11 debconf (filter): <-- GET keyboard-configuration/optionscode
debconf (developer): <-- GET keyboard-configuration/optionscode
debconf (developer): --> 1

I think I finally understand how the whole default-keyboard configuration is done in ubiquity and console-setup. The reason why it's busted is indeed the keyboard-configuration/xkb-keymap debconf variable. The reason everything worked up until artful was that before artful we have been reverting the xkb-keymap pieces from keyboard-configuration.config because of ubiquity (mostly), and this change was dropped with the artful console-setup merge with Debian. I'm trying to ask around if that was intentional or not. If it was, then we need to handle setting of xkb-keymap after performing language selection (possibly in ubiquity). If not, then we'd have to remove it - or at least the pieces where xkb-keymap is overriding all locale/country-based default keyboard layout selection in console-setup.

I talked with cyphermox and it seems the delta was removed intentionally - with the intention of just starting to use xkb-keymap in ubiquity - but this cycle that didn't happen. Since it does require a small amount of design work and we're really close to release, it's best to just re-introduce the changes. I'll be uploading as soon as I have a moment.

This bug was fixed in the package console-setup - 1.178ubuntu2

---------------
console-setup (1.178ubuntu2) bionic; urgency=medium

  * debian/keyboard-configuration.config:
    - Drop the xkb-keymap bits once again as we're not ready for those yet, as
      it's currently causing an invalid default layout in the installer.
      (LP: #1758647)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Thu, 19 Apr 2018 08:53:32 +0200

There was a misunderstanding there: I never suggested we should use xkb-keymap instead (or at least, that's not the message I was trying to convey).

The intent was to reduce delta by not removing so much code, when console-setup merges are already difficult. I didn't foresee that some of that code setting a default would impact ubiquity (because ubiquity does things specially with d-i components, sometimes running them multiple times). There's always a need to update ubiquity after changing d-i components; it's inevitable since some of them are embedded in ubiquity source.

The removals of xkb-keymaps aren't ubiquity-specific: we don't use this code in d-i, and instead use multiple screens to change keyboards. That's a conscious decision.

The point for supporting xkb-keymap "in ubiquity" is more about making sure that we either "rebuild" the value based on layoutcode/variantcode, or otherwise unset it before running the configuration script -- ie. leave it in, but just explicitly ignore its values.

I was just saying that if the decision was made to keep xkb-keymap then there had to be a decision to start using it implicitly in ubiquity. And by 'using' I didn't just mean the literal sense of us using the value explicitly, but acting accordingly to what consequences it brings - as it requires us to do one of the two as you mentioned in your post. So I don't think there's much of a misunderstanding here. It's all good and I guess for the next merge we should proceed with adjusting things so that ubiquity is happy with the changes.