VMWare "Easy Install" not working in groovy

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1899632

I can confirm this, we've to go through the normal ubiquity installation

I have never heard of VMWare "Easy Install" and am not aware that we ever did enablement work for this. How does VMWare decide what is the "first disc" of an Ubuntu install?

VMWare Easy Install has certainly been supported in the past. It's been part of the QA test cases for many years [1] - As mentioned, it does work up until Focal, Groovy seems to have lost it somehow.

How does VMWare decide what is the "first disc" of an Ubuntu install? I don't know.

[1] http://iso.qa.ubuntu.com/qatracker/milestones/389/builds/171121/testcases/1304/results

On Tue, Oct 13, 2020 at 08:58:21PM -0000, Marcus Tomlinson wrote:
> VMWare Easy Install has certainly been supported in the past. It's been
> part of the QA test cases for many years [1] - As mentioned, it does
> work up until Focal, Groovy seems to have lost it somehow.

Sure, but I believe this Just Worked rather than having been something that
ever involved explicit enablement work.

We have recently done some work on the groovy image builds to drop various
useless / unused files on the image. And it appears that VMWare is relying
on one of these files to detect the "first disc".

> How does VMWare decide what is the "first disc" of an Ubuntu install? I
> don't know.

I don't see how we can commit to supporting this without that information.

The only changes to the top-level directory structure of the image in
groovy are:

 - removal of the isolinux directory
 - removal of the boot.catalog file
 - EFI/ directory has been renamed to efi/

The first two of these are not a change we are going to revert, these
changes are intrinsic to the transition from isolinux to GRUB for BIOS boot.

The third change could be reverted; there's no good reason for a system that
is looking for this directory to be case-sensitive, but that's one thing we
could change.

The functional regression seems explained, should the bug be marked as confirmed instead of incomplete or does the foundation team require the report to figure out what the feature is relying on to be working first?

On Wed, Oct 14, 2020 at 03:31:59PM -0000, Sebastien Bacher wrote:
> The functional regression seems explained, should the bug be marked as
> confirmed instead of incomplete or does the foundation team require the
> report to figure out what the feature is relying on to be working first?

Yes, we need some insight into how VMWare is detecting this. I can
speculatively make the change to cd-boot-images-amd64 to change the case of
the EFI directory, but if this doesn't work, we either need guidance from
VMWare, or someone who has access to VMWare to hack their images to pinpoint
the source of regression, or to just mark this wontfix (since we're not
reverting isolinux).

please insert the first cd smells a lot like message from detecting debian CD1 vs others.

have we removed any other files from the iso that are typically found on CD1, which VMWare is looking for....

Can somebody run strings on all the VMWare binaries, to see if they can grep for isolinux, bootx64.efi, efi, etc? and check the case & filepaths that smell like ISO detection?

https://people.canonical.com/~xnox/isos/

i've built 3 more isos, please let me know if any of these are any better.

@xnox, no better with those 3 isos I'm afraid.

we now suspect that VMware tries to repack the .iso and expects it to be isolinux bootable.

we will try to get in touch with VMware team, as we are not going to be using isolinux on the iso going forward. Thus they will need to fix VMware Player to be compatible with Groovy's cd-boot-images-amd64 way of buiding images.

https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/cd-boot-images-amd64/tree/debian/rules#n62

or see /usr/share/cd-boot-images-amd64/xorriso-cmd.txt in the cd-boot-images-amd64 package.

There is nothing ubuntu can do to fix this.

FTR: we're trying to get in contact with the VMware team, to find out what they are doing and figure out together how best to make this work.

VMware internal bug for this is:
https://bugzilla.eng.vmware.com/show_bug.cgi?id=2663128

-john

Hi, Lain Lane

I am a developer from VMware, thank you for providing information about this Easy Install bug for VMware product.
Let me introduce something about part of the Easy Install process. Easy Install depend on the removed isolinux directory for two things:
1) If find isolinux/isolinux.cfg, then Easy Install doesn't need secondary disc to continue, otherwise we can see the "..... To use Easy Install, insert the first disc of the set." warning message.
2) Easy Install search isolinux.cfg, if exist, then get parent directory isolinux, then search every .cfg files in this directory to find and determine 3 parameters including kernel path, ram disk path and kernel args like below example:
kernelPathRet = "/casper/vmlinuz" (this is kernel path)
ramdiskPathRet = "/casper/initrd" (this is ram disk path)
kernelArgsRet = "file=/cdrom/preseed/ubuntu.seed quiet splash --- " (this is kernel args)
Before Ubuntu 20.10, we can get those 3 parameters from isolinux/txt.cfg finally. However, as we know, the isolinux directory was removed, so Easy Install failed to get them then finally fail.

I already have a solution and verified it can Easy Install Ubuntu 20.10 successfully, that is to hard code those 3 parameters for Ubuntu 20.10 and newer iso, and I assume that they will not change in the future(please correct me if Canonical plan to change them in later releases).

If you have any concern, please contact me by <email address hidden> or add comments.
Thank you.

Hi Tian Lan!

1) We have removed isolinux, and use /boot/grub/grub.cfg for all flavours.

2)
kernelParthRet = "/casper/vmlinuz" is true for majority of Ubuntu flavours
ramdiskPathRet = "/casper/initrd" is true for majority of Ubuntu flavours
kernelArgsRet = well, it depends on what you do.

By default, Ubuntu in groovy boots with "file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---" => but that is meant for interactive user to choose to either try the live session or install the system.

Without maybe-ubiquity as you show, the ISO will boot straight to the live session.

Xubuntu Desktop ISO for example uses this by default:

linux /casper/vmlinuz file=/cdrom/preseed/xubuntu.seed maybe-ubiquity quiet splash ---
initrd /casper/initrd

Note that /boot/grub/grub.cfg exists across all Ubuntu ISOs going back to 2012, and it is planned to continue to be available.

Maybe it makes sense to parse /boot/grub/grub.cfg and strip out / append options as needed by easy-install. But I don't know if easy-install does automatic unattended preseeded install, or if it expects to boot the ISO into live desktop.

Obviously if easyinstall provides/modifies file=/cdrom/preseed/ubuntu.seed always for the autoinstall to complete, then it should work with all the flavours that use ubiquity.

Adding myself and Tian Lan to the CC list

I'm happy to review things if you want, publically or privately. I am launchpad.net/~xnox here and github.com/xnox on github too. or @ubuntu.com. As you wish.

Thank you very much for this information, /boot/grub/grub.cfg also contain the necessary parameters, so I think it will not break Easy Install Ubuntu 20.10 image after some fix work.

Internal PR says this is fixed in 16.0.1 and 12.0.1.

On Tue, Nov 03, 2020 at 01:03:05AM -0000, John Savanyo wrote:
> Internal PR says this is fixed in 16.0.1 and 12.0.1.

Thanks so much guys!

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

Is there some way for me to get a build of 16.0.1 to test?

Hi, John

As Marcus requested, could you help to send below build to him?
https://buildweb.eng.vmware.com/sb/41464310/
This build is based on WS 16.0.0 with Ubuntu 20.10 Easy Install supported patch.
Thank you very much.

Regards,
Tian
-----Original Message-----
From: <email address hidden> <email address hidden> On Behalf Of Marcus Tomlinson
Sent: Thursday, November 5, 2020 6:45 PM
To: Tian Lan <email address hidden>
Subject: [Bug 1899632] Re: VMWare "Easy Install" not working in groovy

Is there some way for me to get a build of 16.0.1 to test?

--
You received this bug notification because you are subscribed to the bug report.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.launchpad.net%2Fbugs%2F1899632&amp;data=04%7C01%7Ctlan%40vmware.com%7C7625f90d4e0642d143cf08d88178aa5e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637401702545748184%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=XMnwaVoyVuZWWd7DM%2BE%2Br79cBWI7ye7ltPXZ9uLUZRE%3D&amp;reserved=0

Title:
  VMWare "Easy Install" not working in groovy

Status in Ubuntu CD Images:
  Invalid

Bug description:
  Groovy Desktop 20201013.1

  Upon selecting the iso, VMWare Player should display:

  "This operating system will use Easy Install"

  Instead, you get:

  "Ubuntu 64-bit 20.10 detected. To use Easy Install, insert the first
  disc of the set."

  (See screenshot)

  I'm testing this on VMWare Player 16. Focal Daily works. Groovy Daily
  does not.

To manage notifications about this bug go to:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.launchpad.net%2Fubuntu-cdimage%2F%2Bbug%2F1899632%2F%2Bsubscriptions&amp;data=04%7C01%7Ctlan%40vmware.com%7C7625f90d4e0642d143cf08d88178aa5e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637401702545748184%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=p12nyV73kxMdKnTGNHO9N%2FWp24V2OjrxOe1NdVz8JdM%3D&amp;reserved=0

Marcus,

Unfortunately VMware's policy is to not release sandbox builds and any pre-release of an official build of a product requires a Pre-Release Binary Access Agreement (PRBA) to be in place between the two companies.

The good news is that the fix is in Workstation and Player 16.1.0; the anticipated 16.0.1 was renumbered as 16.1.0. And the WS 16.1.0 and Fusion 12.1.0 products have been publicly release.

Thanks again for bring this issue to VMware's attention.

Excellent, I've just tested 16.1.0 and can confirm that Easy Install is now working with the latest Groovy ISO. Good job guys!