grub-install trashes the ntfs partition that starts at sector 63 if one or more sectors in the embedding area is occupied by Adobe FlexNet

https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/701806

Maybe a duplicate?
I have Adobe Acrobat X installed now which seems to be shipped w/Flexnet too.

May I mark mine a duplicate?

NTFS maintains two copies of MFT, so recovering the first sector of the partition is enough to let it finish bootstrapping and self-recovery through chdksk.exe "fsck" utility.
Users already encountered with the issue can boot up their computer with a Windows 7 installation DVD, launch Command Prompt, and run chkdsk c: /f to have the partition fixed.
This will NOT overwrite the MBR so you can still boot into Ubuntu, and running update-grub will also put Windows back into the Grub menu.

ps.dd is dangerous to normal users and since not only one sector is trashed it's still recommended to perform a thorough check using tools released by Microsoft itself.

I investigated this duplicate https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/716865
Its FlexNet sector was sector 50.
I compared that sector50 with this sector32, the first 8 bytes are identical:
d8 41 a0 f5 02 00 02 00
So this is the signature of FlexNet.

If the command below finds the signature you are at risk:
sudo dd if=/dev/sda count=63 2>/dev/null | hd | grep "d8 41 a0 f5 02 00 02 00"

Hi bsfmig and billbear,

In my case, chkdsk didn't work as even Windows XP couldn't recognise my partition. For chkdsk to work, a partition must first be recognised as a Windows partiton, be it NTFS, FAT16 or FAT32. My corrupted partition was recognised as a raw filesystem.

Also, I do not have FlexNet installed.

In my case, the only utility that worked to repair the partition was TestDisk (https://secure.wikimedia.org/wikipedia/en/wiki/TestDisk).

Using this utility, I recovered the MFT.

Perhaps this might offer a clue as to what is going on.

Hi Wei-Yee Chan,

FlexNet is not a independent software, it's part of many proprietary Windows-based softwares (in most cases huge Adobe products: Photoshop, Dreamweaver, After Effects, Flash Catalyst, Acrobat Pro ...), it may install without your acknowledgement or authorization. Once any of these Adobe products is installed, FlexNet silently and secretly writes code in the boot track. The code will permanently be there even if you uninstall, format partitions, delete partitions, or repartition. Its purpose is to prevent people from gaining another trial period even if you reinstall the whole OS.

Though you don't know it, FlexNet is there at sector 50. You can find the signatures from the files (sda.1, sda.2) you attached at https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/716865:
ubuntu@ubuntu:~/Downloads$ cat sda.1 | hd | grep "d8 41 a0 f5 02 00 02 00"
00006400 d8 41 a0 f5 02 00 02 00 49 d3 61 f0 50 50 15 84 |.A......I.a.PP..|
ubuntu@ubuntu:~/Downloads$ cat sda.2 | hd | grep "d8 41 a0 f5 02 00 02 00"
00006400 d8 41 a0 f5 02 00 02 00 49 d3 61 f0 50 50 15 84 |.A......I.a.PP..|

I did not try the chkdsk thing but did you boot from a windows cd to run it?

By design, the last sector of ntfs is a backup of its boot sector. By copying the last sector to its first sector I successfully recovered the NTFS file system on my memory stick. There's no operating systems on the memory stick so I don't know if this recovery method can fix the boot issue of Windows. ( If boot sector is the only sector destroyed it is expected to fix the boot issue but unfortunately several sectors are overwritten ) I would appreciate it if you could test this for me:

First check if FlexNet is still there ( Earlier versions of grub2 overwrites it)
sudo dd if=/dev/sda count=63 2>/dev/null | hd | grep "d8 41 a0 f5 02 00 02 00"
If the command above returns nothing it's gone.

If it finds FlexNet, try ruin your C drive by installing grub2 from 11.04(backup your data first of course):
sudo grub-install /dev/sda
You will see a warning "Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track."

Then run "sudo parted -l", if the file system of sda1 is not shown you successfully ruin your C drive.

Now run "sudo fdisk -lu", from the file you attached its output was
   Device Boot Start End Blocks Id System
/dev/sda1 * 63 85160564 42580251 7 HPFS/NTFS
/dev/sda2 85160626 1953520064 934179719+ 5 Extended
/dev/sda5 85160628 419441084 167140228+ 7 HPFS/NTFS
/dev/sda6 419441148 461386799 20972826 83 Linux
/dev/sda7 461386863 1846141604 692377371 83 Linux
/dev/sda8 1846141700 1953520064 53689182+ 7 HPFS/NTFS

If the partition layout has not been changed since then, recover the boot sector of sda1 from the backup sector:
sudo dd if=/dev/sda of=/dev/sda count=1 seek=63 skip=85160564
Be very careful here no to type anything wrong. Both "if" and "of" devices are /dev/sda, not /dev/sda1. And if the start or end sector changes, modify seek="start sector" skip="end sector" correctly.

Now /dev/sda1 should be mountable again. Reboot to see if windows boots. Thank you!

Hi billbear,

I don't know how I got "infected" with FlexNet. All the while, grub hadn't be causing any problems and nothing on my Windows system had changed, with the exception of a few updates.

You do think that changes in grub2 caused this, don't you? Well, I think so too.

Yes, I did try chkdsk. I booted into the recovery console using the Windows setup disk. The partition was corrupted.

At this moment, I do not have access to that dual boot machine, but I'll let you know once I've tried your suggestion. Thanks.

To Wei-Yee Chan,
As of my experiments, only Windows 7's chdksk (incl. its installation disk and Windows PEs based on Win7) can successfully fix the partition. All older versions, i.e. XP's and Vista's, only report a "RAW disk" error, while Win7's say "NTFS boot sector corrupted, using the second MFT instead" and finally, "Windows has made corrections to the filesystem".

Uuuuuuuuuuuuuuuuuuuuuh, this is a very nasty bug!

Same problem here installing grub on a dual boot system with Windows 2000 on /dev/sda1 and Linux on /dev/sda2. Doing an "update-grub" does not recognize the windows partition and does not create an entry in the grub menu. Doing an "grub-install" kills the master boot record of the Windows 2000 partition and prints out the following:

/usr/sbin/grub-setup: warn: Sector 32 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
/usr/sbin/grub-setup: warn: Sector 56 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.

After that, the Windows 2000 partition cannot be booted anymore. So I inserted the Windows CD and started up the repair console. After doing an "fixboot" Windows 2000 does boot up again - but Ubuntu does't anymore!!!

I hope this bug will be fixed soon, altough I am aware, that this is caused by crappy Windows software.

Hi André,
Please run this command for me
sudo dd if=/dev/sda count=63 2>/dev/null | hd | grep "d8 41 a0 f5 02 00 02 00"
I need more proof that "d8 41 a0 f5 02 00 02 00" is the signature.

Wei-Yee Chan wrote on 2011-03-13:

>I don't know how I got "infected" with FlexNet. All the while, grub hadn't
>be causing any problems and nothing on my Windows system had changed,
>with the exception of a few updates.

You or other people may have installed some adobe product or some other software and then uninstalled to leave this sector there.

>You do think that changes in grub2 caused this, don't you? Well, I think
>so too.

Yes. This is caused by the bad way the new grub2 responds to https://bugs.launchpad.net/ubuntu/maverick/+source/grub2/+bug/441941
Earlier versions of grub2 just overwrites the flexnet sector(s). But in your case, flexnet sector 50 locates after the end of the core image and survived.
In my opinion, grub2 shouldn't dance with these windows softwares just to add complexities and difficulties. If such sectors are detected, just refuse to install grub2 and use grub legacy instead.

billbear: GRUB recognises two signatures: d8 41 a0 f5 02 00 02 00 and d4 41 a0 f5 03 00 03 00. More may be added in the future if I hear of more.

Your opinion is noted, but I disagree. GRUB Legacy needs to die. In any case, this problem can affect GRUB Legacy too, just a bit more rarely - it depends what sector the software in question happens to choose.

This bug is fairly simple - I'll have it fixed soon.

Colin,
Are the two signatures both of FlexNet?
Nice to see fix committed:)

This bug was fixed in the package grub2 - 1.99~rc1-4ubuntu1

---------------
grub2 (1.99~rc1-4ubuntu1) natty; urgency=low

  * Resynchronise with Debian. Remaining changes:
    - Adjust for default Ubuntu boot options ("quiet splash").
    - Default to hiding the menu; holding down Shift at boot will show it.
    - Set a monochromatic theme and an aubergine background for Ubuntu.
    - Apply Ubuntu GRUB Legacy changes to legacy update-grub script: title,
      recovery mode, quiet option, tweak how memtest86+ is displayed, and
      use UUIDs where appropriate.
    - Fix backslash-escaping in merge_debconf_into_conf.
    - Remove "GNU/Linux" from default distributor string.
    - Add crashkernel option.
    - Bypass menu unless other OSes are installed or Shift is pressed.
    - Allow Shift to interrupt 'sleep --interruptible'.
    - Reduce visual clutter in normal mode.
    - Remove verbose messages printed before reading configuration.
    - Suppress kernel/initrd progress messages, except in recovery mode.
    - Handle filesystems loop-mounted on file images.
    - Ignore devices loop-mounted from files in Linux grub.d scripts.
    - Show the boot menu if the previous boot failed.
    - Don't generate device.map during grub-install or grub-mkconfig.
    - Adjust upgrade version checks for Ubuntu.
    - Suppress "GRUB loading" message unless Shift is held down.
    - Adjust versions of grub-doc and grub-legacy-doc conflicts.
    - Fix LVM/RAID probing in the absence of /boot/grub/device.map.
    - Look for .mo files in /usr/share/locale-langpack first.
    - Make sure GRUB_TIMEOUT isn't quoted unnecessarily.
    - Build-depend on qemu-kvm rather than qemu-system for grub-pc tests.
    - Use qemu rather than qemu-system-i386.
    - Add a grub-rescue-efi-amd64 package.
    - On Wubi, don't ask for an install device, but just update wubildr
      using the diverted grub-install.
    - Enable lua from grub-extras.
    - Incorporate the bitop library into lua.
    - Add enum_pci function to grub module in lua.
    - Check hardware support before using gfxpayload=keep.
    - Build part_msdos and vfat into EFI boot images.
    - Put second and subsequent Linux menu entries in a submenu.
    - Preferred resolution detection for VBE.
    - Set vt.handoff=7 for smooth handoff to kernel graphical mode.
    - Don't ask where to install if upgrading and core.img doesn't exist.
    - Drop the default priority of grub2/linux_cmdline to medium.
    - Build with gcc-4.5 on ppc64.
    - Add apport hook for ProblemType = 'Package'.

grub2 (1.99~rc1-4) unstable; urgency=low

  * Don't touch /boot/grub/grub2-installed if using the --root-directory
    option to grub-install (thanks, Nicolas George; closes: #614927).
  * Update branch_devmapper.patch, adding partitioned MD RAID support
    (untested) and support for probing multipath disks.
  * Update ntldr-img from grub-extras:
    - Only call ntfs_fix_mmft if the attribute to find is AT_DATA. This
      matches GRUB's NTFS module.
    - Install grubinst as grub-ntldr-img.
  * Fix loading GRUB from lnxboot (LP: #693671).
  * Update branch_embed-sectors.patch to avoid straying into first partition
    when emb...

On Tue, Mar 15, 2011 at 01:09:45PM -0000, billbear wrote:
> Are the two signatures both of FlexNet?

Yes. There are a couple of other signatures too. Look at
debian/patches/branch_embed-sectors.patch in the grub2 source package if
you want details.

Yesterday I upgraded my 10.10 x64 to the beta of 11.04 x64 using update-manager.

Upon reboot got error in grub about missing xputs and dropped into grub-recovery.

I have followed the instructions here about reinstalling grub: http://ubuntuforums.org/showthread.php?t=1581099

Upon reinstalling grub the error about flexnet appears and when you reboot you are dumped back into grub-recovery.

I cannot boot into Ubuntu as when I followed the instructions in the above thread typing 'insmod linux' resulted in an error about not being able to find memset.

To get back into windows I ended up purging grub via chroot through a live cd, and then installing lilo and recovering my windows MBR (see page two of the thread linked).

My next option would be to trash the sector that flexnet was on but as I use Adobe CS3 products under windows I am reluctant to do this.

Just trash it using dd. Flexnet should be rewriting a new copy of signature in a new place next time related application runs.
Or you can do the following workaround:
cat /boot/grub/lnxboot.img /boot/grub/core.img > /somewhere_you_have_write_access/grldr
then copy the file to a Windows partition add a corresponding item to boot.ini (might not work) or bcd (verified by myself) that points to grldr and
dpkg-reconfigure grub-pc
when prompted to select on which disk or partition will grub be installed on, uncheck the disk (i.e./dev/sda) but check the partition (i.e. /dev/sda4 or so) on which you install Ubuntu.
By doing all those above, you'll let Windows boot managers take over Ubuntu's. But remember, when you encounter with "grub" and "grub-pc" software package updates in Ubuntu, it's suggested to re-generate grldr afterwards.

I have just freshly installed the new Ubutntu 11.04 Beta 2 on my office computer with Windows 2000 on it. And the process went smoothly. Grub2 recognized the Windows 2000 partition with the Flexnet-infected boot sector and integrated it into the boot menu. The bug therefore seems to be resolved ... Good work!

I have reinstall grub in my note book many times and no "error message" like this one,
-------------
tbuntu@tbuntu:~$ sudo mount /dev/sda11 /mnt
[sudo] password for tbuntu:

tbuntu@tbuntu:~$ sudo grub-install --boot-directory=/mnt/boot /dev/sda
/usr/sbin/grub-setup: warn: Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.

tbuntu@tbuntu:~$ grub-install -v
grub-install (GRUB) 1.99-12ubuntu5
-------------
I will see if my window XP is destroyed or not.

I have Ubuntu 10.04, 11.04 and Lubuntu 11.10 installed with window XP.
The previous grub reinstall was done in Ubuntu 11.04.
And after that I did experiment with Boot Repair, and set Ubuntu 11.04 as default boot.
Current Lubuntu 11.10 (sda11) was a clone of Lubuntu 11.10 (sad4) by Clonezilla CD,
my first experiment on Cloning by Clonezilla.
Then I format sda4 for next ubuntu installation.
And after that I did experiment with Boot Repair, and set Ubuntu 11.04 as default boot.

And today,
I reinstall grub2 since I want default /boot to be change to lubuntu 11.10(sad11).
Then I have the above error.

I have not install any new adobe software in windows at all for long time, and the last time I turn on my windows,
to check window files system of usb-hdd. and did not update anything.

The only Adobe program is AR Reader installations were done to USB-stick Ubuntu,
or external drive and to some of my Ubuntu experiment.

Though Photoshop and AI and other Adobe update in windows XP were done years ago,
since I install Ubuntu 8.04,
and never have this issue until lubuntu 11.10 "grub reinstall".
The installation of lubuntu 11.10 did not produce any error too.

When
Bug Watch Updater (bug-watch-updater) on 2011-03-16
Changed in grub2 (Debian):
status: New → Fix Released

Dose it suppose to still produce the error above?

I will report on my XP later.
Thank you.

Disaster,

It actually destroy Grub,
reboot went into grub rescue command,
 grub>

I fresh install Ubuntu 11.04 to sda4.
Successful installation and boot with all grub options correctly listed.

The root partition is in sda4,
I test the same command,
---------------------------------------------------
tbuntu@tbuntu:~$ sudo mount /dev/sda4 /mnt
[sudo] password for tbuntu:
tbuntu@tbuntu:~$ sudo grub-install --boot-directory=/mnt/boot /dev/sda
/usr/sbin/grub-setup: warn: Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.
tbuntu@tbuntu:~$ grub-install -v
grub-install (GRUB) 1.99~rc1-13ubuntu3
---------------------------------------------------
grub is destroyed,
and it will go to grub rescue again
==============================

How ever I used exact same command form live usb hardisk,
It produce the same error

tbuntu@tbuntu:~$ sudo mount /dev/sda4 /mnt
[sudo] password for tbuntu:
tbuntu@tbuntu:~$ sudo grub-install --boot-directory=/mnt/boot /dev/sda
/usr/sbin/grub-setup: warn: Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.
------------------------------------

But this time successful grub re installation.

It may be because I am in the root partition and I should not
mount /dev/sda4 or sda11 when using as root in that partition.

How ever grub should not produce the same reports for the unsuccessful installation.

Thank you.

But for Windows XP dual boot.

After successful fresh install of Ubuntu 11.04,
Grub has no problems to start my windows XP.

Only the main Grub Menu was destroyed.

Thank you

with the same hard disk,
I use usb stick Ubuntu 11.04, install boot-repair,
use boot-repair to reinstall grub2 to the sda.

with sellection to default boot in to Lubuntu 11.10

Successful re-installation and no error message.

Successful boot into Lubuntu 11.10 on sda11.

It is confirm after 6 times of grub-install with the error "Sector 50 is already in use by FlexNet",

1. you can successfully reinstall grub only if you use - usb live cd ubuntu or other installed ubuntu partitions.
with exact the same messages as No.2
------------------------
/usr/sbin/grub-setup: warn: Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.
====================

2. grub-install will destroy "grub boot menu" and you will get "grub command prompt",
 if you use the command grub-install in the following ways

you boot into "current partition", in my case sda4 contain ubuntu 11.04
(I also test with the same result in Lubuntu 11.10)

sudo mount /dev/sda(current partition) /mnt
in my case sudo mount /dev/sda4 /mnt

sudo grub-install --boot-directory=/mnt/boot /dev/sda
in my case I want to reinstall grub to sda4 so when the computer boot up it will auto boot into sda4,
which is the ubuntu partition I am using to execute the grub-install command.

It will produce this message.
---------------------------------
/usr/sbin/grub-setup: warn: Sector 50 is already in use by FlexNet; avoiding it. This software may cause boot or other problems in future. Please ask its authors not to store data in the boot track.
Installation finished. No error reported.
====================
you will get grub>
and not your grub menu any longer.

I don't know "if I should post this as new bug or not".
========================================

But certainly it is not correct when grub report "No error reported".
And you will get a "destroyed grub menu" after reboot.

note== the reasons
 I start to do grub-install from Lubuntu 11.10 to force grub to start Lubuntu 11.10 as default os.
because the "startup-manager" fail to change grub menu as it did for previous version.
which I don't know why, bugs or just incompatible grub version in sda.
But the above "bugs" I have tested and found to be on both Ubuntu 10.04 and Lubuntu 11.10

I don't know if we do not have "Sector 50 is already in use by FlexNet...."
can we do grub-install from the current partition of root (and boot) to force it's partition to be default boot partition.

I don't dare to do "dd" command to try to delete that "Sector 50" so I get stuck here for the time being.
if anyone knows or have different result please comment.

Please remove all my above posts from this thread, if you belive it is out of context.

since it has nothing to do to "grub-install trashes the ntfs partition"

and nothing to do with the ".....Sector 50 is already in use by FlexNet......."
as I thought at the beginning.

Because I found in later experiments that

"grub-install trashed grub-menu if run on current partition to make current partition to be a default boot up OS"

So I have decided to start report this bug at
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/898655

I would like to confirm that my above experiences with grub-install and boot-repair on
the troublesome ".....Sector 50 is already in use by FlexNet......."
has actually fixed the bugs,
and no problems with any ntfs partions at all.