Non trivial grub2 installs no longer fit in small embed areas

Looking at current duplicates show only 3 cases :

1) Bug #491740 : disk with abnormally small (<32kb) embed area. GRUB Legacy OK , GRUB2 KO --> regression

2) Bug #1004376 : RAID+LVM : 10.04 ok , 12.04 KO --> regression

3) BTRFS:
KO on 11.04 (Bugs #782543 , #781010 , #774217)
KO on 11.10 (Bug #885264 , #881342 , #879109 , #873392 , #842247 , #813259 , #800840)
KO on 12.04 (Bugs #990558 , #988848 , #953559 , #945449 , #928128 , #917212 ),
KO on 12.10 (Bug #1045178)
---> not a regression, unless we show it worked with a previous version of GRUB.

4) LVM ok on 10.04, KO on 12.04 --> regression ( http://askubuntu.com/questions/196246 )

When you say partitioning disk with modern tools... ok, let's said we have modern tools on 12.04.1 boot CD (that is not really the case since LVM and RAID are only on alternate CD... why ? is libre-office more relevant than a complete installing system ?)

When a disk is double partitionned, for example LVM over RAID, did it need to partition from the basement, that meens RAID level, or only the top final partitions, LVM level ?

If it needs to partition from the low level, and if it is not going to be fixed, I think it will need a tool, a script or whatever to get around this bug.

I tried installing both 12.04 and 12.10 using BTRFS for both / and /home and Grub failed to install with each of them. Tried on both my Netbook as well as my test rig(P4). When I dig a little deeper, it seems Grub complains about core.img being "unusually large".....

An interesting observation I made today though.
On my test rig I have a number of installations of numerous distros. Havent fired this up in a couple of years mind you so I still had the Lucid development install as the latest on here....lol.
On the box I had a install of Fedora 13(which was the latest stable at the time IIRC) on BTRFS. Installed as both / and /home. Fedora 13 still had legacy Grub and I remember Lucid with Grub2 not being able to boot Fedora 13.
Well last night I replaced the Lucid install with 12.04 on Ext4(since BTRFS wounldn't work) and then today decided to check if Fedora would boot. Sure enough it boots! So I KNOW they can play nice.

So not sure what has changed in Grub2 between Lucid and Quantal but its strange that it will boot F13 just fine but not a current Ubuntu version on BTRFS.

BTRFS support takes a larger disk space, so you either must leave a couple (say 4) MB of free space between the start of he disk and first partition, or create a small "BIOS GRUB" partition there (that you don't mount or otherwise use, grub will take care of it).

Possible duplicate: Bug #1066324

That is incorrect Swami. The bios_grub partition is required when using GPT, and 1MB is quite sufficient either for it, or for the embed area before the first partition on an MBR partitioned disk.

I have a msdos partition table though. Not GPT.

@ YannUbuntu

Bug #1066324 is a duplicate of this one. At the bottom of that bug it says to comment under this one. I have that same issue but also using 12.04.

I have this issue also, trying to upgrade Xubuntu 12.04 to 12.10 on a machine that uses LVM but not RAID.

This bug is especially bad for people installing ubuntu as a secondary os on a windows system. Some factory windows installs do not leave space at the beginning of the hdd beyond the 62 sectors.
I hope this observation might get this bug escalated.

Such a setup wouldn't be hit by this problem; it is only exotic setups that require things like raid or lvm that won't fit in 62 sectors.

Stating that either LVM or RAID are "exotic" is simply bullshit.

And BTW it's not only LVM or RAID that suffer from this issue, but also BTRFS installs, ZFS installs... Everything that is a bit more complex than ext3/4 on plain good ol'e DOS disk partitions...

Escalation won't particularly help as this is just plain difficult to fix: there's no single cause for the code growing beyond the relevant size, but rather gradual accretion of bug-fixes has meant the code is now just slightly larger than will fit. I don't regard LVM or RAID as exotic, and in that case this is a regression; they're close enough to the size limit that we may be able to squeeze things back down again in a future upstream version, but it will take some slow and steady work.

btrfs and ZFS are over the size limit by a very considerable margin, and it is unlikely that we will ever be able to squeeze support for those into 62 sectors. They were never there in the first place, so in that case this isn't a regression.

I managed to get this problem by creating a little partition - sda2 in the list below - trying to make use of that 'unused' space. This worked well enough when grub was on a regular partition, but it does not work when setting up /boot and grub on a LVM. The fix was to delete the partition with fdisk and run partprobe.

   Device Boot Start End Blocks Id System
/dev/sda1 * 2048 15624191 7811072 83 Linux
/dev/sda2 3 1727 862+ 83 Linux
/dev/sda3 15624192 625142447 304759128 5 Extended
/dev/sda5 15624195 625142447 304759126+ 8e Linux LVM

It would be good for grub to display "there are 3 sectors before the first partition available for core.img, and we need 65" (or whatever the case is) rather than to only say that the core image is large - it's not particularly large.

That's the problem; the partition starts at sector 63 so that doesn't leave enough room for grub and the btrfs module.

Ok I reinstall, format the root / partition with the ext4 file system. If you still have not found the solution, the installer should warn about.this. https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1270729

There is no need to reformat with Ext4. If you wish to use BtrFS for the Boot partition, it's recommended. Just tell the partitioner to start at the 1MB boundary instead of the first sector. To reiterate, you need an empty 1MB at the beginning of your drive to write to. I would call this an error with the partitioner/Ubuntu Installer instead of Grub2.
To do this properly, start GParted and format it using this method.

It's not an error with the partitioner or installer since it does start the partition at 1 MiB. The problem is when you have an old disk that you partitioned with say, Windows XP, which started the partition at sector 63.

The installer should detect that the partition starts at sector 63 and display a warning.

My netbook comes with Windows 7 preinstalled.

Actually, Phillip, BtrFS requires 2MB for a Grub2 installation. So, while you are correct that it does start at 1MB, the partitioner does have an error in not including that the rules for BtrFS are different. The normal 1MB is a part of the normal MBR rules. Since Grub2 requires some extra installation space for the BtrFS partition information, GParted should warn when BtrFS is used and a partition is started less than 2MB from the beginning of the drive. For everything else, 1MB works fine.
So, this would be a problem with the partitioner not realizing the rules are different and the installer not seeing that the partitioner made a mistake and that Grub2 can not be properly installed inside of the 1MB space for a BtrFS partition.

Ext4 and other partitions that are not BtrFS have no problem with the 1MB start point.

No, it doesn't need anywhere near 1, let alone 2 MB. It is something like 45k.

Sorry, I was talking about an MBR system, not a UEFI. Core.img is too large to fit into the first 64 sectors (starting from 0). And, so, the next best place to start would be the 127 - 128 boundary. Also known as the 2MB boundary.

You're not making sense. 2 MB is 4096 sectors. The default start location for the last several years has been sector 2048, or 1 MB, which leaves plenty of room for the core to fit.

I think the installer should be added to this bug. What is the name of the package that will install ubuntu? Or partition tool?

I do not know well the technical details of the problem but I agree with "cryo-rebirth" .. the installer or partitioner should warn when a partition BtrFS is used and is started partition at sector 63.

Sorry for my bad english...

Status changed to 'Confirmed' because the bug affects multiple users.

Sorry, I did get a little confused over the sectors thing. The 63rd LBA sector is the starting point for the 0MB boundary of partitions usually. I wasn't thinking straight. I'm used to using the MB settings for partitioning instead. Anyhow, the normal Grub2 install for any partition that's not BtrFS can fit into the MBR area with no problem normally. However, a larger area is required if you're going to use a BtrFS partition since the data on how to read such a partition in Grub2 makes the boot program a bit larger than the MBR size. Recall that I said Core.img is particularly large. I'll have to check on my installation to see where I had to start it at later. But it is a problem that should be addressed or at least warned about in the installation.
As for the installation report, it should mention that Grub2 failed to install properly and should return to the partitioning step to address the issue. Highlighting the issue may also be beneficial to users. However, with newer hard drives, they utilize 4K sector sizes to increase capacity. This should be taken into account in case the 512b emulation ceases. It's known as Advanced Format and can be a pain in the rear if you're not already familiar with it. This will make 256 sectors equal to 1 MB instead.
Anyhow, I'll be sure to check on how I've managed to install BtrFS and where my partitions start and end so that I can toss together a sort of guideline for failsafes.

I hit this bug in a different way with 14.04:

Use fdisk to create a new mbr and single partition on a USB-connected drive
mkfs.btrfs the partition
Run installer and select the partition as root, and drive device as grub target

After grub fails the installer exits. fdisk reports the partition starts at 2048. gdisk reports that the drive has a valid MBR and valid GPT main header but corrupt GPT backup. At no point did I format the drive as GPT - I wonder if ubiquity automatically adds a GPT header causing the subsequent grub install with btrfs to fail?

Your issue is unrelated to this bug Chris. You must have used GPT on the disk in the past, and used fdisk to write a new MBR, leaving parts of the GPT behind.

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1059827

Please don't mark bugs as fixed unless you actually fixed it.

I finally upgraded my 12.04LTS server to 14.04LTS last week, and I really regret having done this. I am suprised this bug is considered as unimportant, when it completelly spoils the booting system. Nothing seems to work.

@Javier: Take a bigger drive, format it with first partition starting at sector 2048 and transfer partitions content. Then you will have enough space to fit GRUB bootloader into.

@Arie: thank you, but I hoped that a software issue could be resolved without the need of
additional hardware (I don't have a bigger drive right now). But I can survive: I can boot
from a live USB with Super Grub2. Anyway, I think that such gross errors should not
take place in an upgrade. This makes me wonder whether Ubuntu is a good choice, particularly
when this type of bugs are not considered as important.

@Javier: I don't think it is a bug per se. And definitely not ubuntu related. GRUB2 is complex piece of sotfware that keeps developing. I don't quite understand how they managed to put ext2 mounting and kernel+initrd loading into mere 31744 bytes back then. You aren't complaining you can't install today's ubuntu onto i386 processor, aren't you?

No, I am complaining about the fact that I had a perfectly working ubuntu server which stopped booting after upgrading. That is a bad thing for an operating system. This should not happen.

This still seems to exit. I am trying to install Ubuntu Server 18.10 on a Intel C202 based server, and although the installer sees and activates the MDADM it finds, installing the boot loader simply fails.

Regression after upgrade disco -> eoan, booted from btrfs disks. Worked normally before upgrade, cannot install grub after upgrade. Ended up with non-bootable system, had to plug an old disk exclusively to put /boot there.

Had to reinstall Kubuntu 19.10. Used BTRFS for root on a 1TB SSD. Grub-install fails. I think this bug is regressed.