Published: Thu 30 April 2015
By lxgr
In misc .
tags: linux software
For an upcoming project, I will have to use Ubuntu 14.04, and since I didn't want to downgrade my main Ubuntu install on my laptop, I decided to install the second version on a spare partition of my primary harddisk. Sounds easy, right? That's what I thought too, and I was very wrong.
I expected to be able to simply point the Ubuntu installer to the spare partition and wait for the automatic setup to complete, like I used to when I was using plain old BIOS and MBRs to boot my system. My current laptop, however, supports something called UEFI and Secure Boot, and since the "secure" part piqued my interest, I had decided to give it a try. I have been using this setup for dual-booting Ubuntu and Windows 8 for more than a year without any problems now.
After watching the installer copy all the necessary files of Ubuntu 14.04 to the disk and installing the bootloader, I booted into the new system, and everything worked as expected so far – I saw the new, old 14.04 desktop and was even able to open my main, encrypted 15.04 partition using Nautilus. But when I wanted to boot back into my main system, I realized that something had gone wrong: My 15.04 install didn't show up anywhere in the boot process. Not in the EFI boot menu of my laptop I can access by pressing F12 during the boot process (I used that to boot the Windows, because somehow the secure boot stuff interfered with grub's chain loading), and also not in grub itself – it looked as if the installation had never existed in the first place (except for the fact that I was able to see all of its file in 14.04, of course).
After the initial rush of panic had subsided, I was able to restore the system to its original state by the "usual" process of mounting and chrooting my previous system and executing grub-install. This worked, but now I was not able to boot into the new system anymore, besides executing all those steps again from within the old system. I realized I had to dig into the details of the Linux boot process on EFI (and secure boot) if I wanted to accomplish the triple-boot setup I had in mind. If you are unfamiliar with UEFI, I would recommend you to read an introduction before reading the following paragraphs.
The main problem seemed to be that the grub-install utility of both Ubuntu systems (which is also executed from the installer) wrote to the same location – I just didn't know what that location was. I started to dig into the details of the EFI boot process – from previous triple-boot experiments on a Mac I vaguely remembered something about a so-called "EFI system partition" where the initial boot loaders of all installed operating systems are stored, and also that there were some parts of the UEFI configuration that were stored in a non-volatile memory on the mainboard. This is very different from the "legacy" boot process, where the trusty BIOS simply loads a chunk of code stored in the MBR of some disk (which can be configured in the BIOS setup, but usually defaults to a sequence of CD-ROM/USB/primary HDD) and executes it. That process is both simple (it doesn't depend on any stable configuration storage within the PC) and robust (in my experience, it was possible to migrate to a new machine simply by swapping the hard drive!), but is also showing its age – things like a dual-boot setup of uncooperating operating systems quickly become a mess, as everybody who has ever installed Windows after Linux on the same machine probably knows. Additionally, there is no mechanism that allows checking the integrity of the system before it is booted, enabling malware that hooks itself into the boot process and is virtually undetectable by any software or operating system mechanism.
UEFI solves the problem of multiple operating systems by specifying the "EFI system partition", which is basically just a plain old FAT partition with a special partition flag and a standardized folder structure where every operating system on the disk or even machine (more on that later) can store its initial bootloader as an executable file. This is a much more robust and future-proof way of storing the first-stage bootloader than the very limited MBR scheme that basically only allows a single primary bootloader which has to locate and execute all secondary boot loaders of all operating systems on the drive. However, it is unfortunately not enough (at least on my computer!) to just dump a bootloader in the correct location (which would have been a really nice EFI feature!) – the corresponding operating system also has to tell the EFI about the new bootloader (both the EFI disk's UUID it resides on and the path on that disk), which then stores that information in its non-volatile configuration memory (a.k.a. NVRAM).
To sum the EFI boot process up, you need a folder on the EFI system partition containing your operating system's boot loader as an EFI binary (which in turn might be the first stage of a multi-stage boot loader that simply locates and executes its remaining parts) and a pointer (i.e. disk and path) to that file in the NVRAM. In the case of secure boot, the EFI binary will also be signed by some "trusted" entity, which could be your operating system vendor or amusingly Microsoft (even though you aren't actually using their operating system – this is because many hardware vendors opted to include their keys in their firmware, which was cause for much political discussion when secure boot was initially introduced).
Fortunately, grub-install takes care about all of that automatically as long as all the correct flags are supplied to it – but unfortunately, in the case of Ubuntu and secure boot, this only works for a single installation per machine (i.e. not per disk, which kept me puzzling for hours!). I'm no expert on secure boot, but I think that this might not be easily fixable by Ubuntu, depending on how exactly the signature mechanism is implemented.
When invoked with no parameters, Ubuntu's grub-install, on an EFI system, installs its signed bootloader to the EFI system partition that is mounted in /boot/efi (which might or might not be the one you want to use on a multi-disk setup!), in the folder /EFI/ubuntu/. The signed loader consists of a shim signed by Microsoft that subsequently executes the actual EFI loader called grubx64.efi in the same directory. Finally, there is a grub.cfg configuration file which contains a pointer (both as a disk UUID and as a GPT number) to the "regular" grub boot disk, which is usually your Linux root partition (or a separate boot partition if you are using an encrypted root device).
Initially, the problem only seemed to be that both Ubuntu installs were trying to install their bootloader and configuration to the same EFI subdirectory – I thought that if I were to somehow convince grub-install to install the EFI loader to some other subdirectory of /EFI, I would be able to select the Ubuntu I wanted from the EFI boot screen. grub-install conveniently has an option for exactly that; you can either change the value of the GRUB_DISTRIBUTOR variable in /etc/default/grub, or directly supply it using the --bootloader-id parameter. When invoking grub-install this way, you can see that a new folder in /EFI will be created using the supplied name (and registered with the EFI NVRAM). Unfortunately, while I was able to boot from the newly created boot entry, I didn't seem to be able to change the disk that grub was booting from in any way. It took me hours to find out why.
Remember that grub uses the value stored in /EFI/<loadername>/grub.cfg to determine the disk where it will continue loading. With a lot of trial-and-error experimentation, I was finally able to determine that regardless of which boot entry I was using in the EFI boot manager, grub would always read the same grub.cfg from /EFI/ubuntu, regardless of the actual bootloader location (i.e. subfolder of /EFI)! This location is actually hardcoded in the grubx64.efi binary, which can be verified by using strings or simply opening it with a hex editor. This means that regardless of the Ubuntu install from which grub-install was executed, only the system that most recently installed the loader in the default location /EFI/ubuntu was actually able to change the partition that grub would boot from. (I think I found out about that hard-coded string from some bug report, which I will try to find and reference here.)
If the hard-coded string is modified to reflect the actual location of the boot entry in the /EFI directory everything works as expected (with secure boot enforcing disabled)! Now why would the Ubuntu team be so stupid to hard-code a string that obviously would better be supplied by a parameter? The answer is secure boot: If you enable signature enforcing in the EFI configuration, the modified bootloader stops working. It seems that the string within the binary is covered in the asymmetric signature Canonical uses to certify their bootloader; they could either modify it (and break all systems where secure boot is enforced) or leave it as it is (and break multi-booting). They seem to have decided on the latter. (Maybe there is also a third way, where the configuration file location could be encoded relative to the binary location, i.e. ./grub.cfg, but I don't know enough about EFI to say whether such a thing is possible.)
As I later realized, there is an easier way than modifying the signed grub binary. Since secure boot doesn't work with the modified loader anyway, I tried to invoke grub-install with the --no-uefi-secure-boot parameter and examined the resulting bootloader: Without secure boot, there is only a single EFI executable that is also called grubx64.efi (which confused me to no end, since the other files are not cleaned up by grub-install, and I assumed that the configuration file was still working), but that has a much simpler internal structure and importantly has the boot disk location hardcoded. This wasn't as easy to find as the suspicious /EFI/ubuntu string – it is only some kind of relative disk ID like (,gpt2), if your boot partition is the second partition of the volume on which the EFI partition resides, but a complete disk UUID if the boot partition is located on a different disk.
Finally, here is the complete guide on how to install two Ubuntu versions on a single disk:
Disable secure boot in your EFI settings.
Install the first Ubuntu system on the disk. (If it already exists and you have spare space on your disk, you can obviously skip this.)
Backup the boot entry of the first disk by reinstalling it from within the system using a different name, without secure boot:
grub-install --bootloader-id=myfirstubuntu --no-uefi-secure-boot
Install the second system using the regular Ubuntu installer where you want it. This will break the boot entry of the first system called ubuntu, but not the backup you just created.
Boot the second system and create a backup of the bootloader, e.g. grub-install --bootloader-id=mysecondubuntu --no-uefi-secure-boot
(only if you want to primarily boot the first system) Boot the first system using your computer's EFI boot menu and execute grub-install without any parameters.
Congratulations, you now have two Ubuntus running on a single machine!
If you want to use a similar setup, but using more than one disk, you can basically use the same steps if you don't mind that the same EFI partition of the first disk will be used for both systems, which means that you can never format or remove that disk without also disrupting the system on the other disk. grub will just put a pointer to the second disk in its binary that is executed from the EFI partition on the first disk, which should theoretically even survive partition and disk renumbering (but don't count on it!).
If that is a problem for you, there is also the possibility of using a second EFI partition on the second disk, but the Ubuntu installer will make your life even harder by stubbornly insisting to use the EFI partition on the first disk; I was able to solve this only by creating a backup of the first system's bootloader as described above, installing the second system, mounting the second EFI partition in /boot/efi instead of the first one and rerunning grub-install --bootloader-id=....
You can verify if everything has been setup as you want it by examining the EFI directory on the EFI system partition(s) on your disk(s) as well as the output of efibootmgr -v, which lists the content of the boot list in the EFI NVRAM.
There is also an option --removable which supposedly sets up the EFI directory on a removable device, which looks a bit different than for internal devices and importantly doesn't create an NVRAM entry (which wouldn't be available on different machines anyway). You might be able to use that to boot from an internal disk too, but I have not tried that approach, however.
Of course, if that sounds like a lot of headache and your computer still supports the legacy BIOS boot process (a.k.a CSM in EFI parlance), you can just install the second system on a different disk with an MBR bootloader and configure your EFI for both CSM and EFI booting if it supports that; then you can just select the EFI entry of the first system or the second disk (which will start the second system's boot loader) in the EFI boot menu.
Let me know if you actually made it through that big wall of text and were able to solve your EFI boot problems in the comments!
Update (2015-12-11):
There is an interesting discussion about the whole topic in the Kubuntu forums. Apparently, it should also be possible to use multiple EFI partitions to get multiple instances of Ubuntu working with secure boot. Thanks for that idea, and sorry for being unreachable. I will probably have to add my mail address here sometime. In the meantime, you can try "me" at lxgr dot net.
There are comments .