I regularly have the need to try things out on Linux. Sometimes a virtual machine won't cut it for me typically due to memory, disk and performance limitations. Moreover, a decent, up-to-date, bootable Linux environment is a great backup in case all of my other computers are broken, infected or stolen. That entails having the Linux installation on an external, USB-attached hard disk drive which can boot with relative ease on any UEFI-enabled PC (driver compatibility notwithstanding). Moreover, all the preparatory work has to be performed using a single-boot Windows computer without ending up having a dual boot system. It sounds tough. It is tough, but I'm writing this from my portable Ubuntu Linux installation running off a USB-attached SSD!

What you need

  • An empty external (USB) hard drive. I used a USB 3 drive enclosure with a cheap 256Gb SSD. For those of you worrying about performance, the USB 3.0 port is faster than the maximum transfer rate of any SSD I've seen to this date.
  • Ubuntu Linux bootable USB drive. Very easy to create using Rufus on Windows. I used a cheap, promotional flash drive. Reduce, reuse, recycle FTW.
  • Windows System Repair Disc (a bootable USB drive with Windows recovery tools which you can make yourself) or a Windows installation or rescue CD-ROM / DVD-ROM. I used another promotional flash drive.

Remember to take backups of all your critical files and folders before continuing. You are going to make changes to your computer. If it breaks don't cry, I warned you. Also test both your bootable media. Twice. You will need them both!

Caveats

Having a dual- or multi-boot system isn't as straightforward as it used to be a decade ago. Nowadays, in the interest of security, performance and backwards compatibility, there are many semi-hidden options and features which can get in your way. Given enough experience and patience you can work around them. Below is a selection of problems that got me moderately stumped along the way.

Secure boot caveat

I have only tested these instructions with Secure Boot turned off. Even though Ubuntu does support Secure Boot (it comes with signed bootloaders) I have no idea if my method uses the signed bootloaders or not. I suggest turning off Secure Boot if possible.

FastBoot caveats

Many boards come with some sort of "fast boot" or "boot optimization" options. For example, my Intel NUC has an option called Fast Boot which won't let me choose an alternative boot device at startup. It also has an option to support Intel Rapid Start Technology which does get in the way of booting to multiple OS. I had to disable both.

It's worth noting that Windows 8 and 10 have an Fast Startup or Fast Boot feature. This feature takes some shortcuts when it comes to booting and also makes the NTFS filesystem remain in a dirty state, making it unwriteable from Linux. It's best to understand what it does and disable it if you plan on writing to your Windows drive from Linux.

USB host controller caveat

Some firmwares will present the USB host controller as UHCI (USB 1.1) at boot time. When Linux probes for an xHCI (USB 3) host controller during the boot process they will respond positively. At this time Linux loads the xHCI driver and the USB host controller resets itself.

However, your root filesystem is inside a device attached to this USB controller. Therefore the controller resetting means that Linux can no longer communicate with the USB-attached hard drive. Therefore the Linux boot will hang forever without any further indication as to what went wrong.

Most affected boards (including my Intel NUC) have an option to enable the xHCI host controller interface by default. Enabling the xHCI option in the BIOS fixes the hanging boot issue. If you are only using modern operating systems with USB 3 support (anything newer than and including Windows 8.1 and Ubuntu Linux 15.04) you can safely enable that option.

Installing Linux

Boot your computer from the Ubuntu Linux bootable USB drive. Remember that you may have to enter your computer's boot manager to do that (on my Intel NUC I have to press F10; on most other BIOS I've seen it's F9; consult your BIOS documentation).

Install Ubuntu Linux regularly. When it prompts you about the disk layout choose Something Else and partition your external HDD the way you want. I chose to create a modestly sized root partition (about 40Gb), a swap partition that's as big as the biggest RAM configuration I am going to be using this installation with plus one Gb (my computers max out at 16Gb so I made a 17Gb swap partition) and the rest of the disk went to a massive /home partition.

Caveat: I chose to use btrfs which – as I learned along the way – makes things a bit more complicated down the line. For your sanity's sake I recommend using ext4. This guide assumes the use of btrfs and will point out the caveats with this approach.

Fix Windows 10 boot

Unfortunately the Ubuntu Installer assumes that you want a dual booting configuration alongside Windows. Therefore it adds itself (actually, the bootloader it uses, GRUB2) to your computer's UEFI configuration. This causes two problems. For starters, the external HDD is not portable as you cannot boot with it on another computer.

Secondly, if you remove this external HDD your Windows won't boot. Bummer. We have to fix that.

  • Shut down your computer.
  • Disconnect the Ubuntu HDD
  • Boot from the Windows system repair disc USB drive (or a Windows installation or rescue CD-ROM / DVD-ROM).
  • Select Repair your computer.
  • Select the operating system and click Next.
  • Choose Command Prompt.
  • In the command prompt run
    diskpart
    sel disk 0
    list vol
  • Verify that the EFI partition is using the FAT32 file system. It will have a volume ID, let's say 99. Now we need to assign a drive letter to it. Back in the command prompt type:
    sel vol 99
    assign letter=z:
    exit
  • Now we need to fix the boot record. Again in the command prompt type:
    z:
    cd EFI/Microsoft/Boot
    bootrec /FixBoot
  • Finally, we need to re-create the BCD store which tells the Microsoft boot loader where to find Windows so it can boot it. From our trusted command prompt:
    ren BCD BCD.old
    bcdboot c:\Windows /l en-us /s z: All
  • If this didn't work try
    ren BCD BCD.old
    bootrec /RebuildBcd

At this point exit the command prompt and shut down your computer.

Create an ESP on the Ubuntu HDD

A hard drive is not bootable with UEFI unless it has an ESP (EFI System Partition). An ESP is simply a FAT32 partition with a special flag that tells the EFI BIOS to look inside it for boot information. We have to create one on your hard drive.

  • Plug in your external HDD and the Ubuntu Linux bootable USB stick.
  • Boot with the Ubuntu Linux bootable USB stick using the option to try Ubuntu before installing.
  • Open a Terminal (CTRL-ALT-T)
  • Run sudo fdisk -l to get a list of partitions.
  • Identify from them the drive that has the Linux partitions, in my case /dev/sdb. I'll call it /dev/sdX from now on.
  • Also identify the partition that contains the root filesystem. I will call it /dev/sdXY from now on.
  • Launch GParted from the Terminal:
    sudo gparted /dev/sdX
    Why not just click on GParted on your desktop? Well, I kept receiving errors about the Ubuntu Linux bootable USB stick because it was already in use. Of course it is, I am using it to run the computer off it, duh!
  • Resize the first partition on disk to have another 200 Mb of free space after it.
  • Create a new partition on the free space, changing the file system to fat32.
  • Apply operations. You need to do that now for the next step to be possible.
  • Right click the new partition.
  • Click on Manage Flags.
  • Set the boot and esp flags. This is what makes the partition "special" to the EFI BIOS.
  • One more thing! Note down the the partition that contains the ESP filesystem. I will call it /dev/sdXZ from now on.

Make sure the Ubuntu installation on the external HDD can see the ESP

The new ESP on the external drive must be visible by the Ubuntu installation in the HDD. Otherwise GRUB2, the Linux bootloader, won't be able to update itself, making your system unbootable after the next kernel update at the latest.

  • Launch GParted from the Terminal, as we saw above:
    sudo gparted /dev/sdX
  • Double click the partition with your Linux root (/) filesystem on the external HDD
  • Note down the UUID, e.g. 01234567-89ab-cdef-0123-4567890abcde
  • Double click the new FAT32 partition and note down the UUID, e.g. 0123-ABCD
  • Close GParted
  • Open a Terminal

The process is different depending on the format of your root partition on the external hard disk.

If you DID NOT use btrfs (e.g. you used ext4)

sudo umount /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde 
sudo mount /dev/sdXY /mnt

If you DID use btrfs

If you DID use btrfs, you made your life complicated. We need to mount the btrfs subvolume containing the root partition instead of the entire partition. Otherwise you'll never be able to install GRUB and you'll probably lose an entire day, like me.

btrfs subvolume list /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde

This will give you a line with a numeric ID. Let's say 123. Note it down.

umount /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde
mount /dev/sdXY -o subvolid=123 /mnt

The rest of the instructions are common, no matter if used btrfs, ext4 or something else

  • sudo nano /mnt/etc/fstab
  • There is a line with /boot/efi already in this file. Comment it by placing a # in front of it.
  • Add the following line:
    UUID=0123-ABCD /boot/efi vfat defaults 0 1

Install GRUB2 on the external drive's EFI System Partition

Right now our external drive has an empty ESP. We need to put a bootloader in it to make it actually, well, bootable.

First caveat: all the instructions you find on-line assume you are using a dual boot system with Windows or macOS. When you have an external drive it is critical that you use the --removable option in the last step. This installs the EFI bootloader under the special "fallback path" EFI\Boot\bootx64.efi in the ESP. Normally this not supposed to be used for permanently installed Operating Systems. It's the mechanism used by EFI BIOS to boot arbitrary external media. Technically, that's exactly what our external hard drive is: arbitrary external media!

Second caveat: installing the bootloader is only possible from inside the Linux installation we want to boot. However, we need the bootloader to boot that installation, leading to a Catch-22 issue. The solution is to run the bootloader installation through a chroot jail. The actual caveat that got me stumped for a day comes from the fact that I am using btrfs (because it's so much better for SSDs!). btrfs has subvolumes. If you mount the entire partition instead of a subvolume the grub-install script can't figure out the mapping between paths and devices, therefore failing to install itself on the ESP, returning the cryptic error

/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).

The error is misleading! /dev is mounted if you follow my instructions below. The actual problem, as I understand it, is that there is a discrepancy between the mounted device and the path to the chroot root. That's why I had you mount only the subvolume containing the root filesystem in the steps above. If you were not paying attention, you are not following the instructions step-by-step, you rebooted before this step or just came here directly looking for a solution to your problem about GRUB not installing look above for instructions on mounting the correct btrfs subvolume.

  • We need to prepare the chroot environment. The ESP must be mounted in the correct place and we have to bind system mount point for some special trees (most notably /dev). Moreover, we will copy the resolv.conf file to let the chroot environment have network access should it need it.
    mount /dev/sdXZ /mnt/boot/efi
    for i in /dev /dev/pts /proc /sys; do sudo mount -B $i /mnt/$i; done
    cp /etc/resolv.conf /mnt/etc/
    modprobe efivars
  • Finally we enter the chroot environment and install Grub in a way suitable for a removable device (see the first caveat above).
    sudo chroot /mnt
    grub-install -d /usr/lib/grub/x86_64-efi --efi-directory=/boot/efi/ --removable /dev/sdX

Now your external HDD is bootable. Reboot your computer, select it from the boot media selection of your UEFI BIOS and you're done!

201 comments

        • When I wrote the article the Ubuntu installer would detect the EFI partition on the main drive and install its EFI stub there.

          Remember that the EFI System Partition (ESP) and the boot volume (what is mounted on /boot) are NOT the same and have a different purpose in life. The device firmware looks in the ESP of each attached hard disk for EFI stubs. These are presented in the device's boot menu. The GRUB EFI stub (installed by Ubuntu) will find your boot partition and load GRUB from it. GRUB will then present its own boot menu. Once you make a selection, GRUB loads the kernel (vmlinuz) and mounts the initial RAM disk which initializes the system and its devices etc.

          While you can create an ESP in the Ubuntu installer it's not necessary that this is the one which will be used, i.e. the one where Ubuntu will install its EFI stub. Even if it does, it installs a regular stub which is NOT meant for removal drives. So you'd still need to follow the advice to enter the chroot jail on the external installation and install GRUB with the --removable option.

          That bit would be very difficult to describe. The only people who'd get it are the ones who don't need to read this article. Therefore I chose the easy method of describing how to create an ESP on the external drive after the fact.

          Predictably, people try to be "smart" and do things differently than I described without understanding or researching the repercussions. Little wonder people end up telling me "it doesn't work for me, oh by the way I did Y instead of X". Yeah, sure, you can create an ESP during installation but if you don't reinstall GRUB with the --removable option it will only ever work on the machine you originally built the installation since you don't have a removable media EFI loader. If you had Secure Boot enabled it will also not work anywhere else because you're locking the boot loader to the specific hardware configuration; if it changes it invalidates the boot loader. If you don't have Windows the instructions won't work because the installer is likely doing something silly, like replacing your main installation's boot loader with its own. If you unplug your main hard disk it might not work because you are neither reinstalling GRUB with the --removable option nor do you account for most machines recording the new ESP in their NVRAM, breaking your main installation. I have run into these problems and that's why I don't do any of that.

          Follow my instructions to the letter and everything works. I have done that three times with different machines and SSDs. I know these instructions work, if you actually follow them.
          • thanks for very detailed and informative reply. As I wrote in my first post, I did exactly as per your instructions and everything worked. The doubt why not create the ESP partition while creating other partitions was only due to the fact that it is easier and faster to create an additional partition when all the other partitions are empty compared to carving out a partition from the root partition which has system already installed on it. I accept your reply and the matter ends here.
  • Great tutorial, I think, but I have a very preliminary issue.  I cannot get Ubuntu installed on the SSD.  I keep getting an offset from the beginning of the drive, which results in an error message "the partition assigned to "/" starts with an offset ...please go back and fix". Gparted can delete the offset, or at least it does not show, but the installer recreates it, result can't install.  Any suggestions. Thanks.
      • Okay, got that problem solved.  I had to use an external partition program to set up the ssd, no problem.  Installed Ubuntu, seemingly no problem.  Worked well in its installed state.  However, when I shut down, there was no impact on the windows bootmanager. Booted into windows with, or without, ssd connected.  Should I go ahead and make the changes in Ubuntu on the ssd, or would they be futile?  Thanks for your help.  I'm an old, old dog trying to learn new tricks -- think Commadore 64.
        • Yes, you need to make the Ubuntu changes to the SSD. This is what makes it bootable on other computers as well.
  • I wondered if all the hassle would be avoided if I disconnected the internal hard drive of an old computer (not the one I plan to use the drive on often) that had windows 10 on it. If the only drive the computer recognizes is the external one, then would GRUB be installed on the drive like normal? That is just using the "normal install" option on the drive. or would i have to create a swap, root-home, and efi partition with grub, etc. And would this approach still work on any computer, fully portable? I would love to here your response, as I don't want to be using the virtual machine for class any longer and am worried about the process you detailed adversely affecting any of my machines
      • well if that doesn’t work apparently, then can i do the tutorial steps on said old windows pc  and still be able to use it everywhere?
        • Considering that this is quite literally what the post describes and that I am actually posting this from the bootable portable installation attached to a different computer than the one I originally used I would say that yes, it will work.
  • Did the whole tutorial from start to finish, everything seemed to work fine, the drive booted from the pc with no problems. however when i try to boot it from the computer i plan on using it on (the other was an old pc i didn’t care to do the process on) i get a multitude of ACPI errors. after these errors the shell reads “you are in emergency mode. after logging in...etc” “press enter for maintenance (or press control-d to continue):”. i would like to know what this means and how i can fix it, as i feel i was so close to getting it to work.
  • Hi Nicholas, I followed all the steps in the post and succesfully installed the bootloader on the HDD, but when I try loading ubuntu from it, an error pops up that say: Linpus lite boot failed.
    I have no idea why is this coming up. Please help.
    • My blog post is written about Ubuntu. I have not used Linpus Lite. The only thing I understand from the message is that you have probably not updated /etc/fstab with the UUIDs of your partitions.
  • It is probably an issue with the BIOS of my system. I used the same HDD on other system and it booted perfectly. No Linpus Lite error or anything. I have updated the BIOS on my system but still can't boot it from there. Any suggestions ?
  • Hello Nicholas!

    Thank you very much for all your work and responses!

    I have just one question:
    Is it possible to circumvent the Windows-fixing-boot issue, if I do the whole procedure on a Linux machine in the first place?
    • Yes, of course. That step is only required when you carry the procedure on a Windows machine. If your machine is Linux you don't need to do that.