towboot part 3

So, how do you actually use this thing?

I’m writing a bootloader (see the first post for, why) and now you’ll learn how it actually works.

how (implementation and usage)

Boot process

The complete process from entering the bootloader’s efi_main function to jumping into the loaded kernel can be split up into the following steps:

  1. initialize the bootloader

  2. determine what to boot

    1. parse the command line parameters

    2. parse the configuration file

    3. display a menu

  3. boot the operating system

    1. load the kernel

    2. load the modules

    3. configure the graphics output

    4. prepare the information to pass to the kernel

    5. deinitialize most parts of the bootloader

    6. jump to the kernel’s entry point

1. Initialize the bootloader

The firmware calls the bootloader’s efi_main function with two parameters: a handle that represents the executable that the application was loaded from and a reference to the System Table.

Together, we pass them to uefi-rs (which will initialize the memory allocator and the logging framework) and (by using UEFI’s Loaded Image Protocol) use them to acquire more information about the current state of the system: the options passed on the command line1 and the device that the application was loaded from (which is most likely going to be the ESP2).

2. Determine what to boot

The configuration that contains what kernel and modules to load and what parameters to pass to them can come from the command line or from a file (which itself may be specified on the command line).3

So, we first use miniarg (see the miniarg section down below for details) to parse the command line parameters to determine whether to load a configuration file (and which one). We then load the configuration file using toml-rs and display a simple menu. If there is only one entry, it is booted directly.

3. Boot the operating system

3.1 Load the kernel

The kernel image can be a flat binary (AOut) or an ELF file. In both cases, we load the file completely into memory. The Multiboot header then specifies the type and contains more information (see the previous post for details).

Then, we copy parts of the file to memory (which ones to load is specified either by the Multiboot header or by the ELF Program Header). In case of an ELF file, we also copy the Section Header and symbols.

3.2 Load the modules

We load the specified module images to newly allocated memory pages.

3.3 Configure the graphics output

The kernel’s Multiboot header may specify a preferred text or graphics mode but UEFI’s Graphics Output Protocol only allows a graphical mode to be used by writing to a framebuffer in memory (and this is not even guaranteed to work on all setups) — text mode, resolution change and more graphics features are only available via the protocol’s methods which the kernel may not have support for.

So, we list the possible modes of the first GPU and compare them to the kernel’s preferred resolution. If there is a match we use it, else we keep using the mode the system is already in (in many cases, this will be the system’s native resolution).

3.4 Prepare the information to pass to the kernel

We allocate a MultibootInfo struct and fill it with information about the kernel command line, the loaded modules (and their command line), the ELF Section Header and symbols (if possible) and the framebuffer.

Information about the memory configuration is not passed here because it could change with further allocations or deallocations.

3.5 Deinitialize most parts of the bootloader

Exiting UEFI’s Boot Services frees some parts of the memory and produces a description of the memory map of the whole system, but also causes us to lose access to the file system, console and memory allocator.

So this is the point of no return: There is no possibility to go back to a menu or exit with an error code. The only error handling still possible is panicking: printing a message to the console (which may itself fail at this point), waiting and resetting the machine.

We then convert the memory map returned by the firmware into the respective Multiboot structs: everything except broken memory, the UEFI Runtime Services’ memory, memory-mapped IO or memory containing ACPI tables is marked as available, then adjacent entries are joined.

We also compute the two fields describing the legacy “lower” and “upper” memory from the generated memory map (see the previous post for more details.)

Finally, we move the kernel to the correct position in memory if this failed initially when loading the kernel.

3.6 Jump to the kernel’s entry point

With the help of inline assembly, we set up the machine state the Multiboot standard requires (see the previous post). Afterwards, we jump to the kernel’s entry point while passing the Multiboot signature and a pointer to the MultibootInfo struct via the EAX and EBX registers.

Configuration

The bootloader needs to know which kernel and which modules to load and which command line to pass to them. This information could be entered by the user at runtime (and it would be useful for debugging), but the default values should come from some sort of static configuration source, so that the boot can be performed automatically without any user input.

There seem to be three possibilities here:

NVRAM

The configuration could be serialized to binary with an appropriate encoding (for example CBOR or JSON) and stored inside an EFI variable (see section 8.2 of the UEFI specification) that is read at boot. This would be the cleanest solution but sadly, it comes with a few drawbacks:

  • Binary data inside EFI variables cannot be easily edited by hand.

    There would need to be a program to edit this configuration (for example, some sort of command line tool). Also this program would need to be able to run under different operating systems which would add complexity.

  • Configuration is bound to a system.

    This would make live booting impossible (or difficult, at least).

  • Bad firmware implementations might “forget” stored configuration.4

Command line parameters

The information about the kernel and the modules can come from the command line arguments of the bootloader itself. One could invoke the bootloader with (for example) towboot.efi -kernel "kernel.elf quiet" -module "module1.bin –verbose" -module "module2.bin –quiet"5.

Another possibility would be to pass the path to a configuration file (see the next section): towboot.efi -config config.toml

Parsing these command line arguments can be difficult when filenames or arguments contain spaces or quotes.

This would be useful for either manually booting from the EFI shell or in combination with another bootloader that can by itself read its configuration and display a menu but has no support for Multiboot such as rEFInd or systemd-boot.

It could also be used to configure one (or multiple) Boot#### (e.g. Boot0001) entries in the firmware and use the firmware’s boot manager (see section 3.1 of the UEFI spec). But this approach has two downsides:

  • In most cases the firmware’s bootloader implementations are not very user-friendly.

    They mostly display less details and lack the possibility to edit options.

  • The firmware can “forget” configured entries (see above).

Configuration file

This is the most simple and most straight-forward approach: most bootloaders read their configuration from a file. Using a text file has the advantage of it being easy to read and modify by hand. See the previous post for an explanation of why TOML was chosen as the configuration file format.

A sample configuration file looks like this:6

default = "entry1"
timeout = 10
# log_level defaults to 'info'

[entries]

  [entries.entry1]
    name = "Operating System"
    image = "kernel.elf"
    argv = "quiet"

    [ [entries.entry1.modules] ]
      image = "module1.bin"
      argv = "--verbose"

    [ [entries.entry1.modules] ]
      image = "module2.bin"
      argv = "--quiet"

Towboot is going to use a configuration file as described in the previous section. The other two configuration sources could be added later, but that would need some amount of work for no real benefit.

Memory Management

There are multiple ways to allocate memory and they are all used in various situations.

Stack

Placing variables on the Rust stack by using local variables does always work, but it is limited to structures that have their size known at compile time. We use this for most runtime data, for example the MultibootInfo struct that is passed to the kernel. This memory is tracked by rustc at compile time.

Heap

uefi-rs binds Rust’s global memory allocator to the UEFI Boot Services’ allocate_pool, so this memory is tracked both by rustc at compile time (to some extend) and by the firmware at runtime.

We used this for everything with a dynamic size and no further special requirements, for instance the configuration file or strings passed to the kernel.

Whole pages

There are allocations with such special requirements, however: The kernel code has to be placed at the exact same spot which it was built to be placed at.7 Modules may need to be loaded page-aligned, so that the kernel can simply map them into its paging.8

In these cases, the UEFI Boot Services’ allocate_pages function is wrapped by the mem::Allocation struct. It contains a custom core::ops::Drop implementation which calls the Boot Services’ free_pages function to propagate freed memory back from Rust to UEFI.

There is one caveat: The address at which the kernel has to be loaded may not be available when loading the kernel. For example, the address the GNU Mach kernel has to be loaded to is in most cases already in use by UEFI’s Boot Services. We solve this by loading the kernel to a different address first and then copying it to the correct address after exiting Boot Services, if the destination address is not marked as reserved in the memory map passed to the kernel. (If we could not immediately load it to the correct address, that is.)

This may have severe side effects up to overwriting the bootloader’s code.

References passed to the kernel

Information is passed to the kernel by writing the address of a MultibootInfo struct in a register before jumping to the kernel’s entry point.9 Some parts of this information are scalar values contained in the passed struct but other parts are pointers to additional structs. rustc is not able to determine that these structs are being used by the kernel, so we intentionally leak them by calling core::mem::forget to make sure that they are not preemptively freed.

Workarounds

With the UEFI targets being Tier 2 (see the previous post for more details), some breakage was to be expected.

With Rust 1.49.0-nightly, stack probes were broken on i686-unknown-uefi. This has been fixed in compiler-builtins 0.1.36.

There is also currently no support for floating point operations on these targets. Towboot itself does not need them, but some dependencies (the TOML library, for instance) contain code with floating point operations, but they are never executed.

So, in order to successfully compile the code, there are stubs like this one:

#[no_mangle]
pub extern "C" fn fmod(_x: f64, _y: f64) -> f64 {
    unimplemented!();
}

These workarounds are placed inside the hacks module. They can hopefully be removed when the underlying support in Rust improves.

Changes made to other software

Where it was possible, I tried to either upstream changes to other software or release parts as individual crates.

rust-multiboot

The multiboot crate provides safe wrappers for the Multiboot data structures, but it was designed to be used by kernels, so it only had support for parsing the Multiboot Information struct and for retrieving data out of it.

I added code for parsing the Multiboot header, in addition to support for setting values. Integrating these setters with Rust’s (and UEFI’s) memory management was the most difficult part as the crate was designed to differentiate between physical addresses (referenced in the Multiboot data structures) and virtual addresses (the pointers used to actually access the data).

Also, I added tests containing Multiboot data structures to parse and generate.

hhuOS

hhuOS is Multiboot-compatible, but it made several expectations on the memory positions of the Multiboot structs that are correct when being loaded by GRUB, but not when being loaded by towboot:

It assumed the Multiboot structs to reside between the beginning of the (physical) memory and the beginning of the kernel in memory. If the structs have been placed anywhere else by the bootloader, the kernel has not been able to access them after enabling paging. I changed the code to reserve a fixed amount of memory in the BSS for this data and to copy it there as one of the very first actions performed by the kernel.

The kernel also assumes the free memory available for the kernel to use to be between the end of the data passed by the bootloader and the end of the physical memory of the system. So, when a module is placed just below the end of the physical memory, the kernel is barely able to allocate any memory.

miniarg

Parsing a string such as program -foo "bar baz" to command line parameters is not specific to UEFI applications, but there do not seem to be any existing no_std-compatible crates for doing this, so I created miniarg.

It can process relatively simple command lines without any heap allocations and also supports configuration by implementing a custom trait (it can generate a usage text automatically in that case).


  1. UEFI calls these “load options”. 

  2. EFI System Partition, a FAT partition where installed UEFI applications reside 

  3. See the config section for more details. 

  4. RedHat’s shim bootloader has a separate fallback executable for this case. 

  5. This would cause towboot to load the kernel kernel.elf with the command line quiet and the modules module1.bin with the command line –verbose and module2.bin with the command line –quiet

  6. It has the same effect as the first command line in the last section. It should be named towboot.toml and placed in the top-level directory of the ESP

  7. ELF files could contain relocations or be position independent, but neither towboot nor GRUB support this, so there should not be many position independent Multiboot-compatible kernels or ones with relocations out there. 

  8. We do this at all times with no regards whether the kernel actually requires this because it simplifies the control flow. 

  9. See page the previous post for more information about the passed information. 


Kommentare

Die eingegebenen Daten und der Anfang der IP-Adresse werden gespeichert. Die E-Mail-Adresse wird für Gravatar und Benachrichtungen genutzt, Letzteres nur falls gewünscht. - Fragen oder Bitte um Löschung? E-Mail an (mein Vorname)@ytvwld.de.