towboot part 4

tests, tests, tests

• software
• towboot

I’m writing a bootloader (see the first post for, why) and now we’ll see how we can check that it actually works.

Evaluation

There are multiple ways to verify that a software works the way it should.

Components

Usually, unit tests are used exactly for this. They can be used (to some extend) here as well.

The multiboot crate has unit tests using specially created structs. Integration tests using whole kernel images could probably be added later.

The miniarg crate also has unit tests.

Bootloader

To test the whole software end to end, we have to test the following parts:

1. loading and parsing the configuration

2. loading and parsing the kernel

3. loading the modules

4. passing information to the kernel

5. booting the kernel

The Multiboot specifications (version 1 section 4.3.3 or version 2 section 4.4.3) contain example kernels that display the passed information and draw a diagonal blue line on the screen.

Sadly, they display the information they have received from the bootloader using the VGA video memory. This is not guaranteed to be initialized or even available on current hardware since it has been replaced with the Graphics Output Protocol (see the section about GOP in a previous post, the UEFI specification section 12.9 and these slides). A simple way to make the text output available is to write it to the serial port instead:

#define SER(v) __asm__("outb %b0, %w1"::"a" (v), "d"(0x3f8))
static void putchar(int c) {
    SER(c & 0xff);
    if (c == '\n') SER('\r' & 0xff);
}

So, I perform end-to-end tests by providing towboot with a configuration file and the example kernel, running it and comparing the resulting content of the screen and the text printed to the serial output to the ones printed when executed by GRUB in the same environment. These integration tests will miss loading the modules because the example kernels do not need any modules. I also load a random binary file as a module to test the module load process (at least partially).

Booting a whole operating system provides a more comprehensive test.

I tried to check the compliance to the Multiboot specification with regards to the machine state by attaching a debugger, pausing at the moment of the jump to the kernel and observing the control registers.

These tests require running the bootloader with an appropriate configuration, kernel and modules on a machine with UEFI-compatible firmware. The probably easiest way to realize this is by using Mtools and mkgpt to build a hard disk image and booting it in QEMU with OVMF.

I have run the virtual machines on two different x86_64 machines, using QEMU 4.2 and OVMF 20190501 (i686) or 20191122 (x86_64). I didn’t perform a test on bare metal.