Lesson #2: A bootloader is to load and boot Linux

On the first day of FOSDEM I sat through a presentation on what could be called another "U-Boot derivative". One of the greatest asspains at Openmoko was the various kinds of Hell caused by the U-Boot bootloader and its philosophy, which can be summed up as "I wanna be Linux when I grow up".

Configure system is a bad alternative to good bootloader design

First, it has a config system. That should be good though, right? The problem with the config system is that if anything differs from your current config, you must build another incompatible binary with another config and take care of that. When you have more than a handful of different boards, you are in a maze of incompatible bootloaders. Openmoko took it one step further, they mandated a different bootloader binary per PCB revision, so left unchecked there would have been a continuous proliferation of incompatible bootloaders, all basically the same.

All persistent bootloader private state is EVIL

Second, U-Boot thinks it's a good idea to have these environment "scripts", because it's "configurable". Actually, the job of a bootloader is to Load, then Boot Linux. You don't need any configurability for that if the bootloader can figure out what it's running on and therefore where the memory is and how much there is. These scripts expose a really deadly trap I call "private bootloader state". It means the bootloader stores stuff in nonvolatile memory on the PCB and acts different according to what it hides there. The end result is that two boards from the same factory may act totally different even with the same rootfs due to "bootloader secrets". This is totally needless and ALL private bootloader state can be eliminated by correct design of the bootloader leading to completely deterministic boot action per rootfs. A good example how that lead you to the path to hell is hardcoding in the U-Boot environment of the amount of kernel image you will copy from somewhere. People commonly set it to 2MBytes, forget about it and one day they generate a 2.1MB kernel image and wonder why decompress blows up. Actually, that whole procedure is insane, the kernels are uImages that report their length in a header. The bootloader should examine the header and compute the length of image to pull. But that doesn't fit with this "environment" nonsense.

Do Linux Stuff In Linux

In any of these bloated U-Boot style bootloaders, is there even one feature they do better than the same feature in Linux? The startup time should be better by a few 100ms. Other than that, no, every single bloated "I will add it to the bootloader beacuse I can" feature is shittier than you get in Linux. Every single feature! If you need some advanced capability or backup / recovery boot action, check for a button held down at boot-time in the bootloader and go fetch a different Linux partition + kernel. Use standard Linux tools and shells. In return, get really high quality network stack, proper USB support, NAND access that's compatible to your main Linux system access in BBT / ECC terms, and all the other advantages of Linux.

Do your peripheral bringup in drivers in Linux

Typically you do not need ANY bringup in the bootloader except SDRAM controller and chip init, since it's a prerequisite to put Linux in the RAM that it's initialized. That's right, all the megabytes of source spent in U-Boot providing support for so many kinds of peripheral is a waste of time, effort and maintenance. I am being kind saying "maintenance", because the drivers in U-Boot are typically "dumbed down" versions of the equivalent Linux driver that were forked irretrievably the moment all the Linux APIs were ripped, so there's no coherent effort to keep them up to date with the Linux ones . Lately I saw that they try to ape some Linux APIs there... why not go the whole hog and just load and boot real Linux? After all, modern CPUs can be running your driver probes in Linux in ~2 seconds from power using a bootloader that doesn't get in the way. You typically don't even need to talk to the PMU in the bootloader, after all, you are running code fine already, right? Otherwise you wouldn't be able to run the bootloader code itself.

Fat girl in Ibiza

At least at Openmoko, code quality inside U-Boot was awful bad. I called U-Boot on the lists there "the fat girl in Ibiza" because you know she's going to do anything you want. All kinds of constant-only code, weird new scripting keywords were added for test undocumented, you name it. Hardware guys felt up to writing such code secretly by themselves once they learned the software engineering marvel that is *((unsigned int *)0x...) = 0x...;

Your bootloader just tests SDRAM

There's only one test action your bootloader is suited to do, and that is SDRAM test. Once you are in Linux, it can't perform a full SDRAM test while it's running. But the bootloader is typically starting from on-CPU SRAM, it can actually run a true SRAM test from there. Otherwise, the bootloader should be completely absent from the test plan. All other tests should be performed in Linux via standard driver and rootfs tools. More about board and test and board bringup will feature in another report of a lesson learned.

Qi

While at Openmoko (mainly) I wrote a bootloader that meets these ideals, you can find it in git here One of the nicest things about it is that unlike the bloated bootloaders whose job never finishes trying to become Linux cargo cult style, Qi has been pretty much complete for a few months. It's a new job to support a new CPU, a much smaller job to add a new board and it doesn't want to talk to your peripherals anyway so no problem there. Qi creates one binary per CPU, that supports all boards with that CPU. That sounds like a big job but we don't care about your peripherals so all boards with the same CPU look almost identical. You have to find something that can detect your particular board at runtime, for example NOR device ID read check. So there is zero build-time config and Qi generates all CPU support when it's buit, it takes 3 sec or so typically. Typical bootloader binary size per CPU is 28-30KBytes. That supports VFAT, ext2/3/4 typcially the SD controller as well. The single Qi image also supports being booted from NAND, JTAG or SD Card on processors that support it just by being copied into place and without any changes. There is zero bootloader private state, however Qi can look in the rootfs and append kernel commandline text from the content of a filesystem file. This maintains the rule that boot should be completely deterministic per rootfs.