Comparing Yocto and OSBuild

Some personal views I have on using Yocto build a Linux distro for embedded systems vs using a managed distro like Fedora IoT or AutoSD.

Disclaimer: This is a living document I’m putting up here that I’ll add to and edit over time.

Disclaimer: The opinions/information expressed in this post are my own and not representative of those of my employer.

Yocto and OSBuild

I have been working in the automotive space for quite some time, specifically in the area of embedded Linux distro build ops. In the past, I was involved with using Yocto to construct operating system images. Yocto was typically the defacto standard for this kind of space but a new open source project named AutoSD has come out to try enter the automotive Linux space and do things a bit differently to the Yocto way.

I wanted to quickly touch on a tool currently being suggested by the AutoSD community for use by their users, named OSBuild. I haven’t used the tool to a deep level but I wanted to put some notes online comparing this to the Yocto way of doing things.

How do they differ

If I directly compare the two, they actually do quite different things.

| Yocto                      | OSBuild                  |
| -------------------------- | ------------------------ |
| Fetches sources            | Constructs rootfs images |
| Builds packages            |                          |
| Constructs package repos   |                          |
| Constructs rootfs images   |                          |

OSBuild is more akin to mkosi, and is (from what I understand) only really handling the amalgamation of packages from a package repo into bootable system images. Meanwhile, Yocto is doing the whole shebang, from source to rootfs image (in one step).

So if Yocto is building from source to packages to rootfs images, how does AutoSD using OSBuild replace those parts? Well, AutoSD is a distribution so it ships an rpm repository. This means that the source to package build steps are done by the distribution elsewhere and the results shared with all users as a base point.

In my view, there are pros and cons to this. The Yocto way was typical for embedded systems that need to recompile the whole software stack because low level changes in base libraries would be needed to get the most out of low level embedded systems and the ABI may be broken easily while doing that. This lets a developer squeeze an embedded system but is a lot of work and is holding the bag maintaining essentially a custom distribution for their hardware. The AutoSD way is more like standardising on the package set and using prebuilt software that is compiled for more generic use. This makes it more effort to get hardware specific customizations in (have to carefully build ABI compatible packages) but the work of maintaining all the other generic packages is handled by the distribution. Which can be a lot of work.

I do think we’re moving further away from the hardcore way of building embedded Linux distributions as the hardware gets more powerful and more generic. In my experience, the work of maintaining a custom Linux distribution is normally underestimated and being able to build on top of a trusted distribution for an edge device or embedded device may be the more typical way to go in the future.

Some other things I would like to note about OSBuild vs Yocto is the developer runtimes.

• Yocto does not require root and in fact blocks itself from being run as root. OSBuild however requires root access. This also means, OSBuild cannot be run inside an unprivileged container and so it is difficult to fit into a cloud-native environment.
• Yocto allows for cross-architecture package builds and rootfs image generations. This means running on an x86 machine but creating aarch64 build artefacts. OSBuild does not support this and requires running on the same architecture that the build artefacts it creates. So when building an aarch64 rootfs image using OSBuild, it needs to run on an aarch64 host machine.

OSBuild

I skimmed the OSBuild documentation and it seems it works like this: OSBuild itself is a python based tool that takes as input some json or yaml manifest files that define a series of steps to perform to build a rootfs image. Such as downloading rpm packages, doing the rpm install, applying selinux labels to the rootfs directory, tarballing it, etc. There is a component/service named osbuild-composer that acts as a simplified api interface on top of osbuild that generates this json or yaml manifest files and triggers obuild. (More layers sit above osbuild-composer, like the image builder that defines a user interface to interact with osbuild-composer).

Looking at the AutoSD documentation, the distribution is however using a tool named osbuild-mpp to do things not possible via the standard osbuild-composer api. I believe this is more of a debug tool that can generate very custom yaml/json manifests for osbuild from another simplified yaml/json manifest (mpp files).