Supporting multi-platform ARM kernels

The diversity of the ARM architecture is one of its great strengths: manufacturers have been able to create a wide range of interesting system-on-chip devices around the common ARM processor core. But this diversity, combined with a general lack of hardware discoverability, makes ARM systems hard to support in the kernel. As things stand now, a special kernel must be built for any specific ARM system. With most other architectures, it is possible to support most or all systems with a single binary kernel (or maybe two for 32-bit and 64-bit configurations). In the ARM realm, there is no single binary kernel that can run everywhere. Work is being done to improve that situation, but some interesting decisions will have to be made on the way.

On an x86 system, the kernel is, for the most part, able to boot and ask the hardware to describe itself; kernels can thus configure themselves for the specific system on which they are run. In the ARM world, the hardware usually has no such capability, so the kernel must be told which devices are present and where they can be found. Traditionally, this configuration has been done in "board files," which have a number of tasks:

• Define any system-specific functions and setup code.

• Create a description of the available peripherals, usually through the definition of a number of platform devices.

• Create a special machine description structure that includes a magic number defined for that particular system. That number must be passed to the kernel by the bootloader; the kernel uses it to find the machine description for the specific system being booted.

There are currently hundreds of board files in the ARM architecture subtree, and some unknown number of them shipped in devices but never contributed upstream. Within a given platform type (a specific system-on-chip line from a vendor), it is often possible to build multiple board files into a single kernel, with the actual machine type being specified at boot time. But combining board files across platform types is not generally possible.

One of the main goals of the current flurry of work in the ARM subtree is to make multi-platform kernels possible. An important step in that direction is the elimination of board files as much as possible; they are being replaced with device trees. In the end, a board file is largely a static data structure describing the topology of the system; that data structure can just as easily be put into a text file passed into the kernel by the boot loader. By moving the hardware configuration information out of the kernel itself, the ARM developers make the kernel more easily applicable to a wider variety of hardware. There are a lot of other things to be done before we have true multi-platform support—work toward properly abstracting interrupts and clocks continues, for example—but device tree support is an important piece of the puzzle.

Arnd Bergmann recently asked a question to the kernel development community: does it make sense to support legacy board files in multi-platform kernels? Or would it be better to limit support to systems that use device trees for hardware enumeration? Arnd was pretty clear on what his own position was:

There was a surprising amount of opposition to this idea. Some developers seemed to interpret Arnd's message as a call to drop support for systems that lack device tree support, but that is not the point at all. Current single-platform builds will continue to work as they always have; nobody is trying to take that away. The point, instead, is to make life easier for developers trying to make multi-platform builds work; multi-platform ARM kernels have never worked in the past, so excluding some systems will not deprive their users of anything they already had.

Some others saw it as an arbitrary restriction without any real technical basis. There is nothing standing in the way of including non-device-tree systems in a multi-platform kernel except the extra complexity and bloat that they bring. But complexity and bloat are technical problems, especially when the problem being solved is difficult enough as it is. It was also pointed out that there are some older platforms that have not seen any real maintenance in recent times, but which are still useful for users.

In the end, it will come down to what the users of multi-platform ARM kernels want. It was not immediately clear to everybody that there are users for such kernels: ARM kernels are usually targeted to specific devices, so adding support for other systems gives no benefit at all. Thus, embedded systems manufacturers are likely to be uninterested in multi-platform support. Distributors are another story, though; they would like to support a wide range of systems without having to build large numbers of kernels. As Debian developer Wookey put it:

In response, Arnd amended his proposal to allow board files for subarchitectures that don't look likely to support device trees anytime soon. At that point, the discussion wound down without any sort of formal conclusion. The topic will likely be discussed at the upcoming Linaro Connect event and, probably, afterward as well. There are a number of other issues to be dealt with before multi-platform ARM kernels are a reality; that gives some time for this particular decision to be considered with all the relevant needs in mind.

Index entries for this article
Kernel	Architectures/Arm
Kernel	Device tree

---

to post comments

Supporting multi-platform ARM kernels

Posted May 10, 2012 11:21 UTC (Thu) by arnd (subscriber, #8866) [Link] (1 responses)

Thanks Jon for this summary! Over the last week, I've actually tried out building kernels for multiple platforms combined to get a better feeling for the remaining problems. The result is in the testing/multiplatform branch of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc.git and it can build arbitrary combinations of four of the five subarchitectures that the Linaro organization is most interested in (imx, omap, ux500 and vexpress, but not exynos for now). Most other platforms should actually be simpler to convert.

However, this kernel is not yet going to be useful on real hardware because I had to disable or add hacks for a number of features (SMP, sparseirq, clocks) that are still being worked on, but as soon as one platform has all that work done, we can actually build a kernel with everything enabled and run on that particular platform and see what else breaks.

Unlike I suggested in the email thread, this kernel at the moment actually allows enabling all the board files including non-DT ones because that was easier to do with the Kconfig dependencies, but I found two real technical issues that would be solved by making the combined kernel DT-only:

1. The exynos platform defines static platform devices from files shared with five other Samsung platforms that are mutually exclusive because the device definitions depend on platform specific compile-time constants. Using DT probing we can just drop those static platform device definitions and make the conflict go away.

2. With sparse IRQs, a lot of the hardcoded interrupt numbers in static platform device definitions are broken, while the definitions from DT still work. Sparse IRQs are currently needed to build multiplatform kernels and I expect that requirement to stay.

Supporting multi-platform ARM kernels

Posted May 10, 2012 12:57 UTC (Thu) by dgilmore (subscriber, #40144) [Link]

I know from Fedoras point of view we want to build only a single arm kernel. right now we are building kirkwood, imx, omap, tegra, highbank and vexpress. im ok with leaving kirkwood as its own but in order to add exynos I only want to do it in a combined image, and as quickly as possible merge the kernel variants into the base kernel. then have us adding support for other boards only in the base kernel. I like the idea of having DeviceTree only support to make things smaller and simpler if it has DeviceTree support. however i think that the vendors should be on the hook to ship the dtb for a board.

Winter is coming

Posted May 11, 2012 13:53 UTC (Fri) by Aissen (subscriber, #59976) [Link] (1 responses)

And by Winter, I mean Windows 8 on ARM:
http://blogs.msdn.com/b/b8/archive/2012/02/09/building-windows-for-the-arm-processor-architecture.aspx (now named, confusingly for us, Windows RT)

It will be very interesting to compare the different approaches both OSes will use to support multiple SoCs families in one binary kernel. Microsoft is already planning to use device trees system tables, so it's a very similar solution to the one chosen by the Linux kernel community.

It's also interesting, that for once positions are reversed: Linux is the leading OS on ARM devices, and Windows is the challenger. The net result is that Microsoft has the opportunity to start from a clean plate and have a clean implementation. It also means that Microsoft might have the opportunity to be the first to ship products with multi-platform support.

Linux, on the other side has the burden to continue supporting the dozens of different SoC families, and thousands of different boards. That's why I believe that Arnd's proposal to only support multi-platform booting with machines that have DT support is very reasonnable. It's a new feature after all, so there's no possible regression.

It's great that after starting this years ago (DT is not exactly new), the ARM Linux community, Linaro and distributors are finally taking on this problem at full speed. But who will win the race to the multi-platform graal ?

Winter is coming

Posted May 16, 2012 22:30 UTC (Wed) by mwsealey (subscriber, #71282) [Link]

The interesting part will be when embedded systems start coming with UEFI and the whole device tree concept is given a companion for ARM in favor of those system tables. It would be rather stupid to take a hard, fast stand against system tables in favor of device trees when an industry standard firmware simply doesn't supply them (or force UEFI firmware implementations to implement device tree just to support Linux..)

After all, ARM (as in the company) are pushing UEFI not just as a way to boot Windows, but as a better, more unified and standardized bootloader which U-Boot and many other "alternatives" simply cannot do (since on every SoC there tend to be different ways of handling precisely the same thing in U-Boot unfortunately).

Personally I would rather we did away with the device tree concept as SoCs are painful to implement in a device tree - what usually happens is Linux drivers are written then a device tree spec is written for that device, which includes values which should never need to be put in a device tree. What is missing from the device tree is usually hardcoded into the driver, and usually that's stuff you may actually want to be 'dynamic' in a device tree. It worked great for m68k, SPARC and PowerPC but those implementations really WERE dynamic, and supplied by the bootloader (OpenPROM or later OpenFirmware) based on a real-time introspection of the hardware at hand, not preconfigured and unchanging except for a few notable hacky bits which some bootloaders can shove values into. Static device trees built at kernel build time rarely, if ever, match the exact board design, and the usual theme from China is you get a board, write a device tree and 3 weeks later a new board is out which needs a different device tree. Plugging in a single kernel with 10 device trees on flash or disk, to select in the bootloader depending on board revision, is quite a painful concept.

We have an opinion round here at work that the device tree should be written by the board designer - not the SoC vendor (although they should be involved) and certainly not the driver author. The board designer is the only one who knows what is connected and what information is truly, truly important. The board designers knows what he WILL change later. And worse, the current specification of pinmux details in the device tree is truly, truly moronic from an electronics design point of view. The vast majority of hardware designers would rip a hole in spacetime if you told them you are going to make them wait 15 seconds to configure I/O direction and pad configuration. This is stuff that should be configured at a very low level right as the chip is powered on, not at Linux kernel init time.. the restriction this puts on hardware designers to only use the "default" pin configurations until some late time in boot for fear of driving an output pin on the SoC to an output pin on a peripheral chip where this is not electrically sound, is not a fun thing to have to work out. The device tree SHOULD report the CURRENT configuration of the harware, not dictate what the configuration SHOULD be in order for everything to work.. in supplying configuration data for changing the operation of the chip, the flattened device tree concept has essentially proven that it is completely flouting the very concept that made device trees (as a specification of running hardware and API) such a great idea in the first place.

Supporting multi-platform ARM kernels

Posted May 18, 2012 4:46 UTC (Fri) by slashdot (guest, #22014) [Link]

Why not just remove all board files from Linus' kernel tree?

Then those using them will have an incentive to convert to device tree.

Supporting multi-platform ARM kernels

Posted May 17, 2013 21:29 UTC (Fri) by robert1356 (guest, #91030) [Link]

In looking at many discussions, tutorial and even the code, it appears that the term "device tree" is used in a generic way to refer to specific hardware device branches in the kernel sources. How does one go about separating the generic "device tree" reference from the OF DeviceTree discussed here and at devicetree.org?