Archive for the ‘Software design’ Category

New NXP LPC32x0 in Qi bootloader

Monday, November 29th, 2010

LPC3250 from scratch

NXP’s new LPC32x0 is a very cheap and feature-filled ARM926.  According to Digikey anyway, it’s the cheapest ARM chip with at least v5 instruction set that’s going.  That’s important not just because of the extra processor strength over older ARM9 core, but because ARM Fedora is built requiring armv5 or newer instruction set.  Being able to use ARM Fedora and RPM as a basis means freedom from compromise and having to own the building of an integrated, self-consistent rootfs; you can just focus on doing your specialized code on top using the reliable Fedora quality basis.

There are four chips in the series, they differ in having an LCD controller and Ethernet MAC or not; also the smallest guy LPC3220 has “only” 128KBytes of Static IRAM and the others 256KBytes.  Well, having worked with the 2KBytes of internal static RAM on the iMX31 for SD boot on Qi, having to shoehorn an SD card driver in there, even 128KBytes is crazy amounts.

They have support for resistive touchscreen, USB OTG, NAND controller and Mobile DDR, and up to 266MHz CPU clock at 1.4V Vcore (208MHz at 1.2V Vcore but as we will see that is not entirely true).  They don’t support SD Card boot from ROM, but that can be solved for about US$0.30 as will be shown.

In short they’re ready to do some serious embedded work at a budget price.

Embedded Artists EA3250 Dev kit

There are a few dev kits around for LPC32x0, Hitex have a cheap USB stick format one that has been permanently two weeks away from availability since I first looked at it a month or so ago, and it still is two weeks away.

NXP anoited two real dev boards they evidently worked with the vendors for during development, they don’t actually make an NXP branded dev board, it’s Phytec and Embedded Artists.  Since the EA one is in Digikey, that’s what I ended up with.

The dev board is well made but there are some problems with it: like many dev boards it comes in two halves, a cheaper, large breakout board and a 8-layer DIMM type board that has the actual CPU BGA and memory.  In an act of supreme lunk-headedness, the large breakout board re-uses the Pn.m nomenclature that the CPU uses for GPIO, with no care to retain the CPU mapping.  So for example a header is marked with having a pin P1.27, very confusingly this is nothing to do with the CPU GPIO P1.27.  This is also true in the schemtatics for the baseboard and CPU board, complete confusion trying to trace a signal between the two boards or looking for a misnamed signal on the baseboard.

DDR trouble #1

There’s also a more serious problem, the DDR on the CPU card is marginal and Embedded Artists have made a recall where they will replace the board with one with a different DDR DRAM for free.  The CPU board I got was affected but not at room temperature; they want the old card sending back and I am not finished with it yet, so I will take advantage of this recall later.

DDR trouble #2

There’s another problem with DDR, NXP issued an errata confessing their inverted signal for the differential DDR clock is skewed by no less than 1.2ns from the uninverted partner of the differential pair, a huge skew.  This issue removes a lot of comfort zone from designing with DDR and means only some memory devices will tolerate it.  However in the EA board case, they have not used the workaround suggested by NXP which is to nuke the inverted output entirely and make the clock unipolar, so the situation can’t be that bad.

DDR trouble #3

The last problem with DDR… operation at 208MHz with 1.2V Vcore is fine for the CPU, in fact while screwing with the PLL I had the CPU running fine at 400MHz, although there is no way to divide anything useful down for the memory clock at that speed and it’s illegal for the PLL over temperature, which tops out at 320MHz.  However at 1.2V and 208MHz, the CPU side of the DDR bus is unreliable: it requires cranking to 1.4V to operate DDR even at 104/208MHz.  That’s annoying because since 1.2V is needed anyway for other circuitry, it could have saved a regulator.

Unbrickability of LPC32x0

LPC32x0 chips feature UART-based bootloader injection… if you pull down the SERVICE_N pin, then next boot the ROM in the CPU will bring up UART5 at 115200 n81 and issue a simple protocol byte allowing for bootloader download.

Since I couldn’t find a Linux tool for injecting bootloaders, just a Windows one, I wrote a commandline tool for it and added it to Qi build.

http://git.warmcat.com/cgi-bin/cgit/qi/tree/tools/lpcboot.c?h=lpc

No matter how broken your nonvolatile image gets, it’s still possible to recover the device via this UART scheme with a USB <-> LVTTL serial cable.

Bootloader Hell

The LPC32x0 bootloader situation is ugly.  Basically NXP provided a huge suite used for chip verification called CDL (“common driver library”), this is a sort of chopped down OS in bootloader form.  It has all kinds of functions to drive the chip peripherals and test memory, but nothing to actually boot Linux!

What EA shipped, and what you are meant to do as a system integrator, is get an implementation of CDL in the form of “S1L” — stage one bootloader — to load U-Boot, which will then load Linux.  Both U-Boot and S1L — itself like 130KBytes! — store “state” on the board.  It leads to this insane situation that two bootloaders with two kinds of state must be right in order to boot.  Things are further complicated that SPI boot only allows the first 56KBytes to be loaded by ROM into IRAM and executed, but the bloated bootloaders are too big to do this in one step.

Bootloader Heaven

I added support for LPC32x0 to Qi last week, this is a single < 30KBytes image that can boot itself from SPI Flash or UART 5 injection and pull Linux from SD Card in VFAT partition or also via SPI Flash.  Boot from cold, with Qi and Kernel in SPI Flash to Fedora 12 bash prompt is less than 4 seconds.

http://git.warmcat.com/cgi-bin/cgit/qi/log/?h=lpc

This replaces both S1L and U-Boot, and in accordance with Qi philosophy it holds no state at all on the device.

Its strategy is if it finds that it is running via injection on UART5, it copies itself into SPI Flash / EEPROM so it will run next boot from there, and if it finds an SD Card kernel image it will also copy that into SPI Flash.

When it finds it is running from a non-injection source, ie, a normal boot from SPI Flash, it favours any kernel it can find on the first, VFAT, partition of an SD Card if found, otherwise it boots from the kernel also in SPI Flash.

This is why the lack of ROM -> SD Card boot is not critical, the cheapest, smallest SPI EEPROM can be used to contain Qi, which will then load the kernel and rootfs from SD Card if that’s what’s needed as during development.  If SD Card is overkill for the job, then Qi, Kernel and initrd can all be pushed into a single US$2 32MBit SPI Flash.

Since I only have the Embedded Artists board right now it wants to see a kernel image called k-ea3250.img on the SD Card; the way Qi works you add a new file for each supported board in ./src/cpu/lpc32x0/ copied from embart-steppingstone.c in that directory; the bootloaders need some way to identify what they’re running on at runtime since there is only a single image per cpu that supports all devices.  See  http://git.warmcat.com/cgi-bin/cgit/qi/tree/src/cpu/lpc32x0/embart-steppingstone.c?h=lpc for an idea of what’s involved to support a new board in the bootloader image.

Posted in Embedded Linux, Linux peripherals, Qi bootloader, Software design | Comments Off

Don’t let Production Test Be Special

Friday, February 12th, 2010

Lesson 3: Test is not special

Commonly in embedded work test is the “red-haired stepchild”, nobody wants to take care of it and by common, silent consent it is always left until last.  Eventually the need for a test plan becomes overwhelming as the date to go to the factory nears, and the task is assigned to the most junior engineers available, since everybody knows that test is the death knell of your career.

Coming cold to and excluded from being inside an already-existing project, the engineers try to create some kind of test coverage the best way they can.  At openmoko two giant test suites were created, DM1 and DM2, written by people who were learning C for the first time.  I got the job of modernizing this code so I know from experience the code was already truly terrible and bitrotted at an alarming rate.  However I had to admire the guys who wrote it, with everything against them and little experience they did manage to create something that did provide test coverage at the factory, however much it was on life-support.

Totentanz

Similarly, Openmoko used production test jigs, special additional PCBs that formed a kind of custom test environment for the PCB under test.  At one version of GTA03 there were so many test points added it was a serious concern that the board would break down under the overall pressure needed to mate the spring-loaded test probes to the test points.

Jigs and test points have an obvious advantage in terms of test throughput, but there are some big disadvantages.

First, you have to design and build the jig, and track changes to the actual device with it.  This effort is completely disconnected from moving your actual product on, except that it’s meant to help in production.

Second, test points don’t test your connectors; the test point may be connected OK but not the connector pin the user actually accesses.

Third, you need something else outside the device to assess what is happening on the test points, the code for that also has to be written and maintained against changes in the actual product.  It also means that it’s not possible for the tests to be casually performed outside the factory, or maybe by the original engineers if they have access to the ATE gear themselves.

Pain into torture

Additionally the bringup of GTA02 required special versions of U-Boot and kernel which had added “test magic” created by the test guys and unknown to anyone else.  These versions were seldom uplevelled.

Since GTA02 had raw NAND, it needed filling up at the factory with the rootfs.  The way to do this was via a very fragile OpenOCD using a custom USB – serial based device that was bitbanged.  It only worked with certain versions of the usb library needed to talk to it.

All of these quirks and requirements at the factory made production runs difficult and expensive to get right.

I only hurt you because I love you

I spent a lot of time thinking about how to avoid this end result next time I would design something.  The mistakes started in having anything special for test I concluded.  The jig: special, and so evil.  Test kernels or bootloader: special -> evil.  Test rootfs -> Evil.  test software, like Openmoko’s DM1 and DM2, evil.  The device should naturally be able to test itself with the arrangements that already exist inside it to operate at all.

The answer to the problem of “production test” is to completely subsume it into the rest of the design.  So it is the responsibility of Linux drivers to provide enough functionality by probe errors, or sysfs features, that one can perform test and diagnosis.  The “test suite” should boil down to a bash script that is using features exposed in a normal shipping rootfs and kernel.  Bash is ideal because most of the test action will be calling existing commandline tools like ifconfig, ping, l2ping and grepping or looking at their return code, this is what bash is best at.  It’s also easily understood and edited by anyone who has worked with Linux for a while.

The bootloader is required for test in only one capacity, it is the only part of the system that is capable to run the SDRAM tests; once you enter Linux you can’t perform a full SDRAM test any more.  But even that should be done by the one shipping bootloader image.

In many cases, device interfaces can be tested by external loopback connectors, this proves connectivity through the connectors and it leaves open the possibility of end-users being able to run the same tests on the shipping rootfs.

Posted in Embedded Linux, Hardware design, Openmoko Lessons, Software design | Comments Off

Bootloader Envy

Monday, February 8th, 2010

Lesson #2:  A bootloader is to load and boot Linux

On the first day of FOSDEM I sat through a presentation on what could be called another “U-Boot derivative”.  One of the greatest asspains at Openmoko was the various kinds of Hell caused by the U-Boot bootloader and its philosophy, which can be summed up as “I wanna be Linux when I grow up”.

Configure system is a bad alternative to good bootloader design

First, it has a config system.  That should be good though, right?  The problem with the config system is that if anything differs from your current config, you must build another incompatible binary with another config and take care of that.  When you have more than a handful of different boards, you are in a maze of incompatible bootloaders.  Openmoko took it one step further, they mandated a different bootloader binary per PCB revision, so left unchecked there would have been a continuous proliferation of incompatible bootloaders, all basically the same.

All persistent bootloader private state is EVIL

Second, U-Boot thinks it’s a good idea to have these environment “scripts”, because it’s “configurable”.  Actually, the job of a bootloader is to Load, then Boot Linux.  You don’t need any configurability for that if the bootloader can figure out what it’s running on and therefore where the memory is and how much there is.  These scripts expose a really deadly trap I call “private bootloader state”.  It means the bootloader stores stuff in nonvolatile memory on the PCB and acts different according to what it hides there.  The end result is that two boards from the same factory may act totally different even with the same rootfs due to “bootloader secrets”.  This is totally needless and ALL private bootloader state can be eliminated by correct design of the bootloader leading to completely deterministic boot action per rootfs.

A good example how that lead you to the path to hell is hardcoding in the U-Boot environment of the amount of kernel image you will copy from somewhere.  People commonly set it to 2MBytes, forget about it and one day they generate a 2.1MB kernel image and wonder why decompress blows up.  Actually, that whole procedure is insane, the kernels are uImages that report their length in a header.  The bootloader should examine the header and compute the length of image to pull.  But that doesn’t fit with this “environment” nonsense.

Do Linux Stuff In Linux

In any of these bloated U-Boot style bootloaders, is there even one feature they do better than the same feature in Linux?  The startup time should be better by a few 100ms.  Other than that, no, every single bloated “I will add it to the bootloader beacuse I can” feature is shittier than you get in Linux.  Every single feature!

If you need some advanced capability or backup / recovery boot action, check for a button held down at boot-time in the bootloader and go fetch a different Linux partition + kernel.  Use standard Linux tools and shells.  In return, get really high quality network stack, proper USB support, NAND access that’s compatible to your main Linux system access in BBT / ECC terms, and all the other advantages of Linux.

Do your peripheral bringup in drivers in Linux

Typically you do not need ANY bringup in the bootloader except SDRAM controller and chip init, since it’s a prerequisite to put Linux in the RAM that it’s initialized.

That’s right, all the megabytes of source spent in U-Boot providing support for so many kinds of peripheral is a waste of time, effort and maintenance.  I am being kind saying “maintenance”, because the drivers in U-Boot are typically “dumbed down” versions of the equivalent Linux driver that were forked irretrievably the moment all the Linux APIs were ripped, so there’s no coherent effort to keep them up to date with the Linux ones .  Lately I saw that they try to ape some Linux APIs there… why not go the whole hog and just load and boot real Linux?  After all, modern CPUs can be running your driver probes in Linux in ~2 seconds from power using a bootloader that doesn’t get in the way.

You typically don’t even need to talk to the PMU in the bootloader, after all, you are running code fine already, right?  Otherwise you wouldn’t be able to run the bootloader code itself.

Fat girl in Ibiza

At least at Openmoko, code quality inside U-Boot was awful bad.  I called U-Boot on the lists there “the fat girl in Ibiza” because you know she’s going to do anything you want.  All kinds of constant-only code, weird new scripting keywords were added for test undocumented, you name it.  Hardware guys felt up to writing such code secretly by themselves once they learned the software engineering marvel that is *((unsigned int *)0x…) = 0x…;

Your bootloader just tests SDRAM

There’s only one test action your bootloader is suited to do, and that is SDRAM test.  Once you are in Linux, it can’t perform a full SDRAM test while it’s running.  But the bootloader is typically starting from on-CPU SRAM, it can actually run a true SRAM test from there.  Otherwise, the bootloader should be completely absent from the test plan.  All other tests should be performed in Linux via standard driver and rootfs tools.

More about board and test and board bringup will feature in another report of a lesson learned.

Qi

While at Openmoko (mainly) I wrote a bootloader that meets these ideals, you can find it in git here One of the nicest things about it is that unlike the bloated bootloaders whose job never finishes trying to become Linux cargo cult style, Qi has been pretty much complete for a few months.  It’s a new job to support a new CPU, a much smaller job to add a new board and it doesn’t want to talk to your peripherals anyway so no problem there.

Qi creates one binary per CPU, that supports all boards with that CPU.  That sounds like a big job but we don’t care about your peripherals so all boards with the same CPU look almost identical.  You have to find something that can detect your particular board at runtime, for example NOR device ID read check.  So there is zero build-time config and Qi generates all CPU support when it’s buit, it takes 3 sec or so typically.

Typical bootloader binary size per CPU is 28-30KBytes.  That supports VFAT, ext2/3/4 typcially the SD controller as well.  The single Qi image also supports being booted from NAND, JTAG or SD Card on processors that support it just by being copied into place and without any changes.

There is zero bootloader private state, however Qi can look in the rootfs and append kernel commandline text from the content of a filesystem file.  This maintains the rule that boot should be completely deterministic per rootfs.

Posted in Embedded Linux, Linux peripherals, Openmoko Lessons, Software design | Comments Off

Fosdem and the Linux Cross Niche

Monday, February 8th, 2010

fosdem

I was at Fosdem over the weekend, there were several interesting talks I attended but the most interesting one for me was a roundtable about the future of Cross distributions.  I was invited to give a 5 minute talk there which I gave, but unfortunately it was right at the end and the people before had overrun, so there was no time to make much of a coherent case.   So I am going to write some articles covering the issues involved here.

Cross as a niche

Cross itself remains absolutely necessary for systems below a certain level of horsepower.  For example, 8051, ARM7, cortex M3 are not really capable to consider native build.  But processors get faster each year, a lot of things we would have used an 8051 on use an ARM7 or cortex M3 now, in a few years it is likely that baseline has moved further up and it’s an ARM9 equivalent.  What I am suggesting then is that over time, the niche where you need cross is shrinking.

All four of the cross distros at FOSDEM target a CPU that’s powerful enough to run Linux, but not powerful enough to build its own binaries.  That is the niche that I believe will shrink to the point that it won’t support all these cross Linux distro projects, possibly none of them in the end.

My background with cross Linux

A few years ago I created an RPM-based cross distro singlehanded, and used it on a product for a customer  This was AT91RM9200-based, a 200MHz ARM9 with 32MBytes of SDRAM.  The amount of effort needed to create a set of cross packages sufficient to create a workable rootfs was huge, it took me many weeks.  Some packages like perl were just so cross-unfriendly that they were basically out of reach (although I later saw other people have done the invasive magic necessary).  It did work well, and I added patches for busybox RPM support that allowed it to do more useful things like erase and keep a package database.  The packaging was valuable in itself but a nice advantage was the source RPMs it generated ensuring GPL compliance.

My background with Openmoko

Subsequently I spent 14 months as (mainly) the kernel maintainer for Openmoko.  Openmoko had an OpenEmbedded basis for it’s rootfs, also a cross system.  I attempted to use it for “hello world” while I was at Openmoko, but it broke because I was on a newly released Fedora.  How it broke was very revealing, the official way to get started with it was to run a huge script that wgetted and locally built 1100 packages.  It died due to some assumption somewhere breaking while it tried to build host dbus libraries.

What I wanted was a cross toolchain that would let me package “hello world”.  What I got was a massive host build action including host dbus libs.  I have perfectly good host dbus libs in my Fedora install, I enquired about it and was told they were the “wrong” libs for the expectation of the rest of the packages, so they had to be rebuilt.

I gave up on trying to use OpenEmbedded, as I guess most of Openmoko’s customers did.

After Openmoko imploded, I designed the software architecture (and influenced the hardware design in some aspects) for the txtr reader device.  On this device, I put into action various lessons I had learned in how not to do things from Openmoko.  I will write further about the other lessons in future articles, but here’s the first one:

Lesson #1: Don’t compile your own rootfs

I was told by a manager at Openmoko that Openmoko had hired most of the main devs of OpenEmbedded and were paying for that accordingly.  This was a pretty big drain on their resources over a long period.

In contrast, nowadays you can head over to http://fedoraproject.org/wiki/Architectures/ARM and download a generic rootfs tarball of prebuilt binaries for ARMv5 and above[1].  It’s made from unpacking prebuilt binary packages.  Once you boot into it, you can install further packages with the usual yum install type action.  You can be up in a high quality rootfs in five minutes flat.

You do not need to go around compiling everything personally when binary packages exist from a reputable distro already.  Normal distros provide -dev and -devel packages for you to link against too, so you do not need to recompile the universe just because you want to build “hello world” either.  That’s how we do things on desktop and server systems, as the processors involved get stronger embedded does not have to be different.

If you want to cross-build specific packages, you install the Fedora ARM Cross Toolchain RPMs on you host via yum and you are ready to go in a couple of minutes.  This is very useful for cooking the kernel on your host both to get started and during development; you can’t native-build the bootstrap stuff needed to boot your platform.  But that’s just a cross compiler and related pieces, it’s not a cross distro.  (The guy from emdebian at this FOSDEM talk also made this point that you do not need to get into making your own toolchain, your distro should have one you can just install).

Fedora ARM’s strategy is native build.  So you install gcc and other dependencies into the actual device, and use standard rpmbuild to build your package there; you can also just configure ; make ; make install for development too down there.  If something’s missing on the rootfs you can yum install it.

(1 To make the comparison fair to openmoko Fedora ARM came along too late for them to choose it from the start, and the GTA02 s3c2442 was not a v5 class processor, they would have been into a distro recook after changing the distro-level compile options.  However my worry is not repeating Openmoko’s errors and today Fedora ARM is available.)

Quality and Quantity

Another major issue is distro quality.  I was so surprised to hear at Fosdem Dr Mickey Lauer of OpenEmbedded boast about the number of devices that managed to use that distro (including the sad shape of the GTA02) and say that unlike the other cross distros, OpenEmbedded focused on “Quantity not Quality”.  From my experience I think he’s right alright about not focusing on quality, and he did go on to explain there are problems with OpenEmbedded they are trying to address.

In the near future, there will be a carcrash between these difficult cross distros that have relatively poor quality and strange requirements to use them and standard, “proper distros” like Fedora ARM, because on higher-end ARMv5s say 400MHz and above, it is already perfectly possible to compare the two worlds on the same device.  I think many devs currently are trained by their experience with buildroot type systems to assume they have to personally build everything Gentoo style.  However as CPUs increase in power at the same price point, the ways of working with these systems efficiently change, and desktop / server “treat it like a PC” lessons like the value of packaging start to really show their traditional advantages over rootfs tarballs.

Like Debian, Fedora has all kinds of rules and requirements about packaging to ensure high quality, there are a huge number of users of these two normal distributions that leads to tested and debugged basic packages and their dependencies.  OpenEmbedded’s boast about number of users is not even a blip in comparison to Fedora or Debian’s consumers and contributors.

Cross distros are locked into local patch hell

A worse problem against their quality even than not many users is the patch load these projects are carrying, I think all of the cross distro projects bemoaned that they were carrying huge patchsets across a large number of packages to get them to build cross at all, and that most upstreams did not care to take them (I assume they don’t want to have to get into testing them).  To uplevel packages, which distros have to daily when they have a large package universe, it can become a nightmare of breakage because of the private patchsets being dragged around.

(BTW I also saw in another presentation that the limo foundation are carrying around more than 80MBytes of diff between their distro and the upstream projects, and these are the guys who sent out a whitepaper explaining the massive cost of delaying sending patches upstream in dollar terms.)

There was proposed a unified crossbuild patch promoting effort, but the effort seemed only to consist of a domain like “sends-patches.org” that you could use when sending patches instead of your own project name, which seems to just be tea and sympathy rather than a solution.

It’s clear that quality will tend to be higher if you are getting packages built with normal distro specfiles and no pile of local patches to get them to build cross (because they were built native).  Combined with higher quality thresholds at the project level and sheer number of users, native Fedora (or Debian) rootfs basis will provide Quantity and Quality if your processor is appropriate.

A couple of hours after the talk I had an interesting conversation with OpenInkpot dev Mikhail Gusarov, who I found also shared my lack of enthusiasm for OpenEmbedded, although he is trapped still in the cross niche generally by the weak processors he targets at the moment.

[update Feb 10 09:00] Mikhail has written his own response, he still likes the speed of cross (and still hates OpenEmbedded).  But there’s some confusion about what Fedora ARM offers, it’s a generic ARMv5 rootfs, it doesn’t care what exact kind of CPU, vendor or peripherals available.  Build farms are less of a requirement when you are no longer building your rootfs but installing it from distro binary packages.  Sheevaplug makes available a 1.2GHz Marvell ARM compatible with 512MBytes of SDRAM that Fedora ARM can work on if you need a native build machine.  Shortly fast dual processor Cortex A9 machines will become available.

Tags:
Posted in Embedded Linux, Openmoko Lessons, Software design | Comments Off

The Alignment Monster

Friday, May 25th, 2007

insanity at the laptopCurrently I am working on an embedded Linux ISDN-2 device I have designed… the hardware works fine but it’s clear that the challenges lie in the software stack. ISDN uses a cryptic, stateful protocol called LAPD to manage call state and features many layers of protocol stacks to get the job done. You know you are dealing with the old school when they refer to 64kbps log coded PCM as “3.1kHz voice”, meaning the audio bandwidth.

Naturally at this.. mature… stage of ISDN development (ie, I am plundering the ancient dusty tombs of a dead protocol that happens to be in wide use) I am not anxious to become a guru capable of winning arguments at the bar on ISDN protocol trivia, instead I need the freaking thing to work. If it’s a new technology, exploring the byways and understanding it closely can often pay off in the future, but there is much less chance of that when dealing with something old and basically deprecated (cf ADSL). So I chose to use mISDN, an attractive proposition with a driver for the chipset I am using and capable of working in both NT and TE modes — basically allowing acting as the exchange and the customer.

mISDN is getting some reasonable use as part of Asterisk via chan_mISDN, so I hoped for an easy ride. However it is clear that mISDN has not had much of a life outside of x86. The Makefile is not set up for crosscompile and indeed the thing from git when I started on it would not compile against a current kernel source. In fact the thing caused a segfault in the kernel build process on contemporary kernels, the two-line fix for which represents my first contribution to the actual Linux kernel tree. (Rather a weird bug… the text CONFIG_MODULE appearing in any source file will cause the build to fail with a segfault in a script. This text appeared accidentally in mISDN — CONFIG_MODULES was meant which did not trigger the bug)

(more…)

Posted in Software design | Comments Off

GPL2 “or later” distributor sends mixed signals when distributing as GPL2

Tuesday, September 19th, 2006

A couple of people have pointed out something that is important about the GPL2 “or later” license: the distributor chooses the license version they distribute under. (Actually the busybox maintainer Rob Landley pointed it out last week too). The actual language used on sources licensed as GPL2 “or later” looks like this:

* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of
* the License, or (at your option) any later version.

Note that this text uses “you” and “your” to apply equally to everyone who gets distributed to and who is distributing.

In the “distributor chooses” reading of this text:

* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of
* the License, or (at your option) any later version.

a new class of information is generated about the distribution action which I did not see expressed anywhere so far, that is a sort of distribution action license versioning metadata. For example if Nokia have a GPL 2 “or later” package on a secure (crypto-locked) phone platform, in the “distributor chooses license version” reading when distributing they can specify “Nokia gives you this specifically under the GPL2, so no keys”. If the guy then distributes the package on, he can choose the terms afresh, but since he has no keys for the Nokia platform that has no impact on Nokia.

The distributor cannot make a GPL V2 “or later” distribution action stick when he gives it out under, say, GPL V3. If he says, here, have this GPL V2 “or later” package under GPL V3 only terms, the recipient need only redistribute it to himself or through a friend to have it validly on GPL V2 terms again. So it seems this distribution action license choice is only useful for one thing, getting the distributor out of distribution conditions found in later license versions, it seems it can’t be used as a forced upgrade off older license versions.

Simply by stressing ‘modify’ rather than ‘distribute’ from the and/or:

* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of
* the License, or (at your option) any later version.

the text can equally be read another very different way, which is that the distributed-to person is granted the right to modify the sources “under the terms of” the GPL version he chooses. Someone who makes this reading of the text can believe they had a right to apply for example GPL v3 duties on their distributor, and ask them for the keys needed to allow installation as specified by the current GPL3 draft[1]. Folks can argue the distributor saying (where? when subsequently challenged?) that he distributed on the terms of GPL v2 beats out this reading, but it’s not that clear to me because it was also the distributor himself that gave the recipient the license telling him he had the right to modify and distribute according to the terms in GPL V2 “or later”; if modification and installation according to GPL V3 needs the distributor’s keys the GPL V3 gives a way to demand it of the distributor. There are at the least mixed signals coming from the distributor in this case it seems to me.

These proposed considerations only apply to GPL V2 “or later” licenses, stuff like Linux which is GPL2-only seem to have no risk that GPL V3 provisions might be claimed as duties for the distributor.

[1] http://gplv3.fsf.org/gpl-draft-2006-07-27.html

”1. Source Code … The Corresponding Source also includes any encryption or authorization keys necessary to install and/or execute modified versions from source code…”

Posted in Software design | 1 Comment »

I’ll make you free if I have to lock you up!

Monday, September 18th, 2006

Interesting goings-on over on the Busybox list. Busybox is a single app that masquerades as a large set of common unix tools like ls, a shell and so on. The maintainer is planning to, well, sort of migrate the project to being GPL v2-only. It’s a bit complex because there are a variety of copyright notices floating around in there at the moment. He discussed it on the list for some time and although there are some people that want to have GPL2+ (meaning, GPL v2 or any later version at the user’s discretion) the proposal seemed to be gaining traction.

The issue is significant, because in the current GPL3 drafts there is language that would require any signing keys to be given up with the sources. If you plan to design a device which, for the security of the customer, would reject code, eg, updates, that were not signed by the manufacturer, then the GPL3 would appear to disallow using GPL2+ or GPL3-only licensed code with such a scheme.

Linus has already come out against this idea as one of the reasons he will be sticking with GPL v2 for Linux. But of course Linux is just one part of the puzzle, and Linux is fairly unusual in having changed the default language of the GPL v2 license from “version 2 or later” to fix it firmly with v2 only. There is a lot of code out there that may suddenly inherit this “I demand your crypto keys because I am treating your code as licensed under GPL v3″ problem simply because they left the default “v2 or later” in there.

Bruce Perens, who was in at the start of Busybox but left it many years ago, argued on the busybox mailing list against changing the license to GPL v2-only, but I think he is mistaken. I think GPL v2 “or later” licenses may turn out to be a very ugly can of worms. In any event the result was the maintainer a few hours later announced that busybox was going v2-only.

The problems over on busybox are the first sign for me of what may be a major licensing train crash brought on by the thoughtless handing over of the author’s licensing terms to Richard Stallman. Great man that he is, that is a lot of power he is channelling right now, he alone, through the FSF, can randomly dictate the licensing terms of a vast body of “GPL v2 or later” code that is currently in use. Under the banner of “increasing freedom”, he is in the position to disallow current usage of existing code that is currently used in accordance with the license. For an unlikely but illuminating example, if he decides that the GPL v4 requires the distributor to donate $10 to the FSF, then recipients who decide they will use “GPL v2 or later” code on the GPL v4 terms can force the distributor to do this. And this is on code that is currently used based on the fairly well understood GPL2 terms where there is nothing like that.

Now you may scoff at such a wild straw man argument, but I discover over the weekend there are people that believe that the GPL v2 requires you to give up any signing keys you may use on a binary created from it! In a subthread starting here, Rich Felker proposes the idea as fact and I argue against it. Later in the thread Rich insists he will take people to court if they fail to deliver such keys; I bring up Redhat’s own signing of GPL’d packages as a case where he should attack according to his principle. He deflects this by saying that since it is possible to install unsigned packages, he will not need to sue Redhat. However, yum by default will not install unsigned packages, and besides you cannot do so without the root password for the box. For many reasons, a user may not have the root password. Does Rich propose that everyone with a box with GPL v2 software on it must be given root access? There has been no reply.

Anyway, it is all getting a bit chilling this talk of negating the possibility of actions of users of free software in order to make them free. It’s starting to sound a little bit like the start of a tortured logic found in Socialist states, where the workers must build palaces “to be free”, grub around in the fields to feed people in offices “to be free”. Purely by that innocent sounding “or later” found in the default GPL2, a huge amount of power, proportional to the amount of software with “or later” in the license, has landed on one person, Richard Stallman. Was everyone really aware they had elected a Great Leader, no matter how trustworthy, and that their package was part of a mobilization force under orders from above?