Losing the magic

Losing the magic

The use of magic numbers is something that I learned from Multics. One advantage of structure magic numbers is that it also provides protection against use-after-free bugs, since you can zero the magic number before you free the structure, and even if you don't, when it gets reused, if everyone uses the magic number scheme where the first four bytes contain a magic number, then it becomes a super-cheap defense a certain class of bugs without needing to rely on things like KMSAN, which (a) is super-heavyweight and so won't be used on production kernels, and (b) didn't exist in the early days of Linux.

Like everything, it's a trade-off. Yes, there is overhead associated with magic numbers. But it's not a lot of overhead (and it's certainly cheaper than KMSAN!) and the ethos of "trying to eliminate an entire set of bugs" which is something is well accepted for making the kernel more secure, is someting that could be applied for magic numbers as well.

I still use magic numbers in e2fprogs, where the magic number is generated using the com_err library (another Multicism; where the top 24-bits identify the subsystem, and the low 8-bits is the error code for that subsystem). This means it's super easy to do things like this:

In lib/ext2fs/ext2fs.h:

#define EXT2_CHECK_MAGIC(struct, code) \
	  if ((struct)->magic != (code)) return (code)

In lib/ext2fs/ext2_err.et.in:

	error_table ext2

ec	EXT2_ET_BASE,
	"EXT2FS Library version @E2FSPROGS_VERSION@"

ec	EXT2_ET_MAGIC_EXT2FS_FILSYS,
	"Wrong magic number for ext2_filsys structure"

ec	EXT2_ET_MAGIC_BADBLOCKS_LIST,
	"Wrong magic number for badblocks_list structure"

The compile_et program generates ext2_err.h and ext2_err.c, for which ext2_err.h will have definitions like this:

#define EXT2_ET_BASE                             (2133571328L)
#define EXT2_ET_MAGIC_EXT2FS_FILSYS              (2133571329L)
#define EXT2_ET_MAGIC_BADBLOCKS_LIST             (2133571330L)
...

Then in various library functions:

errcode_t ext2fs_dir_iterate2(ext2_filsys fs,
			      ext2_ino_t dir,
...
{
	EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
        ...

And of course:

void ext2fs_free(ext2_filsys fs)
{
	if (!fs || (fs->magic != EXT2_ET_MAGIC_EXT2FS_FILSYS))
		return;
       ...
	fs->magic = 0;
	ext2fs_free_mem(&fs);
}

Callers of ext2fs library functions then will do things like this:

 	errcode_t    retval;

	retval = ext2fs_read_inode(fs, ino, &file->inode);
	if (retval)
		return retval;

		retval = ext2fs_read_bitmaps (fs);
		if (retval) {
			printf(_("\n%s: %s: error reading bitmaps: %s\n"),
			       program_name, device_name,
			       error_message(retval));
			exit(1);
		}

This scheme has absolutely found bugs, and given that there is a full set of regression tests that get run via "make check", I've definitely found that having this kind of software engineering practice increases developer velocity, and reduces my stress when I code since when I do make a mistake, it generally gets caught really quickly as a result.

Personally, I find this coding discipline easier to understand and write than Rust, and more performant than using things like valgrind and MSan. Of course, I use those tools too, but if I can catch bugs early, my experience is that it allows me to generate code much more quickly and reliably.

Shrug. Various programming styles go in and out of fashion. And structure magic numbers goes all the way back to the 1960's (Multics was developed as a joint project between MIT, GE, and Bell Labs starting in 1964).