vulns.xyz

2025 wrapped

Wed, 31 Dec 2025 00:00:00 +0000

Same as last year, this is a summary of what I’ve been up to throughout the year.

See also the recap/retrospection published by my friends (antiz, jvoisin, orhun).

Uploaded 467 packages to Arch Linux
- Most of them being reproducible, meaning I provably didn’t abuse my position of compiling the binaries
- 35 of them are signal-desktop
- 29 of them are metasploit
Made 53 uploads to Debian
- All of them being related to my work in the debian-rust team, that I’ve been a part of since 2018
- Also applied for Debian Developer status (with 4 Debian Developers advocating for me)
Made 14 commits in Alpine Linux’ aports
- 13 of them being package releases
Made 2 commits in NixOS’ nixpkgs
- Also joined their Github org
Made 4 commits in homebrew-core
- With special focus on polishing the Rust development experience for the RP2040 microcontroller
Lost Onion, my cat of 13 years, to inoperable cancer. He has been with me throughout my entire open source journey (sometimes being credited as co-author) and who looked after me for my entire adult life. You won’t be forgotten. 🐈‍⬛
Developed 6 hand-held games with embedded Rust, most of them being birthday gifts for people close to me
- game-taco-burglar
  - A motorcycling lockpicker
- game-antifa-syndikitty
  - A nurse with a secret double life
  - At that point the longest and most in-depth game I built throughout my life
- game-chop-chop
  - A French tetris-spinoff, this was one of my Fusion projects this year
  - The hardware was specifically designed to be easy to solder/make from readily available parts (~€5 per unit)
  - I gave away a few devices I made, some people successfully built one on their own
- game-ratatat
  - A space-invader like game about a very enthusiastic seamster
- game-octo-space-irs
  - As an employee of the intergalactic revenue service, you tax the rich through reversing and cracking computer programs
  - I gifted another copy to a Tor directory authority operator I’m friends with, who was very excited about the concept and levels I designed
- game-the-curse-of-the-headless-goose
  - A turn-based game about an underground kickboxing club
  - This one was meant to be a rogue-lite (which I needed the savegame library for), but only managed to build the introduction/tutorial unfortunately
Picked up work on apt-swarm again
- Replaced the old database code with a custom engine, reducing RAM usage from multiple gigabytes down to ~9MB
- Ran a small p2p network all over the world, with ~10-15 locations/countries on average
- As part of this, found a bug in tokio that could lead to silent data loss in some cases
Had 2 of my projects explicitly mentioned in the Debian release notes in their “What’s new in Debian 13” summary
Was mentioned in multiple academic papers on arxiv.org:
- Reproducible Builds and Insights from an Independent Verifier for Arch Linux (explicitly in the “Acknowledgments” section)
- Beneath the Mask: Can Contribution Data Unveil Malicious Personas in Open-Source Projects?
- Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack
- Causes and Canonicalization of Unreproducible Builds in Java
- Reproducible Builds for Quantum Computing (mentions rebuilderd)
Was referenced twice on LWN:
- Hash-based module integrity checking (mentions me directly)
- Fedora change aims for 99% package reproducibility (doesn’t mention me, but rebuilderd 10x)
Published a draft version of PlatypOS, an “Experimental toy unix-like userspace operating system with strong preference towards Rust”. As part of this:
- Developed custom pacman-database tooling in Rust instead of bash
- During this project I found and reported issues in uutils’ install (uutils/coreutils#8033) and mv (uutils/coreutils#8044) (both fixed shortly after)
- The project stalled because it’s too big to side-quest
Had the first ever CVE issued for software I wrote: CVE-2025-52926
- Found, reported and fixed by a c’t Open Source editor
Published 9 repositories related to my embedded Rust work
- embedded-mono-img for all the graphics in my games
- rp2040-psp-joystick to demo use of an analog joystick input
- rp2040-demo-st7789 to demo a higher resolution screen I experimented with
- rp2040-demo-w25qxx to demo how to store data in NOR flash
- rp2040-demo-at24cxx to demo how to store data in an EEPROM
- embedded-graphics-colorcast a library I developed so I can keep using embedded-mono-img on ST7789/ILI9486 screens - I used tinybmp in one project but it was fairly slow
- ch32v003-demo to demo and document the lowend ch32v003 RISC-V microcontroller, with devboards that are commonly sold for €0.50-0.70 on AliExpress (it’s cute but lacks the required 5.1kΩ resistors on the USB-C configuration pins that tell the host to provide 5V, so it won’t work with many USB-C chargers, which is quite annoying)
- embedded-savegame an atomic/transactional savegame library, with powerfail-safety and wear-leveling, optimized for flash and EEPROM storage
- djb2 a very lightweight non-cryptographic checksum algorithm that replaced my use of CRC32 in the embedded-savegame library, to make it more suitable for the ch32v003
Contributed to the Reproducible Builds mailing list 30 times
Developed repro-threshold, an integration for apt to act as a rebuilderd client, enforcing a reproducible builds trust policy of your choice
- The feature was suggested/requested by a CCC member during MiniDebConf Hamburg 2025
Collaborated with an openSUSE engineer I’ve known for several years to debug and fix an issue in gtk-rs that caused indeterministic build output for many desktop programs
Volunteered at a soldering workshop for beginners for the 4th year in a row
Completed the first year of volunteering in an awareness team
Wrote 1 blog post (besides this one)
Attended FOSDEM, MiniDebConf, Fusion, the Reproducible Builds summit, the Arch Summit and 39c3
- Hosted sessions at both FOSDEM (1st time) and Fusion (2nd time)
Grew and harvested 2 plants
Traveled to
- Denmark
- Sweden
- Turkey, visiting a good friend
- Belgium
- Austria
Made and printed
- 2 new sticker designs
- 2 new hoodie designs
Changed my medication plan
Got 4 new tattoos

Thanks to everybody who has been part of my human experience, past or present. Especially those who’ve been closest.

Release: rebuilderd v0.25.0

Thu, 25 Sep 2025 00:00:00 +0000

rebuilderd v0.25.0 was recently released, this version has improved in-toto support for cryptographic attestations that this blog post briefly outlines. 😺

As a quick recap, rebuilderd is an automatic build scheduler that emerged in 2019/2020 from the Reproducible Builds project doing the following:

Track binary packages available in a Linux distribution
Attempt to compile the official binary packages from their (alleged) source code
Check if the package we compiled is bit-for-bit identical
1. If so, mark it GOOD, issue an attestation
2. In every other case, mark it BAD, generate a diffoscope

The binary packages in question are explicitly the packages users would also fetch and install.

This project has caught the attention of Arch Linux, Debian and Fedora.

Before this version

The original in-toto integration was added 4 years ago by Joy Liu during GSoC 2021, with help from Santiago Torres and Aditya Sirish (shoutout to the real ones!). Each rebuilderd-worker had its own cryptographic key and included a signed attestation along with the build result that could then be fetched from /api/v0/builds/{id}/attestation.

Since these workers are potentially ephemeral, and the list of worker public keys wasn’t publicly known, it was difficult to make use of those signatures.

Since this version

This version introduces the following:

The rebuilderd daemon itself generates a long-term signing key
All attestations signed by a trusted worker also get signed by the rebuilderd daemon
The rebuilderd daemon gets a new endpoint that can be used to query the public-key this instance identifies with: /api/v0/public-keys

An example of this new endpoint can be found here:

https://reproducible.archlinux.org/api/v0/public-keys

The response looks something like this (this is the real long-term signing key used by reproducible.archlinux.org):

{
    "current": [
        "-----BEGIN PUBLIC KEY-----\r\nMCwwBwYDK2VwBQADIQBLNcEcgErQ1rZz9oIkUnzc3fPuqJEALr22rNbrBK7iqQ==\r\n-----END PUBLIC KEY-----\r\n"
    ]
}

It’s a list so keys can potentially be rolled over time, and in future versions it should also list the public keys the instance has used in the past.

I haven’t develop any integrations for this yet (partially also to allow deployments to catch up with the new release), but I’m planning to do so using the in-toto crate.

Closing words

To give credit where credit is due (and because people pointed out I tend to end my blog posts too abruptly), rebuilderd is only the scheduler software, the actual build in the correct build-environment is outsourced to external tooling like archlinux-repro and debrebuild.

For further reading on applied reproducible builds, see also my previous blogpost Disagreeing rebuilders and what that means.

Also, there are currently efforts by the European Commission to outlaw unregulated end-to-end encrypted chat, so this may be a good time to prepare for (potential) impact and check what tools you have available to reduce unchecked trust in (open source) software authorities, to keep them operating honest and accountable.

Never lose the plot~

Sincerely yours

2024 wrapped

Tue, 31 Dec 2024 00:00:00 +0000

Dear blog. This post is inspired by an old friend of mine who has been writing these for the past few years. I meant to do this for a while now, but ended up not preparing anything, so this post is me writing it from memory. There’s likely stuff I forgot, me being gentle with myself I’ll probably just permit myself to complete this list the next couple of days.

I hate bragging, I try to not depend on external validation as much as possible, and being the anarcho-communist anti-capitalist that I am, I try to be content with knowing I’m “doing good in the background”. I don’t think people owe me for the work I did, I don’t expect anything in return, and it’s my way of giving back to the community and the people around me. Consider us even.

That being said, I:

Uploaded 689 packages to Arch Linux
- Most of which being reproducible, meaning I provably didn’t abuse my position of compiling the binaries
- 59 of those are signal-desktop
- 34 of those are metasploit
Made 28 commits in Alpine Linux’ aports
- 24 of those being package releases
Made 43 uploads to Debian
- All of them being related to my work in the debian-rust team, that I’ve been a part of since 2018
Made 5 commits in NixOS’ nixpkgs
Made 1 commit in homebrew-core
Was one of the people involved in rolling out _FORTIFY_SOURCE=3 compiler hardening in Arch Linux, for the entire operating system. I wrote lists, tools, patches and my work got me quoted in an “Additional Considerations” section of the OpenSSF compiler hardening guide for C and C++. There are now more, stricter buffer-overflow checks at runtime that hopefully make your computer harder to exploit in 2025.
Was one of the people behind the launch of reproduce.debian.net which is analogous to reproducible.archlinux.org that I also helped create 5 years ago. Reproducing these packages (and allowing anybody else to do the same) proves the binaries have not been backdoored by the build server (or whoever compiled them), and if there’s a backdoor, you can likely find it in the source code.
Integrated librustls, a memory safe TLS implementation, into Arch Linux’ C dynamic linking ecosystem and became one of the authors of the rustls curl TLS backend
In response to the XZ Jia Tan incident I created whatsrc.org, a source code indexing project. It doesn’t solve anything in itself, but it’s framing the concept of source code inputs and how to reason about them in a way that I consider promising. It also documents and makes it very apparent what specifically is the source code we’re putting into our computers, that would benefit from code reviews.
Contributed to the Reproducible Builds mailing list 33 times
Volunteered at a soldering workshop for beginners for the 3rd year in a row, with people describing me as a good teacher, giving very calm vibes and having endless patience (qualities that I value deeply)
Reverse engineered the signal username and QR-code feature
Rewrote my tooling for apt.vulns.xyz to use repro-env, the .deb files can now be verified through reproducible builds, and I switched to static Rust binaries because I had trouble targeting multiple Debian/Ubuntu releases with my previous tooling
Wrote 0 blog posts (besides this one)
Wrote 5.937 messages in irc channels
Got mentioned 1.664 times on irc
Attended FOSDEM, Fusion, the Reproducible Builds summit, Hackjunta 2024#2 and 38c3
Made and printed 8 new sticker designs, and a custom hoodie
Mastered the art of pragmatic zaza cultivation and processing
Got 2 new piercings and 2-3 new tattoos (depending on how you count them)

Thanks to everybody who has been part of my human experience, past or present. Especially those who’ve been closest.

cheers,
kpcyrd ✨

Writing a Linux executable from scratch with x86_64-unknown-none and Rust

Tue, 28 Mar 2023 00:00:00 +0000

I recently mentioned on the internet I did work in this direction and a friend of mine asked me to write a blogpost on this. I didn’t blog for a long time (keeping all the goodness for myself hehe), so here we go. 🦝 To set the scene, let’s assume we want to make an exectuable binary for x86_64 Linux that’s supposed to be extremely portable. It should work on both Debian and Arch Linux. It should work on systems without glibc like Alpine Linux. It should even work in a FROM scratch Docker container. In a more serious setting you would statically link musl-libc with your Rust program, but today we’re in a silly-goofy mood so we’re going to try to make this work without a libc. And we’re also going to use Rust for this, more specifically the stable release channel of Rust, so this blog post won’t use any nightly-only features that might still change/break. If you’re using a Rust 1.0 version that was recent at the time of writing or later (>= 1.68.0 according to my computer), you should be able to try this at home just fine™.

This tutorial assumes you have no prior programming experience in any programming language, but it’s going to involve some x86_64 assembly. If you already know what a syscall is, you’ll be just fine. If this is your first exposure to programming you might still be able to follow along, but it might be a wild ride.

If you haven’t already, install rustup ^{(possibly also available in your package manager, who knows?)}

# when asked, press enter to confirm default settings
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

This is going to install everything you need to use Rust on Linux (this tutorial assumes you’re following along on Linux btw). Usually it’s still using a system linker (by calling the cc binary, and errors out if none is present), but instead we’re going to use rustup to install an additional target:

rustup target add x86_64-unknown-none

I don’t know if/how this is made available by Linux distributions, so I recommend following along with rust installed from rustup.

Anyway, we’re creating a new project with cargo, this creates a new directory that we can then change into (you might’ve done this before):

cargo new hack-the-planet
cd hack-the-planet

There’s going to be a file named Cargo.toml, we don’t need to make any changes there, but the one that was auto-generated for me at the time of writing looks like this:

[package]
name = "hack-the-planet"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

There’s a second file named src/main.rs, it’s going to contain some pre-generated hello world, but we’re going to delete it and create a new, empty file:

rm src/main.rs
touch src/main.rs

Alrighty, leaving this file empty is not valid but we’re going to walk through the individual steps so we’re going to try to build with an empty file first. At this point I would like to credit this chapter of a fasterthanli.me series and a blogpost by Philipp Oppermann, this tutorial is merely an 2023 update and makes it work with stable Rust. Let’s run the build:

$ cargo build --release --target x86_64-unknown-none
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-none` target may not support the standard library
  = note: `std` is required by `hack_the_planet` because it does not declare `#![no_std]`

error[E0601]: `main` function not found in crate `hack_the_planet`
  |
  = note: consider adding a `main` function to `src/main.rs`

Some errors have detailed explanations: E0463, E0601.
For more information about an error, try `rustc --explain E0463`.
error: could not compile `hack-the-planet` due to 2 previous errors

Since this doesn’t use a libc (oh right, I forgot to mention this up to this point actually), this also means there’s no std standard library. Usually the standard library of Rust still uses the system libc to do syscalls, but since we specify our libc as none this means std won’t be available (use std::fs::rename won’t work). There are still other functions we can use and import, for example there’s core that’s effectively a second standard library, but much smaller.

To opt-out of the std standard library, we can put #![no_std] into src/main.rs:

#![no_std]

Running the build again:

$ cargo build --release --target x86_64-unknown-none
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error[E0601]: `main` function not found in crate `hack_the_planet`
 --> src/main.rs:1:11
  |
1 | #![no_std]
  |           ^ consider adding a `main` function to `src/main.rs`

For more information about this error, try `rustc --explain E0601`.
error: could not compile `hack-the-planet` due to previous error

Rust noticed we didn’t define a main function and suggest we add one. This isn’t what we want though so we’ll politely decline and inform Rust we don’t have a main and it shouldn’t attempt to call it. We’re adding #![no_main] to our file and src/main.rs now looks like this:

#![no_std]
#![no_main]

Running the build again:

$ cargo build
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error: `#[panic_handler]` function required, but not found

error: language item required, but not found: `eh_personality`
  |
  = note: this can occur when a binary crate with `#![no_std]` is compiled for a target where `eh_personality` is defined in the standard library
  = help: you may be able to compile for a target that doesn't need `eh_personality`, specify a target with `--target` or in `.cargo/config`

error: could not compile `hack-the-planet` due to 2 previous errors

Rust is asking us for a panic handler, basically “I’m going to jump to this address if something goes terribly wrong and execute whatever you put there”. Eventually we would put some code there to just exit the program, but for now an infinitely loop will do. This is likely going to get stripped away anyway by the compiler if it notices our program has no code-branches leading to a panic and the code is unused. Our src/main.rs now looks like this:

#![no_std]
#![no_main]

use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

Running the build again:

$ cargo build --release --target x86_64-unknown-none
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
    Finished release [optimized] target(s) in 0.16s

Neat, it worked! What happens if we run it? 👀

$ target/x86_64-unknown-none/release/hack-the-planet
Segmentation fault (core dumped)

Oops. Let’s try to disassemble it:

$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64

Ok that looks pretty “from scratch” to me. The file contains no cpu instructions. Also note how our infinity loop is not present (as predicted).

Making a basic program and executing it

Ok let’s try to make a valid program that basically just cleanly exits. First let’s try to add some cpu instructions and verify they’re indeed getting executed. Lemme introduce, the ✨ INT 3 ✨ instruction in x86_64 assembly. In binary it’s also known as the 0xCC opcode. It crashes our program in a slightly different way, so if the error message changes, we know it worked. The other tutorials use a #[naked] function for the entry point, but since this feature isn’t stabilized at the time of writing we’re going to use the global_asm! macro. Also don’t worry, I’m not going to introduce every assembly instruction individually. Our program now looks like this:

#![no_std]
#![no_main]

use core::arch::global_asm;
use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

global_asm! {
    ".global _start",
    "_start:",
    "int 3"
}

Running the build again (ok basically from now on the build is always going to be expected to work unless I say otherwise):

$ cargo build --release --target x86_64-unknown-none
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
    Finished release [optimized] target(s) in 0.11s

Let’s try to disassemble the binary again:

$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64

Disassembly of section .text:

0000000000001210 <_start>:
    1210:	cc                   	int3

And sure enough, there’s a cc instruction that was identified as int3. Let’s try to run this:

$ target/x86_64-unknown-none/release/hack-the-planet
Trace/breakpoint trap (core dumped)

The error message of the crash is now slightly different because it’s hitting our breakpoint cpu instruction. Funfact btw, if you run this in strace you can see this isn’t making any system calls (aka not talking to the kernel at all, it just crashes):

$ strace -f ./hack-the-planet
execve("./hack-the-planet", ["./hack-the-planet"], 0x74f12430d1d8 /* 39 vars */) = 0
--- SIGTRAP {si_signo=SIGTRAP, si_code=SI_KERNEL, si_addr=NULL} ---
+++ killed by SIGTRAP (core dumped) +++
[1]    2796457 trace trap (core dumped)  strace -f ./hack-the-planet

Let’s try to make a program that does a clean shutdown. To do this we inform the kernel with a system call that we may like to exit. We can get more info on this with man 2 exit and it defines exit like this:

[[noreturn]] void _exit(int status);

On Linux this syscall is actually called _exit and exit is implemented as a libc function, but we don’t care about any of that today, it’s going to do the job just fine. Also note how it takes a single argument of type int. In C-speak this means “signed 32 bit”, i32 in Rust.

Next we need to figure out the “syscall number” of this syscall. These numbers are cpu architecture specific for some reason (idk, idc). We’re looking these numbers up with ripgrep in /usr/include/asm/:

$ rg __NR_exit /usr/include/asm
/usr/include/asm/unistd_64.h
64:#define __NR_exit 60
235:#define __NR_exit_group 231

/usr/include/asm/unistd_x32.h
53:#define __NR_exit (__X32_SYSCALL_BIT + 60)
206:#define __NR_exit_group (__X32_SYSCALL_BIT + 231)

/usr/include/asm/unistd_32.h
5:#define __NR_exit 1
253:#define __NR_exit_group 252

Since we’re on x86_64 the correct value is the one in unistd_64.h, 60. Also, on x86_64 the syscall number goes into the rax cpu register, the status argument goes in the rdi register. The return value of the syscall is going to be placed in the rax register after the syscall is done, but for exit the execution is never given back to us. Let’s try to write 60 into the rax register and 69 into the rdi register. To copy into registers we’re going to use the mov destination, source instruction to copy from source to destination. With these registers setup we can use the syscall cpu instruction to hand execution over to the kernel. Don’t worry, there’s only one more assembly instruction coming and for everything else we’re going to use Rust.

Our code now looks like this:

#![no_std]
#![no_main]

use core::arch::global_asm;
use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

global_asm! {
    ".global _start",
    "_start:",
    "mov rax, 60",
    "mov rdi, 69",
    "syscall"
}

Build the binary, run it and print the exit code:

$ cargo build --release --target x86_64-unknown-none
$ target/x86_64-unknown-none/release/hack-the-planet; echo $?
69

Nice. Rust is quite literally putting these cpu instructions into the binary for us, nothing else.

$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64


Disassembly of section .text:

0000000000001210 <_start>:
    1210:	48 c7 c0 3c 00 00 00 	mov    $0x3c,%rax
    1217:	48 c7 c7 45 00 00 00 	mov    $0x45,%rdi
    121e:	0f 05                	syscall

Running this with strace shows the program does exactly one thing.

$ strace -f ./hack-the-planet
execve("./hack-the-planet", ["./hack-the-planet"], 0x70699fe8c908 /* 39 vars */) = 0
exit(69)                                = ?
+++ exited with 69 +++

Writing Rust

Ok but even though cpu instructions can be fun at times, I’d rather not deal with them most of the time (this might strike you as odd, considering this blog post). Instead let’s try to define a function in Rust and call into that instead. We’re going to define this function as unsafe (btw none of this is taking advantage of the safety guarantees by Rust in case it wasn’t obvious. This tutorial is mostly going to stick to unsafe Rust, but for bigger projects you can attempt to reduce your usage of unsafe to opt back into “normal” safe Rust), it also declares the function with #[no_mangle] so the function name is preserved as main and we can call it from our global_asm entry point. Lastely, when our program is started it’s going to get the stack address passed in one of the cpu registers, this value is expected to be passed to our function as an argument. Our function declares ! as return type, which means it never returns:

#[no_mangle]
unsafe fn main(_stack_top: *const u8) -> ! {
    // TODO: this is missing
}

This won’t compile yet, we need to add our assembly for the exit syscall back in.

#[no_mangle]
unsafe fn main(_stack_top: *const u8) -> ! {
    asm!(
        "syscall",
        in("rax") 60,
        in("rdi") 0,
        options(noreturn)
    );
}

This time we’re using the asm! macro, this is a slightly more declarative approach. We want to run the syscall cpu instruction with 60 in the rax register, and this time we want the rdi register to be zero, to indicate a successful exit. We also use options(noreturn) so Rust knows it should assume execution does not resume after this assembly is executed (the Linux kernel guarantees this). We modify our global_asm! entrypoint to call our new main function, and to copy the stack address from rsp into the register for the first argument rdi because it would otherwise get lost forever:

global_asm! {
    ".global _start",
    "_start:",
    "mov rdi, rsp",
    "call main"
}

Our full program now looks like this:

#![no_std]
#![no_main]

use core::arch::asm;
use core::arch::global_asm;
use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

global_asm! {
    ".global _start",
    "_start:",
    "mov rdi, rsp",
    "call main"
}

#[no_mangle]
unsafe fn main(_stack_top: *const u8) -> ! {
    asm!(
        "syscall",
        in("rax") 60,
        in("rdi") 0,
        options(noreturn)
    );
}

After building and disassembling this the Rust compiler is slowly starting to do work for us:

$ cargo build --release --target x86_64-unknown-none
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64


Disassembly of section .text:

0000000000001210 <_start>:
    1210:	48 89 e7             	mov    %rsp,%rdi
    1213:	e8 08 00 00 00       	call   1220 <main>
    1218:	cc                   	int3
    1219:	cc                   	int3
    121a:	cc                   	int3
    121b:	cc                   	int3
    121c:	cc                   	int3
    121d:	cc                   	int3
    121e:	cc                   	int3
    121f:	cc                   	int3

0000000000001220 <main>:
    1220:	50                   	push   %rax
    1221:	b8 3c 00 00 00       	mov    $0x3c,%eax
    1226:	31 ff                	xor    %edi,%edi
    1228:	0f 05                	syscall
    122a:	0f 0b                	ud2

The mov and syscall instructions are still the same, but it noticed it can XOR the rdi register with itself to set it to zero. It’s using x86 assembly language (the 32 bit variant of x86_64, that also happens to work on x86_64) to do so, that’s why the register is refered to as edi in the disassembly. You can also see it’s inserting a bunch of 0xCC instructions (for alignment) and Rust puts the opcodes 0x0F 0x0B at the end of the function to force an “invalid opcode exception” so the program is guaranteed to crash in case the exit syscall doesn’t do it.

This code still executes as expected:

$ strace -f ./hack-the-planet
execve("./hack-the-planet", ["./hack-the-planet"], 0x72dae7e5dc08 /* 39 vars */) = 0
exit(0)                                 = ?
+++ exited with 0 +++

Adding functions

Ok we’re getting closer but we aren’t quite there yet. Let’s try to write an exit function for our assembly that we can then call like a normal function. Remember that it takes a signed 32 bit integer that’s supposed to go into rdi.

unsafe fn exit(status: i32) -> ! {
    asm!(
        "syscall",
        in("rax") 60,
        in("rdi") status,
        options(noreturn)
    );
}

Actually, since this function doesn’t take any raw pointers and any i32 is valid for this syscall we’re going to remove the unsafe marker of this function. When doing this we still need to use unsafe { } within the function for our inline assembly.

fn exit(status: i32) -> ! {
    unsafe {
        asm!(
            "syscall",
            in("rax") 60,
            in("rdi") status,
            options(noreturn)
        );
    }
}

Let’s call this function from our main, and also remove the infinity loop of the panic handler with a call to exit(1):

#![no_std]
#![no_main]

use core::arch::asm;
use core::arch::global_asm;
use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    exit(1);
}

global_asm! {
    ".global _start",
    "_start:",
    "mov rdi, rsp",
    "call main"
}

fn exit(status: i32) -> ! {
    unsafe {
        asm!(
            "syscall",
            in("rax") 60,
            in("rdi") status,
            options(noreturn)
        );
    }
}

#[no_mangle]
unsafe fn main(_stack_top: *const u8) -> ! {
    exit(0);
}

Running this still works, but interestingly the generated assembly didn’t change at all:

$ cargo build --release --target x86_64-unknown-none
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64


Disassembly of section .text:

0000000000001210 <_start>:
    1210:	48 89 e7             	mov    %rsp,%rdi
    1213:	e8 08 00 00 00       	call   1220 <main>
    1218:	cc                   	int3
    1219:	cc                   	int3
    121a:	cc                   	int3
    121b:	cc                   	int3
    121c:	cc                   	int3
    121d:	cc                   	int3
    121e:	cc                   	int3
    121f:	cc                   	int3

0000000000001220 <main>:
    1220:	50                   	push   %rax
    1221:	b8 3c 00 00 00       	mov    $0x3c,%eax
    1226:	31 ff                	xor    %edi,%edi
    1228:	0f 05                	syscall
    122a:	0f 0b                	ud2

Rust noticed there’s no need to make it a separate function at runtime and instead merged the instructions of the exit function directly into our main. It also noticed the 0 argument in exit(0) means “rdi is supposed to be zero” and uses the XOR optimization mentioned before.

Since main is not calling any unsafe functions anymore we could mark it as safe too, but in the next few functions we’re going to deal with file descriptors and raw pointers, so this is likely the only safe function we’re going to write in this tutorial so let’s just keep the unsafe marker.

Printing text

Ok let’s try to do a quick hello world, to do this we’re going to call the write syscall. Looking it up with man 2 write:

ssize_t write(int fd, const void buf[.count], size_t count);

The write syscall takes 3 arguments and returns a signed size_t. In Rust this is called isize. In C size_t is an unsigned integer type that can hold any value of sizeof(...) for the given platform, ssize_t can only store half of that because it uses one of the bits to indicate an error has occured (the first s means signed, write returns -1 in case of an error).

The arguments for write are:

the file descriptor to write to. stdout is located on file descriptor 1.
a pointer/address to some memory.
the number of bytes that should be written, starting at the given address.

Let’s also lookup the syscall number of write:

% rg __NR_write /usr/include/asm
/usr/include/asm/unistd_64.h
5:#define __NR_write 1
24:#define __NR_writev 20

/usr/include/asm/unistd_32.h
8:#define __NR_write 4
150:#define __NR_writev 146

/usr/include/asm/unistd_x32.h
5:#define __NR_write (__X32_SYSCALL_BIT + 1)
323:#define __NR_writev (__X32_SYSCALL_BIT + 516)

The value we’re looking for is 1. Let’s write our write function (heh).

unsafe fn write(fd: i32, buf: *const u8, count: usize) -> isize {
    let r0;
    asm!(
        "syscall",
        inlateout("rax") 1 => r0,
        in("rdi") fd,
        in("rsi") buf,
        in("rdx") count,
        lateout("rcx") _,
        lateout("r11") _,
        options(nostack, preserves_flags)
    );
    r0
}

Now that’s a lot of stuff at once. Since this syscall is actually going to hand execution back to our program we need to let Rust know which cpu registers the syscall is writing to, so Rust doesn’t attempt to use them to store data (that would be silently overwritten by the syscall). inlateout("raw") 1 => r0 means we’re writing a value to the register and want the result back in variable r0. in("rdi") fd means we want to write the value of fd into the rdi register. lateout("rcx") _ means the Linux kernel may write to that register (so the previous value may get lost), but we don’t want to store the value anywhere (the underscore acts as a dummy variable name).

This doesn’t compile just yet though

$ cargo build --release --target x86_64-unknown-none
   Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error: incompatible types for asm inout argument
  --> src/main.rs:35:26
   |
35 |         inlateout("rax") 1 => r0,
   |                          ^    ^^ type `isize`
   |                          |
   |                          type `i32`
   |
   = note: asm inout arguments must have the same type, unless they are both pointers or integers of the same size

error: could not compile `hack-the-planet` due to previous error

Rust has inferred the type of r0 is isize since that’s what our function returns, but the type of the input value for the register was inferred to be i32. We’re going to select a specific number type to fix this.

unsafe fn write(fd: i32, buf: *const u8, count: usize) -> isize {
    let r0;
    asm!(
        "syscall",
        inlateout("rax") 1isize => r0,
        in("rdi") fd,
        in("rsi") buf,
        in("rdx") count,
        lateout("rcx") _,
        lateout("r11") _,
        options(nostack, preserves_flags)
    );
    r0
}

We can now call our new write function like this:

write(1, b"Hello world\n".as_ptr(), 12);

We need to set the number of bytes we want to write explicitly because there’s no concept of null-byte termination in the write system call, it’s quite literally “write the next X bytes, starting from this address”. Our program now looks like this:

#![no_std]
#![no_main]

use core::arch::asm;
use core::arch::global_asm;
use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    exit(1);
}

global_asm! {
    ".global _start",
    "_start:",
    "mov rdi, rsp",
    "call main"
}

fn exit(status: i32) -> ! {
    unsafe {
        asm!(
            "syscall",
            in("rax") 60,
            in("rdi") status,
            options(noreturn)
        );
    }
}

unsafe fn write(fd: i32, buf: *const u8, count: usize) -> isize {
    let r0;
    asm!(
        "syscall",
        inlateout("rax") 1isize => r0,
        in("rdi") fd,
        in("rsi") buf,
        in("rdx") count,
        lateout("rcx") _,
        lateout("r11") _,
        options(nostack, preserves_flags)
    );
    r0
}

#[no_mangle]
unsafe fn main(_stack_top: *const u8) -> ! {
    write(1, b"Hello world\n".as_ptr(), 12);
    exit(0);
}

Let’s try to build and disassemble it:

$ cargo build --release --target x86_64-unknown-none
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet

target/x86_64-unknown-none/release/hack-the-planet:     file format elf64-x86-64


Disassembly of section .text:

0000000000001220 <_start>:
    1220:	48 89 e7             	mov    %rsp,%rdi
    1223:	e8 08 00 00 00       	call   1230 <main>
    1228:	cc                   	int3
    1229:	cc                   	int3
    122a:	cc                   	int3
    122b:	cc                   	int3
    122c:	cc                   	int3
    122d:	cc                   	int3
    122e:	cc                   	int3
    122f:	cc                   	int3

0000000000001230 <main>:
    1230:	50                   	push   %rax
    1231:	48 8d 35 d5 ef ff ff 	lea    -0x102b(%rip),%rsi        # 20d <_start-0x1013>
    1238:	b8 01 00 00 00       	mov    $0x1,%eax
    123d:	ba 0c 00 00 00       	mov    $0xc,%edx
    1242:	bf 01 00 00 00       	mov    $0x1,%edi
    1247:	0f 05                	syscall
    1249:	b8 3c 00 00 00       	mov    $0x3c,%eax
    124e:	31 ff                	xor    %edi,%edi
    1250:	0f 05                	syscall
    1252:	0f 0b                	ud2

This time there are 2 syscalls, first write, then exit. For write it’s setting up the 3 arguments in our cpu registers (rdi, rsi, rdx). The lea instruction subtracts 0x102b from the rip register (the instruction pointer) and places the result in the rsi register. This is effectively saying “an address relative to wherever this code was loaded into memory”. The instruction pointer is going to point directly behind the opcodes of the lea instruction, so 0x1238 - 0x102b = 0x20d. This address is also pointed out in the disassembly as a comment.

We don’t see the string in our disassembly but we can convert our 0x20d hex to 525 in decimal and use dd to read 12 bytes from that offset, and sure enough:

$ dd bs=1 skip=525 count=12 if=target/x86_64-unknown-none/release/hack-the-planet
Hello world
12+0 records in
12+0 records out

Execute our binary with strace also shows the new write syscall (and the bytes that are being written mixed up in the output).

$ strace -f ./hack-the-planet
execve("./hack-the-planet", ["./hack-the-planet"], 0x74493abe64a8 /* 39 vars */) = 0
write(1, "Hello world\n", 12Hello world
)           = 12
exit(0)                                 = ?
+++ exited with 0 +++

After running strip on it to remove some symbols the binary is so small, if you open it in a text editor it fits on a screenshot:

updlockfiles: Manage dependency lockfiles in PKGBUILDs for upstreams that don't ship them

Sun, 16 Oct 2022 00:00:00 +0000

I’ve released a new tool to manage dependency lockfiles for Arch Linux packages that can’t use a lockfile from the official upstream release. It integrates closely with other Arch Linux tooling like updpkgsums that’s already used to pin the content of build inputs in PKGBUILD.

To use this, the downstream lockfile becomes an additional source input in the source= array of our PKGBUILD (this is already the case for some packages).

source=("git+https://github.com/vimeo/psalm.git#commit=${_commit}"
        "composer.lock")

You would then add a new function named updlockfiles that can generate new lockfiles and copies them into $outdir, and a prepare function to copy the lockfile in the right place:

 prepare() {
   cd ${pkgname}
   cp ../composer.lock .
}

updlockfiles() {
  cd ${pkgname}
  rm -f composer.lock
  composer update
  cp composer.lock "${outdir}/"
}

To update the package to the latest (compatible) patch level simply run:

updlockfiles

This can also be used in case upstreams lockfile has vulnerable dependencies that you want to patch downstream. For more detailed instructions see the readme.

Thanks

This work is currently crowd-funded on github sponsors. I’d like to thank @SantiagoTorres, @repi and @rgacogne for their support in particular. ♥️

auth-tarball-from-git: Verifying tarballs with signed git tags

Sat, 28 May 2022 00:00:00 +0000

I noticed there’s a common anti-pattern in some PKGBUILDs, the short scripts that are used to build Arch Linux packages. Specifically we’re looking at the part that references the source code used when building a package:

source=("git+https://github.com/alacritty/alacritty.git#tag=v${pkgver}?signed")
validpgpkeys=('4DAA67A9EA8B91FCC15B699C85CDAE3C164BA7B4'
              'A56EF308A9F1256C25ACA3807EA8F8B94622A6A9')
sha256sums=('SKIP')

This does:

✅ authentication: verify the git tag was signed by one of the two trusted keys.
❌ pinning: the source code is not pinned and git tags are not immutable, upstream could create a new signed git tag with an identical name and arbitrarily change the source code without the PKGBUILD noticing.

In contrast consider this PKGBUILD:

source=($pkgname-$pkgver.tar.gz::https://github.com/alacritty/alacritty/archive/refs/tags/v$pkgver.tar.gz)
sha256sums=('e48d4b10762c2707bb17fd8f89bd98f0dcccc450d223cade706fdd9cfaefb308')

❌ authentication: there’s no signature, so this PKGBUILD doesn’t cryptographically verify authorship.
✅ pinning: the source code is cryptographically pinned, it’s addressing the source code by it’s content. There’s nothing upstream can do to silently change the code used by this PKGBUILD.

Personally - if I had to decide between these two - I’d prefer the later because I can always try to authenticate the pinned tarball later on, but it’s impossible to know for sure which source code has been used if all I know is “something that had a valid signature on it”. This set could be infinitely large for all we know!

But is there a way to get both? Consider this PKGBUILD:

makedepends=('auth-tarball-from-git')
source=($pkgname-$pkgver.tar.gz::https://github.com/alacritty/alacritty/archive/refs/tags/v$pkgver.tar.gz
        chrisduerr.pgp
        kchibisov.pgp)
sha256sums=('e48d4b10762c2707bb17fd8f89bd98f0dcccc450d223cade706fdd9cfaefb308'
            '19573dc0ba7a2f003377dc49986867f749235ecb45fe15eb923a74b2ab421d74'
            '5b866e6cb791c58cba2e7fc60f647588699b08abc2ad6b18ba82470f0fd3db3b')

prepare() {
  cd "$pkgname-$pkgver"

  auth-tarball-from-git --keyring ../chrisduerr.pgp --keyring ../kchibisov.pgp \
    --tag v$pkgver --prefix $pkgname-$pkgver \
    https://github.com/alacritty/alacritty.git ../$pkgname-$pkgver.tar.gz
}

✅ authentication: auth-tarball-from-git verifies the signed git tag and then verifies the tarball has been built from the commit the tag is pointing to.
✅ pinning: the source code and the files containing the public keys are cryptographically pinned.

In this case sha256sums= is the primary line of defense against tampering with build inputs and the git tag is “only” used to document authorship.

For more infos on how this works you can have a look at the auth-tarball-from-git repo, there’s also a section about attacks on signed git tags that you should probably know about.

Thanks

This work is currently crowd-funded on github sponsors. I’d like to thank @SantiagoTorres, @repi and @rgacogne for their support in particular. ♥️

Reproducible Builds: Debian and the case of the missing version string

Wed, 19 Jan 2022 00:00:00 +0000

If you’ve been following my twitter recently you probably noticed there’s now a rebuilderd based Debian rebuilder run by the Purdue Trustworthy Software Ecosystems Lab. The rebuilder backend - the code that’s actually re-creating the build environment and running the build - is debrebuild.py, written by Frédéric Pierret from the QubesOS project. The setup as a whole automatically monitors packages in Debian unstable, then downloads the source code, build-dependencies and attempts to compile a bit-for-bit identical binary package. If this succeeds, the package is marked as “reproducible”.

The 62.89% reproducible number is currently significantly lower than the 94.6% reproducible number¹ reported at tests.reproducible-builds.org/debian/. This blogpost is diving into why that is and why there are different challenges in “rebuilding” done in this setup vs “build environment fuzzing”² done by tests.reproducible-builds.org.

This is partially due to complexity about versions, let’s look into how other distros are approaching reproducible builds first:

Arch Linux

pkg version -> [ svntogit-*.git ] +-> source code
   |                              `-> build instructions
   |
   |
   `--> [ archive.archlinux.org ] -> *.pkg.tar.zst -> .BUILDINFO -> build dependencies (exact versions)

To build a reproducible binary we need a canonical (source code, build instructions, build dependencies with exact versions) combination that both the original build and all verification rebuilds are using.

In Arch Linux this information is spread out over multiple places (which is fine). There’s a git repository that can be used to locate the PKGBUILD, it links to the source code (pinned by cryptographic checksums) and contains the build instructions, but the PKGBUILD only specifies the build dependencies by name, not by version. We can’t use this alone for reproducible builds because different compiler versions are almost guaranteed to result in different binaries (unreproducible packages caused by mirror drift over time).

The “resolved dependencies” (aka “the build dependencies with exact versions”) are effectively tracked on archive.archlinux.org inside the packages. The (name, version) tuple of a package is required to be unique/canonical within Arch Linux, so we can use the package archive to resolve a pkg version to a canonical binary package that contains the buildinfo file. This file, together with the source code and build instructions we already had, gets us our required (source code, build instructions, build dependencies with exact versions) combination.

Neat!

Debian

pkg version -> [ source packages ] +-> source code
   |                               `-> build instructions
   |
   |
   `--> [ buildinfos.debian.net ] -> build dependencies (exact versions)

This looks very similar to what Arch Linux does (slightly simpler, even). Every binary package is built out of something called a “source package”, which contains both the source code, the build instructions and the dependencies (also by name, with no version).

The resolved dependencies are tracked on buildinfos.debian.net³. Using the extra info in the buildinfo file we get our canonical (souce code, build instructions, build dependencies with exact versions) combination and are able to implement a rebuilder.

Debian version strings (the happy path)

We still need to locate the buildinfo file though. Let’s look at the happy-path first using a package called “sniffglue”:

The Packages.xz is the index that’s used by apt to determine which packages are available in Debian. It contains entries like this:

Package: sniffglue
Source: rust-sniffglue
Version: 0.14.0-2
Installed-Size: 2344
Maintainer: Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net>
Architecture: amd64
Depends: libc6 (>= 2.32), libgcc-s1 (>= 4.2), libpcap0.8 (>= 1.5.1), libseccomp2 (>= 0.0.0~20120605)
Description: Secure multithreaded packet sniffer
Multi-Arch: allowed
Built-Using: rust-nix (= 0.23.0-1), rust-pktparse (= 0.5.0-1), rust-seccomp-sys (= 0.1.3-1), rustc (= 1.56.0+dfsg1-2)
Description-md5: e7f1183e49341488d3bd8fbe63b63f37
X-Cargo-Built-Using: rust-aho-corasick (= 0.7.10-1), rust-ansi-term (= 0.12.1-1), rust-anyhow (= 1.0.44-2), rust-arrayvec (= 0.5.1-1), rust-atty (= 0.2.14-2), rust-base64 (= 0.13.0-1), rust-bitflags (= 1.2.1-1), rust-block-buffer (= 0.9.0-4), rust-block-padding (= 0.2.1-1), rust-bstr (= 0.2.17-1), rust-byteorder (= 1.4.3-2), rust-cfg-if-0.1 (= 0.1.10-2), rust-cfg-if (= 1.0.0-1), rust-clap (= 2.33.3-1), rust-cpuid-bool (= 0.1.2-4), rust-dhcp4r (= 0.2.0-1), rust-digest (= 0.9.0-1), rust-dirs-next (= 2.0.0-1), rust-dirs-sys-next (= 0.1.1-1), rust-dns-parser (= 0.8.0-1), rust-enum-primitive (= 0.1.1-1), rust-env-logger (= 0.9.0-1), rust-generic-array (= 0.14.4-1), rust-humantime (= 2.1.0-1), rust-itoa (= 0.4.3-1), rust-lazy-static (= 1.4.0-1), rust-lexical-core (= 0.4.8-3), rust-libc (= 0.2.103-1), rust-log (= 0.4.11-2), rust-memchr (= 2.4.1-1), rust-memoffset (= 0.6.4-1), rust-nix (= 0.23.0-1), rust-nom (= 5.0.1-4), rust-num-cpus (= 1.13.0-1), rust-num-traits (= 0.2.14-1), rust-opaque-debug (= 0.3.0-1), rust-pcap-sys (= 0.1.3-2), rust-phf (= 0.8.0-2), rust-phf-shared (= 0.8.0-1), rust-pktparse (= 0.5.0-1), rust-quick-error (= 1.2.3-1), rust-reduce (= 0.1.1-1), rust-regex-automata (= 0.1.8-2), rust-regex (= 1.5.4-1), rust-regex-syntax (= 0.6.25-1), rust-rusticata-macros (= 2.0.4-1), rust-ryu (= 1.0.2-1), rust-seccomp-sys (= 0.1.3-1), rust-serde (= 1.0.130-2), rust-serde-json (= 1.0.41-1), rust-sha2 (= 0.9.2-2), rust-siphasher (= 0.3.1-1), rust-static-assertions (= 1.1.0-1), rust-strsim (= 0.9.3-1), rust-structopt (= 0.3.20-1), rust-strum (= 0.19.2-1), rust-syscallz (= 0.15.0-1), rust-termcolor (= 1.1.0-1), rust-textwrap (= 0.11.0-1), rust-time (= 0.1.42-1), rust-tls-parser (= 0.9.2-3), rust-toml (= 0.5.8-1), rust-typenum (= 1.12.0-1), rust-unicode-width (= 0.1.8-1), rust-users (= 0.11.0-1), rust-vec-map (= 0.8.1-2), rustc (= 1.56.0+dfsg1-2)
Section: net
Priority: optional
Filename: pool/main/r/rust-sniffglue/sniffglue_0.14.0-2_amd64.deb
Size: 732980
MD5sum: 177f9229266ad5eef3fb42fff0c07345
SHA256: 448c781a9e594227bc9f0d6c65b8beba2b3add68d3583020de188d4cfa365b40

Package: - this is the name of the binary package as used by apt.
Source: - this is the name of the source package. There can be multiple source packages with the same name in Sources.xz at the same time, if that’s the case this line may contain a version string in parentheses.
Version: - this is the version of the binary package as used by apt.
Architecture: - this is the architecture this package was built for.
Filename: - this value is used by apt to generate a url in order to download the .deb

The Sources.xz is an index of all source packages, it’s only relevant if you want to compile Debian packages yourself. It contains entries like this:

Package: rust-sniffglue
Binary: librust-sniffglue-dev, sniffglue
Version: 0.14.0-2
Maintainer: Debian Rust Maintainers <pkg-rust-maintainers@alioth-lists.debian.net>
Uploaders: kpcyrd <git@rxv.cc>
Build-Depends: debhelper (>= 12), dh-cargo (>= 25), cargo:native, rustc:native, libstd-rust-dev, librust-ansi-term-0.12+default-dev, librust-anyhow-1+default-dev, librust-atty-0.2+default-dev, librust-base64-0.13+default-dev, librust-bstr-0.2+default-dev (>= 0.2.12-~~), librust-dhcp4r-0.2+default-dev, librust-dirs-next-2+default-dev, librust-dns-parser-0.8+default-dev, librust-env-logger-0.9+default-dev, librust-libc-0.2+default-dev, librust-log-0.4+default-dev, librust-nix-0.23+default-dev, librust-nom-5+default-dev, librust-num-cpus-1+default-dev (>= 1.6-~~), librust-pcap-sys-0.1+default-dev (>= 0.1.3-~~), librust-pktparse-0.5+default-dev, librust-pktparse-0.5+serde-dev, librust-reduce-0.1+default-dev (>= 0.1.1-~~), librust-serde-1+default-dev, librust-serde-derive-1+default-dev, librust-serde-json-1+default-dev, librust-sha2-0.9+default-dev, librust-structopt-0.3+default-dev, librust-syscallz-0.15+default-dev, librust-tls-parser-0.9+default-dev, librust-toml-0.5+default-dev, librust-users-0.11+default-dev
Architecture: any
Standards-Version: 4.5.1
Format: 3.0 (quilt)
Files:
 cdb6bf3fb7a8725986b313a65dd3339b 3079 rust-sniffglue_0.14.0-2.dsc
 80a7d2ab6becacf69213d9ce57d29274 134805 rust-sniffglue_0.14.0.orig.tar.gz
 9902c121e4f5cf268b19ee8b88201979 7816 rust-sniffglue_0.14.0-2.debian.tar.xz
Vcs-Browser: https://salsa.debian.org/rust-team/debcargo-conf/tree/master/src/sniffglue
Vcs-Git: https://salsa.debian.org/rust-team/debcargo-conf.git [src/sniffglue]
Checksums-Sha256:
 b9a77f9f918769ecded338c07a344257b17d1112918b1597a8c939a719444ea4 3079 rust-sniffglue_0.14.0-2.dsc
 f056bfa09e8fae5f4cc0e1d4e8ae3619050644b321800d0d6a8cc778eb80aaf3 134805 rust-sniffglue_0.14.0.orig.tar.gz
 cb3498dd85e18e7b2c7ad5cbef2ac56e4c598df0a0ac5024aa480f97de79b096 7816 rust-sniffglue_0.14.0-2.debian.tar.xz
Package-List: 
 librust-sniffglue-dev deb net optional arch=any
 sniffglue deb net optional arch=any
Testsuite: autopkgtest
Testsuite-Triggers: dh-cargo
Directory: pool/main/r/rust-sniffglue
Priority: extra
Section: misc

Package: - this is the name of the source package (this is matched with Source: from the previous file).
Binary: - this is the list of binary packages that are built out of this source package, in this case our source package results in two binary packages when built.
Version:- this is the version of our source package, here it’s identical to the version of the binary package but this isn’t always the case, more on that later.

Ok now using all of these together we try to locate the buildinfo file like this:

"https://buildinfos.debian.net/buildinfo-${Source:Directory}/${Source:Package}_${Source:Version}_${Binary:Architecture}.buildinfo"

This results in⁴:

https://buildinfos.debian.net/buildinfo-pool/r/rust-sniffglue/rust-sniffglue_0.14.0-2_amd64.buildinfo

We try to open the link and get something that looks like this:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.0
Source: rust-sniffglue
Binary: librust-sniffglue-dev sniffglue sniffglue-dbgsym
Architecture: amd64
Version: 0.14.0-2
Checksums-Md5:
 6d57946f2f56b1b58d906eb29d6a021f 125404 librust-sniffglue-dev_0.14.0-2_amd64.deb
 20fd28a9824a2c485d2e9b155b0a073c 8995232 sniffglue-dbgsym_0.14.0-2_amd64.deb
 177f9229266ad5eef3fb42fff0c07345 732980 sniffglue_0.14.0-2_amd64.deb
Checksums-Sha1:
 9fdde2501245e7db1eb2eaa104922b1b2b05fe57 125404 librust-sniffglue-dev_0.14.0-2_amd64.deb
 0dc99414756c846aadee63484642958c1ca7ff60 8995232 sniffglue-dbgsym_0.14.0-2_amd64.deb
 d2e4f34a46527effd3375764463d2c1bbe3eeecc 732980 sniffglue_0.14.0-2_amd64.deb
Checksums-Sha256:
 c452054c216359ef44adc9a5d35870d707f47e503051dfcb736f47df17058961 125404 librust-sniffglue-dev_0.14.0-2_amd64.deb
 214817662f43ec4ae0766dd23700a694c45985cb03d28fe82a791a61202e0705 8995232 sniffglue-dbgsym_0.14.0-2_amd64.deb
 448c781a9e594227bc9f0d6c65b8beba2b3add68d3583020de188d4cfa365b40 732980 sniffglue_0.14.0-2_amd64.deb
Build-Origin: Debian
Build-Architecture: amd64
Build-Date: Mon, 06 Dec 2021 21:35:27 +0000
Build-Path: /build/rust-sniffglue-1KDXF6/rust-sniffglue-0.14.0
Installed-Build-Depends:
 autoconf (= 2.71-2),
 automake (= 1:1.16.5-1.1),
 autopoint (= 0.21-4),
 autotools-dev (= 20180224.1+nmu1),
 base-files (= 12),
 base-passwd (= 3.5.52),
 bash (= 5.1-5),
 binutils (= 2.37-10),
 binutils-common (= 2.37-10),
 binutils-x86-64-linux-gnu (= 2.37-10),
 bsdextrautils (= 2.37.2-4),
 bsdutils (= 1:2.37.2-4),
 build-essential (= 12.9),
 bzip2 (= 1.0.8-5),
 cargo (= 0.57.0-3),
 coreutils (= 8.32-4.1),
[...]

This is the full list of dependencies with exact versions we need!

Debian version strings (the not-so-happy path)

Let’s look into a different package next, one that’s slightly quirky. This package is going to be mariadb-server, which is currently on version 1:10.6.5-2.

First of all, there are multiple entries starting with Package: mariadb-server but when filtering by version we get the one we’re looking for:

Package: mariadb-server
Source: mariadb-10.6
Version: 1:10.6.5-2
Installed-Size: 66
Maintainer: Debian MySQL Maintainers <pkg-mysql-maint@lists.alioth.debian.org>
Architecture: all
Depends: mariadb-server-10.6 (>= 1:10.6.5-2)
Description: MariaDB database server (metapackage depending on the latest version)
Homepage: https://mariadb.org/
Description-md5: 810926b6e0e35edd4901d8603857b214
Tag: devel::lang:c++, devel::lang:sql, devel::library, implemented-in::c++,
 interface::commandline, interface::daemon, network::server,
 protocol::db:mysql, role::devel-lib, role::metapackage, role::program,
 works-with::db
Section: database
Priority: optional
Filename: pool/main/m/mariadb-10.6/mariadb-server_10.6.5-2_all.deb
Size: 34360
MD5sum: e32900fc549e3be67c98578e185a2eb9
SHA256: 4a37a29f28b7e8e81f9d432acb96265258e9dd5e24af8026a4c8f234d1485334

This is referencing source package mariadb-10.6, let’s try to find that next:

Package: mariadb-10.6
Binary: libmariadb-dev, libmariadb-dev-compat, libmariadb3, libmariadbd19, libmariadbd-dev, mariadb-common, mariadb-client-core-10.6, mariadb-client-10.6, mariadb-server-core-10.6, mariadb-server-10.6, mariadb-server, mariadb-client, mariadb-backup, mariadb-plugin-connect, mariadb-plugin-s3, mariadb-plugin-rocksdb, mariadb-plugin-oqgraph, mariadb-plugin-mroonga, mariadb-plugin-spider, mariadb-plugin-gssapi-server, mariadb-plugin-gssapi-client, mariadb-plugin-cracklib-password-check, mariadb-test, mariadb-test-data
Version: 1:10.6.5-2
Maintainer: Debian MySQL Maintainers <pkg-mysql-maint@lists.alioth.debian.org>
Uploaders: Otto Kekäläinen <otto@debian.org>
Build-Depends: bison, cmake, cracklib-runtime <!nocheck>, debhelper (>= 10), dh-exec, gdb <!nocheck>, libaio-dev [!linux-any], libboost-dev, libcrack2-dev (>= 2.9.0), libcurl4-openssl-dev | libcurl4-dev, libedit-dev, libedit-dev:native, libjemalloc-dev [linux-any], libjudy-dev, libkrb5-dev, liblz4-dev, libncurses5-dev (>= 5.0-6~), libncurses5-dev:native (>= 5.0-6~), libnuma-dev [linux-any], libpam0g-dev, libpcre2-dev, libpmem-dev [amd64 arm64 ppc64el], libsnappy-dev, libssl-dev, libssl-dev:native, libsystemd-dev [linux-any], liburing-dev [linux-any], libxml2-dev, libzstd-dev (>= 1.3.3), lsb-release, perl:any, po-debconf, psmisc, unixodbc-dev, uuid-dev, zlib1g-dev (>= 1:1.1.3-5~)
Architecture: any all
Standards-Version: 4.5.0
Format: 3.0 (quilt)
Files:
 6ee9e176b9b8b380b5de85af20720083 4573 mariadb-10.6_10.6.5-2.dsc
 3049b2c7f83f5e99eab9fb871c7743cc 81853489 mariadb-10.6_10.6.5.orig.tar.gz
 7b80c984b8ec0e1d6a7ea4791087bb59 221456 mariadb-10.6_10.6.5-2.debian.tar.xz
Vcs-Browser: https://salsa.debian.org/mariadb-team/mariadb-server
Vcs-Git: https://salsa.debian.org/mariadb-team/mariadb-server.git
Checksums-Sha256:
 114e04d6916437218fd995deb90af216b20c387c6e22136af16964c96c4e3ee0 4573 mariadb-10.6_10.6.5-2.dsc
 0831debda6ff6f2942d756732a9e5886ef2c5526ad360119502ae3e03b13e013 81853489 mariadb-10.6_10.6.5.orig.tar.gz
 6c33c0e43f6e24d07af754b960c8b0e7473bfa7c6bece7488125cd13eb260206 221456 mariadb-10.6_10.6.5-2.debian.tar.xz
Homepage: https://mariadb.org/
Package-List: 
 libmariadb-dev deb libdevel optional arch=any
 libmariadb-dev-compat deb libdevel optional arch=any
 libmariadb3 deb libs optional arch=any
 libmariadbd-dev deb libdevel optional arch=any
 libmariadbd19 deb libs optional arch=any
 mariadb-backup deb database optional arch=any
 mariadb-client deb database optional arch=all
 mariadb-client-10.6 deb database optional arch=any
 mariadb-client-core-10.6 deb database optional arch=any
 mariadb-common deb database optional arch=all
 mariadb-plugin-connect deb database optional arch=any
 mariadb-plugin-cracklib-password-check deb database optional arch=any
 mariadb-plugin-gssapi-client deb database optional arch=any
 mariadb-plugin-gssapi-server deb database optional arch=any
 mariadb-plugin-mroonga deb database optional arch=any-alpha,any-amd64,any-arm,any-arm64,any-i386,any-ia64,any-mips64el,any-mips64r6el,any-mipsel,any-mipsr6el,any-nios2,any-powerpcel,any-ppc64el,any-sh3,any-sh4,any-tilegx
 mariadb-plugin-oqgraph deb database optional arch=any
 mariadb-plugin-rocksdb deb database optional arch=amd64,arm64,mips64el,ppc64el
 mariadb-plugin-s3 deb database optional arch=any
 mariadb-plugin-spider deb database optional arch=any
 mariadb-server deb database optional arch=all
 mariadb-server-10.6 deb database optional arch=any
 mariadb-server-core-10.6 deb database optional arch=any
 mariadb-test deb database optional arch=any
 mariadb-test-data deb database optional arch=all
Testsuite: autopkgtest
Testsuite-Triggers: eatmydata
Directory: pool/main/m/mariadb-10.6
Priority: optional
Section: misc

The version is 1:10.6.5-2 in both cases, but this doesn’t check out with the .buildinfo file url. Using our previous scheme we’d get:

https://buildinfos.debian.net/buildinfo-pool/m/mariadb-10.6/mariadb-10.6_1:10.6.5-2_all.buildinfo

This link 404’s, the correct url is:

https://buildinfos.debian.net/buildinfo-pool/m/mariadb-10.6/mariadb-10.6_10.6.5-2_all.buildinfo

Note the missing 1: in the version. Unfortunately there’s no field containing just 10.6.5-2 so there’s no clean way to get this value. Whatever, we just cut off everything before the : and call it a day, the link is now working correctly:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.0
Source: mariadb-10.6
Binary: mariadb-client mariadb-common mariadb-server mariadb-test-data
Architecture: all
Version: 1:10.6.5-2
Checksums-Md5:
 f06a4f7d1a760131b2e94d88d011e4a4 34244 mariadb-client_10.6.5-2_all.deb
 5863a24052e165e68565b5e310e3051d 35768 mariadb-common_10.6.5-2_all.deb
 e32900fc549e3be67c98578e185a2eb9 34360 mariadb-server_10.6.5-2_all.deb
 f8ab5df076102d28fafb4b3c4c62396a 17958780 mariadb-test-data_10.6.5-2_all.deb
Checksums-Sha1:
 c472cef1dc83f77d5f267d3f48e34b6b7518f52e 34244 mariadb-client_10.6.5-2_all.deb
 338b14e8e96e2e261bf4d1f669b86a5321c5ea06 35768 mariadb-common_10.6.5-2_all.deb
 629f8c52395f3e59ebbfa095059277d06f541eaa 34360 mariadb-server_10.6.5-2_all.deb
 bacd0f09f0d465133b213bd992320b3af78fb5c4 17958780 mariadb-test-data_10.6.5-2_all.deb
Checksums-Sha256:
 6c198e06111c1de8389ca798bd24d712b8c2776ad8b7c97052cba6d2b8a8f675 34244 mariadb-client_10.6.5-2_all.deb
 d2f17ff78af73dea232a6890b12becaaf9292100a515f63935d370df0c2515ff 35768 mariadb-common_10.6.5-2_all.deb
 4a37a29f28b7e8e81f9d432acb96265258e9dd5e24af8026a4c8f234d1485334 34360 mariadb-server_10.6.5-2_all.deb
 f603d7f330f4e8988e899e5e0ab85ff7eb68caf1967b8a7d6d0a4c8c91c5a06b 17958780 mariadb-test-data_10.6.5-2_all.deb
Build-Origin: Debian
Build-Architecture: amd64
Build-Date: Wed, 15 Dec 2021 05:41:50 +0000
Build-Path: /build/mariadb-10.6-kGWtxs/mariadb-10.6-10.6.5
Installed-Build-Depends:
 autoconf (= 2.71-2),
 automake (= 1:1.16.5-1.1),
 autopoint (= 0.21-4),
 autotools-dev (= 20180224.1+nmu1),
[...]

The .deb file is also using this truncated version, but the Packages.xz has an entry that contains a full path for every .deb so we don’t really worry about that:

Filename: pool/main/m/mariadb-10.6/mariadb-server_10.6.5-2_all.deb

Debian version strings (the miserable path)

I didn’t really mention this yet, but there are essentially three version strings so far:

The “binary package version”
The “source package version”
The “source package version but without the epoch”

The first two are fully independent from one-another and can have completely different values (although they didn’t in the examples so far). But there’s a 4th one you possibly never heard of, even if you’re fairly familiar with Debian:

The binNMU version⁵.

But what’s a binNMU? It stands for “binary non-maintainer-upload” and it’s essentially recycling a past source upload and building new binary packages out of it in a more recent build environment. This is useful if you want to re-resolve the dependencies in the source package to more recent dependencies that have been uploaded in the meantime, for example if you want to compile again with a more recent compiler. In other distros like Arch Linux this would be done with a pkgrel bump, although a pkgrel bump also allows you to change the build instructions. I don’t know why it’s implemented like this in Debian, but it’s potentially been like that for a really long time⁶.

Let’s look into the courier-imap package next. It’s built out of the courier source package (the source package version is 1.0.16-3):

Package: courier
Binary: courier-base, courier-mlm, courier-mta, courier-faxmail, courier-webadmin, sqwebmail, courier-pcp, courier-pop, courier-imap, courier-ldap, courier-doc
Version: 1.0.16-3
Maintainer: Markus Wanner <markus@bluegap.ch>
Build-Depends: automake, courier-authlib-dev (>= 0.66.4-5~), debhelper-compat (= 13), default-libmysqlclient-dev, dh-exec, dh-apache2, expect, ghostscript, gnupg2, gnutls-bin, groff-base, libcourier-unicode-dev (>= 2.1-3~), libgamin-dev, libgcrypt-dev, libgdbm-dev | libgdbmg1-dev, libgnutls28-dev, libidn11-dev, libldap2-dev, libpam0g-dev, libpcre3-dev, libperl-dev, libpq-dev, libsasl2-dev | libsasl-dev, libtool-bin | libtool, mgetty-fax, mime-support, netpbm, po-debconf, procps, wget, zlib1g-dev
Build-Conflicts: automake1.4
Architecture: any all
Standards-Version: 4.5.1
Format: 3.0 (quilt)
Files:
 dc395743184bd43c6bef647e29907439 3874 courier_1.0.16-3.dsc
 25f1c97a9ee74b7b264402b52d424ea9 7644196 courier_1.0.16.orig.tar.bz2
 f421e270aa2bb0eef69883b8dfc67661 866 courier_1.0.16.orig.tar.bz2.asc
 d82c935aa59897dc40adf85d8cc951b0 108396 courier_1.0.16-3.debian.tar.xz
Vcs-Browser: https://salsa.debian.org/debian/courier
Vcs-Git: https://salsa.debian.org/debian/courier.git
Checksums-Sha256:
 dd89bad1059adfba65b6f06be895b97c7d3d28d4f177d6a4f055407d374ef683 3874 courier_1.0.16-3.dsc
 87fc35ddff4f273aa04f43fdffc73f9236abf39bc3234a449eab88742d885ebb 7644196 courier_1.0.16.orig.tar.bz2
 e2d574353654d2a3e473d481a1354f2d8eb6412e77277a489d6545ef41e6122d 866 courier_1.0.16.orig.tar.bz2.asc
 565912449f530457892ccee787585184d644bac76f7055698d7f41600308519f 108396 courier_1.0.16-3.debian.tar.xz
Homepage: http://www.courier-mta.org/
Package-List: 
 courier-base deb mail optional arch=any
 courier-doc deb doc optional arch=all
 courier-faxmail deb mail optional arch=any
 courier-imap deb mail optional arch=any
 courier-ldap deb mail optional arch=any
 courier-mlm deb mail optional arch=any
 courier-mta deb mail optional arch=any
 courier-pcp deb mail optional arch=any
 courier-pop deb mail optional arch=any
 courier-webadmin deb mail optional arch=any
 sqwebmail deb mail optional arch=any
Testsuite: autopkgtest
Testsuite-Triggers: default-mta
Directory: pool/main/c/courier
Priority: source
Section: mail

But the version of the binary package is sorta wild (5.0.13+1.0.16-3+b1):

Package: courier-imap
Source: courier (1.0.16-3)
Version: 5.0.13+1.0.16-3+b1
Installed-Size: 593
Maintainer: Markus Wanner <markus@bluegap.ch>
Architecture: amd64
Replaces: courier-imap-ssl (<< 4.16.2+0.75.0-1~), imap-server
Provides: imap-server
Depends: courier-base (= 1.0.16-3+b1), debconf | debconf-2.0, gamin, default-mta | mail-transport-agent, sysvinit-utils (>= 2.88dsf-50) | init-d-script, courier-authlib (>= 0.71), libc6 (>= 2.15), libcourier-unicode4 (>= 2.1.2), libgamin0 | libfam0, libgdbm6 (>= 1.16), libidn12 (>= 1.13)
Suggests: courier-doc, imap-client
Conflicts: imap-server
Breaks: courier-imap-ssl (<< 4.16.2+0.75.0-1~)
Description: Courier mail server - IMAP server
Homepage: http://www.courier-mta.org/
Description-md5: aedad44242f18297b70663ef077f0e63
Tag: interface::daemon, mail::imap, network::server, network::service,
 protocol::imap, role::program, works-with::mail
Section: mail
Priority: optional
Filename: pool/main/c/courier/courier-imap_5.0.13+1.0.16-3+b1_amd64.deb
Size: 277952
MD5sum: f8eb4b0c42f191581189b2aee4c31793
SHA256: 67acfd8593f6c0a12a2681906e43dfe2872a694bd74e5438726646ec0e2af0a6

Let’s naively use our scheme again, so the buildinfo link we get is:

https://buildinfos.debian.net/buildinfo-pool/c/courier/courier_1.0.16-3_amd64.buildinfo

This link works, but hold on:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.0
Source: courier
Binary: courier-base courier-base-dbgsym courier-faxmail courier-imap courier-imap-dbgsym courier-ldap courier-ldap-dbgsym courier-mlm courier-mlm-dbgsym courier-mta courier-mta-dbgsym courier-pcp courier-pcp-dbgsym courier-pop courier-pop-dbgsym courier-webadmin courier-webadmin-dbgsym sqwebmail sqwebmail-dbgsym
Architecture: amd64
Version: 1.0.16-3
Checksums-Md5:
 15a8c74a2a05ae71c433e40693c7fe6e 460316 courier-base-dbgsym_1.0.16-3_amd64.deb
 b0570504e8e76072260e4b954b0d2c1e 327972 courier-base_1.0.16-3_amd64.deb
 d0d6cedf35b6555cd3e22be5dc9845fa 137208 courier-faxmail_1.0.16-3_amd64.deb
 5712d505d26c9d121ab7f233df825af6 505112 courier-imap-dbgsym_5.0.13+1.0.16-3_amd64.deb
 a099c775cf8801b00106974307c9fb9d 277672 courier-imap_5.0.13+1.0.16-3_amd64.deb
 3df62f2c9c29843cbb22699316fac2d8 31956 courier-ldap-dbgsym_1.0.16-3_amd64.deb
 56de5a9a609e1c098abe717279fe81c2 142028 courier-ldap_1.0.16-3_amd64.deb
 e1f1e737ca49da98fe1121ccaf331d07 2975104 courier-mlm-dbgsym_1.0.16-3_amd64.deb
 a80662978d886ae2feee732e3c0f8abd 390368 courier-mlm_1.0.16-3_amd64.deb
 4f81470483ceafdeb7a9d06971baa923 3235056 courier-mta-dbgsym_1.0.16-3_amd64.deb
 76fe544615420d64092b335c9dc1a965 634624 courier-mta_1.0.16-3_amd64.deb
 f55197352abe8bd6aa39f1e8252e9159 148224 courier-pcp-dbgsym_1.0.16-3_amd64.deb
 f922ee7b466cdbf7c24e953b5ff44c2d 169028 courier-pcp_1.0.16-3_amd64.deb
 620698dda8139f0d8012e59f89d5e3d0 135076 courier-pop-dbgsym_1.0.16-3_amd64.deb
 701684c8508858dd26e6fce133737c54 179620 courier-pop_1.0.16-3_amd64.deb
 6d96808e88e071ce2da86185d1e87f5a 4076 courier-webadmin-dbgsym_1.0.16-3_amd64.deb
 5c45649f63422f17c5a19187c2bacbac 147616 courier-webadmin_1.0.16-3_amd64.deb
 4073e99b9e047d263256a1a07428c596 1009160 sqwebmail-dbgsym_6.0.5+1.0.16-3_amd64.deb
 55319a910101a4742bebe7476bb93258 496464 sqwebmail_6.0.5+1.0.16-3_amd64.deb
[...]

It’s for the wrong package! This one describes the build environment for courier-imap_5.0.13+1.0.16-3_amd64.deb. The correct buildinfo file is:

https://buildinfos.debian.net/buildinfo-pool/c/courier/courier_1.0.16-3+b1_amd64.buildinfo

This link shows the correct buildinfo file for courier-imap_5.0.13+1.0.16-3+b1_amd64.deb (note the +b1):

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.0
Source: courier (1.0.16-3)
Binary: courier-base courier-base-dbgsym courier-faxmail courier-imap courier-imap-dbgsym courier-ldap courier-ldap-dbgsym courier-mlm courier-mlm-dbgsym courier-mta courier-mta-dbgsym courier-pcp courier-pcp-dbgsym courier-pop courier-pop-dbgsym courier-webadmin courier-webadmin-dbgsym sqwebmail sqwebmail-dbgsym
Architecture: amd64
Version: 1.0.16-3+b1
Binary-Only-Changes:
 courier (1.0.16-3+b1) sid; urgency=low, binary-only=yes
 .
   * Binary-only non-maintainer upload for amd64; no source changes.
   * Rebuild against libidn12
 .
  -- amd64 / i386 Build Daemon (x86-ubc-01) <buildd_amd64-x86-ubc-01@buildd.debian.org>  Sun, 22 Aug 2021 22:12:19 +0000
Checksums-Md5:
 224dacec94be1775639ef61ff01efea9 459456 courier-base-dbgsym_1.0.16-3+b1_amd64.deb
 06d0be1d5211f0d425803633806663a5 328148 courier-base_1.0.16-3+b1_amd64.deb
 24f9a41c6380ea2ca7559ce9e6f6f54a 137492 courier-faxmail_1.0.16-3+b1_amd64.deb
 56f61b3dd8a9b855a688bf10fe982e9a 504672 courier-imap-dbgsym_5.0.13+1.0.16-3+b1_amd64.deb
 f8eb4b0c42f191581189b2aee4c31793 277952 courier-imap_5.0.13+1.0.16-3+b1_amd64.deb
 56ab0d8f58360e7f9e9cdd1e550ef4aa 31896 courier-ldap-dbgsym_1.0.16-3+b1_amd64.deb
 fda5c8d7f98633778de1beb7197d257c 142284 courier-ldap_1.0.16-3+b1_amd64.deb
 786868f86bedc808f60f185b7e1ca2a1 2975168 courier-mlm-dbgsym_1.0.16-3+b1_amd64.deb
 4f8cc681fee4364d1a89e118c16bb5a7 390264 courier-mlm_1.0.16-3+b1_amd64.deb
 bab1bb694fe552a42472b2b0c9cf39bb 3233060 courier-mta-dbgsym_1.0.16-3+b1_amd64.deb
 788546e04c005d285cc084c1dea6b2ab 634364 courier-mta_1.0.16-3+b1_amd64.deb
 1e94cc602065e1df6b0a57212da60ffc 148160 courier-pcp-dbgsym_1.0.16-3+b1_amd64.deb
 16d6b5a9f33b6da3e8cbe0fb52105aa5 169328 courier-pcp_1.0.16-3+b1_amd64.deb
 97c64a4c770f9b613ac9eca381ddf162 134468 courier-pop-dbgsym_1.0.16-3+b1_amd64.deb
 97bbb2d0a27f34d1b48482cfd003b081 179872 courier-pop_1.0.16-3+b1_amd64.deb
 04031603ee4c00f5d972bd7b61c89bc3 3992 courier-webadmin-dbgsym_1.0.16-3+b1_amd64.deb
 146605f7e774c98d94da493cc3c19c4e 147916 courier-webadmin_1.0.16-3+b1_amd64.deb
 594fb250bfd24cd9b570cd8881ab88df 1008928 sqwebmail-dbgsym_6.0.5+1.0.16-3+b1_amd64.deb
 5fa070104394a53254b244b16685508e 496944 sqwebmail_6.0.5+1.0.16-3+b1_amd64.deb
Checksums-Sha1:
 5c9481e660beb92d12a315e1eee3174ed0024bcf 459456 courier-base-dbgsym_1.0.16-3+b1_amd64.deb
 47f73e25e38bcc966bd9db4f01b20a89cacdac38 328148 courier-base_1.0.16-3+b1_amd64.deb
 a7df6d382b73c34bb22d14cca2c6228d5a4ef0e0 137492 courier-faxmail_1.0.16-3+b1_amd64.deb
 da99a63aa0653506414a167b93cd55b801b1a45c 504672 courier-imap-dbgsym_5.0.13+1.0.16-3+b1_amd64.deb
 e9ac59293bcad55ae1a8e884510633f0b2387fc7 277952 courier-imap_5.0.13+1.0.16-3+b1_amd64.deb
 cae9c8e7ec94c448844a29531e5c6ad0a3dc0527 31896 courier-ldap-dbgsym_1.0.16-3+b1_amd64.deb
 1c4550c6dcca151a94b5bd23e859d5ec325d84f9 142284 courier-ldap_1.0.16-3+b1_amd64.deb
 5a0f79081975dc7fd91b0a84664b6a32639ba509 2975168 courier-mlm-dbgsym_1.0.16-3+b1_amd64.deb
 1faa6b2ba5e02e349cbb9bbb0f5c18bd196722da 390264 courier-mlm_1.0.16-3+b1_amd64.deb
 b9a1eb39e916317db01a9b5df760066736b8db51 3233060 courier-mta-dbgsym_1.0.16-3+b1_amd64.deb
 0e1b9802303b48bab4b2f0377c40b18b0464991a 634364 courier-mta_1.0.16-3+b1_amd64.deb
 38cfe238d668a31a24356af200380dd96160c38e 148160 courier-pcp-dbgsym_1.0.16-3+b1_amd64.deb
 31f33ecfd6b8f59a1826c9b695bdd2f3bf37bdd5 169328 courier-pcp_1.0.16-3+b1_amd64.deb
 f701fe7cf62ca401161a92353bebf9d3892611e7 134468 courier-pop-dbgsym_1.0.16-3+b1_amd64.deb
 2ab1cb53c878452b40d9b266bd41b5fbe5a23c87 179872 courier-pop_1.0.16-3+b1_amd64.deb
 f0a7436e84b642562f50659aa8c0b772a934077f 3992 courier-webadmin-dbgsym_1.0.16-3+b1_amd64.deb
 a00ddf85da6ad81ba1db25317779165f273bd6c4 147916 courier-webadmin_1.0.16-3+b1_amd64.deb
 14288aedb222350809795ca82502c9e36078b82b 1008928 sqwebmail-dbgsym_6.0.5+1.0.16-3+b1_amd64.deb
 34d298eaa1065ad5cff91bdec577cb7a91142654 496944 sqwebmail_6.0.5+1.0.16-3+b1_amd64.deb
Checksums-Sha256:
 786673efeab0460e94d1a1e60f36aa8732bb4535ae85746dee471a6db49fb676 459456 courier-base-dbgsym_1.0.16-3+b1_amd64.deb
 688b7c11b8ec92514929d37e207681e4b9ac754db9e8cf0ab0632374433eed7e 328148 courier-base_1.0.16-3+b1_amd64.deb
 6cb78e731f845dd98ab792c43fc01a6dc3416140b08d2db00e7415eff5973527 137492 courier-faxmail_1.0.16-3+b1_amd64.deb
 c90cc58b7c957b90120b606f39422315bca05293defacc1d4cf42ce4aa178128 504672 courier-imap-dbgsym_5.0.13+1.0.16-3+b1_amd64.deb
 67acfd8593f6c0a12a2681906e43dfe2872a694bd74e5438726646ec0e2af0a6 277952 courier-imap_5.0.13+1.0.16-3+b1_amd64.deb
 a146803c918d1160ca4681bd62b3bc61e14f15bed0cd483ae66916e7b279c57a 31896 courier-ldap-dbgsym_1.0.16-3+b1_amd64.deb
 733c1e1f620b416fb107e115fc8fa3cbe511d1e47ce1e171e1ffb8b5b1cecd05 142284 courier-ldap_1.0.16-3+b1_amd64.deb
 aaf96264630a4526c7aff2502a8015c0d409ca6e772213d7b9014cce3f1ecb63 2975168 courier-mlm-dbgsym_1.0.16-3+b1_amd64.deb
 209018400fd2dfa4caf4ede13a8865a110a9dec1c387e8eb9e45e2cd0550b771 390264 courier-mlm_1.0.16-3+b1_amd64.deb
 5a513509987478410e297f99be9d0d644f999928c7bc9ad1e854ad25473eeee0 3233060 courier-mta-dbgsym_1.0.16-3+b1_amd64.deb
 a05a38e8aa0986067b40d1f82dbf59732c6f4697ea91d8717ef0ea88b388ae6a 634364 courier-mta_1.0.16-3+b1_amd64.deb
 254defd40412e5f6a09d4ca4784b07255f617d2a1bb2ed1f7deac6d67a1b9771 148160 courier-pcp-dbgsym_1.0.16-3+b1_amd64.deb
 7b118060d17983ff0a070825819a5b6ac46bad0262288e5f63c7bd0baefcf72d 169328 courier-pcp_1.0.16-3+b1_amd64.deb
 655cd79a41636291ed2eb4be416828d14d7d44c55cebeb139ddce65d33390dcf 134468 courier-pop-dbgsym_1.0.16-3+b1_amd64.deb
 efc865b63f19efda4feb15917789b9390667d973ac18d7aa6e641290a7ba8461 179872 courier-pop_1.0.16-3+b1_amd64.deb
 e7f7abf143674c7ac1fe5a6c0c21ac2d9505c3ade37625e7623a6d13ccc6e45f 3992 courier-webadmin-dbgsym_1.0.16-3+b1_amd64.deb
 bc65ecb8eac668c0c5fe18c6784217dd67541d2347899e0096b4b2f9f2ab0059 147916 courier-webadmin_1.0.16-3+b1_amd64.deb
 e48e7fb6a38b6dd2057f61db875368cf5d175bc5b012c5b88eba8f4c15b4321e 1008928 sqwebmail-dbgsym_6.0.5+1.0.16-3+b1_amd64.deb
 51ebf109a5257b34a521a68967db5b328d2301ad9e7585ccf797dfb549ed5e6e 496944 sqwebmail_6.0.5+1.0.16-3+b1_amd64.deb
Build-Origin: Debian
Build-Architecture: amd64
Build-Date: Sun, 22 Aug 2021 22:24:21 +0000
Build-Path: /build/courier-DEOrho/courier-1.0.16
Installed-Build-Depends:
 adduser (= 3.118),
 apache2-dev (= 2.4.48-4),
 autoconf (= 2.69-14),
 automake (= 1:1.16.4-1),
 autopoint (= 0.21-4),
 autotools-dev (= 20180224.1+nmu1),
 base-files (= 11.1),
 base-passwd (= 3.5.51),
 bash (= 5.1-3+b1),
 binutils (= 2.37-4),
 binutils-common (= 2.37-4),
[...]

So the binnmu-version we’re looking for is 1.0.16-3+b1, which, again, is not easily available in any fields of either index.

Ok but why is this important?

Excellent question! Of course there’s the even better question “is anything important” that I’m occasionally asking myself, but for the version string there’s an easier answer:

It seems there are currently two ways to locate the correct 1.0.16-3+b1 version string:

Option #1: courier-imap_5.0.13+1.0.16-3+b1_amd64.deb contains a file called usr/share/doc/courier-imap/changelog.Debian.amd64.gz, this file contains a changelog entry that contains the binnmu-version:

courier (1.0.16-3+b1) sid; urgency=low, binary-only=yes

  * Binary-only non-maintainer upload for amd64; no source changes.
  * Rebuild against libidn12

 -- amd64 / i386 Build Daemon (x86-ubc-01) <buildd_amd64-x86-ubc-01@buildd.debian.org>  Sun, 22 Aug 2021 22:12:19 +0000

One could download all packages in Debian and parse the changelog file inside of them to get an accurate binnmu-version for every package.

Option #2: Crawl buildinfos.debian.net, download all buildinfo files and generate a lookup table. There’s an implementation of this at builtin-pho but unfortunately no publicly available database.

With reproducible builds taking a somewhat central role in supply-chain security there’s quite a bit of power around the answer to “What’s inside of a distro and how is it built”. The security of Packages.xz and Sources.xz is fairly well understood at this point, I’d like to keep the missing link as simple as possible and the barrier of running a rebuilder as low as possible.

There’s a related video by José Miguel Parrella called Traversing deb packages upstream provenance and downstream integrity that I highly recommend looking into, it goes into further detail how these trust-chains work.

Thanks

This work is currently crowd-funded on github sponsors. I’d like to thank @jvoisin, @SantiagoTorres, @repi and @rgacogne for their support in particular, this work wouldn’t be possible without them. ♥️

Footnotes

isdebianreproducibleyet.com, also, yey, I figured out how to do footnotes! ↩
build environment fuzzing is attempting to build something twice and varying things in the environment you don’t want to end up in the binary. For example the clock of the second build server could be off by multiple years or have a different username for the build server, if the binary of the second build server is bit-for-bit identical you’ve successfully confirmed there are no non-normalized values leaking into the binary. ↩
Historically there’s also buildinfo.debian.net (buildinfo without the s vs buildinfos) but at this point of the reproducible Debian project it’s fairly irrelevant. ↩
Actually, this results in https://buildinfos.debian.net/buildinfo-pool/main/r/rust-sniffglue/rust-sniffglue_0.14.0-2_amd64.buildinfo, but the main/ part needs to be removed from the Directory: field. This is an implementation quirk of buildinfos.debian.net. ↩
Or at least that’s the name somebody suggested when I asked in #debian-devel. ↩
Since at least 2006. ↩

Release: rebuilderd v0.15.0

Mon, 18 Oct 2021 00:00:00 +0000

rebuilderd 0.15.0 very recently released, this is a short intro into what it is, how it works and how to build our own integrations!

rebuilderd monitors an index of artifacts and parses it into a datastructure that looks like this. In the most basic case, based on the distro field it’s going to pick the right build script and attempt to generate an artifact identical to the file linked to in url.

We’re starting with a script that generates a json. In our case we’ll simply hard-code all values for demonstration purpose. Most of these values can be arbitrary strings and are simply used to sort packages into different buckets.

Let’s run the script and pipe it through jq for formatting and pretty colors. This describes one build that produces a single output-artifact.

Next up we’re going to import it into rebuilderd! Unless you’re on a distro that ships official rebuilderd packages, the easiest way to get started is cloning the repo and using docker-compose up daemon.

Test everything is working correctly with REBUILDERD_COOKIE_PATH=secret/auth cargo run --bin rebuildctl -- status you might have to set the permissions of ./secret/auth to 644 because the default permissions are fairly strict. If you don’t get errors everything is good to go.

Next up we’re going to import our hello-world package with the script we wrote in the first tweets. The arguments for this command need to match with the json.

If everything has worked correctly rebuildctl pkgs ls should now print that there’s one known package that’s in “unknown” state. This means the rebuild did neither succeed nor fail yet.

This has also been scheduled in the build queue with rebuildctl queue ls. Next up we need a worker that’s going to pick up this job, attempt the build and then report back the result. This is split into a different process so can run the builds on different servers.

These are usually wrapper shell scripts that wrap around the heavy lifting that’s implemented in rebuilder backends like archlinux-repro or debrebuild, but can be anything you want. This script does some basic math and then creates a file with hello world in the output folder.

Testing the script is working correctly…

Next edit the example config file at contrib/confs/rebuilderd-worker.conf, set enabled = true in the diffoscope section (or leave at false if you don’t want to install diffoscope). We also need to configure our script as rebuilder backend for our example “foo” distro.

Usually you’d use absolute paths but because we’re running the worker from our shell it should be fine this time. Remember that we’ve provided a url for our hello.txt artifact? We need an http server that hosts this for us. We’re going to let this fail intentionally first.

Open another terminal and change the directory into the rebuilderd checkout again. Then run the rebuilderd-worker and instruct it to connect to the local instance. If something gues wrong you can requeue the package with rebuildctl pkgs requeue.

The rebuild has been marked as BAD, as expected. Let’s have a look at the diffoscope. With REBUILDERD_COOKIE_PATH=secret/auth cargo run --bin rebuildctl -- pkgs diffoscope --name hello-world we see both the line from our http server and the line from our script.

Usually we wouldn’t use rebuilderd with basic text files and diffoscope is useful to diff complex nested packages. Let’s put the correct file on our local http server and try again.

It worked! The artifact that was generated by our build script was bit-for-bit identical with the reference file we downloaded.

Wrapping up for today, this work is currently funded by: myself and github sponsors! I’ve also setup a patreon account. Thanks! <3

Monthly Report (September 2021)

Thu, 30 Sep 2021 00:00:00 +0000

This is the monthly report of what I’ve been up to in September 2021. 🙌

Reproducible Builds

There have been 3 releases of rebuilderd this month, 0.14.0, and two minor bugfix releases, 0.14.1 and 0.14.2.

The 0.14.0 release introduced experimental support to rebuild Tails images in #66. Tails is a portable operating system that’s known for it’s strong focus on privacy and security, and commonly used by activists, journalists and various human-rights NGOs. It already had reproducible images for a long time (since around 2017), but you had to reproduce the images manually. Starting with this release you can setup rebuilderd to monitor Tails for new releases and automatically attempt to recreate the release from source, on your own independent build system. If this succeeds, the result looks like this:

Since the Tails build uses virtual machines it has special requirements for the build system, specifically /dev/kvm needs to be available. The VPS hoster I usually use didn’t offer nested-kvm and I instead used a bare-metal server to develop this.

Special requirements for rebuilders are common, for example archlinux-repro uses systemd-nspawn to build in a clean environment and debrebuild uses mount(2) to setup a build container. mount(2) is considered a very privileged action in Linux and requires the CAP_SYS_ADMIN kernel capability, which is effectively equivalent to root access. Requirements like this are now documented in the README, along with a link to the rebuilder-backend that’s being used:

Because the rebuilderd and rebuildctl binaries have previously only been packaged in Arch Linux I’ve also created a package for Alpine Linux that’s now available in the [testing] repository (aports!25695).

Cryptographic rebuild attestations with in-toto

One of the most notable changes in rebuilderd 0.14.0 is support for in-toto attestations, contributed by Joy Liu during GSoC 2021 (#65, #67). Attestations are human readable and can be fetched like this:

% rebuildctl -H https://wolfpit.net/rebuild/ pkgs attestation --name zstd | jq .
{
  "signatures": [
    {
      "keyid": "585a2a5c5efec5fc22d84f7fa7a4a22cc1c62507cf97b6d8e7df7aaea8e5f659",
      "sig": "0b22ed56c69d362a66bd247a206e7f97e0208e45f2d24bf16bf2e7a3ba69bc73ddd1bf4a457bb57039dc570dc3827ea6350bc4972f899cc9c78f43b41b0d960c"
    }
  ],
  "signed": {
    "_type": "link",
    "byproducts": {},
    "env": {},
    "materials": {
      "/tmp/rebuilderdyeQ5vC/inputs/zstd-1.5.0-1-x86_64.pkg.tar.zst": {
        "sha256": "0414f4baa43aef409b235184e6079ee0dac8e692912e1da61a8ae264c7606670",
        "sha512": "21a56b70b072673159277a9049ac61831ada6712d00ea1f45b4ca1207196d8005332e9c1b6371b6c8615df7dcfa878a8d440c1d85985b83d63eb4e9f970f9e4a"
      }
    },
    "name": "rebuild zstd-1.5.0-1-x86_64.pkg.tar.zst",
    "products": {
      "/tmp/rebuilderdyeQ5vC/out/zstd-1.5.0-1-x86_64.pkg.tar.zst": {
        "sha256": "0414f4baa43aef409b235184e6079ee0dac8e692912e1da61a8ae264c7606670",
        "sha512": "21a56b70b072673159277a9049ac61831ada6712d00ea1f45b4ca1207196d8005332e9c1b6371b6c8615df7dcfa878a8d440c1d85985b83d63eb4e9f970f9e4a"
      }
    }
  }
}

An in-toto attestation is a signed data structure that (in this case) cryptographically documents a successful rebuild. From a reproducible builds point of view, the essentials you need to know about in-toto:

    "materials": {
      "/tmp/rebuilderdyeQ5vC/inputs/zstd-1.5.0-1-x86_64.pkg.tar.zst": {
        "sha256": "0414f4baa43aef409b235184e6079ee0dac8e692912e1da61a8ae264c7606670",
        "sha512": "21a56b70b072673159277a9049ac61831ada6712d00ea1f45b4ca1207196d8005332e9c1b6371b6c8615df7dcfa878a8d440c1d85985b83d63eb4e9f970f9e4a"
      }
    },

materials are the input files that have been used. Since this is an attestation for an Arch Linux package we’re going to need the buildinfo file that describes the build environment that has been used (which compiler version, which library versions, etc). Arch Linux embeds this info inside the package, that’s why the pre-compiled package that’s used by all Arch Linux users is also listed as a build input. The content of this file is identified using two cryptographic checksums, sha256 and sha512. This is mostly for documentation (similar to some of the unused sections like env and byproducts).

    "products": {
      "/tmp/rebuilderdyeQ5vC/out/zstd-1.5.0-1-x86_64.pkg.tar.zst": {
        "sha256": "0414f4baa43aef409b235184e6079ee0dac8e692912e1da61a8ae264c7606670",
        "sha512": "21a56b70b072673159277a9049ac61831ada6712d00ea1f45b4ca1207196d8005332e9c1b6371b6c8615df7dcfa878a8d440c1d85985b83d63eb4e9f970f9e4a"
      }
    }

products are the most important part of the attestation, it records the artifacts that have been built from source by the rebuilder. In this case there’s only one and since both the sha256 and sha512 checksums are identical, this indicates that this rebuilder has successfully reproduced the binary package from source.

    "name": "rebuild zstd-1.5.0-1-x86_64.pkg.tar.zst",

name describes what this signature is actually about. In this case it indicates that this is a rebuild attestation and which package has been rebuilt. This helps avoid signature-reuse attacks that are common in more traditional systems like OpenPGP where a) signing-intent is implied b) keys are tied to identity instead of purpose.

For example, my pgp key is present in both the Arch Linux and Debian keyring, from a cryptographic point of view my signed Arch Linux packages are also valid signed Debian uploads. The only defense against signature-reuse attacks with systems like this are the file format parsers (Debian wouldn’t accept a tar file as a valid upload, and pacman wouldn’t accept the Debian upload text-format as a valid package). Fingers crossed I never sign a polyglot. 🤞

Recording the purpose in the signature (like in-toto does) prevents attacks like this.

  "signatures": [
    {
      "keyid": "585a2a5c5efec5fc22d84f7fa7a4a22cc1c62507cf97b6d8e7df7aaea8e5f659",
      "sig": "0b22ed56c69d362a66bd247a206e7f97e0208e45f2d24bf16bf2e7a3ba69bc73ddd1bf4a457bb57039dc570dc3827ea6350bc4972f899cc9c78f43b41b0d960c"
    }
  ],

signatures is the cryptographic portion of the attestation. It documents which key created the signature, and the signature itself. Because signatures work on binary blobs instead of data structures, the in-toto spec defines how to serialize the data structure into a canonical binary representation. Serializing the same data structure is always going to give you the same serialized binary representation.

There are currently slightly over 1.000 attestations that have been issued so far across all rebuilders at the time of writing:

for x in 'https://reproducible.archlinux.org' 'https://r-b.engineering.nyu.edu' 'https://wolfpit.net/rebuild'; do
    echo -n "$x "; curl -sSfm30 "$x/api/v0/pkgs/list" | jq '.[] | select(.has_attestation) | .name' | wc -l
done

Having cryptographic attestations like this is useful to design more secure package authentication systems for package updates that rely on confirmations from multiple rebuilders. I’m looking forward to the first package-manager integrations that fetch in-toto attestations from rebuilderd!

Acknowledgments

This project was funded by Google, The Linux Foundation, and people like you and me through GitHub Sponsors. Without this support I wouldn’t be able to do all of this, thanks! ♥️♥️♥️

Monthly Report (August 2021)

Tue, 31 Aug 2021 00:00:00 +0000

This is the monthly report of what I’ve been up to in August 2021. 🙌

Reproducible Builds

There are many different reasons to be interested in Reproducible Builds. When I originally got involved in the project I wasn’t a maintainer in any Linux distribution yet, instead I was wondering if there’s a way to distribute pre-compiled artifacts as an independent open source dev without carrying all the responsibility alone.

A few years later I’ve now published a manual called i-probably-didnt-backdoor-this. It contains a hello world program and instructions on how to reproduce the various pre-compiled artifacts, explains all build instructions and why these controls are effective.

A similar project has been published in response by Michael Lieberman. This project also distributes binaries that can be rebuilt with very simple commands using NixOS and Bazel.

The project also got a shout-out at debconf21 in a talk about Reproducible Builds by Holger Levsen (around 11:23).

Reproducible Arch Linux

Quite a few (but not all) of the remaining 13% unreproducible packages on Arch Linux are unreproducible due to python bytecode files (.pyc). Debian doesn’t distribute .pyc files but because both Arch Linux and Alpine do I’ve published a blog post about Reproducible Python Bytecode. Levente ‘anthraxx’ Polyak of the Arch Linux Security Team was also involved in the investigation and a patch for pacman-6.0.1 was suggested by Allan McRae on the pacman-dev email list. The blog post has also been featured in Python Weekly!

The rebuilder setup of Reproducible Arch Linux has been mentioned in Looking Forward to Reproducible Builds (around 3:19) by Vagrant Cascadian. I’d recommend watching the whole video if you’re interested in rebuilders, regardless of the distribution you’re using!

I found two bugs in archlinux-repro and submitted patches #101, #102.

The following packages have been fixed:

v2ray-domain-list-community (upstream) - Sort list before marshaling into dlc.dat
gitlab-workhorse (upstream) - Add SOURCE_DATE_EPOCH support for build-time
podman (upstream) - Fix embedded build-paths with -trimpath
ktoblzcheck - Don’t download new data at build time
calibre - PYTHONHASHSEED=0
notion - Disable embedded build time
gauche (upstream) - Disable embedded timestamps in info page compression
swift.im (upstream) - Sort directory contents at build time

rebuilderd

rebuilderd is one of the components powering reproducible.archlinux.org. It monitors release artifacts (like the packages of a Linux distribution) and schedules rebuilds using supported rebuild backends.

Joy Liu worked on in-toto for Google Summer of Code 2021 and developed code for both in-toto-rs and rebuilderd to add in-toto attestation capabilities. This allows cryptographically verifying rebuild attestation in-toto/rebuilderd#1, #65.

Santiago Torres has contributed major fixes for the work-in-progress debian integration with debrebuild #59.

Aditya Sirish who’s operating the NYU rebuilder has discovered that rebuilderd can’t handle some compression formats for pacman database files, this has been fixed in #62.

Rebuilderd is now able to track which rebuilder-backend a worker needs to rebuild the package. A worker that is setup to rebuild an Arch Linux package might not have the setup to rebuild Debian packages too #64. The systemd unit for the rebuilder worker has been updated to use idle cpu and io priority #60. The debrebuild flag to specify the output directory is now passed correctly #63.

Binary Transparency

This isn’t directly related to Reproducible Builds but an important component for supply chain security as well. Binary Transparency helps if an attacker has gained control over an update signing key. Since a compromised update signing key is a very valuable asset for an attacker, they might hesitate to officially upload a malicious update to the distro’s archive. Instead they could use the key to target specific high-profile individuals directly to avoid burning the key. This kind of attack is very likely to go unnoticed.

Because in this scenario the attacker controls a signing key we can’t trust the signature alone, but if the update system requires Binary Transparency proofs the attacker is forced to log their malicious update to the transparency log. This doesn’t necessarily prevent the install of the malicious package, but it would likely get flagged as an incident by a transparency log monitor, making it a rather unappealing approach for the attacker.

I’ve released pacman-bintrans which acts as an experimental download plugin for pacman that performs additional security checks. I had an old codebase from a failed attempt to implement this with certificate transparency and SCTs, I’ve removed this part and replaced it with the rekor transparency log from the sigstore project.

You can already use this on your Arch Linux system today to ensure all updates you’re downloading have been properly logged to the transparency log, but keep in mind there’s no monitor auditing the log yet (although all data to implement this is public). The transparency signatures are hosted on https://pacman-bintrans.vulns.xyz/ and http://2iz5fzvuwjapcv5v2msvhlr5oqbwriznwu7hnsccrsho47ljqynrgryd.onion/. The transparency log used for this is the public rekor instance at https://rekor.sigstore.dev.

Luke Hinds from Red Hat sent a shout-out in his talk OCB: sigstore, Software Signing for All (around the 34:40 mark).

The rekor package in Arch Linux has been updated to include shell completions. I found a very minor bug that is now fixed upstream as well! I’ve also started working on an alpine package for rekor.

Acknowledgments

This project was funded by Google, The Linux Foundation, and people like you and me through GitHub Sponsors. Without this support I wouldn’t be able to do all of this, thanks! ♥️♥️♥️