Bugfixes

The most important fixes for bugs in the generated code are:

• The implementation of peek and poke for bitfields was broken, which could lead to segfaults.
• Duplicate record fields are now usable also in Template Haskell mode.
• Patterns for unsigned enums now get the right value.

We have also resolved a number of panics during code generation, but those would not have resulted in incorrect generated code (merely in no code being generated at all).

New features

• Implicit fields arise when one struct (or union) is nested in another, without any field name or tag:

  struct outer {
    int x;
    struct {
      int y;
      int z;
    };
  };

  We now support such implicit fields; both the inner (anonymous) struct as well as the corresponding field of the outer struct will be named after the first field of the inner struct¹:

  data Outer = Outer {
      x :: CInt
    , y :: Outer_y
    }
  
  data Outer_y = Outer_y {
      y :: CInt
    , z :: CInt
    }

  For this particular case we could also have chosen to flatten the structure and add y and z directly to Outer, but that does not work in all cases (for example, when we have an anonymous struct inside a union), so instead we opt for consistency and always generate an explicit type for the inner struct.

• Unnamed bit-field declarations, which are used to control padding, are now supported:

  struct bar {
    signed char x : 3;
    signed char   : 3;  // Explicit padding
    signed char y : 2;
  };

• We used to distinguish between parse predicates (which files should hs-bindgen parse at all?) and selection predicates (for which C declarations should we generate Haskell declarations?). This was confusing, and as we are getting better at skipping over declarations with unsupported features (and that list is dwinding anyway), parse predicates are not that useful anymore. Parse predicates therefore have been removed entirely; we simply always parse everything (selection predicates are still very much an important feature of course).

• Some infrastructure for and around binding specifications has been improved. For example, we now distinguish between macros and non-macros of the same name, and our treatment of arrays has changed slightly. For example, given

  typedef char T [];
  void foo (T xs);

  we now generate

  foo :: Ptr (Elem T) -> IO ()

  We do not use Ptr CChar, because T might have an existing binding in another library (with an external binding specification), and we don’t know what the type of the elements of T are (it could for example be some newtype around CChar). Elem is a member of a new IsArray class, part of the hs-bindgen-runtime.

• Top-level anonymous enums are now supported. For example,

  enum {A, B};

  results in

  pattern A :: CUInt
  pattern A = 0
  
  pattern B :: CUInt
  pattern B = 1

  (Normally an enum results in a newtype around the enum’s underlying type, and the patterns are for that newtype instead.)

• We now generate bindings for static global variables (such globals are sometimes used in headers that also contain static function bodies).

• All definitions required by the generated code are now (re-)exported from hs-bindgen-runtime, so that it becomes the only package dependency that needs to be declared (no need for ghc-prim or primitive anymore).

This list is not complete; some other less common edge cases have also been implemented.

Conclusions

Although we are still working on some finishing touches before we can release the first official version of hs-bindgen, it is already being put to good use on various projects. There are only a handful of missing C features left, all of which low priority edge cases (though if you have a specific use case for any of these, do let us know!). So if you are interested, please do try it out, and let us know if you find any problems. There should be no major breaking changes between now and the first official release.

You can find the previous editions collected under the haskell-ecosystem-report tag.

Sponsorship

We offer Haskell Ecosystem Support Packages to provide commercial users with support from Well-Typed’s experts while investing in the Haskell community and its technical ecosystem including through the work described in this report. To find out more, read our announcement of these packages in partnership with the Haskell Foundation. We need funding to continue this essential maintenance work!

Many thanks to our Haskell Ecosystem Supporters: Standard Chartered, Channable and QBayLogic, as well as to our other clients who also contribute to making this work possible: Anduril, Juspay and Mercury; and to the HLS Open Collective for supporting HLS release management.

Team

Matthew Pickering announced that he will be leaving the company and moving to a non-Haskell role at the end of March. Working with Matt has been a joy – more than his deep technical insight or sharp intuition, it’s the warmth of his vision for how to work together and his generosity that has made him such a force within the team. He was also a beacon that could rally the community in difficult times, perhaps most memorably with his technical and social contributions in consolidating Haskell IDEs with the creation of the Haskell Language Server. His dedication to tooling has also been an inspiration, with his work on ghc-debug and on profiling an invaluable contribution to our understanding of memory usage of Haskell programs.

The Haskell toolchain team at Well-Typed currently includes:

• Andreas Klebinger
• Hannes Siebenhandl
• Magnus Viernickel
• Mikolaj Konarski
• Rodrigo Mesquita
• Sam Derbyshire
• Zubin Duggal

In addition, many others within Well-Typed contribute to GHC, Cabal, HLS and other open source Haskell libraries and tools. This report includes contributions from Alex Washburn, Duncan Coutts, Wen Kokke and Wolfgang Jeltsch in particular.

We are active participants in community efforts for developing the Haskell language and libraries. Rodrigo joined the GHC Steering Committee in December, alongside Adam Gundry. Wolfgang joined the Core Libraries Committee in February.

Highlights

Interactive step-through debugging

The Haskell Debugger (hdb) has been made more robust and more features were implemented by Rodrigo, Matthew, and Hannes. Most notably, the debugger now:

• Displays stack traces for bytecode and compiled code frames (provided the program and dependencies were compiled with -finfo-table-map for the latter)
• Displays source locations and callstacks for exception breakpoints
• Uses the external interpreter by default
• Can be run on GHC itself!

To run hdb you need to use GHC 9.14 and to configure the IDE accordingly. Please refer to the installation instructions. Apart from that, if HLS just works on your codebase, so should the debugger!

Live monitoring using the eventlog

GHC’s eventlog already lets Haskell programs emit rich runtime telemetry, but the workflow has historically been to run the program to completion and inspect the eventlog afterwards. eventlog-live allows us instead to monitor the program as it is running. Wen continued work on this project, taking significant steps towards making it production-ready, including:

• extending eventlog-live with support for the OpenTelemetry protocol (#119),

• bringing the underlying eventlog-socket library closer to being ready for general use, by adding a testsuite (#27). fixing a litany of issues with the C code (#38), and finalising the user-facing API (#43),

• adding support for custom commands in eventlog-socket (#36).

Trees That Grow

The Language.Haskell.Syntax module hierarchy is intended to be a stable, public API for the Haskell AST — one that external tools could eventually depend on without coupling themselves to GHC internals, reducing ecosystem breakage. Right now, that goal is undermined by lingering dependencies on internal modules under the GHC hierarchy.

Alex, with help from Rodrigo, has been systematically removing these edges in the dependency graph:

• Language.Haskell.Syntax.Type no longer depends GHC.Utils.Panic (!15134, #26626).
• Language.Haskell.Syntax.Decls no longer depends on GHC.Unit.Module.Warnings (!15146, #26636), nor on GHC.Types.ForeignCall (!15477, #26700) or GHC.Types.Basic (!15265, #26699).
• Language.Haskell.Syntax.Binds no longer depends on GHC.Types.Basic (!15187, #26670).

Once this work is done, it will be possible to consider moving the AST into a separate package, and taking further steps towards increasing modularity of the compiler.

Towards a standalone base package

Historically, the base package was used as both the user-facing standard library and a repository of GHC-specific internals, with much special treatment in the compiler. This means GHC and base versions are tightly coupled, and makes upgrading to new compiler versions unnecessarily difficult.

GHC developers have made significant progress towards making base a normal Haskell package: ghc-internal has been split out as a separate library, base no longer has a privileged unit-id in the compiler, and Cabal now allows reinstalling it.

Matt posted a summary of progress and outlined possible next steps to seek community consensus on the direction of travel. The reinstallable-base repository collects documents and discussion on the effort.

Wolfgang continued various pieces of technical groundwork:

• cleaning up many unused known-key names in the compiler (!15184, !15190, !15211, !15213, !15217, !15218, !15219, !15215),

• finishing the process of removing GHC.Desugar from base (!15433),

• refining the import list of System.IO.OS to aid in modularity (!15567).

Wolfgang improved the public API of base relating to OS handles, to make the API more stable across platforms and avoid the need for users to depend on GHC-internal implementation details (!14732, !14905). While in the area, he fixed a bug in the implementation of hIsReadable and hIsWritable for duplex handles (#26479, !15227), and a mistake in the documentation of hIsClosed (!15228).

Incorrect absence analysis in GHC

GHC bug #26416 has occupied the attention of the team for quite some time. Initially thought to be an issue with specialisation, a reproducer that Sam and Magnus created showed that the issue is in fact a bug in absence analysis — an optimisation that identifies and removes unused function arguments — in which GHC would erroneously conclude that a used argument was in fact absent.

Andreas helped investigate the root cause, before Zubin finally took the torch and put up a solution (!15238).

GHC changelogs

GHC’s changelogs have not always been as complete or reliable as the community deserves. Keeping changelogs accurate across backports has also been a major source of frustration for release managers.

This is why, after a discussion initiated by Teo Camarasu in #26002, we have decided to adopt the changelog.d system — already in use by the Cabal project — in which each change is a separate file in the changelog directory. This eliminates the merge conflicts that make backporting painful, and makes it easier to associate MRs with changelog entries.

Zubin has been spearheading the effort, with the intention to switch to this new method of changelog generation right after the fork date for GHC 10.0.

GHC

GHC Releases

• Zubin worked on 9.12.3, backporting patches and preparing release candidates, with a final release on the 27th of December.
• Magnus and Zubin worked on backports for 9.12.4.
• Zubin worked on 9.14.1, putting out the final release on the 19th of December.

Frontend

• Sam reviewed the implementation of the QualifiedStrings extension by Brandon Chinn (!14975). This allows string literals of the form ModName."foo" (interpreted as ModName.fromString ("foo" :: String)).

• Sam made several changes to the treatment of Coercible constraints in the typechecker (!14100):

  • Defaulting of representational equalities to nominal equalities, functionality previously added to GHC by Sam, is now more robust (#25825).
  • Error messages involving unsolved Coercible constraints are greatly improved, an oft-requested improvement (#15850, #20289, #23731, #26137). Error messages now consistently mention relevant out-of-scope data constructors, provide import suggestions, and include additional explanations about roles (when relevant).

• Magnus implemented several fixes to the implementation of ExplicitLevelImports:

  • a missing check for types (#26098, !15119),
  • a GHC panic in the driver (#26568, !15118).

• Sam improved the reporting of “valid hole fits”, adding support for suggesting bidirectional pattern synonyms (#26339) and properly dealing with data constructors with linear arguments (#26338).

• Sam investigated a typechecking regression starting in GHC 9.2 with the introduction of the Assert type family to improve error messages involving comparison of type-level literals (#26190), posting his analysis to the ticket. To tackle this, he opened GHC proposal #735, which is still in need of further community feedback.

• Sam minimised a bug with rewrite rules (#26682), which allowed Simon Peyton Jones to identify and fix the bug (!15208).

• Sam improved how existential variables are displayed in Haddock documentation (!15099, #26252).

Determinism

• Matt identified and fixed several ways in which GHC compilation was not deterministic:
  • an issue with non-deterministic documentation information (#26858, !15482).
  • non-determinism of constraint solving impacting generated Typeable evidence (#26846, !15442).
  • issues with the Template Haskell machinery of the singletons library producing non-deterministic names (singletons #629, th-desugar #240).

Plugins

• Sam finished up and landed a long-standing MR by Chris Wendt (!10133) which fixed a plugin-related issue.

• Sam fixed a regression in ghc-typelits-natnormalise in which the plugin would cause GHC to fall into an infinite loop (ghc-typelits-natnormalise #116, #118).

Backend

• Rodrigo announced that work described in #23218 evolved into the POPL 2026 paper “Lazy Linearity for a Core Functional Language”, which presents a way to type linearity in GHC Core that is robust to almost all GHC optimisations, together with a GHC plugin validating programs at each optimisation stage.

• With the oversight of Andreas, Sam carefully reconsidered the treatment of register formats in the register allocator and liveness analysis. This culminated in !15121:

  • Keep track of register formats in liveness analysis (#26526).
  • Use the right format when reloading spilled register (#26411).
  • Enforce the invariant that writes to a register re-defined the format that this register is used at for the purposes of liveness analysis, fixing another bug reported by @aratamizuki on !15121.

• Sam put up a small fix for the mapping of registers to stack slots, fixing an oversight in the case that registers start off small and are subsequently written at larger widths (#26668, !15185).

• Sam reviewed a GHC contribution by @sgillespie adding SIMD primops for abs and sqrt operations (!15236), suggesting more efficient implementations of certain operations.

• Andreas investigated potential missed specialisations, which allowed Simon Peyton Jones to make further progress in improving the specialiser (#26831, !15441).

• Sam investigated several bugs to do with the interactions of join points with ticks (#14242, #26157, #26642, #26693) and casts (#14610, #21716, #26422). He fixed the main bug (#26642, !15538), which was due to incorrect transformations in mergeCaseAlts. He also undertook a general refactor of the area and, pinning down the overall handling of casts and ticks under join points in a Note.

Runtime system and linker

• Matt fixed a decoding failure for stg_dummy_ret by using INFO_TABLE_CONSTR for its closure (#26745, !15303).

• Duncan fixed long-standing inconsistencies in eventlog STOP_THREAD status codes (#26867, !15522).

• Andreas improved the documentation of the -K RTS flag in !15365 (#26354).

Exception backtraces, stack annotations and stack decoding

• Matt and Hannes improved the reporting of backtraces when using error (!15306, !15395, #26751). This involved opening two CLC proposals (CLC #383, CLC #387).

• Hannes continued working on the implementation of stack annotations and stack decoding (#26218), including:

  • integrating ghc-stack-profiler, a profiler that relies on stack annotations instead of heavier profiling mechanisms, with the eventlog-socket library; and
  • working on the ghc-stack-annotations compatibility library for annotating the stack.

• Rodrigo removed an incorrect assertion that fired when decoding a BCO whose bitmap has no payload (#26640, !15136).

Build system and packaging

• Zubin fixed a GHC 9.14.1 build issue due to missing .cabal files for ghc-experimental and ghc-internal in the source tarball (#26738, !15391).

• Andreas investigated the use of Cabal’s --semaphore feature to speed up GHC builds slightly (#26876, !15483). There are some issues preventing us from enabling this unconditionally (#26977, Cabal #11557).

CI and testing

• Magnus ensured the user’s guide can be generated with old versions of Python to fix CI build failures on some older containers (!15127).

• Magnus updated the Debian images used for CI (ci-images !183, !178).

• Sam finished up the work of Sven Tennie on testing floating point expressions in the test-primops test framework for GHC (test-primops !19). This is preparatory work for improving the robustness of GHC’s handling of floating point (#26919).

• Andreas updated the nofib GHC benchmarking suite to fix issues that Sam ran into when trying to use it, updating the CI in the process (nofib !81, !82, !83).

Infrastructure

• Magnus worked on the infrastructure for the GitLab instance used for the GHC project, bringing up new runners for CI and switching to a new verification system to approve new users which makes it easier for new contributors to open issues.

• Magnus and Andreas helped the Haskell infrastructure team address Gitlab outages on short notice in order to improve availability of the GHC Gitlab instance.

• Andreas and Magnus organized temporary CI capabilities sponsored by WT during a temporary outage of one of GHC’s CI runners.

Cabal

• Sam added support for setting the logging handle via the library interface of Cabal, a significant milestone in updating cabal-install to compile packages with the Cabal library without invoking external processes (Cabal #11077).

• Matt helped Matthías Páll Gissurarson to fix a bug in which cabal haddock was looking for files in the wrong directory (Cabal #11475, #11476).

• Matt fixed a bug with broken Haddocks locally due to non-expanded ${pkgroot} variable (Cabal #11217, #11218).

• Matt fixed some issues with cabal repl silently failing (Cabal #11107, #11237).

HLS

• In collaboration with Zubin and Andreas, Hannes investigated the root cause of HLS issue #4674, posting his analysis in this comment. In short, the problem was that the hlint plugin was using an incompatible version of ghc-lib-parser, and a version mismatch in this library was causing segfaults due to changes to the GHC.Data.FastString implementation between the versions. Hannes disabled the hlint plugin on GHC 9.10 to work around this issue (HLS PR #4767).

• Hannes reviewed and assisted with HLS PR #4856 by @vidit-od. This PR makes HLS use the stored server-side diagnostics for code actions, in order to make them more responsive. This fixes HLS issue #4805.

• Hannes helped land long-running HLS PR #4445 by @soulomoon, which allows files to be loaded concurrently in batches in order to improve responsiveness of HLS.

• Zubin and Hannes worked together to update HLS to work with GHC 9.14 (HLS PR #4780).

• Hannes worked on general maintenance of the HLS project:

  • Prepared release 2.13.0.0 (HLS PR #4785)
  • Tackled various CI issues (HLS PR #4863, HLS PR #4812, HLS PR #4811)
  • Updated the advertised range of supported GHC versions (HLS PR #4801, HLS PR #4799)

• Hannes and Zubin implemented some fixes to Windows CI (HLS PR #4800, HLS PR #4768).

• Hannes merged the hls-module-name-plugin into hls-rename-plugin in HLS PR #4847.

• Hannes improved the robustness of the hls-call-hierarchy-plugin-tests in HLS PR #4834 by using VirtualFileTree.

• Hannes also worked on hie-bios:

  • Preparing release 0.18.0.0 (hie-bios PR #496)
  • Updated the supported GHC versions (hie-bios PR #495)
  • Adapted to GHC migrating some parts of its codebase to use OsPath (hie-bios PR #493)

Haskell Debugger

Rodrigo continued work on the new Haskell Debugger.

• Matt and Rodrigo introduced a DSL for evaluation on the remote process, which allows the debuggee to be queried from a custom instance, making it possible to implement visualisations which rely on e.g. evaluatedness of a term (#139).

• Matt improved support for exceptions: break-on-exception breakpoints now provide source locations (#165).

• Rodrigo allowed call stacks to be inspected in the debugger (#158).

• Hannes introduced support for stack decoding and viewing custom stack annotations (#172).

• Rodrigo made the Haskell Debugger use the external interpreter (#170), which paves the way for multi-threaded debugging (see also #140). This change also allowed Rodrigo to implement Windows support (#184) with the help of Hannes.

• Matt fixed a bug in the handling of data constructors with constraints (#175).

• Hannes improved caching in the CI (#173).

ghc-debug

• Matt and Hannes fixed several issues with AP_STACK closures (!79, !80, !86).

• Hannes implemented asynchronous heap traversal in ghc-debug-brick, making the interface more responsive (!78).

• Hannes added history navigation and search caching to the ghc-debug-brick interface (!83).

• Hannes added a summary row to the string counting table view (!81), and fixed the search limit not being honoured during incremental searches (!76).

This blog post will be a brief overview of what hs-bindgen can do for you, as well as a summary of the status of the project. You will find links for further reading at the very end, the most important of which is probably the (draft) manual. The hs-bindgen repository also contains a number of partial example bindings, including one for rpm; the hs-bindgen nix tutorial includes one further partial example set of bindings, for wlroots.

Installation

The alpha version of hs-bindgen is not yet released on Hackage. Instead, add the following to your cabal.project file:

source-repository-package
  type: git
  location: https://github.com/well-typed/hs-bindgen
  tag: release-0.1-alpha
  subdir: c-expr-dsl c-expr-runtime hs-bindgen hs-bindgen-runtime

source-repository-package
  type: git
  location: https://github.com/well-typed/libclang
  tag: release-0.1-alpha

We have however uploaded three package candidates, primarily so that they can be used to lookup Haddocks:

• hs-bindgen-runtime provides runtime support for the code generated by hs-bindgen, and in many cases is also necessary for interacting with that code

• c-expr-runtime provides similar support for bindings generated for CPP macros

• hs-bindgen-the-library for using hs-bindgen in Template Haskell mode.

Introduction

We’ll start very simple. Let’s generate bindings for some library A, which offers the following API:

struct Version {
  int major;
  int minor;
};

void showVersion(struct Version v);

If you want to follow along, you can find the examples in this blog post on GitHub.

Invoking hs-bindgen

There are multiple ways to integrate hs-bindgen into your project (we’ll mention a few others below), but for now we will focus on just running it on the command line:

cabal run -- hs-bindgen-cli preprocess \
  --overwrite-files \
  --unique-id com.well-typed.hs-bindgen-0.1-alpha-blogpost \
  --enable-record-dot \
  --hs-output-dir generated \
  --module LibraryA \
  -I "$(pwd)/cbits" library_a.h

The first few arguments tell hs-bindgen-cli that it is okay to overwrite existing files, specify a unique identifier to avoid generating C name collisions¹, enable the optional record-dot syntax to avoid having to prefix all field names (following the recommendations in Haskell Unfolder #45: Haskell records in 2025), set the output directory, and specify the desired name of the generated Haskell module.

The final line deserves a slightly more detailed explanation. Suppose you are generating bindings for a library that is installed in /opt/library, and that library has a header /opt/library/api/server/secure.h. The bindings generated by hs-bindgen will need to refer to this header using a #include statement; the question is what that #include statement should look like. If we generated

#include </opt/library/api/server/secure.h>

then the generated bindings would only work on machines where that library is installed in that exact location. If this include path should instead be

#include <api/server/secure.h>

with /opt/library in the search path, then hs-bindgen must be invoked with -I /opt/library and main argument api/server/secure.h: the final argument is included precisely as-is in the #include. For our simple example, -I "$(pwd)/cbits" means that the cbits folder of our Haskell package is added to the C include path, and the generated #include will be

#include <library_a.h>

Generated bindings

The above invocation of hs-bindgen will have generated a few Haskell modules. LibraryA.hs contains the translation of the types in the header:

module LibraryA where

data Version = Version {
    major :: CInt
  , minor :: CInt
  }
  deriving stock (Eq, Show)

instance Storable Version where (..)

instance HasField "major" (Ptr Version) (Ptr CInt) where (..)
instance HasField "minor" (Ptr Version) (Ptr CInt) where (..)

(The above code is slightly cleaned up for readability, and various parts of the generated module are omitted: boilerplate such as imports and required language extensions, Haddocks, as well as some more specialized type class instances.)

In addition, module LibraryA/Safe.hs will contain safe imports for all functions in the header:

module LibraryA.Safe where

showVersion :: Version -> IO ()
showVersion = (..)

Since the C function showVersion takes a struct argument by value (rather than as a pointer to the struct), which is not supported by Haskell FFI, hs-bindgen will also have generated a C wrapper; that all happens transparently and automatically.

Module LibraryA/Unsafe.hs contains the same API, but using unsafe imports (if you need a refresher on safe versus unsafe, you might like to watch Haskell Unfolder #36: concurrency and the FFI). Module LibraryA/FunPtr.hs finally contains the addresses of all functions, in case you have C code that works with function pointers.

Using the generated bindings

We can call showVersion very simply as

import LibraryA
import LibraryA.Safe

main :: IO ()
main = showVersion $ Version 2 3

We will come back to the HasField instances for Ptr Version below; no explicit HasField instances for Version itself are necessary, because they are generated by ghc.

Dependencies

Suppose library B defines some kind of API for drivers, and suppose it uses library A:

#include "library_a.h"

struct Driver {
  char* name;
  struct Version version;
};

void initDriver(struct Driver *d);
void showDriver(struct Driver *d);

Main headers

If we run hs-bindgen on this header in the same way as we did for library_a.h, we will get warnings such as

[Warning] [HsBindgen] [select] 'struct Driver' at "../cbits/library_b.h 8:8"
  Could not select declaration (direct select predicate match):
    Transitive dependency not selected:
      'struct Version' at "../cbits/library_a.h 3:8"
      Adjust the select predicate or enable program slicing

When we generate bindings for a header, we need to know which declarations in that header the user wants to generate bindings for; in hs-bindgen this happens by means of selection predicates. The default selection predicate is --select-from-main-headers, which means that any declarations in headers explicitly mentioned on the command line are selected, but any declarations in headers that might be imported by those headers are not. The first way in which we can fix this warning therefore is by explicitly generate bindings for both library_a.h and library_b.h:

cabal run -- hs-bindgen-cli preprocess \
  (..other arguments as before..)
  --module LibraryB \
  -I "$(pwd)/cbits" library_a.h library_b.h

Program slicing

Suppose we are really only interested in library B, and want to generate only those bindings in library A that are required by library B. To do this, we can enable program slicing:

cabal run -- hs-bindgen-cli preprocess \
  (..)
  --enable-program-slicing \
  -I "$(pwd)/cbits" library_b.h

This will pull in only those declarations in library A that are referenced by library B.

Opaque types

The API provided by library B exclusively works with pointers to struct Driver; so perhaps we don’t need a Haskell-side representation of that struct at all. If that is the case, we can configure hs-bindgen through a prescriptive binding specification, and tell it that it should keep the Haskell type opaque:

version:
  hs_bindgen: 0.1.0
  binding_specification: '1.0'
ctypes:
- headers: library_b.h
  cname: struct Driver
  hsname: Driver
hstypes:
- hsname: Driver
  representation: emptydata

This states that the C declaration struct Driver, found in header library_b.h, should be mapped to a Haskell type called Driver (we could pick a different name here if we wanted to, overriding naming decisions made by hs-bindgen), and that the Haskell type Driver be represented as an empty datatype. If we now run

cabal run -- hs-bindgen-cli preprocess \
  (..)
  --prescriptive-binding-spec libraryB.yaml \
  -I "$(pwd)/cbits" library_b.h

then the generated LibraryB is simply

data Driver

It can be quite useful to combine emptydata with program slicing, limiting how many declarations from imported headers are in fact needed.

Composability

The solutions in the previous section all had one important downside: in none of them the generated code for library B reused the generated code for library A. This kind of composability of generated bindings is an important goal of hs-bindgen, influencing many design decisions. Composability is achieved through (external) binding specifications; a binding specification is a .yaml (or .json) file describing a set of generated bindings, a bit like .hi files in Haskell, or a module signature in OCaml or Backpack.

When we generate the bindings for library A we can ask hs-bindgen to additionally generate a binding spec:

cabal run -- hs-bindgen-cli preprocess \
  (..)
  --gen-binding-spec libraryA.yaml

The resulting .yaml file describes the types generated for library A, including which type class instances they have (necessary in order to know which type class instances we can generate when these types are used in other libraries). When generating bindings for library B, we can pass this binding specification along:

cabal run -- hs-bindgen-cli preprocess \
  (..)
  --external-binding-spec libraryA.yaml \
  -I "$(pwd)/cbits" library_b.h

The generated code then looks like

module LibraryB where

import qualified LibraryA

data Driver = Driver {
    name    :: Ptr CChar
  , version :: LibraryA.Version
  }
  deriving stock (Eq, Show)

instance Storable Driver where (..)

instance HasField "name"    (Ptr Driver) (Ptr (Ptr CChar))      where (..)
instance HasField "version" (Ptr Driver) (Ptr LibraryA.Version) where (..)

External binding specifications are useful not only when generating bindings for multiple libraries, but also for structuring bindings for multiple headers of the same library, or when an identical header is included in lots of libraries (such as the rtwtypes.h header generated by MATLAB). We consider external binding specifications to be an essential feature of hs-bindgen.

Pointers

HasField

Suppose we want to override one value deeply nested in some C data structure. We could use the Storable instances to peek the value, then override the appropriate field, and finally poke the updated value:

main :: IO ()
main = do
    alloca $ \(driverPtr :: Ptr Driver) -> do
      initDriver driverPtr

      driver <- peek driverPtr
      poke driverPtr $ driver & #version % #minor .~ 2
      showDriver driverPtr

(The use of lenses here is optional of course.)

However, it may well be undesirable to marshall the entire structure back and forth merely to change a single value. This is why hs-bindgen also generates HasField instances for pointers, so that record dot syntax can be used to index C structures. We can update the minor version number without marshalling the entire structure as follows:

poke driverPtr.version.minor 3

If you prefer to avoid record-dot syntax, you can use the HsBindgen.Runtime.HasCField infrastructure directly.

FunPtr

When dealing with a higher order API, hs-bindgen will generate additional bindings to convert back and forth between C function pointers and Haskell functions, and package these conversions up as instances of two type classes in hs-bindgen-runtime:

class ToFunPtr a where
  toFunPtr :: a -> IO (FunPtr a)

class FromFunPtr a where
  fromFunPtr :: FunPtr a -> a

withFunPtr :: ToFunPtr a => a -> (FunPtr a -> IO b) -> IO b

For example, suppose the driver API in library B additionally contains

struct Driver;
typedef int RunDriver(struct Driver* self);

struct Driver {
  // .. other fields as before ..
  RunDriver* run;
};

int callDriver(struct Driver* d);

then we generate

newtype RunDriver = RunDriver {
    unwrap :: Ptr Driver -> IO CInt
  }

instance ToFunPtr   RunDriver where (..)
instance FromFunPtr RunDriver where (..)

Here’s how we might use this:

counter <- newIORef 0
let run = RunDriver $ \_self -> atomicModifyIORef counter $ \x -> (succ x, x)
withFunPtr run $ \funPtr -> do
  poke driverPtr.run funPtr
  replicateM_ 5 $ print =<< callDriver driverPtr

Macros

Some C libraries make part of their API available as CPP macros. Macros in C don’t have a semantics per se; they are merely a list of tokens that the preprocessor splices in whenever they are used. In order to nonetheless be able to generate “bindings” for macros, hs-bindgen imbues macros with a bespoke semantics. In future versions of hs-bindgen this macro infrastructure will be pluggable (#942), because the default semantics may not suit all applications.

Constants

Low level C libraries (such as this Analog Devices Talise driver) may make certain constants such as bitfields available as CPP macros

#define SIGNALID 0x01

We translate these to Haskell constants:

sIGNALID :: CInt
sIGNALID = 1

Expressions

It may also happen that libraries offer certain functionality as macro functions. For example, a library might provide a definition that provides a pointer offset for devices with memory-mapped I/O:

#define INPUT_PORT(x) x + 4

Since hs-bindgen has no way of knowing if that (+) operator should be interpreted as integer addition or pointer offset (or indeed something else), it generates a very general definition:

iNPUT_PORT :: Add a CInt => a -> AddRes a CInt

Add and AddRes come from c-expr-runtime); one way that we can instantiate this type is to:

inputPort :: Ptr Word8 -> Ptr Word8
inputPort = iNPUT_PORT

Types

Finally, some low-level libraries define types as macros:

#define S16_TYPE short int

In the case (and only in the case) that these can be parsed as the corresponding typedef

typedef short int S16_TYPE;

hs-bindgen will treat these as typedefs, and generate a Haskell newtype:

newtype S16_TYPE = S16_TYPE {
    unwrap :: CShort
  }

Squashing

C structs are often defined using a typedef:

typedef struct Point {
  int x;
  int y;
} Point;

This is typically done for syntactic convenience only, making it possible to write simply Point rather than struct Point. When the only use of a struct is within a typedef in this manner, hs-bindgen will “squash” the typedef, and generate a single type only:

data Point = Point {
    x :: CInt
  , y :: CInt
  }

If however the typedef is not the only use of the struct, then hs-bindgen assumes that this is intended to convey some kind of semantic information, and will not squash. For example, suppose the device driver API includes

typedef struct Driver DeviceDriver;

then we generate

newtype DeviceDriver = DeviceDriver {
    unwrap :: Driver
  }

To avoid confusion, a notice will be emitted whenever a type is squashed. If desired, this notice can be suppressed with --log-as-info select-mangle-names-squashed. In a future release of hs-bindgen squashing will be configurable per typedef using a prescriptive binding spec (#1436).

Build process integration

Preprocessor

The primary way of invoking hs-bindgen is by calling hs-bindgen-cli, as we have discussed in this blogpost, raising the question of how to integrate that into your build process. We don’t have a perfect answer here just yet. You can of course write your own Makefile, or integrate this into a nix derivation (see the Nix tutorial). If you want to use cabal as the main driver, you can use a custom setup script, or the new SetupHooks API; we don’t provide explicit support for either just yet (#1666 tracks adding support for SetupHooks).

We do offer a “literate” mode, abusing Cabal’s support for literate Haskell to trick it into running hs-bindgen instead; see section Cabal preprocessor integration of the hs-bindgen manual for details.

Template Haskell

If you prefer, you can use hs-bindgen in TH mode; this avoids the need for running the preprocessor altogether. Instead, you can #include a C header in a Haskell module:

let cfg :: Config
    cfg = def & #clang % #extraIncludeDirs .~ [Pkg "cbits"]
 in withHsBindgen cfg def $
       hashInclude "library_a.h"

Since we cannot generate more than one module in this way, by default this splices in bindings for all the types in the header along with the safe imports, though you can override that if you wish. Of course, you can also use hashInclude in more than one Haskell module to manually generate multiple modules. See Template Haskell mode in the manual for details.

Cross-compilation

We rely on libclang for all machine-dependent decisions (as well as parsing the C headers in the first place). Therefore if you want to generate code for a target differing from your host platform, in principle it should suffice to provide the appropriate clang arguments. We are working on a guide for doing so; this will become section Cross-compilation in the manual; however, this section is not quite ready yet (#1630).

Non-portability

It is important to note that the bindings generated by hs-bindgen are not portable in general (indeed, it will be rare that they are). The slogan to remember is:

The bindings generated by hs-bindgen should be regarded as build artefacts.

If you do want to distribute generated bindings as part of your package, you can of course do so, but then you are responsible for making the appropriate provisions in your .cabal file (perhaps using SetupHooks) to check machine architecture, choose between different sets of bindings, etc. At present hs-bindgen does not yet provide any explicit support for making this process easier.

Conclusions

Although Haskell provides good FFI features, writing bindings to large C libraries by hand can be a laborious process. The bindings generated by hs-bindgen are low-level: char* is translated to Ptr CChar, not ByteString or Text or something else still; nonetheless, it should make the process of writing bindings significantly easier. Automatically generating high-level bindings is something we’ll soon turn our attention to.

Before we do so, however, there is still some cleanup to be done before we can release version 0.1. As it stands, the alpha release supports nearly all of C, although some missing corner-cases are still missing (such as implicit fields, #1649); see the issue tracker for a list of missing C features. However, we would like to invite you to start using the alpha release now; there should only be very minor backwards incompatible changes introduced in between versions 0.1-alpha and 0.1, so your code should not break when upgrading to version 0.1.

Acknowledgements

Many people at Well-Typed have contributed to hs-bindgen, including Travis Cardwell, Joris Dral, Dominik Schrempf, Armando Santos, Sam Derbyshire, and others (as well as myself, Edsko de Vries).

Well-Typed is grateful to Anduril Industries for sponsoring this work.

Further reading

• Paper on hs-bindgen: Automatic C Bindings Generation for Haskell, published at Haskell 2025. You might also like to watch the presentation.
• The hs-bindgen manual. Note that while many sections of the manual have already been written, some are still empty or out of date; this is one of the things that will be completed prior to the official 0.1 release.
• The hs-bindgen nix tutorial. This tutorial shows how to generate partial bindings for two libraries (pcap and wlroots). Although this assumes nix, most of the information here will be relevant for non-nix users also.
• The examples folder of the hs-bindgen repo, which contains example partial bindings for a bunch of libraries, such as Botan, minisat, QR-Code-generator, rogueutil, rpm and libpcap.

The installation, configuration, and talks can be found in the official website. The tl;dr first step is installing the debugger:

$ ghc --version # MUST BE GHC 9.14
The Glorious Glasgow Haskell Compilation System, version 9.14.1

$ cabal install haskell-debugger --allow-newer=base,time,containers,ghc,ghc-bignum,template-haskell --enable-executable-dynamic # ON WINDOWS, DO NOT PASS --enable-executable-dynamic
...

$ ~/.local/bin/hdb --version # VERIFY IT'S THE LATEST!
Haskell Debugger, version 0.11.0.0

The second step is configuring your editor to use the debugger via the Debug Adapter Protocol (DAP).

• For VSCode, install the haskell debugger extension.
• For Neovim, install nvim-dap and configure it for haskell-debugger
• For other editors, consult your DAP documentation and let others know how!

Bug reports and discussions are welcome in the haskell-debugger issue tracker.

My MuniHac 2025 talk also walks through the installation, usage, and design of the debugger. Do note much has been improved since the talk was given, and much more will still improve.

A little bit more info

The debugger work is sponsored by Mercury. It’s the project in which I’ve spent most of my full working days (for almost a full year now), with the invaluable help from my team at Well-Typed.

The debugger is meant to work both on trivial files and on large and complex codebases.¹ It is a GHC application so all features are supported. Like HLS, it also uses hie-bios to automatically configure the session based on your cabal or stack project.

Robustness is a main goal of the debugger. If anything doesn’t work, or if you have performance issues, or something crashes, please don’t hesitate to submit a bug. We’ve got a small but respectable testsuite, and have tested performance by debugging GHC itself, but there’s much still to be fixed and improved.

Roadmap: There’s a lot to do. I’m currently working on callstacks and multi-threaded support. Do let me know what features would be most important to you, so I can also factor that into the future planning.

You can find the previous editions collected under the haskell-ecosystem-report tag.

Sponsorship

We offer Haskell Ecosystem Support Packages to provide commercial users with support from Well-Typed’s experts while investing in the Haskell community and its technical ecosystem including through the work described in this report. To find out more, read our announcement of these packages in partnership with the Haskell Foundation. We need funding to continue this essential maintenance work!

Many thanks to our Haskell Ecosystem Supporters: Standard Chartered, Channable and QBayLogic, as well as to our other clients who also contribute to making this work possible: Anduril, Juspay and Mercury; and to the HLS Open Collective for supporting HLS release management.

Team

Ben, who has been a key maintainer of GHC for many years, announced in October that he is moving to a new role that will allow less time for GHC work, though he remains a Partner at Well-Typed. We were delighted to welcome Magnus to the team in November.

The Haskell toolchain team at Well-Typed currently includes:

• Andreas Klebinger
• Hannes Siebenhandl
• Magnus Viernickel
• Matthew Pickering
• Mikolaj Konarski
• Rodrigo Mesquita
• Sam Derbyshire
• Zubin Duggal

In addition, many others within Well-Typed contribute to GHC, Cabal and HLS occasionally, or contribute to other open source Haskell libraries and tools. This report includes contributions from Alex Washburn, Tobias Dammers, Wen Kokke and Wolfgang Jeltsch in particular.

GHC

GHC Releases

• Ben worked on backports for GHC 9.12.3. Zubin continued this work and released rc1 and rc2.
• Ben worked on backports for GHC 9.14.1, and released alpha1, alpha2, alpha3 and rc1. Zubin continued this work and released rc2 and rc3.
• An overview of the status of GHC releases is available on the GHC wiki.

Reinstallable Base

• Matthew has facilitated a period of community discussion about plans for making the base library reinstallable, in order to reduce the coupling between GHC upgrades and library version upgrades. This discussion aims to assess what the consensus is about the project and to establish a shared plan amongst all the stakeholders.

• Wolfgang and Tobias have been investigating the technical aspects of reinstallable base, including questions such as what concretely it takes to compile an API-identical base with multiple GHC versions, what the testing story for an external base repository would be and which parts of ghc-internal can be moved back to base.

Frontend

• Sam added error message hints for unsolved HasField constraints. GHC will now provide additional hints for an unsolved constraint of the form HasField fld_name rec_ty fld_ty, such as similar name suggestions for misspelled field names, warnings about record fields that contain existentials or foralls, import suggestions, etc. This is particularly useful for users of the OverloadedRecordDot extension (#18776, #22382, #26480, !14952).

• Magnus worked on fixing several oversights with the initial implementation of ExplicitLevelImports, such as re-instating an erroneously removed level check (#26099, !15053) and addressing a module graph issue (#26568, !15118). He also took the opportunity to better document that area of the codebase (!15117).

• Sam reworked the way error messages involving invisible components (e.g. the k in Proxy @k ty, or RuntimeRep variables as in TYPE r) are reported, bringing the pretty-printing of multiplicities (from LinearTypes) in line with other invisible components (#26335, #26340, !14761).

• Sam cleaned up how GHC pulls out TypeError msg when computing insoluble constraints. This makes the code much more consistent, but also improves detection of redundant pattern matches as in #26400 (!14802).

• Ben added a special case to avoid GHC emitting “unused package” warnings for packages which don’t contain any modules, as such packages are typically used for other purposes such as providing linker flags (#24120, !14965).

• Sam reviewed Vladislav Zavialov’s work implementing type .. and data .. namespace-specified wildcards from GHC proposal #581 (!15014).

Type-checker plugins

• Sam made GHC invoke type-checker plugins when running the pattern-match checker. This greatly improves pattern match warnings when using type-checker plugins and dealing with GADTs that involve natural numbers (such as Vec :: Nat -> Type -> Type) (#26395, !14797).

• Sam added an Outputable Natural instance in the ghc library, useful in particular for type-checking plugins dealing with natural numbers.

Representation polymorphism

• Sam improved the representation-polymorphism checks to allow function applications in which the representation of the argument is hidden under a type family reduction (such as in #26072). As part of this work, Sam undertook a major overhaul of the typechecking of data constructors as it relates to linear types. The end result is that GHC no longer eta-expands saturated applications of data constructors, resulting in reduced compilation times (!14357, !14764).

• Sam fixed an incorrect coercion constructed during representation-polymorphism checking (#26528, !15038).

Other frontend bug fixes

• Matthew improved error handling in the driver when processing linking nodes, resolving some failures in the T9930fail test (!14964).

• Sam fixed a GHC panic that could occur when writing Arrow/Category code in which overloaded code is instantiated to the function arrow (->) (#26277, !14760).

• Sam reverted !14402 in order to fix GHC failing to compile Agda (#26154, !14763).

• Sam fixed a GHC panic that could occur when reporting an invalid record update in which the record constructor was out of scope (#26391, !14804).

• Sam fixed a GHC panic that could occur when typechecking a pattern synonym that uses a view pattern (#26465, !14936).

Backend

• Matthew improved the memory usage of recompilation checking by using OsPath in the getModificationTimeIfExists function (!14917).

• With the help of Andreas, Sam identified and fixed a long standing serious bug in the Cmm Sink optimisation pass, which is responsible for inlining variable declarations at the Cmm level (#26550, !15041).

• Sam implemented several fixes relating to register allocation, in particular in relation to tracking formats of registers for the purpose of spilling to the stack (#26542, !15041, #26411, !15121).

• Sam fixed an oversight in the Cmm liveness analysis pass in which GHC did not properly combine register allocations at different formats (#26595, #26611, !15121).

• Zubin re-engineered the tags used for Uniques within GHC, making GHC use a proper ADT rather than ad-hoc characters to namespace Uniques (#26246, !14639). This also makes it clearer to plugin authors which tag to use when generating Uniques.

• Andreas investigated the performance of occurrence analysis, adding a new performance test from #26425 (!14927). Several improvements were then made, such as avoiding space leaks by being a bit more strict (!14928).

• Andreas added handling of ticks in the sizeExpr computation (!14883).

• Andreas added a regression test for #26056 (!14719), which had been fixed by his earlier work on the interaction of ticks with unsafeCoerce# (!13413).

• Matthew made the loading of bytecode static pointer entries more lazy (!14900).

GHCi and bytecode interpreter

• Matthew added support for bytecode libraries, allowing the bytecode for standard libraries (such as base) to be distributed, which in turns allows the GHCi debugger to step through these libraries (!15062).

• Matthew added support in GHC for generating bytecode object files (.gbc). This allows compiled bytecode to be cached, improving load times of the interpreter (!14717).

• Matthew refactored the code involved in loading bytecode in order to unify two code-paths with a lot of duplication (#26459, !14901).

• Alex worked on issues with the reliance of the bytecode linker on laziness (#25636, !15092).

• Rodrigo improved the performance of the bytecode interpreter by pruning certain useless sequences of instructions (!14576).

• Rodrigo changed the breakpoint index from Word16 to Word32 in order to fix #26325 (!14691).

• Matthew renamed, within the GHC codebase, the “interpreter backend” to the “bytecode backend”, which names what the backend produces rather than the means by which the code will eventually run (!14915).

Trees That Grow

• Alex has been working on fully realising the vision introduced by the “Trees that Grow” (TTG) paper by making the syntax tree used by GHC fully independent of GHC. This aims at making GHC more modular and potentially making it easier for other Haskell tools to reuse GHC’s syntax tree without acquiring a dependency on the full ghc package.

• Alex decoupled Language.Haskell.Syntax.Type from depending on GHC.Utils.Panic (#26626, !15134), and initiated preliminary work to decouple other modules in the Language.Haskell namespace from GHC internal modules.

Runtime system and linker

• With the help of Andreas, Zubin identified and fixed a segfault that would occur on Windows due to DLL name strings getting freed while they were still used (#26613, !15100).

• Ben added dynamic initialisation of built-in RTS closures in order to address MacOS 26 breakage of -undefined dynamic_lookup (#26166, !14902).

• Rodrigo moved certain symbols from ghc-internal to the rts package, as part of the effort to address the aforementioned MacOS 26 breakage (!14956).

• Ben improved how the RTS deals with address space reservations in low memory, avoiding potential loops (#26151, !14466).

• Matthew fixed a deadlock that could occur when the RTS was shutting down while trying to emit eventlogs (#26573, !15068).

• Ben fixed a runtime loop that could occur with the non-moving garbage collector by using atomic operations (#26053, !14519).

• Rodrigo dropped some obsolete logic in the RTS relating to Windows DLLs (!14873).

• Hannes fixed an off-by-one error in an assertion in the RTS (!15019).

Profiling

• Hannes enabled the stack frame annotation mechanisms by default for IPE backtraces (#26218, !14721).

• Hannes moved the implementation of stack decoding logic from ghc-heap to ghc-internal, in order to allow its usage in GHC.Stack.CloneStack in base (!14543).

• Matthew implemented several fixes to the stack decoding logic when using the profiled runtime (#26507, !14991).

• Building on previous work by Finley McIlwaine, Hannes introduced the -fno-distinct-constructor-tables and -fdistinct-constructor-tables-only flags which provide more granular control over which constructors get different info tables when using info-table profiling (#23703, !14705). This is useful as -fdistinct-constructor-tables can lead to unreasonably large binary sizes in some projects.

Template Haskell

• Ben improved the documentation of getQ from the template-haskell library to better reflect its semantics (#26484, !14970).

• Sam improved the Template Haskell compatibility story for the new “expressions in specialise pragmas” patch by using a pattern synonym to avoid breaking existing Template Haskell code (#26356, !14729).

Build system and packaging

• Andreas fixed a bug with long paths on Windows when invoking GHC to gather module dependency information (!15060).

• Rodrigo made further progress on removing global settings files in favour of per-target configuration (#26227, !14554).

• Rodrigo ensures that flags such as CFLAGS apply only to the build phase, and are not carried over to target settings (#25637, !14854).

• Rodrigo deleted some stale settings from the Hadrian bindist configure script (#26478, !14937).

• Ben fixed zstd-enabled builds (#26312, !14693).

• Ben fixed inconsistencies in configure checks relating to C compilers (#26394, !14796).

• Ben fixed a build failure caused by Clang warning on implicit constant arrays (#26502, !14971).

• Ben made a change to the order of arguments when building with Hadrian (#25281, !14318). Thanks to Cheng Shao for helping land this work.

• Andreas added hpc to the release script (!15056).

CI and testing

• In an ongoing general effort to improve CI stability in nightly pipelines, Magnus fixed several issues with the CI to make nightly GHC builds available again. This included fixing the Debian 9 CI job, which was failing due to a small Python issue (!15127).

• Andreas improved the jspace test to be less flaky, fixing a recurrent issue with AArch64 LLVM jobs in CI (#25401, !14837).

• Zubin added support for testsuite output files that depend on the testsuite way, fixing test failures that used the external interpreter (#26552, !15078).

• Zubin addressed an issue in which spurious linker warnings were causing testsuite failures (#26349, !14615).

• Zubin fixed the links to -fno-xxx flags in the user’s guide (#26352,!14794).

• Sam fixed a Unix vs Windows filepath issue with the codes test (which checks test coverage of error codes). This makes the test more robust on Windows (#25178, !14778).

• Ben made whether a job is a release job into a GitLab “input”, facilitating release management for maintainers (!14933).

• Ben disabled split sections on FreeBSD in order to fix CI failures on FreeBSD (#26303, !14782).

• Matthew made some changes to the testsuite driver to allow some tests to run only with GHCi (!14914).

• Rodrigo made some progress on fixing the test-reinstall jobs which tests compilation of GHC using cabal (#26202, !14518).

Infrastructure

• Ben migrated ghcup-metadata jobs to run on the new OpenCape CI runners (!14814).

• Magnus set up new Windows x86 runners and improved stability and observability of the GHC GitLab instance.

Haddock

• Sam made several changes to the information stored in interface files for data constructors, in order to improve Haddock output. This allows Haddock to display the user-written kind signature (instead of aggressively expanding type synonyms), as well as preserve user-written type variable names (#26246, !14868).

Cabal

• Mikolaj took part in the preparation of the upcoming Cabal release 3.16.1.0.

• Matthew helped on the Cabal issue tracker and reviewed PRs from other Cabal contributors.

• Matthew wrote up a Cabal proposal to add support for bytecode library files and bytecode object files.

• Matthew fixed two issues reported in #11107 that occur when starting an interactive session with cabal repl (PR #11237).

Haskell Language Server

• Zubin prepared the release of HLS 2.12.0.0 (PR #4728).

• Hannes fixed a bug in which GHC plugins aren’t loaded for files that are checked in the background (#4631, PR #4749).

• Hannes and Andreas diagnosed a recurrent segfault that turned out to be caused by how the ghc-lib-parser library would re-use shared CAFs defined in the RTS but expecting a different memory layout due to API changes in between GHC versions (HLS issue #4674, GHC tickets #26033 and #26553).

Haskell Debugger

• Rodrigo continued work on the new Haskell Debugger. The debugger will first be compatible with the upcoming GHC 9.14 release, and we’ll be looking forward to you all trying it out.

• Rodrigo presented the debugger at MuniHac in a tutorial oriented for users. The recording is available here.

• A detailed list of improvements to the debugger can be found in the changelog. Notably:

  • Several robustness fixes and bug fixes
  • Support for stepping-out of functions
  • Conditional and hit conditional breakpoints
  • Proxy program which runs on the terminal to allow passing input to the debuggee

• Matthew and Rodrigo worked on allowing customisation of how the debugger displays user values. By instantiating the DebugView type class for your data type, you can control how the debugger expands and displays the value in the debugger.

Live monitoring using the eventlog

• Wen has been working on improvements to eventlog-live, a collection of libraries and tools for the live profiling of Haskell applications using the eventlog-socket library to stream data from the eventlog to an external monitoring process.

• Matthew has implemented support for creating TCP sockets and increased the testing coverage of the eventlog-socket library (PR #18).

• The socket created can now be used bidirectionally using a control protocol to instruct the monitored process for further heap profiling information (PR #20). In future work, the protocol will be extended to allow other libraries to register commands.

Lazy Linearity for a Core Functional Language

Rodrigo Mesquita, Bernardo Toninho

POPL 2026 (PDF)

Abstract. Traditionally, in linearly typed languages, consuming a linear resource is synonymous with its syntactic occurrence in the program. However, under the lens of non-strict evaluation, linearity can be further understood semantically, where a syntactic occurrence of a resource does not necessarily entail using that resource when the program is executed. While this distinction has been largely unexplored, it turns out to be inescapable in Haskell’s optimising compiler, which heavily rewrites the source program in ways that break syntactic linearity but preserve the program’s semantics. We introduce Linear Core, a novel system which accepts the lazy semantics of linearity statically and is suitable for lazy languages such as the Core intermediate language of the Glasgow Haskell Compiler. We prove that Linear Core is sound, guaranteeing linear resource usage, and that multiple optimising transformations preserve linearity in Linear Core while failing to do so in Core. We have implemented Linear Core as a compiler plugin to validate the system against linearity-heavy libraries, including linear-base.

This post explains the debugging process which led to the resolution of this ticket. At the start of the investigation the program used 2 GB memory, at the end, about 200 MB, an improvement of approximately 10x!

First impressions

I first looked at the ticket report to get an idea of the problem.

• The ticket contained a profile generated by eventlog2html which showed a profile of a program which runs in two phases. During these two phases the memory increased before resetting to some baseline and then increasing again.
• Reproduction instructions were provided in the subsequent comment, I could run these easily to reproduce the issue. I altered the options to use the -hT profiling mode to generate a basic heap profile without needing to compile with profiling support. This is a useful technique to get an initial handle on the problem.
• The instructions used profiling-detail: all-functions, which will insert many cost centres into the program which will significantly affect the runtime characteristics of the resulting program. I replaced this with profiling-detail: late.

Most importantly, the ticket lacked a precise description about what the issue was with the profile. It may have been that this was exactly the memory profile that the program should exhibit! When starting to think about memory issues, thinking about memory invariants is a very helpful technique. The first question I ask myself is:

What is the memory invariant that the program should uphold?

This situation was a useful test of this technique, since I had no domain knowledge of what the program did, what the test did or what function the library even aimed to perform. It certainly highlighted to me the importance of knowing your domain and knowing the invariants.

Memory invariants

A memory invariant is a property that your program’s heap should obey. Establishing a memory invariant makes it easier to verify and fix memory issues, since if you have a precise invariant, it is easy to check whether the invariant holds.

A memory invariant consists of two parts:

• A predicate on the heap
• The timeline over which the predicate should hold

For example, some predicates on the heap might be:

• “No constructors of type T are alive”
• “There is an upper bound of 20000 bytestrings”
• “There are exactly 5 live closures of type T”
• “No closures of type T are reachable from closures of type Q”

When paired with a timeline, a memory invariant is formed. Example timelines include:

• “Before the second phase of the program”
• “During the cleanup phase”
• “After initialisation”
• “Between runs 10 and 100”
• “At all points of the program’s execution”

Establishing a memory invariant requires domain knowledge about the program. Without first establishing an invariant (even informally in your head), you can’t begin to debug memory usage of a program. The main challenge for me when investigating this issue was coming up with a memory invariant.

Initial investigation

In order to get an idea of how to proceed, I generated a “Profile by Closure Type” using the -hT runtime system option.

cabal run bittide-instances:unittests -- -p RegisterWb +RTS -hT -l -RTS

The result was a unittests.eventlog file which contains the profiling information.

I rendered this eventlog using eventlog2html and inspected the result in my browser.

eventlog2html unittests.eventlog

The profile shows a coarse breakdown by the type of closures currently alive on the heap. The maximum value reported in the profile is about 600 MB. This value relates to the total memory used by the process (2 GB), but doesn’t include additional memory used by the RTS. The relationship between live bytes and OS memory usage is explained fully in this blog post. Reducing live memory is a good way to reduce the overall memory usage of your program, but it isn’t the only factor.

The top four bands came from

• Clash.Signal.Internal.:-
• Protocols.Wishbone.Wishbone.S2M
• THUNK_1_0
• 2-tuples (,)

and as can be easily seen from the “detailed pane”, the patterns of allocation of these top four bands closely align with each other:

Looking at these correlations in the detailed pane can be invaluable in understanding the root issue, since memory issues are normally about different closures retaining each other, they are allocated together and retained together. Seeing these overall patterns can give you context about what exact kind of thing is using the memory.

Without a clear memory invariant, I wanted to get a better idea about these top 4 bands of allocation, I had a hypothesis at this stage that the THUNK closures were contained within tuples, which were retaining the WishboneS2M and :- constructors.

A more specific profile with info table provenance

I want to know information about the precise source location of the :-, WishboneS2M constructors and thunks. Therefore I enabled a few more debugging options to add this information to the binary:

• -finfo-table-map: Gives a source location to each thunk and data constructor
• -fdistinct-constructor-tables: Distinguishes between allocation sites of each constructor

Then if you make a profile using the -hi option, you get a very fine-grained breakdown about where exactly in your program to start looking. That’s useful for me since I didn’t yet look at any of the source code!

cabal run bittide-instances:unittests -- -p RegisterWb +RTS -hi -l -RTS

After consulting the source code in the relevant places, I quickly realised that this wouldn’t necessarily be as straightforward an investigation as I had hoped. I had hoped that the THUNK_1_0 locations reported in the -hT profile would be clear that my hypothesis about retaining was correct, but the -hi profile didn’t show up anything directly wrong. These locations were normal ways you could construct a Clash circuit.

At this stage my lack of a memory invariant or some domain knowledge was a hindrance. I took the opportunity to consult Ben who knew about the Clash ecosystem and asked on the ticket what the expected profile should look like.

• The program is simulating a digital circuit.
• The :- constructor represents a single time-step of simulation.
• The expected memory profile is to use a constant (or near constant) amount of memory, since the circuit being simulated has bounded size.

For this program, a plausible invariant might have been: the number of :- constructors should remain roughly constant during simulation.

With this knowledge, the number of :- constructors alive seemed to be the biggest unexpected source of memory usage. Knowing that :- is a data constructor with two arguments, Each allocation is 24 bytes, so 240 MB of live :- closures corresponds to roughly 10 million constructors. That is certainly an issue.

Secondly, the number of live WishboneS2M constructors looked wrong. I didn’t have a good idea of the domain still, but by similar arithmetic, many millions of these are also resident on the heap.

These two facts gave me some further avenues to investigate but I was going to need to use ghc-debug to investigate further.

Using ghc-debug to investigate retainers

Using ghc-debug I wanted to establish

• What was retaining :- constructors
• What was retaining WishboneS2M constructors

Therefore I instrumented the test executable, and launched ghc-debug-brick in order to query what was retaining :- and WishboneS2M. This was the start of making progress on the investigation.

To find the retainers of :-, I paused the test program just after the test started and used the “Find retainers” command in ghc-debug-brick. The result was a list of 100 :- closures, and when expanded, each one shows the path which was taken to reach it. It wasn’t very important where I paused, as long as it was in this initial period, since we saw in the profile that the :- closures are alive and linearly increasing for the whole phase.

When looking at retainers of :-, it was immediately noticeable that the program contained very long chains of :- constructors (upwards of 5000 long). This looked wrong to me, since my understanding was that :- was being used as a control operation to drive the simulation of the circuit.

The information about where each :- constructor was allocated is not very informative. That just gives me a location inside the library functions.

The question then becomes, why is :- being retained? I scrolled, for a long while, and eventually get to the point where the chain of :- constructors is retained by a non-:- constructor. That’s the interesting part since it’s the part of the program which led to the long chain being retained.

At the time, I didn’t think this looked so interesting, but also I didn’t know what I was looking for exactly.

So I kept going down the stack, looking for anything which looked suspicious. In the end, I got quite lucky: I found a tuple which was retained by a thunk. Since I had compiled with profiling enabled, I could see the cost centre stack where the thunk was allocated, which pointed to the implementation of singleMasterInterconnectC.

Culprit 1: lazy unzip

In the source code of singleMasterInterconnectC, I worked out this part of the allocation was coming from these calls to unzip.

  go (((), m2s), unzip -> (prefixes, unzip -> (slaveMms, s2ms))) = ((SimOnly memMap, s2m), (\x -> ((),     ((), x))) <$> m2ss)

Then I looked at the definition of unzip, and found it was defined in a very lazy manner.

unzip :: Vec n (a,b) -> (Vec n a, Vec n b)
unzip xs = (map fst xs, map snd xs)

With this definition, the thunk created by applying map to fst and xs retains a reference to xs, which retains a reference to all the bs as well as the as. In a definition which performs a single iteration, if you force either list, both will be evaluated and leave no reference to the other half. I changed this definition to one which performed a single iteration and this had a massive positive effect on memory usage.

unzip xs
  | clashSimulation = unzipSim xs
  | otherwise = (map fst xs, map snd xs)
 where
  unzipSim :: Vec m (a,b) -> (Vec m a, Vec m b)
  unzipSim Nil = (Nil, Nil)
  unzipSim (~(a,b) `Cons` rest) =
    let (as, bs) = unzipSim rest
    in (a `Cons` as, b `Cons` bs)

This issue was hard to spot with the tools, I got lucky, but it was harder to spot since unzip was also marked as INLINE. In the end, I guessed right, but it’s a bit unsatisfying to not have a great story about how I worked it out, but I knew the answer was somewhere in the retainer stack I was looking at, and eventually I looked in the right place.

This problem is similar to one you can encounter when using conduit and similar libraries. In short, by sharing a thunk between two consumers, the input structure can be retained longer than intended. Since one part of the program continues by evaluating the thunk, the other reference is updated to the result of the thunk being evaluated. This is a problem though, since the original structure was intended to be used for control and discarded immediately after driving the next step of execution.

Culprit 2: lazy record update retains old field values

Once the original problem had been fixed, memory usage was much improved. I circled back to the start to look at a modified -hT profile. Perhaps there were still other problems lurking?

The final phase of memory usage looks much better, so I turned my attention to the initial phase. In the initial phase, it looked like OUTPUT was increasing linearly.

I turned to ghc-debug again, inspected the retainers of the OUTPUT constructor and discovered that the fields of WishboneM2S were not being strictly updated, indirectly keeping a reference to the OUTPUT constructor.

I looked at the source location that WishboneM2S was allocated,

and made the update strict:

     toSlaves =
-      (\newStrobe -> (updateM2SAddr newAddr master){strobe = strobe && newStrobe})
+      (\newStrobe -> strictM2S $ (updateM2SAddr newAddr master){strobe = strobe && newStrobe})
         <$> oneHotOrZeroSelected
     toMaster
       | busCycle && strobe =
@@ -152,10 +152,8 @@ singleMasterInterconnect (fmap pack -> config) =
             (maskToMaybes slaves oneHotOrZeroSelected)
       | otherwise = emptyWishboneS2M

+    strictM2S (WishboneM2S !a !b !c !d !e !f !g !h !i) = WishboneM2S a b c d e f g h i

This resulted in another reduction in memory usage:

Culprit 3: lack of sharing in iterate

When I returned to the profile a final time, the memory usage seemed much better, especially in the initial section. I had now eliminated retained :- constructors, so the overall memory usage was much lower, but still increasing slightly. I turned to a -hi profile again to get more information about the THUNK bands.

Looking at the source code for the sat_ssVQ thunks, they come from the Clash.Sized.Vector.map function. Therefore… back to ghc-debug to see what retains these map thunks, and the callstack where they are allocated from. This time I used “Find Retainers (Exact)”, to find closures which are named sat_ssVQ_info.

The first one I looked at, I found was allocated from iterateI by inspecting the cost centre stack.

iterateI was implemented as follows:

iterateI :: forall n a. KnownNat n => (a -> a) -> a -> Vec n a
iterateI f a = xs
  where
    xs = init (a `Cons` ws)
    ws = map f (lazyV xs)

Reasoning about the definition, you can see that iterateI will result in a vector of the form:

a `Cons` map f (a `Cons` (map f (a `Cons` ...)))

As a result, each element of the vector will independently compute f^n a, no intermediate results will be shared, a quadratic amount of thunks for the n applications of f will be allocated.

Defining iterateI in a directly recursive style means only a linear amount of thunks will be allocated and f will be computed only a linear number of times.

iterateU :: UNat n -> (a -> a) -> a -> Vec n a
iterateU UZero _ _ = Nil
iterateU (USucc s) f a = a `Cons` iterateU s f (f a)

Even better, for the specific example, was to use a strict accumulator, so no intermediate thunks were allocated or retained in the early part of the program.

Final result

The final profile shows slightly higher memory usage in the initial phase of the program’s execution than the second phase, but looking at the detailed pane, I could identify why the memory was retained.

Overall, the total memory usage decreased from about 2 GB to 200 MB. There is probably still some improvement which can be made to this profile, but we felt like it was a good place to stop for this post.

Conclusion

The goal of this post is to document the thought process involved in investigating a memory issue. Overall, I feel like I would have found it easier to fix the problem with some domain knowledge. Once I acquired some knowledge about the domain I made much more rapid progress about what to investigate.

The main cause of the memory leak was not obvious, and I got a bit lucky in finding the right place. One issue being the problem was obfuscated by the problematic definition being inlined. In general, with a busy heap, finding the needle can be quite tricky. Debugging the subsequent issues was more straightforward.

In the future, we want to explore more reliable ways to identify and investigate the kinds of memory invariants that were violated in this program. For example, it was crucial to know that :- should not be retained, perhaps additional language design can express that property more clearly. On another note, a logical specification of memory invariants could be useful to automatically detect and pause the program at the exact point a violation was detected. There remains significant potential to improve our memory debugging tooling!

This work was performed for QBayLogic as part of their Haskell Ecosystem Support Package. If your company is using Haskell and from time to time requires expert help in issues like this, our packages fund maintenance on core tooling such as GHC and Cabal, as well as development or support for your specific issues. Please contact info@well-typed.com if we might be able to help you!

The io-sim Haskell library, developed by Well-Typed in partnership with engineers from IOG and Quviq, offers a compelling solution to this problem. The library provides a pure simulation environment for IO computations, enabling deterministic execution of concurrent code with accurate time simulation and detailed execution traces. Unlike other testing approaches, with io-sim one is able to write highly concurrent, real-time systems and verify their timeliness constraints in a deterministic manner, by accurately simulating GHC’s runtime system (e.g. asynchronous exceptions, timeouts & delays, etc.).

This blog post introduces and explores io-sim through a practical example: debugging an elevator controller that violates its response time requirements.

There’s also this great blog post announcing io-sim and it goes a bit more into detail about its features!

The Problem

Consider a simple elevator located in a three-floor building (ground, first , second). It takes roughly 1 second for the elevator to go up and down between each floor. The service requirement is: no passenger should wait more than 4 seconds from pressing the call button until the elevator doors open at their floor. It should be possible to test and verify this requirement when writing our elevator controller.

This ensures a reasonable quality of service and prevents frustration. Given the short distance between floors, 4 seconds is sensible. In the worst case, the elevator must travel from ground to second floor and back again.

Here’s a first attempt at modelling the system. Let’s start with the core data structures:

data Direction = Up | Down | None
  deriving (Eq, Show)

data Floor = Ground | First | Second
  deriving (Eq, Ord, Show, Enum)

data ElevatorState = ElevatorState
  { currentFloor :: Floor
  , moving       :: Direction
  , requests     :: [Floor]
  } deriving (Show)

The elevator’s state tracks three things: where it currently is, which direction it’s moving (if any), and a queue of floor requests.

The system has two main components that run concurrently:

• An elevator controller that continuously processes the request queue
• Button press handler that adds new floor requests

Let’s look at the controller first:

-- | Initialize an empty elevator state.
--
-- The elevator starts on the ground floor
--
initElevator :: IO (TVar ElevatorState)
initElevator = newTVarIO $ ElevatorState Ground None []

-- | Elevator controller logic.
--
-- 1. Read the current 'ElevatorState'
-- 2. Check if there are any requested floors
-- 3.
--    3.1. Block waiting for new requests if there aren't any
--    3.2. If there any requests, move to the floor at the top of the queue.
--
-- Straightforward FIFO elevator algorithm.
--
elevatorController :: TVar ElevatorState -> IO ()
elevatorController elevatorVar = forever $ do
  -- Atomically get the next floor from the queue
  (nextFloor, dir) <- atomically $ do
    state <- readTVar elevatorVar
    case requests state of
      []               -> retry  -- Block until a request arrives
      (targetFloor:rs) -> do
        -- Remove the floor from queue and start moving
        let direction = getDirection (currentFloor state) targetFloor
        writeTVar elevatorVar $ state
          { moving = direction, requests = rs }
        return (targetFloor, direction)

  putStrLn ("Going " ++ show dir ++ " to " ++ show nextFloor)
  moveToFloor elevatorVar nextFloor

The moveToFloor function simulates the physical movement of the elevator.

moveToFloor :: TVar ElevatorState -> Floor -> IO ()
moveToFloor elevatorVar targetFloor = do
  elevatorState <- readTVarIO elevatorVar
  when (currentFloor elevatorState /= targetFloor) $ do
    -- Takes 1 second to move between floors
    threadDelay (1000000 * numberOfFloorsToMove)
    atomically $
      modifyTVar elevatorVar (\elevatorState' ->
        elevatorState' { currentFloor = targetFloor
                       , moving       = Idle
                       }
        )
  putStrLn ("Arrived at " ++ show targetFloor)

The buttonPress function handles both external calls (someone waiting for the elevator) and internal requests (someone inside selecting a destination):

-- | Whenever a button is pressed this function is called.
--
-- There are two scenarios when a button is pressed:
--
-- 1. When a person is calling the elevator to a floor in order to enter it.
-- 2. When a person is inside the elevator and wants to instruct the elevator
--    to go to a particular floor.
--
buttonPress :: TVar ElevatorState -> Floor -> IO ()
buttonPress elevatorVar floor = do
  putStrLn ("Pressing button to " ++ show floor)
  atomically $
    modifyTVar elevatorVar $ \state -> do
      case requests state of
        rs@(nextFloor:_)
          | let mostRecentRequestedFloor = last rs
          ,    nextFloor /= floor
            || mostRecentRequestedFloor /= floor ->
            state { requests = rs ++ [floor] }
          | otherwise -> state
        [] -> state { requests = [floor] }

Consider the following example scenario and timeline:

0. The elevator starts on the ground floor.
1. Person A is on the first floor and presses the button to call the elevator to the first floor.
2. While the elevator is going up, Person B arrives on the ground floor calls it to the ground floor.
3. Elevator arrives at the first floor.
4. Person A enters and presses the button to go to the second floor.
5. Elevator goes to the ground floor to pick up Person B.
6. Person B enters and presses the button to go to the first floor.
7. Elevator goes to the second floor.
8. Elevator goes to the first floor.

-- | This example mimicks the scenario above, pressing buttons in the right
-- order.
elevatorExample :: [Floor] -> IO ()
elevatorExample floors = do
  elevator <- initElevator
  withAsync (elevatorController elevator)
    $ \controllerAsync -> do
        -- Simulate multiple people pressing buttons simultaneously
        forConcurrently_ floors (buttonPress elevator)
        threadDelay (10 * 1000000)
        cancel controllerAsync

elevatorExample [First, Ground, Second, First]

This function spawns the elevator controller and then simulates multiple button presses happening concurrently. Let’s trace through our example:

Pressing button to First
Going Up to First
Pressing button to Ground
Pressing button to Second
Pressing button to First
Arrived at First
Going Down to Ground
Arrived at Ground
Going Up to Second
Arrived at Second
Going Down to First
Arrived at First

Does such a simple implementation adhere to the specified time constraints? The answer is no, a FIFO elevator algorithm is easy to implement but can be inefficient if the requests are spread out across floors, leading to more travel time.

How would one go about to test/verify this? Testing timeliness constraints in concurrent IO is tricky, due to its non-deterministic nature and limited observability.

io-sim: Deterministic IO Simulator

io-sim closes the gap between the code that’s actually run in production and the code that runs in tests. Combined with property based testing techniques it is possible to simulate execution of a program for years worth of simulated time and find reproducible, rare edge-case bugs.

io-sim achieves this by taking advantage of the io-classes set of packages, which offers a class-based API compatible with most of the core Haskell packages, including mtl. In general the APIs follow the base or async

io-sim is a time based, discrete event simulator. Which means, it provides a granular execution trace that can be used from inspecting the commit order of STM transactions to validating a high level, temporal logic property over some abstract trace. The best part is that code requires minimal changes to use io-sim, just polymorphic type signatures that work with both IO and IOSim monads. Here’s the elevator controller code refactored for testing with io-sim:

initElevator :: MonadSTM m => m (TVar m ElevatorState)
initElevator = ...

elevatorController
  :: ( MonadSTM m
     , MonadDelay m
     , MonadSay m
     )
  => TVar m ElevatorState -> m ()
elevatorController elevatorVar =
  ...
  say ("Going " ++ show dir ++ " to " ++ show nextFloor)
  ...

moveToFloor
  :: ( MonadSTM m
     , MonadDelay m
     , MonadSay m
     )
  => TVar m ElevatorState -> Floor -> m ()
moveToFloor elevatorVar targetFloor = do
  ...
  say ("Arrived at " ++ show targetFloor)

getDirection :: Floor -> Floor -> Direction
getDirection from to = ...

buttonPress
  :: ( MonadSTM m
     , MonadSay m
     )
  => TVar m ElevatorState -> Floor -> m ()
buttonPress elevatorVar floor = do
  say ("Pressing button to " ++ show floor)
  ...

elevatorExample
  :: ( MonadSTM m
     , MonadAsync m
     , MonadDelay m
     , MonadSay m
     )
  => [Floor]
  -> m ()
elevatorExample floors = ...

Notice that only type signatures and IO operations needed changes. The core business logic remains identical. When instantiated to IO, say becomes putStrLn, but in the IOSim monad it produces traceable events.

main :: IO ()
main = do
  let simpleExample :: [Floor]
      simpleExample = [First, Ground, Second, First]

  -- Runs the 'elevatorExample' in IO. This outputs exactly the same output
  -- as before
  elevatorExample simpleExample

  -- Runs the 'elevatorExample' in IOSim.
  putStrLn . intercalate "\n"
           . map show
           . selectTraceEventsSayWithTime
           -- ^ Extracts only the 'say' events from the 'SimTrace' and
           -- attaches the timestamp for each event
           --
           -- selectTraceEventsSayWithTime :: SimTrace a -> [(Time, String)]
           --
           -- This function takes a 'SimTrace' and filters all 'EventSay'
           -- traces. It also captures the time of the trace event.
           $ runSimTrace (elevatorExample simpleExample)
           -- ^ Runs example in 'IOSim'
           --
           -- runSimTrace :: (forall s. IOSim s a) -> SimTrace a
           --
           -- This function runs a IOSim program, yielding an execution trace.

Running the program above, the first noticeable thing is that when the program runs in IO, it actually takes 10 real seconds to run due to the threadDelay calls. However, when the program runs in IOSim the output is instantaneous. This is because io-sim operates on simulated time rather than wall-clock time, i.e. only the internal clock advances when threads execute time-dependent operations like threadDelay or timeouts. Between these operations, the simulation executes as if it had infinite CPU speed, i.e. all computations at a given timestamp complete instantly, yet remain sequentially ordered and deterministic.

(Time 0s,"Pressing button to First")
(Time 0s,"Going Up to First")
(Time 0s,"Pressing button to Ground")
(Time 0s,"Pressing button to Second")
(Time 0s,"Pressing button to First")
(Time 1s,"Arrived at First")
(Time 1s,"Going Down to Ground")
(Time 2s,"Arrived at Ground")
(Time 2s,"Going Up to Second")
(Time 4s,"Arrived at Second")
(Time 4s,"Going Down to First")
(Time 5s,"Arrived at First")

This particular scenario doesn’t violate the constraint. To find violations, property-based testing can explore the space of possible request patterns. The only problem is that our say traces are strings which is not a very functional way of tracing things.

contra-tracer: Structured Tracing

While say provides basic, string-based tracing, real systems need structured tracing of domain-specific events. String-based logging quickly becomes inadequate when trying to verify complex properties or analyze system behavior programmatically. Tracing strongly-typed events that can be filtered, analyzed, and used in property tests is much better. The contra-tracer library provides a contravariant tracing abstraction that integrates seamlessly with io-sim.

The key advantages of structured tracing:

• Type Safety: Events are strongly typed, preventing typos and logging errors.
• Composability: Tracers can be filtered, transformed, and combined.
• Testability: Events can be programmatically analyzed in tests.

All one needs to do is to have a custom trace type:

data ElevatorTrace = ButtonPress Floor
                   | Going Direction Floor
                   | ArrivedAt Floor
                   deriving (Eq, Show, Typeable)

And substitute all calls to say for traceWith tracer (ButtonPress floor), for example.

With structured tracing in place, extracting and analyzing traces becomes type-safe and straightforward:

-- | Extract typed elevator events with timestamps

extractElevatorEvents :: SimTrace a -> [(Time, ElevatorTrace)]
extractElevatorEvents =
  selectTraceEventsDynamicWithTime

Property-Based Testing: Verifying Timing Constraints

The elevator system began with a clear requirement: no passenger should wait more than 4 seconds. The FIFO implementation seemed reasonable, but the elevator can end up travelling between the bottom and top floors whilst someone in the middle waits their turn.

With typed traces from contra-tracer and deterministic simulation from io-sim, QuickCheck can systematically explore the space of possible request patterns and verify this property.

To verify our timing constraint, we need to:

1. Generate random sequences of floor requests
2. Run each sequence through the elevator simulation
3. Check that every passenger gets service within 4 seconds

Let’s start with the test data generation:

-- | 'Floor' Arbitrary instance.
--
-- Randomly generate floors. The shrink instance is the most important here
-- since it will be responsible for generating a simpler counterexample.
--
instance Arbitrary Floor where
  arbitrary = elements [Ground, First, Second]
  shrink Second = [Ground, First]
  shrink First  = [Ground]
  shrink Ground   = []

The shrinking strategy is important because when QuickCheck finds a failing case with many floors, it will try simpler combinations to find the minimal reproduction of the original input.

To verify the property that no passenger waits more than 4 seconds for the elevator to arrive to its floor, one needs to track the button presses and measure how long until the elevator arrives.

The property works by maintaining a map of pending requests. Each ButtonPress adds an entry (keeping the earliest if multiple people request the same floor), and each ArrivedAt checks if that floor was requested and whether the wait exceeded 4 seconds:

-- Traverse the event trace and check if there is any gap longer than 4s
-- between requests and the elevator arriving at the request's floor.
--
violatesFourSecondRule :: [(Time, ElevatorTrace)] -> Property
violatesFourSecondRule events = counterexample (intercalate "\n" $ map show events)
                              $ checkViolations events Map.empty
  where
    checkViolations :: [(Time, ElevatorTrace)] -> Map Floor DiffTime -> Property
    -- Fail if there are pending requests
    checkViolations [] pending =
      counterexample ("Elevator never arrived at: " ++ show pending)
                     (Map.null pending)
    checkViolations ((Time t, event):rest) pending =
      case event of
        -- Add request to the pending requests map. Note that if there's
        -- already a request for a particular floor, overwriting the
        -- timestamp is not the right thing to do because there's an older
        -- request that shouldn't be forgotten.
        --
        ButtonPress floor ->
          checkViolations rest (Map.alter (maybe (Just t) Just) floor pending)

        -- The elevator arrived at a floor. Check if it took more than 4
        -- seconds to do so. If not continue and remove the request from
        -- the pending map.
        --
        ArrivedAt floor ->
          case Map.lookup floor pending of
            Nothing ->
              checkViolations rest pending
            Just requestTime
              | let time = t - requestTime
                counterexample (  "Passenger waited "
                               ++ show time
                               ++ " for the elevator to arrive to the "
                               ++ show floor
                               ++ " floor"
                               ) False
              | otherwise -> checkViolations rest (Map.delete floor pending)

        _ -> checkViolations rest pending

Then it is just a matter of running the example for randomly generated inputs, extract the trace and use QuickCheck to assert if the property is true or not.

prop_no_passenger_waits_4_seconds :: [Floor] -> Property
prop_no_passenger_waits_4_seconds floors =
  -- Run the button press sequence and get the execution trace
  --
  let trace = extractElevatorEvents
            $ runSimTrace
            $ elevatorExample (Tracer (emit traceM)) floors

   in violatesFourSecondRule trace

  where

Running this property, QuickCheck quickly finds a counterexample:

*** Failed! Falsified (after 8 tests and 2 shrinks):
[Second,Ground,First]
(Time 0s,ButtonPress Second)
(Time 0s,Going Up Second)
(Time 0s,ButtonPress Ground)
(Time 0s,ButtonPress First)
(Time 2s,ArrivedAt Second)
(Time 2s,Going Down Ground)
(Time 4s,ArrivedAt Ground)
(Time 4s,Going Up First)
(Time 5s,ArrivedAt First)
Passenger waited 5s for the elevator to arrive to the First floor

The counterexample is minimal thanks to QuickCheck’s shrinking. Here, one can imagine three passengers, pressing a button almost at the same time. Since the elevator starts on the Ground floor and it is the Second floor passenger that wins the race, the elevator starts going to the Second floor and queues the Ground and the First floor requests, by this order. It then takes 5 seconds in total for the elevator to arrive at the First floor, violating the timeliness requirement.

With property test in place, it is possible to iterate on better algorithms with confidence. prop_no_passenger_waits_4_seconds property will be able to assert if any of the improvements actually meet the timing requirements.

Using io-sim in the Real World

Real systems don’t explicitly block, they perform actual work that takes time. To make such code testable with io-sim, one can introduce a typeclass abstraction (e.g. MonadElevator m) with methods like moveElevator. In production, this would perform real hardware control; in tests, it would use threadDelay to simulate the operation’s duration.

In this elevator example, in a real system, there would be a sensor which would inform the controller at what time the elevator arrive at a specific floor, at which point the internal logic about the current floor of the elevator would be updated. With suitable abstraction, that implementation could replace our simplification using threadDelay.

io-sim can accurately simulate the standard IO operations, but this additional abstraction also introduces the challenge of verifying that the model accurately describes the real-world interactions. For example, 1 second is actually a very fast elevator, so our model and timeliness requirements may have to be modified slightly. That’s a topic left for another blog post!

Related Tools and Libraries

The Haskell ecosystem offers several libraries to test concurrent systems, each one addresses different aspects of the problem. Here are two of the most popular and known ones:

• dejafu
• quickcheck-state-machine

Each takes a slightly different approach to exploring thread schedules, invariants, or state-space, and all have proven useful in practice.

dejafu explores all possible thread interleavings to find concurrency bugs. The library offers a similar typeclass abstraction to io-classes for concurrency primitives, allowing testing code that uses threads, MVars and STM.

quickcheck-state-machine tests stateful programs using state machine models with pre and post-conditions. The library can find race conditions through parallel testing. It excels at testing APIs with complex state dependencies, e.g. databases or file systems, but focuses on state correctness rather than temporal properties.

io-sim distinguishes itself by being the only time-based simulator. One can’t easily ask “what happens when this operation takes 150ms instead of 15ms?” with dejafu nor quickcheck-state-machine. io-sim enables testing of timeout logic, retry mechanisms, timeliness constraints, etc. The ability to compress years of simulated execution into seconds of test runtime makes io-sim particularly valuable for testing long-running systems where bugs emerge only after extended operation.

Conclusion

The key insight is that io-sim simulates the actual behavior of Haskell’s runtime. STM transactions, thread scheduling, and time passing behave exactly as in production, but deterministically.

For concurrent Haskell systems with timing requirements, e.g. network protocols, distributed systems, or real-time controllers, io-sim allows the verification of time-sensitive properties. The library offers much more than shown here, including thread scheduling exploration testing with partial order reduction.

The complete code examples are available here.

This is a change of name for our GHC activities report, to reflect the fact that it focuses on more than just GHC work. You can find the previous editions collected under the haskell-ecosystem-report tag.

Sponsorship

We offer Haskell Ecosystem Support Packages to provide commercial users with support from Well-Typed’s experts while investing in the Haskell community and its technical ecosystem including through the work described in this report. To find out more, read our announcement of these packages in partnership with the Haskell Foundation. We need funding to continue this essential maintenance work!

Recently we will delighted to welcome Standard Chartered as a Gold Haskell Ecosystem Supporter. Many thanks to Standard Chartered; to our other Haskell Ecosystem Supporters: Channable and QBayLogic; to our existing clients who also contribute to making this work possible: Anduril, Juspay and Mercury; and to the HLS Open Collective for supporting HLS release management.

Team

The Haskell toolchain team at Well-Typed currently includes:

• Andreas Klebinger
• Ben Gamari
• Matthew Pickering
• Rodrigo Mesquita
• Sam Derbyshire
• Hannes Siebenhandl
• Zubin Duggal
• Mikolaj Konarski

In addition, many others within Well-Typed contribute to GHC, Cabal and HLS occasionally, or contribute to other open source Haskell libraries and tools. This report includes contributions from Alex Washburn and Wen Kokke in particular.

GHC

GHC Releases

• Ben worked on the 9.14.1 release, preparing backports and releasing the first alpha.

• Ben and Zubin worked on backports for GHC 9.12.3.

• Zubin worked on the 9.10.3 release, preparing backports and publishing the rc1, rc2 and rc3 release candidates.

Frontend

• Sam made several improvements to the implementation of deep subsumption, allowing GHC to accept programs that it previously rejected (#26225, !14577).

• Matt refactored the treatment of nested Template Haskell splices in GHC, making the code paths more consistent and removing code duplication (!14377).

• Matt fixed some bugs related to level checking (for the ExplicitLevelImports extension), including a crash in the presence of cyclic imports (#26087, !14478) and some missing level checks (#26088, !14479, #26090, !14550).

• Andreas allowed the type-class specialiser to look through type families, exposing more opportunities for specialisation (#26051, !14272).

• Sam identified and fixed a situation in which RULES which were not active could fire nonetheless (#26323, !14687).

• Ben and Andreas investigated various performance issues in the opaque newtype dictionaries patch, in preparation for merging to 9.14 (!10479)

LLVM backend

• Alex fixed incorrect sign-extension and narrowing of bitReverse, byteSwap and pdep primops in the LLVM backend (#20645, #26109, !14609).

• Alex fixed the LLVM backend generating references to non-existent LLVM intrinsics llvm.x86.bmi.{pdep,pext}.{8,16}, replacing them with usage of the appropriate 32-bit operations (#26065, !14647).

• Andreas made GHC allow LLVM versions outside of the supported range, emitting a warning rather than an error (#25915, !14531).

• Ben fixed the treatment of built-in arrays with LLVM, which was necessary to support newer LLVM versions. This fixed a raft of failing tests with the LLVM backend (#25769, !14157).

• Ben implemented a major rework of the Windows Clang toolchain (!14442), with help from long-time Windows contributor Tamar Christina. This was necessary to adapt to changes in newer LLVM versions, such as the use of API sets.

GHCi and bytecode interpreter

• Rodrigo has been working to improve the GHCi debugger, as a step towards better debugger tooling for Haskell programs. In particular, he implemented support for stepping-out of a function when debugging in GHCi, with the new :stepout command (#26042, !14416). He also made breakpoint indices 32 bits instead of 16 bits, as loading large programs in GHCi could overflow the counter (#26325, !14691), and made various other improvements to breakpoints (!14461, !14480, !14534).

• Andreas fixed some issues around endianness in the bytecode interpreter, which could cause the interpreter to produce incorrect results from primitive operations on some platforms (#25791, #23387, !14172).

• Hannes added a new GHCi flag -fno-load-initial-targets, allowing GHCi to be started without immediately loading all the target modules, so the user can selectively load the modules they are interested in with :reload (#26144, !14448). This can significantly reduce GHCi startup times when working on part of a large project.

• Hannes fixed the remaining issues in the interaction of the GHCi :reload command with multiple home units (#26128, !14427).

• Matt fixed some bugs in interpreter statistics calculations (#25756, !13956, #25695, !13879).

Documentation

• Hannes updated the user’s guide to advertise full support for multiple home units (#20889, !14426).

• Wen improved the documentation of eventlog cost centres and sample labels (!14499), and of heap profile IDs (!14506).

• Andreas added @since annotations for the -fexpose-overloaded-unfolding and -fdo-clever-arg-eta-expansion GHC flags (#26112, #26113, !14517).

Runtime system and linker

• Andreas added support for COFF BigObj files in the linker (!14582).

• Ben made the linker less reliant on file extensions to identify archive members (#13103, #24230, !14405).

• Hannes fixed an oversight in the hashing function used in the RTS (#26274, !14651).

Profiling and debugging

• Hannes added a new primop and ghc-experimental API which allows annotating the call stack with arbitrary data, in pursuit of better backtraces (#26218, !14538). See his recent blog post Better Haskell stack traces via user annotations for a more detailed explanation of the new features.

• Hannes improved the implementation of the Backtraces type and extended the backtrace mechanism to allow configuring stack decoders (#26211, !14532). He also reverted a change that would have exposed the internal implementation of the Backtraces type from base, as this needs a CLC proposal (!14587).

• Andreas disabled the -fprof-late-overloaded-calls functionality for join points, as this could cause GHC crashes (#26138, !14460). This is a temporary fix before the root problem can be addressed in full.

• Matt allowed info table entries used for profiling to be distributed separately (#21766, !14465).

• Wen disabled the usage of --eventlog-flush-interval in the non-threaded RTS, to avoid eventlog corruption (#26222, !14547).

• Wen removed the unused hard-coded profile_id from eventlog traces (!14507).

• Ben factored out the constructor ctoi tuple info tables into a data section for re-usability (!14508).

• Ben fixed a regression in zstd compression support for info-table provenance tables (#26312)

Core libraries and ghc library

• Zubin added newNameCache, a version of initNameCache that isn’t prone to being misused (!14446).

  This was in response to a bug in which the weeder library (weeder #194, #26055) was using initNameCache incorrectly.

• Rodrigo added an export of displayExceptionWithInfo to base, implementing CLC proposal #344 (!14419).

• Hannes implemented CLC proposal #212, removing some deprecated heap representation details from GHC.Exts (!14544).

• Ben removed IOPort, an internal datatype that could be replaced by MVar, implementing CLC proposal #213 (!8776).

Build system and packaging

• Zubin fixed an issue in which the user’s guide PDF would not be included in a binary distribution (#24093, !14469).

• Ben added support for otool, install-name-tool and LLVM utilies such as llc, opt to ghc-toolchain (#23675, !14050).

• Ben allowed the CrossCompiling predicate to be overridden (#26236, !14568).

• Ben dropped build-system logic for preferring the now-deprecated ld.gold linker (#25716, !14324).

CI and testing

• Hannes and Zubin upgraded the bootstrap compiler to 9.10.1 on MacOS (!14601), Windows (!14622) and FreeBSD (!14666). This allowed Hannes to update the test-bootstrap job to use 9.10.1 (!14676).

• Zubin improved how the testsuite driver filters out certain spurious linker warnings (#26249, !14615).

Haddock

• Zubin fixed Haddock emitting spurious warnings for undocumented type family axioms (which cannot have documentation attached to them) (#26114, !14447).

Cabal

• Matt helped the Cabal project adopt a formal proposal process by writing up the Cabal Proposals Process. This document was discussed in Cabal #11006, and eventually agreed upon by the existing Cabal maintainers.

• Matt opened a Cabal proposal to add support for bytecode artifacts, which would speed up GHCi usage and allow the GHCi debugger to step through dependencies such as code in base (Cabal Proposals #2).

• Matt helped finish up work by Cabal contributor Julian G (@jgotoh) to migrate the cabal.project parser to use Parsec (Cabal #8889).

• Matt updated the CI release scripts, bumping the boot compiler version and updating platforms (Cabal #11032).

• Matt adapted the Cabal library to the change in exception contexts in base-4.21 (Cabal #11125). This issue arose when helping Phil de Joux investigate a mysterious issue on Cabal #10684.

• Matt made Cabal use response files when starting multi-repl sessions, rather than passing long command-line invocations (Cabal #10995). These changes were subsequently reverted for the Cabal 3.16 release, in order to preserve compatibility with released HLS bindists (Cabal #11101). The plan is to go forward with response files in Cabal 3.18, giving HLS the time to adapt.

• Matt, with help from Hannes, added the --with-repl flag to the cabal-install repl command, allowing external tools such as hie-bios and doctest to easily figure out the correct options for starting a GHCi session (Cabal #10996).

Haskell Language Server

• Hannes made hie-bios use Cabal’s --with-repl command to load the session, which greatly simplifies the implementation and its treatment of multiple home units (hie-bios #466).

ghc-debug

• Hannes made the IPE information display inline for stack closures (ghc-debug #73).

• Matthew allowed the ghc-debug-brick terminal interface to be incrementally updated while a query is run (ghc-debug #68).

• Hannes restructured the ghc-debug-brick module hierarchy (ghc-debug #72), and made it available as a separate library (ghc-debug #74).

• Hannes added support for custom stack annotations (ghc-debug #69).

Infrastructure

• Ben migrated the haskell.org mail delivery and mailing list infrastructure to a more maintainable hosting situation. This involved rebuilding the mail delivery configuration on NixOS, migrating two decades of mailing list data to from mailman-2 to mailman-3, and implementing a scheme to ensure that the previous mailman-2 archives remain available.

• In response to user feedback, Ben carried out a variety of improvements to the Hackage documentation builder, allowing a greater breadth of packages to build.

• Ben coordinated with the Haskell Foundation to provision a set of new CI runners for AArch64/Linux to replace capacity lost to the end of Azure’s open-source program.

Libraries

• Sam migrated the ghc-typelits-natnormalise library to use ghc-tcplugin-api in ghc-typelits-natnormalise #90.

  This eases the maintenance burden of keeping ghc-typelits-natnormalise working across different GHC versions. Several pre-existing bugs were fixed in the process (ghc-typelits-natnormalise #47, ghc-typelits-natnormalise #70, ghc-typelits-natnormalise #71).

• Wen improved the eventlog-socket library, adding support for setting the eventlog writer from C, and providing documented usage examples.

This facility offers a number of advantages over the existing backtrace collection mechanisms:

• It is not necessary modify the function API (unlike HasCallStack)
• A “continuous chain” of modifications is not necessary (unlike HasCallStack)
• The annotations work in all ways of compilation (unlike cost centre stacks)
• The backtrace is expressed in terms of predictable source locations (unlike some IPE backtraces)

In this post we wil introduce the API for stack annotation, give some examples of how to use the annotation functions and discuss some trade-offs we have noticed with the design.

We’re interested in feedback from users on this feature. We’re expecting it to be available from GHC 9.16, as our implementation already landed in GHC HEAD (!14538).

Annotation stack frames

The core of the design is a new primop, annotateStack#, which when executed pushes an “annotation stack-frame” to the stack. Semantically, the frame is a no-op, but the payload contains a pointer to an arbitrary user-defined annotation. When decoding the native Haskell stack the annotation can be rendered to provide the user with additional context about the current location of the program.

The primop annotateStack# is exposed to the user via an IO-based API in GHC.Stack.Annotation.Experimental from the ghc-experimental package:¹

annotateStackIO :: (Typeable a, StackAnnotation a) => a -> IO b -> IO b

This will push the annotation value a onto the stack for the duration of the IO b action. The constraints allow the value to be rendered to a string or have its type inspected, similarly to the Exception class.

There are also specialised variants:

annotateCallStackIO   :: HasCallStack => IO b -> IO b  -- Annotate with the current source location
annotateStackStringIO :: String       -> IO b -> IO b  -- Annotate with an arbitrary String
annotateStackShowIO   :: Show a => a  -> IO b -> IO b  -- Annotate with the result of 'show' on a value

In addition, there are “pure” variants for use in non-IO code. However, these tend to be less intuitive due to the combination of lazy evaluation and imprecise exceptions, so the IO versions will generally produce better stack traces more reliably.

Note, annotateStack# is heavily inspired by annotated-exception and can be used together with annotated-exception for even better stack traces.

Example of the status quo

Let’s use the annotation functions to improve the backtrace for a program reported in a GHC ticket (#26040). The program implements a simple REST API using servant. When the endpoint is requested with a parameter which is larger than or equal to 100, the endpoint will error. topHandler catches all exceptions thrown by the handler and turns them into an HTTP 505 error. Finally, the exception handler prints any exceptions that might be thrown by the endpoint.

main :: IO ()
main = do
  setBacktraceMechanismState IPEBacktrace True
  run 8086 mkServer

type Api = Capture "x" Int :> Get '[PlainText] Text

mkServer :: Application
mkServer =
  serve
    (Proxy @Api)
    (hoistServer (Proxy @Api) topHandler api)

topHandler :: IO a -> Handler a
topHandler action = do
  result <- liftIO $
    (Right <$> action) `catch` \(exc :: SomeException) -> do
      liftIO $ putStrLn $ "Exception: " <> displayExceptionWithInfo exc
      pure $ Left err500

  either throwError pure result

api :: ServerT Api IO
api = handler

handler :: Int -> IO Text
handler x =
  if x >= 100
    then throw $ ErrorCall "Oh no!"
    else pure (pack "handler")

With the current version of GHC, when calling this API via http://localhost:8086/105, this stack trace is printed:

Exception: ghc-internal:GHC.Internal.Exception.ErrorCall:

Oh no!

IPE backtrace:
  Main.liftIO (src/Servant/Server/Internal/Handler.hs:30:36-42)
  Servant.Server.Internal.Delayed.runHandler' (src/Servant/Server/Internal/Handler.hs:27:31-41)
  Control.Monad.Trans.Resource.runResourceT (./Control/Monad/Trans/Resource.hs:(192,14)-(197,18))
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:195:20-22)
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:(195,5)-(203,31))
  Network.Wai.Handler.Warp.HTTP1.http1server.loop (./Network/Wai/Handler/Warp/HTTP1.hs:(141,9)-(157,42))
HasCallStack backtrace:
  collectExceptionAnnotation, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:170:37 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:90:42 in ghc-internal:GHC.Internal.Exception
  throw, called at app/Main.hs:42:10 in backtrace-0.1.0.0-inplace-server:Main

In this example there are two different backtraces:

• The “IPE backtrace” is constructed by decoding the Haskell stack, using information stored in the binary by -finfo-table-map, where each frame is automatically associated with a source location. (The compiler option -finfo-table-map was originally introduced for profiling.)
• On the the other hand, the “HasCallStack backtrace” is built using the implicitly passed HasCallStack constraints, which are automatically supplied by the type-checker, provided HasCallStack appears in the type.

The HasCallStack backtrace seems the most useful, telling us exactly where our program went wrong. However, the backtrace is very brief, as the rest of the program doesn’t have any HasCallStack constraints. As such, this stack trace might be unhelpful in larger programs, if the call to error was placed behind many layers of abstraction.

The IPE backtrace looks impressive, but doesn’t even show us where the exception is thrown! We get more intermediate source locations, but not the source of the exception. The function from which the exception is thrown is not even listed.

The reason the IPE backtrace may be unhelpful lies in the way the Haskell call stack works. We show the IPE info for each stack frame, which doesn’t relate precisely to the original source code and the resulting stack trace feels unintuitive. One reason for this is many function calls are tail-calls which don’t result in stack frames.

For more of an overview of the different backtrace mechanisms consult the discussion section of GHC Proposal #330.

Better stack traces with annotateCallStackIO

The IPE backtrace can be improved by manually annotating important parts of the program which should always appear in a backtrace.

For example, we always want to know in which handler the exception was thrown in, so the handler function is annotated with annotateCallStackIO. Further, we annotate the location where the exception is thrown.

handler :: Int -> IO Text
handler x = annotateCallStackIO $ do
  if x >= 100
    then annotateCallStackIO $ throw $ ErrorCall "Oh no!"
    else pure (pack "handleIndex")

When running this program again, the stack trace will now contain the source location of the handler where exception was thrown from:

Exception: ghc-internal:GHC.Internal.Exception.ErrorCall:

Oh no!

IPE backtrace:
  annotateCallStackIO, called at app/Main.hs:42:10 in backtrace-0.1.0.0-inplace-server:Main
  annotateCallStackIO, called at app/Main.hs:40:13 in backtrace-0.1.0.0-inplace-server:Main
  Main.handler (app/Main.hs:(40,1)-(43,30))
  Main.liftIO (src/Servant/Server/Internal/Handler.hs:30:36-42)
  Servant.Server.Internal.Delayed.runHandler' (src/Servant/Server/Internal/Handler.hs:27:31-41)
  Control.Monad.Trans.Resource.runResourceT (./Control/Monad/Trans/Resource.hs:(192,14)-(197,18))
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:195:20-22)
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:(195,5)-(203,31))
  Network.Wai.Handler.Warp.HTTP1.http1server.loop (./Network/Wai/Handler/Warp/HTTP1.hs:(141,9)-(157,42))
HasCallStack backtrace:
  collectExceptionAnnotation, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:170:37 in ghc-internal:GHC.Internal.Exception
  toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:90:42 in ghc-internal:GHC.Internal.Exception
  throw, called at app/Main.hs:42:32 in backtrace-0.1.0.0-inplace-server:Main

Note the first two entries of the IPE backtrace:

annotateCallStackIO, called at app/Main.hs:42:10 in backtrace-0.1.0.0-inplace-server:Main
annotateCallStackIO, called at app/Main.hs:40:13 in backtrace-0.1.0.0-inplace-server:Main

These have been added due to our manual annotation of our source program via annotateCallStackIO!

They give us precise source location where the exception is thrown, making the IPE backtrace just as useful as the HasCallStack backtrace. However, note, we did not have to change the type signature of handler at all to get a much more informative stack trace.

throwIO vs throw vs error

Some readers may have noticed that we used throw instead of error, which is usually the go to function for throwing example errors (or from within pure code). At the moment, throw and error produce noticeably different stack traces, because error evaluates the exception annotations lazier than throw, which leads to failing to capture the call stack when throwing the exception. This should be possible to resolve; see GHC issue #25430.

On the other hand, throwIO behaves more predictably within IO code and the IPE backtrace includes the source location of the exception throwing:

IPE backtrace:
  Main.handler (app/Main.hs:42:10-45)
  Main.liftIO (src/Servant/Server/Internal/Handler.hs:30:36-42)
  Servant.Server.Internal.Delayed.runHandler' (src/Servant/Server/Internal/Handler.hs:27:31-41)
  Control.Monad.Trans.Resource.runResourceT (./Control/Monad/Trans/Resource.hs:(192,14)-(197,18))
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:195:20-22)
  Network.Wai.Handler.Warp.HTTP1.processRequest (./Network/Wai/Handler/Warp/HTTP1.hs:(195,5)-(203,31))
  Network.Wai.Handler.Warp.HTTP1.http1server.loop (./Network/Wai/Handler/Warp/HTTP1.hs:(141,9)-(157,42))

This means that how the exception is thrown is important to get reasonable stack traces. Unsurprisingly, you should use throwIO whenever you are within the IO monad.

Summary

Annotation stack frames are a lightweight way to add extra information to stack traces. By modifying the execution stack, the information is always available and can be used by the native stack decoder to display informative backtraces to users. We’re interested to hear what users think about this feature and how libraries will be adapted to take advantage of the new annotation frames.

This work has been performed in collaboration with Mercury, who have a long-term commitment to the scalability and robustness of the Haskell ecosystem. Well-Typed are always interested in projects and looking for funding to improve GHC and other Haskell tools. Please contact info@well-typed.com if we might be able to work with you!