Irmin Retrospective

3 months ago 2

is a libary for building OCaml preprocessors. Users can generate OCaml code from

At the core of is the OCaml parsetree; a data structure in the compiler that represents OCaml source code. some

In , the internal AST was bumped from the AST of OCaml 4.14.0, to the AST of OCaml 5.2.0. This post discusses some of the internal mechanisms that allows ppxlib to support multiple compilers and how this bump broke a lot of backwards compatibility.

2025 2 18 Manual Migrations in Ppxlib

It is possible to perform migrations between parsetrees manually using the ppxlib.ast package.

open Ppxlib open Ast_builder.Make struct let loc = Location.none end let pp_expr = Format.printf " " Pp_ast. expression ?config:None let loc = Location.none let str txt = txt; loc module To_408 = Ppxlib_ast.Convert Ppxlib_ast.Js Ppxlib_ast__.Versions.OCaml_408

The Ppxlib_ast.Js module is ppxlib.0.36.0 provided by the Convert functor to copy parts of the AST from one version to another. Usually there is very little difference between the versioned parsetrees.

let func arg body = pexp_fun Nolabel None ppat_var str arg body let add_expr = func "x" func "y" eapply evar "Int.add" evar "x"; evar "y"

The above excerpt of OCaml code produces an expression in the language. More accurately, a function that has two arguments. If we print the simplified AST we can see that the arguments are stored together in a list.

# pp_expr add_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" ; pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "add" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit =

This is a relatively new feature of the parsetree, if we migrate the expression down to 4.08 we see it is represented differently. The output is a little too verbose to be readable, and there's no good way to show the AST, so you will just have to take my word.

let expr_408 = To_408.copy_expression add_expr 2025 2 18 Function Arity in the OCaml Parsetree

OCaml 5.2 introduced a new way to represent functions. Before 5.2, all functions were represented with single arity. Functions with multiple arguments would return functions as their body. For example:

let add x y = x + y

A two-argument add function would be represented in the parsetree as fun x -> fun y -> x + y .

Now, since OCaml 5.2, functions can be expressed with their syntactic arity intact. The function add would look something like . Both arguments are stored as a list.

The constructor Pexp_function was not a new addition to the parsetree. It used to represent pattern-matching expressions like function A -> 1. This can now be expressed as a function body of Pfunction_cases.

Whilst these additions and modifications to the parsetree are arguably a better design, it has caused chaos in terms of 's reverse dependencies!

2025 2 18 Coalescing Function Arguments

In 0.36.0, functions from Ast_builder will help ppx authors produce maximum arity functions. However, functions from Ast_helper will not do this.

let ast_helper_func arg body = Ast_helper.Exp.fun_ Nolabel None ppat_var str arg body let ast_helper_expr = ast_helper_func "x" ast_helper_func "y" eapply evar "Int.equal" evar "x"; evar "y"

Here we recreate our helper function to create function expressions using Ast_helper.Exp.fun_ instead. We can now show the difference between the expr expression and the ast_helper_expr expression.

Above, we see a function expression with arity two. Below, we see nested function expressions each with arity one.

# pp_expr ast_helper_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" , None , Pfunction_body Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "equal" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = 2025 2 18 Ramifications for Ppxlib Users

Many, many, many ppxes broke after migrating the AST to 5.2. Mostly, this is due to pattern-matching. Downstream users of frequently pattern-match directly against the parsetree. However, nodes such as Pexp_fun no longer exist and Pexp_function has completely different constructor arguments!

Pexp_fun x function

Here is a non-exhaustive list of some of the ppxes that have been patched. It may useful for other ppx authors who wish to upgrade to ppxlib.0.36.0:

2025 2 18 The Future of Ppxlib

is not going anywhere anytime soon. We have committed to trying to keep the internal AST up to date with the latest OCaml release. In fact, there is already a PR for !

There have been many discussions about simplifying this process, but so far the time and effort to build a

Thank you to Tarides and Jane Street for funding my time to work on this release and all the chaos it has caused.

Happy preprocessing!

Read Entire Article