Jonas Hietala: Packing Neovim with Fennel

I’ve got lots of stuff to do but I ended up rewriting my Neovim config instead.

… anyone can do any amount of work, provided it isn’t the work he is supposed to be doing at that moment.

My partner Veronica is amazing as she’ll listen to my bullshit and random whims (or at least, pretend to). That’s a big benefit to having a blog: so I have an outlet for rambling about my weird projects and random fixations and spare Veronica’s sanity a little.

I know that Veronica won’t be impressed by another Neovim config rewrite (even when done in Lisp!) so I’ll simply write a big blog post about it.

The rewrite

I wanted to rewrite my Neovim configuration in Fennel (a Lisp that compiles to Lua) and while doing so I wanted to migrate from lazy.nvim to Neovim’s new built-in package manager vim.pack.

This included bootstrapping Fennel compilation for Neovim; replicating missing features from lazy.nvim such as running build scripts and lazy loading; modernizing my LSP and treesitter config; and trying out some new interesting plugins.

Please see my new Neovim config here.

Why Fennel?

Lua has been a fantastic boon to Neovim and it’s a significant improvement over Vimscript, yet I can’t help but raise an eyebrow when I hear people describe Lua as a great language. It’s definitely great at being a simple and fast embeddable language but the language itself leaves me wanting more.

Fennel doesn’t solve all issues as some of Lua’s quirks bleeds through but it should make it a little bit nicer. I particularly like the destructuring; more functional programming constructs; macros for convenient DSL; and the amazing pipe operator.

But the biggest reason is that I’m simply a bit bored and trying out new programming languages is fun.

Why vim.pack?

I don’t rewrite my config often. But when I do, I do it properly.

Folke, maker of many popular plugins such as lazy.nvim and snacks.nvim, recently had a ~5 month break from working on his plugins. Of course, they continued to work and anyone working on open source projects can (and should) take a break whenever they want.

But it exemplifies that core Neovim features will likely be better maintained than standalone plugins and should probably be preferred (if they provide the features you need).

While Neovim’s built-in plugin manager is a work in progress and still a bit too simplistic for my needs I wanted to try it out.

Structuring the config

If you’ve got a small configuration having it all inside a single init.lua is probably fine. Somehow I’ve gathered almost 6k lines of Lua code under ~/.config/nvim so showing it all in one file isn’t that appealing.

I first wanted to separate the configuration into a core/plugin split, where non-plugin configuration happens in core and plugin configuration lives under plugin. However, to support lazy loading with a single call to vim.pack.add I decided to go back to letting the files under plugin/ return plugin specs, like lazy.nvim does for you.

With Fennel support under fnl/ this is how my configuration is structured:

It’s not a perfect system as I’d ideally want the plugins/ to only add packages while I would configure the plugins in config/. But some plugins use lazy loading making it more convenient to do it together with the plugin spec.

Bootstrapping Fennel

There are a handful of different plugins that allows you to easily write your Neovim config in Fennel. I ended up choosing nvim-thyme because it’s fast (it hooks into require and only compiles on-demand) and it allows you to mix Fennel and Lua source files.

nvim-thyme contains installation instructions for lazy.nvim and it references a bootstrapping function to run git to manually clone packages. But we’re going to use vim.pack and it makes the bootstrap a bit cleaner:

(I added lze to my bootstrapping too as I’ll use it later when adding lazy loading support, it was simpler having it in the bootstrap.)

nvim-thyme also instructs us to override require() calls (so it can compile on demand) and to setup the cache path (where it’ll store the compiled Lua files):

And now we’re ready to write the rest of the config with Fennel!

Now we can continue with Fennel fnl/config.fnl or fnl/config/init.fnl:

There’s one last thing we should do to make the bootstrap complete: we should call :ThymeCacheClear when nvim-laurel or nvim-thyme changes. The recommended way is to use the PackChanged event, with something like this:

But if we for example force an update for nvim-laurel (by deleting it with vim.pack.del({"nvim-laurel"}) and restart Neovim) we get this error:

There is no order guarantee for the packages and so PackChanged for nvim-laurel may run before thyme has been loaded. I worked around this with a variable that I check after vim.pack.add, which will guarantee that all packages have been added before we try to require a package:

Building a convenient plugin management system

I wanted to migrate to vim.pack but it’s missing a few key features from lazy.nvim:

• It can’t automatically require all files under a directory.
• There’s no lazy loading support.
• It can’t run build scripts (such as make after install or update).

I could’ve given up and gone back to lazy.nvim but that just wouldn’t do.

Source pack specs from files

I want to be able to create a file under plugins/, have it return a vim.pack.Spec, and have it automatically added. This is similar to the structured plugins approach of lazy.nvim.

To build this I first list all files under plugins/ like so:

This uses Fennel’s -> threading macro, Fennel’s version of the pipe operator. It’s one of my favorite features of Elixir and was stoked to discover that Fennel has it too. (Fennel actually has even more power with the ->>, -?>, and -?>> operators!)

Now we need to loop through and transform the paths to relative paths and evaluate the files to get our specs. (I’m using accumulate to explicitly build a list instead of collect as we’ll soon expand on it):

Now we can populate specs from files under plugins/, for example like this that returns a single spec:

But I also want to be able to return multiple specs:

To support this we can match on the return value to see if it’s a list, and then loop and insert each spec in the list, otherwise we do as before:

Now all that’s left is to call vim.pack.add with our list of specs and our plugins are now automatically added from files under plugins/:

Lazy loading with lze

lze is a nice and simple plugin to add lazy-loading to vim.pack.

We’ve already added it as a dependency in our init.lua so all we need to do is modify the load parameter to vim.pack.add like so:

Now we can specify lazy loading via the data parameter in our specs:

It relies on wrapping configuration under data but that’s annoying, so let’s simplify things a little.

Simplified specifications

The idea here is to transform the specs before we call vim.pack.add.

We can do it easily when we collect our specs by calling the transform_spec function:

I want transform_spec to transform this:

Into this:

By storing keys other than src, name, and version under a data table. This is what I came up with:

Another quality of life feature I’d like is to make it simpler to call setup functions. lazy.nvim again does this well and it’s pretty convenient.

For example, this is how it looks like with lze to add an after hook and call a setup function:

What if we could instead do:

But this is just data and we can transform the second case to the first one fairly easily. In the transform_spec function:

How to figure out the package name to require (since it may differ from the path)? lazy.nvim has a bunch of rules to try to figure this out automatically but I chose to be explicit. lze uses the on_require argument so it can load on a require call (on (require :fyler) for example), which seems like a good idea to reuse.

And just to prevent me from making mistakes, I added a sanity check:

Build scripts via PackChanged events

There’s one last feature I really want from lazy.nvim and that’s to automatically run build scripts after a package is installed or updated.

I basically want to specify this in my specs:

Again, we’ll rely on PackChanged for this:

The above code uses macros from nvim-laurel to define an autocommand that calls the pack_changed function. That function will then run pack_changed when the package is updated or installed:

To run the scripts I use vim.system with some simple printing:

This will now allow us to run build scripts like cargo build --release or make after a package is installed or updated. It’s a bit too basic as there’s no visible progress bar for long running builds (Rust, I’m looking at you!) and it doesn’t handle build errors that well but it works well enough I guess.

But what about user commands or requiring a package? For example with nvim-treesitter you’d want to run :TSUpdate after an update, something like this:

Let’s try it by allowing functions in the build parameter (and bare strings because why not):

If we run this though it doesn’t work:

The problem is that PackChanged is run before the pack is loaded. Maybe we could work around this by calling packadd ourselves but that would shortcut lazy loading. In this instance we’d like to run TSUpdate after the pack is loaded but only if it’s been updated or installed so we don’t run it after every restart.

What I did was introduce an after_build parameter to the spec that’s run after load if a PackChanged event was seen before:

Then in plugins/init.fnl I use a local variable packs_changed that’s updated on PackChanged like so:

Then we’ll call after_build from the after hook we setup before:

With this we can finally specify build actions such as these:

Some Fennel examples

You’ve already seen how Fennel code looks like but what about configuration with Fennel? One of the negative things of moving my configuration from Vimscript to Lua was that simple things such as settings options or simple keymaps is more verbose.

So how does Fennel compare for the simpler, more declarative stuff?

Options

With nvim-laurel macros I think Fennel is decent. Slightly better than Lua but not as convenient as Vimscript.

Keymaps

Not a huge difference to be honest. I like the #(do_the_thing) shorthand for anonymous functions fennel has. Having to (sometimes) split up require and method calls on separate lines in Fennel is annoying.

Overriding highlight groups

One example that was a big step up with Fennel is overriding highlight groups. I’m using melange which is a fantastic and underrated color scheme but I’ve collected a fair bit of overrides for it.

In Lua you use nvim_set_hl to add an override, for example like this:

When you do this 100 times this is annoying so I made an override table to accomplish the job:

In Fennel with the hi! macro this all becomes as simple as:

Autocommands

Here are some autocommands to enable cursorline only in the currently active window (while skipping buffers such as the dashboard):

Plugin specs

One thing I like more in Lua compared to Fennel is how readable tables are. The Fennel formatter fnlfmt might be partly to blame as it has a tendency to use very little whitespace. Regardless, I prefer this Lua code:

Over this corresponding Fennel code:

To me the Lua code is for some reason easier to read.

Similarly I don’t have a problem with this lazy.nvim spec:

But with this new Fennel spec I use—even though it’s simpler in some ways—it’s harder for me to quickly see what table the keys belong to:

Maybe it’s something you’ll get used to?

Notable plugin updates

Neovim is moving quickly and I’ve had a bit of catching up to do in the plugin department. I won’t bore you with an exhaustive list; just a few highlights.

Native undotree

I’ve been using undotree a long time and it’s excellent. This feature was recently merged into Neovim:

Simplified LSP config

Neovim routinely gets shit on for LSPs being so hard to setup. Yes, it could probably be easier but Neovim has recently made some changes to streamline LSP configuration and it’s not nearly as involved as it used to be.

Here’s how my base config looks like:

I also use nvim-lspconfig but it doesn’t do anything magical (anymore). It’s basically a collection of LSP configs, so I don’t have to fill my config with things like this:

If you don’t want to change the keymaps (Neovim comes with defaults that I personally dislike) or customize specific LSPs then there’s not that much left. Mason is also totally optional and if you want to manage your LSPs outside of Neovim you can totally do that. The only thing missing is autocomplete, which blink.cmp provides out of the box.

Automatically install and enable treesitter grammars

Another thing that has changed since my last config overhaul is nvim-treesitter being rewritten and is now a much simpler plugin. The new version lives on the main branch and the old archived one on master and it contains a bunch of breaking changes.

For example, it no longer supports installing and activating grammars automatically. I think I saw a plugin for that somewhere but here’s some Fennel code that sets it up:

If you use nvim-treesitter-textobjects (which you should) remember to migrate to the main branch there too.

Some new fun plugins

1. fyler.nvim, edit a file explorer like a buffer

   oil.nvim is a great plugin that allows you to manage files by simply editing text. fyler.nvim takes it to the next level by combining it with a tree-style file explorer.

2. blink.cmp, faster autocomplete

   I’ve been using nvim-cmp as my completion plugin but I migrated to blink.cmp as it’s faster and more actively maintained. It’s too bad that it broke my custom nvim-cmp source for my blog but it wasn’t too hard to migrate.

3. snacks.nvim, a better picker

   telescope.nvim has been a solid picker but it’s no longer actively developed and the snacks.nvim is the replacement I settled on.

   I tried fff.nvim for file picking but surprisingly it felt really slow compared to snacks.nvim. fzf-lua is another great alternative that I haven’t given enough attention to.

4. grug-far.nvim, global query replace

   I’ve been happy with Neovim’s regular %s/foo/bar for single files (aided by search-replace.nvim for easy population). But query replace in multiple files has always felt lacking. I used to use telescope.nvim to populate the quickfix window and then use replacer.nvim to make it editable, updating multiple files.

   It worked but was a bit annoying so now I’m trying grug-far.nvim as a more “over engineered” solution. I haven’t used it that long to say for sure but I’m hopeful.

Ending thougths

It would be better to gradually evolve your Neovim config over time instead of doing these large rewrites. But afterwards it feels pretty good as I can once more try to claim with a straight face that I know what’s in my configuration and what it’s doing.

The vim.pack migration was more painful than I had expected. It’s still an experimental nightly feature and it’s missing a lot of nice features that lazy.nvim has. I’ll keep using vim.pack as I think I’ve gotten it to a state of good enough but I’m looking forward to vim.pack becoming more feature complete.

Fennel is fun to write in and I will keep using it where I can. To be honest though, for basic configuration I was expecting Fennel to make a bigger difference than it did. It’s nicer for sure but it’s nothing revolutionary.

Then again, it’s the little things in life that matters.