.png)
3 hours ago
2
We're pleased to announce new releases for both River and River Pro, featuring significant performance improvements, new features, and a valuable community contribution. This is River's biggest new release to date.
Performance improvements
This release brings major performance improvements to River and River Pro: A rewrite of the job completion query increased River's benchmark throughput by 15-20%, going from 55k jobs/s to 66k jobs/s on an M4 Pro MacBook Pro, with no changes to functionality or indexes.
We've also made two major performance improvements to River Pro:
Sequences: We fixed a critical performance issue where the Postgres query planner sometimes chose inefficient query plans for sequence operations. This could cause promotion queries to take 10+ seconds. Our solution combines with new indexes with query optimizations that encourage Postgres to actually use them, resulting in:
- >90% faster typical promotion times for customers with many sequences
- ~99.9% faster worst-case scenarios (10s of ms instead of 10s of seconds)
Concurrency Limits: After benchmarking revealed performance issues with sequences, we leveled up our benchmarks on concurrency limits and sought to accelerate those queries. We rebuilt this feature to be significantly faster by:
- Pre-computing partition keys at job insertion time instead of runtime
- Adding new indexes and a client-side cache
- Results: 70% faster typical performance, and 85% faster with frequent fetches
The pre-computed partition keys for concurrency limits mean existing jobs won't be re-partitioned when you adjust partitioning settings, but we believe this tradeoff is worth the substantial performance gains. The new design also makes it easier to build a related feature that we hope to ship soon. 🚀
New features
SQLite driver preview
River v0.23 offers an early preview of SQLite support for River (riverdriver/riversqlite). Although River remains mainly a Postgres-based job queue, this zero-process Postgres alternative may be useful in cases like embedded installations or development environments.
It needs more vetting to be considered fully production ready, but it passes River's comprehensive test suite, and performs quite well despite the existence of a number of clear optimizations to be made in the near future.
import "database/sql"
import "github.com/riverqueue/river/riverdriver/riversqlite"
dbPool, err := sql.Open("sqlite", ":memory:" )
if err != nil {
panic(err)
}
dbPool.SetMaxOpenConns(1)
defer dbPool.Close()
workers := river.NewWorkers()
river.AddWorker(workers, &SortWorker{})
riverClient, err := river.NewClient(riversqlite.New(dbPool), &river.Config{
Queues: map[string]river.QueueConfig{
river.QueueDefault: {MaxWorkers: 100},
},
Workers: workers,
})
If you're a SQLite user, give it a try and let us know how it works out!
Pro: database/sql support
River Pro adds official support for the standard Go database/sql package through a new Pro driver (riverqueue.com/riverpro/driver/riverprodatabasesql) similar to the one found in the main River project. This makes integration with popular ORM libraries like Bun or GORM seamless, significantly enhancing compatibility and flexibility.
import "database/sql"
import "riverqueue.com/riverpro/driver/riverprodatabasesql"
stdPool := sql.Open("pgx", "postgres://...")
defer stdPool.Close()
riverClient, err := riverpro.NewClient(riverprodatabasesql.New(stdPool), &riverpro.Config{
...
})
Like the main package, riverprodatabasesql is tested through pgx and lib/pq, making it useful for a variety of project setups, modern or not.
Pro: Durable periodic jobs
A long standing limitation of River's periodic jobs subsystem is that their bookkeeping is done in memory, and therefore lose state on client restart or election loss. River Pro v0.15 brings in a highly requested feature that addresses this problem: durable periodic jobs.
Durable periodic jobs piggyback off the existing periodic jobs service, but persist run times to the database so they can be reused in the event a client loses its scheduling state. This should be a major help for projects that need maximum scheduling predictability and stronger run guarantees.
riverClient, err := riverpro.NewClient(riverpropgxv5.New(dbPool), &riverpro.Config{
Config: river.Config{
PeriodicJobs: []*river.PeriodicJob{
river.NewPeriodicJob(
river.PeriodicInterval(15*time.Minute),
func() (river.JobArgs, *river.InsertOpts) {
return PeriodicJobArgs{}, nil
},
&river.PeriodicJobOpts{
// inclusion of an ID makes periodic job durable (as
// long as DurablePeriodicJob.Enable below is true);
// must be unique across periodic jobs
ID: "my_periodic_job",
},
),
},
},
DurablePeriodicJobs: riverpro.DurablePeriodicJobsConfig{
Enabled: true,
},
})
Reindexer improvements
The reindexer's timeout is now configurable with Config.ReindexerTimeout to allow for longer reindex attempts. Ths previous default was generally sufficient for most cases, but might trigger in the presence of enormous jobs tables, leaving behind invalid partially built indexes that'd occupy disk space and need to be cleaned up manually.
The default timeout has been increased from 15 seconds to 1 minute so that River is suited to a wider variety of cases without having to touch its default configuration. It's also been made more resilient to repeat failures, gracefully skipping reindex attempts if artifacts are present (e.g. an invalid partially built index) that are suggestive of previous failures.
Introducing rivertui, a River UI for the terminal
Finally, the community has delivered an amazing new addition: a terminal UI for River named rivertui by @almottier. This beautiful, real-time terminal UI allows for easy job monitoring, filtering, job detail inspection, and actions like retry and cancellation—all directly from your terminal. Check it out—it's even cooler in action than it sounds!
Upgrade Recommendations
Remember: Internal APIs have changed between River (v0.23.1) and River Pro (v0.15.1), so both should be updated simultaneously.
go get -u github.com/riverqueue/river riverqueue.com/riverpro
Pro: Version 003 migration
⚠️ The pro line has a new migration, version 003. This migration adds new indexes to optimize sequence queries and concurrency limits, as well as a new table to track durable periodic jobs.
If migrating with the CLI, make sure it's installed @latest so the new migration is present:
go install riverqueue.com/riverpro/cmd/riverpro@latest
riverpro migrate-up --database-url "$DATABASE_URL" --line pro
As always, check out the Pro migration docs for more details.
Full changelogs
There's a lot more in these releases that's not covered here, so check out the full changelogs for River v0.23 and River Pro v0.15 for the exhaustive details.
We're thrilled about these updates and can't wait to see them in action in your projects. Happy queueing!
![Micrographia (1665) [pdf]](https://news.najib.digital/site/assets/img/broken.gif)
