Show HN: Better-Experiments – open-source A/B testing library

4 months ago 16

Developer-first A/B testing framework with built-in dashboard

Better-Experiments makes A/B testing as simple as writing a feature flag. No complex setup, no vendor lock-in, no data science degree required.

🚀 2-minute setup - Get testing in minutes, not hours
🎯 Developer-first - Clean API that just works
🔒 Privacy-focused - Cookie-based, no external tracking
📊 Built-in analytics - View results without third-party tools
🔧 Self-hosted - Your data stays with you
⚡ Lightweight - Minimal dependencies, maximum performance
🌐 Universal - Works in browser, Node.js, edge functions
🛡️ Type-safe - Full TypeScript support with perfect inference

npm install better-experiments # or yarn add better-experiments # or pnpm add better-experiments

import { BetterExperiments } from "better-experiments"; // Initialize the client const ab = new BetterExperiments(); // Test different button colors - returns assignment object const buttonTest = await ab.test("button-color", ["red", "blue", "green"]); // Use the variant in your UI console.log(`User sees ${buttonTest.variant} button`); // Track conversions directly! await buttonTest.convert("click"); await buttonTest.convert("signup");

That's it! 🎉

import { BetterExperiments } from "better-experiments"; const ab = new BetterExperiments(); // Test different headlines const headerTest = await ab.test("homepage-headline", [ "Welcome to Our Amazing Product", "Transform Your Business Today", "The Tool You've Been Waiting For", ]); // Use the selected headline document.querySelector("h1").textContent = headerTest.variant; // Track conversion when user signs up document.querySelector("#signup").addEventListener("click", async () => { await headerTest.convert("signup"); });

const ab = new BetterExperiments(); // Gradual feature rollout const dashboardTest = await ab.test("new-dashboard", [false, true]); if (dashboardTest.variant) { // Show new dashboard loadNewDashboard(); await dashboardTest.convert("feature_enabled"); } else { // Show old dashboard loadOldDashboard(); await dashboardTest.convert("control_group"); }

Complex Variants with TypeScript

Multiple Tests for Same User

import { BetterExperiments } from "better-experiments"; const ab = new BetterExperiments({ storage : CustomStorageAdaptor() debug: false, // Disable in production }); // Create test with full control await ab.createTest({ testId: "enterprise-feature", variants: ["control", "experiment"], weights: [0.8, 0.2], metadata: { name: "Enterprise Feature Rollout", description: "Gradual rollout of new admin features", }, }); const featureTest = await ab.test("enterprise-feature", ["old-ui", "new-ui"]);

const ab = new BetterExperiments(config?: BetterExperimentConfig);

Configuration Options:

storage?: StorageAdapter - Custom storage adapter (defaults to MemoryStorage)
debug?: boolean - Enable debug logging

ab.test<T>(testId, variants, options?): Promise<TestAssignment<T>>

Runs A/B test and returns assignment object
testId: Unique identifier for the test
variants: Array of variant values to test
options: Optional configuration (weights, metadata, userId override)

TestAssignment.convert(event?, metadata?): Promise<void>

Track conversion for this specific assignment
event: Event name (defaults to 'conversion')
metadata: Optional event metadata

Other Methods:

ab.getResults(testId) - Get test results and statistics
ab.getTests() - Get all test configurations
ab.createTest(config) - Manually create a test (optional)
ab.stopTest(testId) - Deactivate a test

interface TestAssignment<T> { variant: T; // The selected variant value assignment: UserAssignment; // Full assignment details convert(event?, metadata?): Promise<void>; // Conversion tracking method }

Perfect for development and testing:

import { BetterExperiments, MemoryStorage } from "better-ab"; const ab = new BetterExperiments({ storage: new MemoryStorage(), });

Implement the StorageAdapter interface for your database:

import { StorageAdapter } from "better-experiments"; class PostgresStorage implements StorageAdapter { // Implement required methods async saveTest(config) { /* ... */ } async getTest(testId) { /* ... */ } async saveAssignment(assignment) { /* ... */ } async getAssignment(testId, userId) { /* ... */ } async saveConversion(event) { /* ... */ } async getConversions(testId) { /* ... */ } async getTestResults(testId) { /* ... */ } // ... other methods } const ab = new BetterExperiments({ storage: new PostgresStorage(), });

# Install dependencies npm install # Build the package npm run build # Run linting npm run lint # Type checking npm run type-check # Clean build files npm run clean

🌟 Why Better-Experiments?

Traditional A/B Testing Libraries

// Complex setup, separate tracking const variant = await abTest.getVariant("test-id", userId); // Later... easy to mess up IDs await abTest.track("test-id", userId, "conversion"); // ❌ Error-prone