What works well and doesn't with AI coding agents in October 2025

This post is about a real refactoring experience using AI coding agents on production code. No hype, no hand-waving—just what actually happened, what worked, what didn’t, and what I learned.

The Challenge

The Testcontainers for Go project needed a major refactoring: migrate all 60 modules from using testcontainers.GenericContainer() to the new testcontainers.Run() API. Each module represents a different technology: PostgreSQL, Redis, Kafka, MongoDB, Elasticsearch, etc.

This wasn’t just a find-and-replace job. Each module had its own quirks:

• Different configuration patterns
• Custom options and validations
• Environment variable inspection
• TLS support and credentials
• Wait strategies and health checks

The scope? Roughly 5,000+ lines of code across modules, tests, and examples. All had to maintain backward compatibility, pass existing tests, and follow our coding conventions.

The Experiment: Learn First, Then Delegate

I decided to try an experiment: do part of the migration manually to understand the patterns, then use an AI coding agent for the rest.

Phase 1: Manual Migration (Sept 26 - Oct 3)

I started alphabetically with aerospike and worked my way through to grafana-lgtm—19 modules in 7 days.

Why go manual first? Because I needed to:

• Understand what patterns actually worked
• Hit all the edge cases myself
• Build intuition for the “right” solution
• Create reference implementations

The learnings from this phase were crucial:

Pattern 1: The Run Function Structure

Pattern 2: Option Types

Pattern 3: Env Variable Inspection

Critical Discoveries:

• ✅ Return struct types, NOT interfaces (testcontainers.CustomizeRequestOption, not testcontainers.ContainerCustomizer)
• ✅ Call built-in options directly: testcontainers.WithFiles(f)(req), not .Customize(req)
• ✅ Use strings.CutPrefix from stdlib, not manual string manipulation
• ✅ Option order matters: defaults → user options → post-processing
• ✅ Always initialize container variable before error check
• ✅ Container naming: use Container, not PostgresContainer

After 7 days and 19 modules, I had:

1. A proven pattern that worked
2. Multiple reference implementations
3. Confidence that this was the right approach

Phase 2: AI-Assisted Migration (Oct 6 - Oct 9)

With the patterns established, I brought in Claude Code to help with the remaining 41 modules in just 3 days.

Here’s what that looked like:

Day 1 (Oct 6): Teaching the Agent

I started by providing a detailed prompt to the agent, a bit clumsy, but it worked.

Me: "Check the git log history in the current branch, and you'll see that a series of pull request with the same commit message/PR title are happening: "chore($MODULE): use Run function". The commits are removing the usage of the GenericContainer function in the "./modules" directory, and each pull request is only changing one module. One module lives in a subdirectory in the modules dir. Whenever you run commands, make sure you are in Go 1.24. Just run this in you shell: eval "$(gvm 1.24.7 --arch=arm64)". The migration to the new API consists in replacing the initialisation of the COntainerRequest and GenericContainerRequest structs, and instead use the functional options that live in the main Go module "testcontainers". The options live in the options.go file in the root of the repository. Please iterate through the modules, and apply the changes one at a time, creating a git commit per module to isolate the changes. Please run "make pre-commit test" from the module directory before accepting any proposed solution. Exclude the "compose" module, as it's not of our interest now. If you have any doubt, prompt the question and I'll try to answer. Please do not cause a breaking change in the module you are modifying, only change the internal implementation. The commit message should follow the pattern described above: "chore($MODULE): use Run function". If the module has been migrated, just check if you see any inconsistency, and mention them if/when found. To identify the modules that have been migrated, pleae compare the existing commits, but also verify that there is no call to the "testcontainers.GenericContainer" function. When doing the changes, use the "moduleOpts" variable name for the default options, and for the error when calling Run, wrap the "run $moduleName" error message, and please use "moduleOpts" as the one receiving the opts from the caller. The "moduleOpts" variable has to be an slice of testcontainers.ContainerCustomizer. Never use the CustomizeRequest option, as it is overkill. Go with granular options, one by one, as each field in the request struct has its own functional option. Skip the following options: WithImage and CustomizeRequestOption. The first one is passed directly to the Run function. The second one is not needed, as you are not using the GenericContainerRequest struct. When multiple wait strategies are added, there is no need to wrap them all into a wait.ForAll, as the option allows a variad argument Do not include "container" in the error message, just "run module" If the module applied its own custom options, keep it in the for loop that applies them. Just remove the testcontainers options evaluation from that for loop. You must never use a Generic container request as you did for mongo. When you need to access the env vars to assign them to a variable, do it by calling the Inspect function in the container after it's been run, and retrieve it from that API. You have an example in the databend module, and also in the mongodb module, which I fixed for you. If the request updated the hostconfig and the tmpfs, do not call the WithTmpfs option, just stay with the hostconfig modifier options. For each module, apply your changes to the code, but let me verify each module. If the changes are good, I'll eventually allow you to continue in auto-mode. Do you have any doubt?" Claude: [Migrates module, runs tests, fixes issues, commits with proper message]

Day 2-3 (Oct 7-8): Building Velocity

I realised that it was important for the agent to understand what to do, then I asked it to write down a plan.txt file with all the knowledge it had gathered, in order to be able to continue after different sessions by simply reading that file. I also asked the agent to update the plan with the discoveries and important patterns it had found, so the plan was always up to date.

This is the plan.txt file that was created:

Once Claude understood the pattern, the workflow became:

Me: "Migrate the kafka module" Claude: [Works on the module following the established pattern] [Runs tests for the module] [Creates a new branch: use-run-claude-kafka] [Creates commit] [Submits PR with proper description] Me: [Reviews PR, merges]

Day 4 (Oct 9): Complex Modules and Documentation

Even the complex ones followed the pattern:

• RabbitMQ: SSL settings, custom config templates, multiple wait strategies
• Redpanda: Listeners, TLS, authentication, bootstrap config (required breaking change to Option type)
• Redis: TLS with dynamic certificate generation, config file handling

I had time to polish the docs, of course, assisted by the agent:

• Updated docs/modules/index.md with comprehensive best practices (#3445)
• Updated module generator templates (#3445)
• Created AI.md file with guidelines for future AI agents (#3446)

Phase 3: Follow-up Consistency Pass (Oct 10)

After completing the main migration, I noticed some modules still had inconsistent option processing patterns. Some were processing custom Option types AFTER building moduleOpts instead of BEFORE.

This was a perfect test case for the pattern we’d established. I asked Claude to:

1. Scan all modules for this specific inconsistency
2. Fix only the modules that had the issue
3. Test each one individually
4. Commit with the pattern: chore(module): apply consistent patterns for options

Please check the results of the PRs in #3447.

Result: 6 modules fixed in a few hours

Modules corrected:

• couchbase - Complex custom options (buckets, services, credentials)
• etcd - Cluster configuration options
• gcloud/firestore - Project ID and datastore mode settings
• dockermcpgateway - Tools and secrets configuration
• azure/eventhubs - Azurite container options
• azure/servicebus - MSSQL container options

Each fix followed the same pattern:

This follow-up work demonstrated that the AI-assisted pattern is repeatable for any consistency improvement across a large codebase.

The Numbers

Let’s be honest about the metrics:

PhaseDurationModulesModules/DayFilesInsertionsDeletionsNet

Manual (Me)	7 days	19	2.7	51	+1,004	-1,184	-180
AI-Assisted (Migration)	3 days	41	13.67	60	+1,357	-1,557	-200
AI-Assisted (Consistency)	0.5 days	6	12	6	+81	-66	+15
Total	10.5 days	66	6.3	117	+2,442	-2,807	-365

The AI multiplier: 5.0x faster (13.67 vs 2.7 modules/day for initial migration)

Why negative net lines? The refactoring simplified code by replacing verbose GenericContainer initialization with cleaner functional options. Less code, same functionality—that’s good refactoring!

Follow-up consistency work: Once the pattern was established, applying it to catch edge cases was even faster—6 modules in half a day, touching just 6 files with minimal changes.

But the raw speed isn’t the full story.

What Actually Happened: The Real Collaboration

What Worked Brilliantly

1. Pattern Application

Once I showed Claude the pattern, it applied it consistently across all modules. No copy-paste errors, no forgotten steps, no variance. Every module got:

• The same 5-step Run function structure
• Proper error handling
• Correct option ordering
• Consistent naming conventions

2. Test-Driven Workflow

The workflow became incredibly efficient:

Claude: "Migrating postgres module..." [Implements migration] [Runs: make pre-commit test] [Test failure] [Analyzes error] [Fixes issue] [Re-runs tests] [All pass] [Commits with proper message] "✅ Postgres migrated, tests passing" [Moves to next module]

3. Learning and Adapting

Mid-migration, I discovered a better pattern:

Me: "Use strings.CutPrefix with early exit for env inspection, don't do manual string manipulation" Claude: [Updates current module] [Applies to ALL subsequent modules] [Never makes the old mistake again] [Update plan.txt]

4. Documentation Quality

Because Claude needs explicit instructions, I was forced to document everything clearly. This created artifacts that benefit everyone:

• docs/modules/index.md: Comprehensive best practices
• AI.md: Guidelines for future AI agents (and new developers!)
• plan.txt: Every pattern and pitfall documented

What Required Human Oversight

1. Architectural Decisions

Breaking changes needed human judgment:

Me: "The redpanda Option type needs to return errors for proper validation. This is a breaking change but it's the right call." Claude: "Understood. Changing type Option func(*options) error and updating all call sites to check errors."

2. Edge Cases and Context

Some modules needed special handling:

Claude: "The k6 module tests are failing..." Me: "Skip k6 tests - the Docker image is broken, not our code. This is documented in the plan." Claude: "Got it. Migrating code but marking tests as skipped."

3. Go Idioms

Me: "Return struct types, not interfaces. This is a Go best practice." Claude: [Updates code] "Changed return type from testcontainers.ContainerCustomizer to testcontainers.CustomizeRequestOption"

4. Git Workflow

Me: "Only git add files from the module you're working on, not the entire repo" Claude: git add modules/postgres/ [Not: git add .]

The Surprises

Positive Surprises

Claude caught bugs I would have missed:

In the rabbitmq module:

Consistent application of optimizations:

Once I mentioned strings.CutPrefix with early exit, Claude applied it perfectly across all 41 modules. No variance, no forgetting.

Better commit messages:

Claude wrote more consistent commit messages than I sometimes do:

Challenges

Security concerns:

Running an AI agent with full access to your codebase and terminal raises legitimate security questions:

• What if the AI runs destructive commands?
• What if it modifies files incorrectly?
• What if it pushes bad code to production?

My approach to mitigate these risks:

1. Strong test suite as a safety net - Every change must pass make pre-commit test before committing
2. Git version control - All changes are tracked, reviewable, and reversible
3. Human review - I reviewed every module migration, especially complex ones
4. Incremental commits - One module at a time, making issues easy to isolate and rollback

Future consideration: Docker sandbox

For even greater safety, running the AI agent in a Docker sandbox would provide:

• Isolated environment - Agent can’t affect host system
• Limited permissions - No access to push code or create PRs from inside container
• Controlled resources - Workspace mounted as volume, host handles git operations
• Audit trail - Container logs everything the agent does

This approach would let the agent do all the code generation and testing in isolation, while a human operator on the host reviews and executes the final git push and PR creation steps. The plan.txt file actually documents this workflow for future use.

Environment setup reminders:

Claude needed occasional reminders:

Me: "Remember to run: eval \"$(gvm 1.24.7 --arch=arm64)\" before any Go command" Claude: "Right, setting Go version first..."

Initial learning curve:

The first few modules with Claude required corrections:

Claude: [Makes attempt] Me: "Almost, but return the struct type, not the interface" Claude: [Corrects immediately] [Never makes that mistake again]

Explicit context needed:

Me: "The conventions.yml workflow enforces commit types. Only use: security, fix, feat, docs, chore, deps" Claude: "Understood. Using 'chore' for module migrations."

Lessons Learned

1. You’re the Architect, AI is the Builder

The human role is critical:

• Define patterns from real experience
• Make architectural decisions (breaking changes, design choices)
• Review and validate AI output
• Provide context the AI can’t have

The AI excels at:

• Applying patterns consistently
• Implementing details without copy-paste errors
• Running tests and fixing failures
• Handling tedious repetitive work

2. Context is Everything

The quality of AI output is directly proportional to the quality of context provided:

What worked:

• Detailed plan.txt with patterns and pitfalls
• Multiple reference implementations
• Explicit instructions about conventions
• Links to existing documentation

What didn’t:

• “Migrate this module” (too vague)
• Assuming AI remembers previous sessions
• Implicit knowledge I had but didn’t share

3. The Learning Phase is Essential

Going manual first was NOT wasted time. It was the foundation that made the AI-assisted phase successful:

1. I discovered what patterns actually worked in production
2. I hit the edge cases and learned how to handle them
3. I built intuition for “right” vs “wrong” solutions
4. I created reference implementations Claude could learn from

Key insight: You can’t delegate what you don’t understand.

4. Documentation Quality Matters for Everyone

Creating clear documentation for the AI had an unexpected benefit: it’s also perfect for human developers.

The AI.md file that was created by the agent includes:

• Exact commit format requirements (with validation rules)
• Common pitfalls and how to avoid them
• Testing workflow and environment setup
• Quick reference patterns with links to detailed docs

I hope that new contributors will love it. Future-me will love it. And yes, future AI agents will love it too. 💖

5. Trust but Verify

AI is impressively consistent, but:

• Always run the test suite. Thankfully, I had a strong test suite
• Review changes, especially for complex modules
• Validate architectural decisions
• Check for subtle bugs (like hardcoded values)

The test suite became my confidence check. If tests pass, the migration is likely correct.

Looking back, the real value wasn’t just migrating 60 modules. It was establishing a scalable collaboration pattern:

1. Learn patterns through hands-on experience (19 modules)
2. Document thoroughly what works and why (plan.txt, AI.md)
3. Create references for AI to learn from (postgres, redis, etc.)
4. Delegate implementation while maintaining architectural oversight (41 modules)
5. Capture learnings in living documentation (docs updates)

This pattern is repeatable for future refactorings.

What Changed My Mind

Before this experience:

• “AI can’t handle architectural refactoring”
• “Too risky for production code”
• “Would need constant supervision”

After this experience:

• AI excels at pattern application and consistency
• With good tests, it’s actually safer than manual work
• Supervision needed for direction, not implementation
• 4.4x faster (13.67 vs 3.1 modules/day) with equal or better quality

The bottleneck shifted from typing code to thinking about architecture.

Practical Takeaways

If You Want to Try This

Prerequisites:

1. Strong test suite - Your safety net
2. Clear patterns - Work out what “good” looks like manually first
3. Good documentation - For both AI and humans
4. Version control - Git makes experiments safe

Workflow:

1. Start with simplest cases yourself
2. Document patterns as you discover them
3. Create 2-3 reference implementations
4. Bring in AI for pattern application
5. Review, test, iterate

Red Flags:

• AI doesn’t understand context → Provide more examples
• Patterns inconsistent → Document explicitly
• Tests failing repeatedly → Review your pattern
• Copying mistakes → Better initial examples needed

The Questions I Asked Most

During the AI-assisted phase, these were my most common interactions:

1. “Read module X and Y, then migrate module Z using the same pattern”
2. “Run tests and fix any failures”
3. “Use [specific technique] not [other technique]”
4. “Commit with proper conventional commit format”
5. “Create a PR with this description”

Simple, directed, specific.

The Future

What This Enables

For large codebases:

• Breaking changes become less scary
• Technical debt pays down faster
• Library updates more feasible
• Consistency improves over time

For development teams:

• Senior developers focus on architecture
• Junior developers learn from perfect consistency
• Code reviews focus on design, not typos
• Documentation quality improves (AI forces clarity)

For open source:

• Contributors can tackle larger refactorings
• Patterns propagate quickly across modules
• Onboarding becomes easier (better docs)
• Velocity increases without sacrificing quality

The Real Insight

This isn’t about AI replacing developers. It’s about:

• Elevating developer work from implementation to architecture
• Amplifying impact through better leverage
• Accelerating evolution without sacrificing quality
• Improving consistency beyond human capability

The future isn’t “AI vs. Humans.” It’s “Senior Developer + AI = Superpowers”.

Conclusion

I migrated and standardized 66 Go modules in 10.5 days:

• 19 modules manually (7 days) - Learning patterns
• 41 modules with Claude (3 days) - Applying migration patterns
• 6 modules with Claude (0.5 days) - Applying consistency patterns

The AI didn’t do my job. It amplified my work:

• I made architectural decisions
• I defined patterns and conventions
• I reviewed and validated results
• Claude handled repetitive implementation

The metrics:

• 3.3x faster with AI assistance
• Zero regression bugs (all tests passing)
• Better consistency than manual work
• Improved documentation as side effect
• Repeatable pattern for future refactorings

The lesson: AI coding agents aren’t about replacing experience—they’re about multiplying it. The key is knowing what to delegate and what to own.

Start with understanding. Document clearly. Delegate confidently. Review thoughtfully.

On security: While giving an AI agent access to your codebase raises valid concerns, proper safeguards make it manageable—strong tests, git version control, human review, and incremental commits. For even greater safety, having a Docker sandbox can isolate the agent’s work while keeping critical operations (push, PR creation) in human hands.

The future of senior development is already here. It’s just not evenly distributed yet.

---

Try It Yourself

Want to see the actual code and patterns? Check out:

• Testcontainers for Go - The full codebase
• AI.md - Guidelines for AI agents (works for humans too!)
• Module Development Docs - Comprehensive best practices
• Sample PRs: #3425 (postgres), #3429 (redis), #3430 (redpanda)

The patterns, conventions, and lessons learned are all open source. Learn from them, adapt them, improve them.

---

Have you used AI coding agents for large refactorings? What worked for you? What didn’t? I’d love to hear your experiences. Find me on LinkedIn or Twitter.