.png)
7 hours ago
2
At the InfoQ Dev Summit in Boston, Phil Calçado, founder and CEO at Outropy, delivered a keynote aimed squarely at software leaders grappling with the realities of building and scaling generative AI products in production. Drawing on three years of hands-on experience and a 30-year career in software architecture, Calçado laid out concrete lessons on which traditional engineering patterns hold up when faced with the stochastic, stateful demands of AI systems.
He opened by acknowledging the varied backgrounds in the room and the evolving nature of generative AI. Calçado contrasted the rich set of IDE automations available to individual engineers with the near-total lack of tooling for managers and reliability engineers. He recalled, “My initial idea back in 2021 was like, okay, can we automate this? Can I create basically VS Code for everything either the manager does or everything an engineer does?”
Despite early success in benchmarks Calçado candidly admitted that “this product failed miserably.” He pinpointed the core issue as a misalignment of incentives: users were more interested in reverse-engineering the assistant’s internal design than relying on it for day-to-day productivity gains. This insight, he explained, revealed the difference between tools that demonstrate technological novelty and those that actually deliver sustained user value.
From these experiences, he distilled three dominant mindsets he’s observed in AI development. In the first, teams chase the latest model releases and funding cycles without addressing current limitations, waiting for “AGI next year” instead of iterating on today’s constraints. In the second, data-science labs undertake year-long projects to eke out incremental accuracy improvements with little product thinking, producing classifiers that only marginally outperform previous versions. In the third, teams adopt a software-engineering-centric workflow, emphasizing rapid, iterative development on lightweight, pluggable components that can be tested and refined in production.
To structure these systems, Calçado introduced two core abstractions: workflows and agents. Workflows are static, predefined pipelines that execute a series of transformations, such as ingesting messages, deduplicating conversations, and generating summaries. Agents, by contrast, are semi-autonomous components endowed with memory, goal-orientation, and the ability to invoke tools or other services dynamically. “Agents have memory, they’re goal-oriented, dynamic, and they like to collaborate,” he said, likening them to stateful objects in traditional object-oriented design.
He warned against treating agents as fine-grained microservices. The stateful, nondeterministic behavior inherent to LLM-driven agents undermines conventional service boundaries and leads to brittle coupling and operational fragility. Instead, he advocates for semantic event buses, where structured, intent-driven events—rather than raw database change streams—decouple components and enable discoverable, pluggable behaviors across the system.
When it comes to assembling AI pipelines, Calçado stressed the importance of decomposing monolithic projects into small, well-defined slices with clear semantic interfaces. In Autopi’s daily-briefing feature, for example, separate components handled Slack ingestion, duplicate-conversation detection, semantic summarization, and calendar context enrichment.
Calçado also challenged the applicability of the Twelve-Factor App model to AI systems. He noted that configuration is no longer static, processes cannot remain stateless, and LLM calls introduce inherent latency bottlenecks. To address these challenges, he urged teams to explore durable-workflow frameworks, such as Temporal, that cleanly separate orchestration from side-effects, handle retries and timeouts transparently, and provide checkpointing for long-running, stateful processes.
In closing, Calçado emphasized that while AI introduces new architectural demands, the core solution lies not in reinventing the wheel but in extending proven software-engineering principles to accommodate stochasticity and statefulness. “We don’t need to rebuild everything from scratch,” he argued. “We need better platforms that integrate these patterns so teams can iterate rapidly without reinventing the plumbing.”
Developers looking to learn more can watch infoq.com in the coming weeks for videos from the event.
