Context is all you need

1 day ago 2

This is the blog post version of the paper we published with expansion on more actionable tactics. You can view the full paper here.

Rewind to an idea that once lit up machine learning: "Attention is all you need," the breakthrough that showed AI models could focus on relevant parts of data without complex memory systems.

Yet, the next decade may prove that attention is only as powerful as the context you feed it. The race for AI dominance has shifted from who has the best models to who controls and understands user context.

Forget the Humane AI Pin — this new AI pendant from Limitless actually  looks useful | Tom's Guide

Startups like Limitless are shipping $99 pendants that clip to your shirt and record every conversation, promising to transcribe meetings and answer questions based on your accumulated interactions. Similar devices like Friend's are betting on much more personalized companions that can provide just-in-time personal interactions based on any situations. The race for context is underway.

This essay distills key insights from our recent collaborative paper into actionable tactics for product builders and designers.

A few definitions to get on the same page:

Context: All signals available to a product, including environmental cues, device state, user history, and intent.

Context Window: The quantity and recency of information processed by a model.

Memory: Persistent context carried forward across sessions.

Personal Context Infrastructure: Systems that aggregate, store, update, and share context between services.

Context Boundary: Rules defining what information is accessible and how memory is managed.

​​Effective context-aware systems operate across four layers:

  • Immediate: Current conversation or task.

  • Session: Today's work or project.

  • Personal: Preferences, patterns, relationships.

  • Environmental: Location, schedule, available devices.

For as long as we’ve dreamed of AI, we’ve longed for machines that not only “get” our explicit asks but seem to understand us, the way a longtime friend finishes your sentence or a creative partner anticipates your next thought.

The 2 Most Important Things Steve Jobs Taught Jony Ive

We want AI to evolve from a mere notetaker to a genuine collaborator, a presence that knows when to nudge, when to listen, and when to step back.

The most valuable resource isn’t algorithms, compute, or even just data. It’s context: the mosaic of real-time signals, history, recent mood shifts, stray calendar invites, and the live-wire pattern that is “you, right now.”

There’s a collective land grab for context. Who gets to see it? Who stores it? Who profits from it? Whoever answers these questions wins the right to shape what AI is for the next generation. Business models are already emerging around context:

Notion AI is here, for everyone
Notion AI
  • OpenAI's GPTs: User generates context and memory by using the product, OpenAI hosts and monetizes

  • Notion's AI: Context stays in user workspace, subscription model

  • Google's Ultra AI: Free context analysis across product surface area that can subsidize ad targeting

  • Apple Intelligence: On-device context processing as premium hardware differentiator

Historically, platforms vacuumed clicks and scrolls into massive datasets. Today, with smarter models and expressive interfaces, AI builds "digital twins," live models of user intent and state. Recent context now vastly outweighs stale historical data.

For example, a health-tracking wearable that alerts you based on your current heart rate, hydration, or stress levels rather than past averages. Smart home devices that adjust lighting, temperature, or music based on immediate activities or conversations, not historical usage patterns. Productivity apps like Linear or Notion highlighting tasks that are actively being discussed in real-time collaboration.

Oura takes part in its first Prime Day sale with viral smart ring reduced  to just $281 - plus more deals to shop now | The US Sun
Oura Ring

TACTIC #1: Prioritize Real-Time, Portable Context

  • Focus on real-time signals over legacy data, like live user interactions, real-time sensor inputs, or immediate environmental conditions.

  • Design context ingestion pipelines optimized for immediacy, such as instant transcriptions or live sentiment analysis.

  • Make "context state" objects explicit and actionable, like real-time dashboards showing current user states or live context feeds for immediate decision-making.

Context isn’t something you can just scrape or surveil. You have to earn it. The interfaces we choose—whether chat, camera, calendar, or code editor—are invitations. They shape not only how much context we’ll willingly share, but also the kind that gets generated. A design tool will surface creative patterning. A health wearable will log your physical rhythms. An intimate AI chat will prompt you to disclose the messy, heartfelt stuff you’d never type into a search box.

The big unlock will come from combining data from different domains. Your Spotify listens, Dexcom data, and Notion scratchpads are all different “eyes and ears” on your life. The winner might just be whoever stitches the most relevant slices together in a way you actually trust.

TACTIC #2: Enable Composable, Multi-Source Context

  • Make it seamless for users to permission and de-permission slices of their context (per product, per task, per agent).

  • Build apps/agents/APIs/protocols that let users stitch context across domains (calendar + health + writing tools + chat).

  • Don’t fight context silos. Design for “cross-context” products that combine the few signals that matter for each job.

Generated image

Success comes from mastering three technical layers working together.

  • Collection Layer: APIs, sensors, and behavioral signals. GitHub Copilot doesn't just read your code—it notices your coding patterns, preferred libraries, and when you're most likely to introduce bugs.

  • Processing Layer: Where raw signals become intelligence. This handles context synthesis (connecting dots across data sources), privacy filtering (what gets remembered vs. discarded), and relevance ranking (what matters right now).

  • Application Layer: Context becomes action. The AI that finishes your sentences, the assistant that avoids scheduling calls during your kid's soccer practice, the design tool that suggests your recent color themes.

The winners either build all three layers (OpenAI's full-stack approach) or dominate one layer while integrating seamlessly with others (Calendly owns scheduling context, Linear owns dev project context).

This architecture explains why context portability is hard. You're not just moving data files, you're losing the processing logic that learned your patterns. Trust becomes a competitive advantage: users share more context with systems they trust, creating a flywheel effect across all three layers.

What is Cursor AI? Everything You Need to Know About the AI-Powered Code  Editor | UI Bakery Blog
  • Precise control of intent and state context.

  • Uses @-symbol syntax for specific context targeting.

  • Minimalist approach, prioritizing relevance.

Cursor proves that context minimalism beats maximalism. "Too much irrelevant context dilutes the signal." Success comes from giving developers precise control over what matters when.

Spotify Debuts a New AI DJ, Right in Your Pocket — Spotify

Spotify's AI DJ demonstrates how layering multiple context signals creates deeply personalized experiences:

  • Combines immediate, session, personal, and environmental contexts.

  • Invisible context collection through seamless interactions.

  • Real-time narratives for personalized user experience.

Spotify succeeded by making context collection invisible while delivering immediate value. Users don't fill out preference forms, they just listen while the system learns.

Several of us in the group played with the idea of a local, personal “MCP” (Model Context Protocol). Tina previously wrote about it on People as Files. Imagine every insight about you, from personality type to skill profile to emotional pulse, is stored in a sandbox you control.

The dream: a portable context engine that travels with you, layerable across services, encrypted and privacy-respecting by default. In theory, you could “train” a new chatbot on your real personality—or let a digital agent represent you—without starting from scratch every time you switch apps.

In practice, privacy and security challenges remain complex. But the direction is clear: a future where your context truly belongs to you, freely movable with permissions tailored to your preferences.

TACTIC #3: Move Toward Portable Context “Profiles”

  • Let users export/import raw context and trained “persona” objects (as files, APIs, or via open protocols).

  • Make it explicit what part of the user’s context your product ingests, and let them audit/edit the feed.

  • Expect the emergence of “context wallets,” products that manage, update, and route slices of user context to downstream agents.

Legacy platforms have no reason to help you escape. The business model is simple: lock in as much context as possible, then charge rent (in money, attention, or both) for access to your own life. Regulation (GDPR, DMA) is starting to force cracks in these walls. The friction is part of the moat.

​​But the challenges run deeper than business incentives. Previous attempts at portable identity, from OpenID to decentralized social protocols, failed not just due to platform resistance, but because context interoperability is genuinely hard. Context isn't just data; it's data plus the processing logic that interprets it. When you export your Spotify listening history, you lose the algorithm that understands what "listening to Radiohead at 2 AM" means for your emotional state. Technical standards can share the what, but struggle with the why and how.

Context exhibits powerful network effects that work against portability. Each additional piece of context about a user doesn't just add linear value,it creates exponential improvements in personalization. A company with 80% of your context can provide dramatically better service than one with 20%, making it economically irrational for users to fragment their context across providers. This creates a natural monopolization force where winner-takes-most dynamics emerge.

The computational costs also become staggering. Maintaining a rich, multi-layered context for millions of users requires massive infrastructure. Every additional context signal exponentially increases processing complexity. This creates a natural centralization force, only tech giants can afford to maintain context at scale, reinforcing the very lock-in we're trying to escape.

Most critically, we face the privacy paradox: users simultaneously crave deep personalization ("know me like a best friend") while fearing surveillance ("but don't watch me"). This isn't irrational, it reflects the genuine risk that context powerful enough to anticipate your needs is also powerful enough to manipulate them. The same context that helps an AI therapist provide better support could be weaponized for targeted exploitation. No technical architecture fully resolves this tension.

So what breaks the cycle? Maybe it's regulation, maybe it's collective consumer demand, or maybe it's "killer apps" that make it genuinely worth your time to aggregate your context somewhere new. If a product or protocol helps you get real value such as better recommendations, more meaningful decision support, even a sense of feeling "seen," you'll move. If not, inertia wins and the silos persist.

TACTIC #4: Design for Context Leverage, not Lock-in

  • Empower users through aggregated, portable context.

  • Build context-aware features that provide immediate value.

  • Be the product users actively choose to bring their context into.

  • Be the “killer use case” that gets users to bring their context in.

People are already more candid with AI than with most humans. When your AI agent “gets” you, it can feel downright magical. But the line is thin. Premonition, overdone, is unsettling or even creepy. Worse, breaches of context trust feel like personal betrayals, whether it’s a chatbot forgetting your favorite color or a search history resurfacing at the wrong moment.

AI systems need to support radically different context preferences across domains. Some users want everything unified, others want sharp boundaries between work and life, or between public and private selves. Agency isn’t just about context control; it’s about the ability to curate which “self” you bring to each interaction, and to see (and edit) what your agents see.

TACTIC #5: Build Context Boundaries and User Controls

  • Let users toggle between “public,” “work,” “private,” or custom context scopes for different agents.

  • Expose “what your agent sees” as an auditable, interactive surface (not a black box).

  • When in doubt, surface ambiguity: ask if the agent should remember/discard/refine certain facts.

The true potential of context-aware AI lies not just in predicting but in autonomously acting on user intentions. Not just “Do you need new vitamins?” but “Should I order them?” All of this depends, again, on context, knowing enough to take action, but not so much as to overstep.

The interfaces will feel more ambient, more “always on,” more gently anticipatory than the rigid forms and feeds of the web 2 era. And as these agentic systems mature, we’ll see both specialized and general-purpose versions, from the domain-specific assistant that reigns in health or finance to the “big brain” generalist that orchestrates across your life.

TACTIC #6: Productize Context-to-Action Loops

  • Enable one-click or agentic follow-through (“Do this on my behalf”).

  • Allow users to preview, veto, or modify actions based on current context state.

  • Instrument the “last-mile” of automation: what context drove which actions, and with what confidence.

We're at an inflection point. The AI race is shifting from model performance to context mastery. The winners won't be those with the biggest models, but those who build the most trusted context infrastructure.

  • If you're a builder: Start with one killer use case that makes context portability irresistible. Focus on immediate value, not abstract promises. Remember: Spotify didn't pitch "context awareness"—they just made music feel personal.

  • If you're a platform: Choose your strategy now. Will you lock in context and milk it or become the trusted context layer others build on (the Stripe playbook)?

  • If you're a user: Your context is your leverage. Support products that give you control, not those that mine you for data.

The technical challenges are real. Interoperability is hard, computation is expensive, and privacy is paradoxical. But these are engineering problems, not laws of physics. The first products to crack the code will trigger an avalanche. Once users experience a truly portable, powerful context, there's no going back to silos.

Context is all you need, but only if you build for user agency, not addiction; for portability, not lock-in; for trust, not surveillance.

If these themes resonate, or if you want to dig into the full slightly more technical and design detail, you can check out the paper here.

Read Entire Article