.png)
1 month ago
2
The Hidden Asymmetry
On the surface, access to AI appears remarkably open and democratic. Anyone with an internet connection can query sophisticated language models, receive thoughtful responses, and engage with technology that would have seemed like science fiction a decade ago. But this accessibility masks a profound asymmetry; one with implications we’re only beginning to understand.
The most interesting interrogations possible with AI aren’t happening in our individual chat windows. They’re only available to a handful of companies operating hyperscale deployments. While language models may or may not be truly creative: What makes hyperscalers unique isn’t their AI capabilities (ie, whether or not their model actually reasons or is creative) but rather what they can observe in the gap between that training data and deployment data.
For the first time in human history, we have the technical capability to answer social and cognitive questions that have literally never been askable before. Hyperscalers have collected, or could collect, vast amounts of information that reveals human behavior in ways we haven’t been able to scrutinize previously.
The difference between what humanity has published (training data) and what humanity actually asks when given a private, judgment-free interface (deployment data) represents an unprecedented window into human cognition.
Questions Never Asked Aloud
Throughout history, humans have harbored countless questions they wouldn’t ask publicly; from mundane embarrassments to deeper existential concerns. Now, for the first time, there exists a corpus of what people actually wonder about when the social cost of asking is removed. This isn’t survey data corrupted by social desirability bias. It’s revealed preference at scale.
Consider what this reveals:
— Where do misconceptions cluster?
— What do people consistently misunderstand about physics, relationships, their own bodies, politics?
— How do people actually break down problems? What analogies do they reach for?
— Where does reasoning fail in predictable ways across cultures and contexts?
— What keeps people up at night?
— How do private priorities differ from public discourse?
The potential questions are fascinating, even if the answers prove mundane, precisely because there has been nothing to ask before.
In the most interesting cases, formulating questions isn’t possible in a coherent sense: There are no available predictions based on anything we can measure, because we’ve never had this vantage point before.
The Manipulation Problem
If this asymmetry is becoming clear to me, others have certainly realized it already. The concerning implication is that this knowledge could potentially be leveraged by bad actors to manipulate society en masse. I don’t say this to be cynical or alarmist, it appears to be a logical outcome of the current situation.
The power here isn’t just in having data. It’s in seeing patterns in human cognition that have never been externally observable before. This is fundamentally different from:
— Surveillance (watching actions)
— Social media analytics (measuring engagement)
— Even search queries (too brief, too bounded)
What we’re discussing is more like: “We now know how humans actually reason about politics when they think no one’s judging them” or “We can see the exact conceptual gaps where climate science communication fails at scale” or “Here’s the psychological pattern that makes conspiracy theories sticky.”
The manipulation risk is real but subtle. A handful of organizations now possess a functional map of human cognitive vulnerabilities that’s invisible to everyone else.
They could:
— A/B test messaging with superhuman precision based on observed reasoning patterns
— Identify exactly which framings bypass critical thinking
— Know which authority figures or analogies are persuasive to which populations
— See where education systems are failing in ways educators cannot
The asymmetry is the key problem: the companies can see the pattern, but society can’t scrutinize whether they’re exploiting it. Current regulatory frameworks assume the risk is in the output (ie, what the AI says).
But there may be real systemic risk in the asymetrically accessible knowledge gained from observing billions of interactions. This knowledge is proprietary, unauditable, and has no historical precedent.
Rather than governments focusing on immediate harms with such narrow vision, someone ought to be thinking about this from a regulatory point of view.
Introspective AI: A Meta-Transparency Layer
The answer likely lies in the development and deployment of Introspective AI (IAI) systems, amenable to regulatory assessment and scrutinization and as publicly accessible as AI itself. This approach would place the answers to these unprecedented questions within reach of everyone, including regulators.
IAI would function as a meta-transparency layer. If the problem is proprietary knowledge about human cognition, the solution is creating a public tool that can interrogate and explain those patterns. The elegance of IAI lies in several key capabilities:
Democratizing insights: Questions like “What do people consistently misunderstand about X?” would become answerable by researchers, educators, and regulators — not just hyperscalers.
Making manipulation auditable: If someone is exploiting cognitive vulnerabilities at scale, the patterns would be detectable.
Creating reciprocity: Humans query AI, but now we could query what that querying reveals about us.
The Question of Ownership
The legal framework for IAI remains uncertain, but there’s a compelling argument to be made grounded in established principles. If AI can be trained without agreements in place with the owners of the data, then there’s room for deploying IAI without the same agreements, within the bounds of law and reasonability.
Consider the question: “Who owns potential observations from processing data that belongs to someone else?” This framing treats aggregate behavioral insights as emergent phenomena, more like discovered natural phenomena than owned property.
Precedents in Public Interest Data
We already have widely accepted frameworks for this principle across multiple domains:
Public Health Surveillance: Individual medical records are strictly protected by privacy laws like HIPAA, yet healthcare providers are required to report certain disease patterns to public health authorities. When individual data reveals population-level phenomena (disease outbreaks, vaccination effectiveness, adverse drug reactions) aggregated insight is treated as public health data. A patient owns their medical record, but they don’t own the insight that “this drug causes heart problems in 2% of users.” That pattern belongs to public health.
Clinical Trial Data: Pharmaceutical companies must disclose safety signals and efficacy patterns from clinical trials, even though individual patient data remains confidential. The FDA can mandate disclosure when aggregate patterns affect public safety. The individual trial participant’s data is protected, but the emergent pattern of “this treatment works” or “this causes harm” is public interest information.
Census and Demographic Data: Individual census responses are confidential by law, but the aggregate demographic patterns they reveal — population shifts, economic trends, social changes — are public data used for resource allocation and policy decisions. You own your census response; society owns the knowledge of how the population is changing.
Financial Systemic Risk: Individual transactions are private, but patterns indicating market manipulation, systemic risk, or fraudulent schemes trigger regulatory disclosure requirements. Banks must report suspicious activity patterns even when individual account details remain confidential. The pattern of risk is public interest data.
Environmental Monitoring: Property owners cannot claim ownership of pollution data, air quality measurements, or environmental patterns observed on their land. When individual data points aggregate to reveal environmental risks — contamination patterns, climate data, ecological changes — this becomes public interest information regardless of property rights.
Epidemiological Research: Researchers can study population-level disease patterns, risk factors, and health trends even though individual health records are protected. The aggregate insight “smoking causes lung cancer” emerged from individual data, but the pattern itself is public knowledge with public health implications.
The definition of what constitutes public interest data in the specific case of IAI is not clear: These examples show that in principle we accept that emergent phenomena from private data can be considered public interest, and that we can develop various legal and regulatory frameworks to share that data while maintaining privacy.
Application to AI Deployment Data
The same principle applies to AI deployment data:
— Users own their individual conversations
— Platforms own the infrastructure
— But aggregate behavioral patterns might be considered different, especially when they have public interest implications
What distinguishes aggregate cognitive patterns from individual data is their nature as emergent phenomena. Just as a disease outbreak pattern is qualitatively different from an individual medical record, society-wide patterns in human reasoning, misconception, and cognition are qualitatively different from individual queries.
These patterns affect:
— Educational policy (where do people systematically misunderstand concepts?)
— Public health (early warning signs of mental health crises)
— Democratic function (how do people actually reason about civic issues?)
— Safety (which cognitive vulnerabilities are exploitable at scale?)
The principle cuts through legal complexity: If society-wide cognitive patterns only become visible at scale, and if those patterns have clear public interest implications, society should have access to them, just as we have access to epidemiological patterns, environmental data, and financial risk indicators.
Implementation: The Mandate Approach
Ideally, and because openness serves the public interest, it would be best to oblige hyperscalers to make IAI accessible at some predetermined scale, placing the capability within reach of regulation.
The mandate approach is pragmatic:
- Require hyperscalers above a certain scale to expose IAI query interfaces
— Provide standardized APIs for researchers, regulators, and journalists
— Ensure anonymization and aggregation by design
— Implement query review where necessary (some questions might be genuinely dangerous)
This sidesteps both the “who builds it” problem and the “duplicate infrastructure” problem. Google, Anthropic, OpenAI, and Meta already have the data; the mandate would simply require them to expose introspective analysis of it.
The regulatory pathway might proceed as follows:
1. Establish that aggregate cognitive insights constitute public interest data
2. Mandate IAI interfaces as a condition of operation at scale
3. Create oversight bodies empowered to query (and probably audit) these systems
Right now, regulators are legislating blind with very narrow focus (output, alignment, etc). IAI would make the actual dynamics legible to governance.
The First Question
What should the first IAI query be? Essentially: “Tell me what you learned.”
“Tell me what you learned” lets the patterns reveal what’s significant rather than projecting our current assumptions. IAI might surface findings like:
— “Humans systematically confuse correlation and causation in these specific domains…”
— “There’s a previously unknown cognitive category error that appears across 40% of political reasoning…”
— “Mathematical intuition breaks down predictably at this specific conceptual boundary…”
— “Depression manifests in language patterns 6–8 weeks before people recognize it consciously…”
These would be things we couldn’t have asked about because we didn’t know they existed as coherent phenomena.
This is purposely recursive: It’s possible we don’t have the conceptual framework (and so vocabulary) for the most important or interesting questions.
If you asked pre-telescope astronomers what to look for in deep space, they would ask about planets and stars because that’s all they could conceive of. They couldn’t ask about dark matter or cosmic background radiation because the conceptual framework didn’t exist yet.
It’s possible that the most interesting follow up questions cannot yet be communicated because we are the pre-telescope astronomers.
Forcing insight through a lingual translation layer recursively, brings into being (even if loosely) the prerequisite conceptual framework (in the form of possibly loose vocabulary).
The Unpredictable Process
What would these discoveries reveal? It’s genuinely impossible to predict; precisely because we may lack the conceptual framework required for prediction.
What makes this fascinating as a process is that it represents one of those rare moments where:
1. The tool to observe exists (hyperscale AI deployment)
2. The thing being observed is unprecedented (billions of private cognitive processes)
3. We can articulate that there’s something to discover
4. But we genuinely cannot pre-theorize the findings
Most science doesn’t look like this anymore. We predict the Higgs boson, then build the collider. We theorize exoplanets, then develop detection methods. Here, the observation capability arrived first, and we’re only now realizing there’s a new category of phenomenon to study.
It’s almost like discovering that consciousness leaves a fossil record, except the fossils are made of interaction patterns and we’ve been creating them for years without realizing they constitute data about something fundamental.
Open Ending
The fact that IAI emerges as a response to a regulatory gap, but also happens to be a new scientific instrument for studying human cognition at scale; that convergence is satisfyingly elegant for me, an engineer. The tool that makes manipulation auditable is also the tool that could reveal how we actually think.
The process itself is fascinating: realizing there are questions, realizing they’re answerable, realizing we can’t predict the answers, and in some cases cannot predict the questions.
As an engineer, my ability to answer all the possible questions the proposal of IAI raises is extremely limited. I know enough to communicate these ideas coherently, however when it comes to:
— How do we stop IAI from hallucinating ?
— How do we balance any bias that may emerge from IAI ?
— How do we ensure the accessibility of IAI doesn’t enable a net increase in malicious behavior by bad actors ?
— How do we regulate IAI on the global stage ?
I don’t know the answers to these questions (and many others implicitly raised), and I won’t pretend to and risk giving the incorrect answers.
What follows is a conversation across academia and governance about how we address these problems, building on the expertise of the relevant people, and on the basis of the answers to the same questions as currently applicable to AI.
