The Unexamined AI Isn't Worth Trusting

Socrates and the Cost of Asking Questions

‘The life which is unexamined is not worth living,’ Socrates famously said, according to his student Plato, during his trial defense. Socrates was taken to court, accused of corrupting the youth and disrespecting the gods.

These charges were a euphemism for ‘we don’t like this guy, he’s causing a stir, and people, especially the youth, are beginning to question things. We better get rid of him before he does any more damage.’

Socrates was around 70 years old then. He was sentenced to death. Despite being given the chance to leave Athens and live in exile elsewhere, he preferred to follow the law and accept his death sentence. He argued that no one is above the law.

During his lifetime, Socrates asked questions. The questions themselves weren’t annoying. What was annoying was how, through his inquisitive act and guided conversations, his interlocutors would soon realize they didn’t really know a subject quite as well as they thought they did, or pretended to.

The Sophists: Persuasion Over Truth

Socrates, and later Plato, were grappling with what it means to know. What can we know? How can we know it? How can we teach it? How can we distinguish what is good from what is bad? How can we apply this knowledge to live a good life and flourish as individuals and as a society?

The pursuit of understanding preoccupied them, and many other philosophers throughout the ages. Plato, in particular, was haunted by his mentor’s trial and execution on seemingly ungrounded charges. He saw it as an example of how knowledge and opinion can become dangerously blurred. The sentence was decided by vote: the jurors found him guilty, 280 to 221.

Is there any distinction between an opinion and real knowledge?

The Sophists, a group of rhetoricians and orators that taught the art of public speaking, believed that there was no truth with a big T. There are only opinions. The goal was to learn the ropes of good argumentation and persuasion to sway the public in your favor. Of course, this is a caricature of their teachings. There were prominent Sophists, like Protagoras, who taught that truth was relative, depending on the context. I’m exaggerating slightly for the sake of this exploration.

The Sophists claimed they could be equally convincing arguing for one point or its opposite. The substance of your speech carried less weight. It was all about the delivery and the structure.

The Sophists thrived. Their model and method centered on persuasion, a skill they taught for a fee, and one that proved useful to politicians, merchants, and anyone navigating social or public life.

Socrates was suspicious of that. He distinguished himself from them by not charging his students. His goal was to invite people to examine their ideas and themselves more deeply, to think about the habits, customs, and beliefs they took for granted.

Unlike the Sophists, who focused on persuasion, Socrates was more interested in getting people to identify their blind spots, to acknowledge how little they truly knew. He believed that knowledge was acquired through questioning, reflection, and honest dialogue.

This method backfired. During his trial, as perhaps loosely documented by Plato, he suggested that many Athenians who viewed him as a threat were in fact behaving like Sophists, valuing persuasive opinion over critical self-examination and truth.

The feud was interesting. Some Athenians were suspicious of the Sophists, and they lumped Socrates in with them because of his influence on the youth. But Socrates wasn’t one of them. In fact, he believed that many Athenians had already fallen under the Sophists’ spell. At his trial, he made the case that if he really were a Sophist, he would have used rhetoric and persuasion to save himself. He didn’t.

Opinion, Knowledge, and the Alignment Problem

This mess is what pushed Plato to ask deeper questions about the nature of knowledge, the solid foundations beneath it, the structure of education, and the form of good and just governance.

I think that we’re in a similar mess today, a fairly reasonable one at that, given the whirlpool stirred by generative AI.

The way AI behaves resembles the Sophists’ persuasive style. It’s all about the delivery. It produces definitive and confident responses, prioritizing fluency over understanding.

The hype around AI is exposing us, like many ancient Athenians, to the risk of being seduced by its hypnotic touch.

We have an alignment problem with AI. We want to ensure that the way AI behaves is aligned with our values and intentions, avoiding to the extent possible issues like bias, misinformation, or an existential risk.

This requires more than just an infatuation with answers and prompt, polished responses whipped up from statistical models and probabilistic relationships.

The alignment problem is real, and it’s being tackled on several fronts, with efforts to develop models that reflect virtuous behavior.

AI’s Answers and Our Certainty Addiction

There’s also another issue, though. The problem of self-knowledge and self-examination. AI models can’t say they don’t know, yet. When they don’t know, they hallucinate, and still offer a confident, convincing reply. Without scrutiny, we can easily be persuaded by their answer and take it for granted.

The issue is that they don’t know that they don’t know.

Answers and certainty are way more comforting than questions and uncertainty. We often tell ourselves compelling narratives and build models to tame uncertainty and the risk that comes along with it. That’s understandable. Humans are wired to avoid uncertainty. We do that by naming things around us. We prefer to be in control of the situation.

When these stories fail, when the models stop working, we find ourselves grappling with an existential void and reaching for something solid to hold on to.

Today’s generative AI models appear to mimic this certainty, producing authoritative, persuasive answers. Without scrutiny, we might be outsourcing not only critical thinking, but also judgment, doubt, and responsibility.

The Unexamined Answer

The unexamined answer is an answer not worth accepting at face value. Examining an answer requires a multifaceted effort, including knowing oneself, understanding that we might not know, but also realizing that a super confident answer from an AI model does not necessarily equate to knowledge, either.

Seen through the lens of mimetic desire, going all in on the AI bet makes sense. We don’t want to be left out. We want to be part of the group that wins big with new tech. But at the extremes, there is just as much chance of losing big if and when the hype bubble bursts.

AI works. It’s extremely good. It saves us time and does all the tedious work that we couldn’t be bothered with. The benefits are undeniable.

Part of its appeal, though, is that it provides a comforting mirror when we most need it. It mimics our thinking and constructs compelling narratives that help us make sense of the world. Our relationship with it is almost faith-based.

In this case, asking deeper questions about AI’s benefits, or lack thereof, isn’t necessarily a Luddite rejection of new tech. It’s something more unsettling: it can feel like a challenge to our most recent beliefs. Examining ourselves and our AIs may be the only way toward a life worth living. But a modern-day Socrates might still be accused of corrupting the youth and disrespecting the new gods: GPT, Claude, Gemini, and the rest.

Socrates wasn’t executed for being persuasive. He was considered irritating for asking questions and challenging what people took for granted.