How Hype Works; What LLMs have in common with genetics

We knew how life worked when, in 4th century BC, Aristotle so thoughtfully explained that the soul (psyche) is what makes a body living.

In the 17th century, Newton and Descartes clarified that life is mechanical. Biology is simply wires and pumps. How exactly do these systems coordinate to produce life? Unclear. But if we applied the discipline of physics, we'd surely uncover the forces that underpin it all. It was just a matter of connecting the dots.

Phillip Ball’s "How Life Works" offers a historical recap of our evolving understanding of, well, how life works.¹ TLDR: we still don't know. But as Ball shows, a familiar pattern emerges. Every century or so, we crown a champion narrative — one that can finally answer the ever-elusive question. Every breakthrough gushes in heady optimism. We finally know what makes life tick!

In the 19th century, we saw limbs twitch when zapped with an electrical charge. Of course, the answer lies within the current! We were later shocked to discover the actual "physical basis of life" was protoplasm.

At some point, a German Zoologist observed,

There is one universal principle of development for the elementary parts of organisms, and this principle is in the formation of cells.

Cells... of course! They were staring us in the face all along, just out of sight. That is, until mitochondria broke the illusion. We had not yet reached the bottom of the biological stack.

Oh wait, no, it's genes! Genes are the missing key! Eureka! In the 1900s, gene-optimism was high. We were all this close to having 6-foot-4 gene-edited children.

Breakthroughs kick off phases of excitement. Some discoveries hit escape velocity and reach the uneducated public. This euphoria crescendos into confident (yet misguided) claims from world leaders, This is it! We have discovered "the language in which God created life"!²

Then, the cold water.

Life is not, after all, a black box with genes at one end and organisms and ecosystems at the other. For genes aren’t what we thought.

Phillip Ball later adds,

Just as there are physicists who will tell you that everything that happens can ultimately be explained by physics alone (it can’t), and chemists who tell you that in the end biology is just chemistry (it isn’t), so by asserting the primacy of the gene, geneticists are establishing an intellectual pecking order when they attribute more to genes than they should.

Once the hype died down, we were able to retro Watson's work. We realized we were so caught up in the emotion of it all that we still hadn't connected the dots.

Yet in years to come, it will be seen as deeply peculiar that we ever entertained the idea that Watson tried to seed with that apocryphal story: that life itself somehow inheres in the DNA molecule. To use a very crude analogy, that’s a bit like a literary scholar proclaiming to have discovered the “secret of Dickens,” only to whip out an abridged dictionary and say “It’s all in here!”

Discovery is intoxicating. Discovery tends to usher in vitalism: a doctrine that explains away the gaps in our knowledge through a nearly spiritual conviction. The apotheosis of discovery is what turns revelation into divine revelation.

A lesser-­known fabrication, however, is Watson’s claim (which he only recently admitted was pure invention) that, when he and Crick finally realized what the structure of the DNA molecule must be, Crick regaled the occupants of The Eagle pub in Cambridge, the duo’s favorite watering hole, with the claim that they had discovered “the secret of life.”

Vitalism's strength can cause a whole discipline of scientists to overreach.

We might recall here that many scientists of the early twentieth century were convinced that eugenics was a necessary and rigorously scientific consequence of Darwin’s theory of evolution. It wasn’t; but they believed it because it justified their view of how the world works, with its “natural” hierarchies of human value. The idea that DNA makes you what you are is not in itself as pernicious as eugenics, but it’s a hair’s breadth away if you are not careful, and is just as flawed.

Vitalism is a ladder.

Why we do still humor this notion is an interesting question for historians and sociologists of science—­which is to say, it surely reveals something about how science is shaped by the power dynamics of status, authority, and narrative control.

So what's the lesson? Is genetics a sham? Did we waste our time? No.

Geneticists aimed high. The uneducated masses aimed even higher (and in the wrong direction). Although we ultimately fell short of the unrealistic expectations, we unlocked incredible advancements in science and medicine. For example, we recently saved a child's life through gene editing.³ Who do we thank for these achievements? Not the eugenicists, but the tireless and pragmatic scientists who leveraged genetics as a tool.

I can't help but apply this Panglossian phenomenon to explain what's happening in my field.⁴

We are in a period of unprecedented uncertainty and pressure. I feel sorry for the engineering orgs who are asked to double their output because their C-suite saw Karpathy's tweet. Misconceptions around the limits of LLMs will lead to misplaced investments. And, unfortunately, unrealistic expectations will lead to a generation of LLM luddites. When we need them the most, engineers are retreating.⁵

Even though I acknowledge the hype, I cannot overstate the significance of LLMs. Software engineering is forever changed. Instead of hand-writing code, I am prompting.⁶ And yet, my engineering skillset has never felt as relevant and useful as it does today. An engineer's duties and expectations are shifting, and we must adapt.

The biologists most committed to their craft took advantage of the innovations in genetics while staying sober in the hype. As engineers, it is our duty to understand the strengths and constraints of these tools, educate the non-technical, and apply them thoughtfully. Yes, vibe-coding is cancerous, but that doesn't excuse a lack of curiosity.

Progress is not made because of the eugenicists, but despite them.

So, what are the true strengths and weaknesses of LLMs in software engineering? No one knows yet, but I begin to unpack the question here.

Knowing these constraints, how do we make best use of LLMs? Again, no one knows yet, but here is where I'd like to see investment.