AI doesn't make you more efficient

Since everyone is in the business of giving their opinion on the whole AI craze at the moment I thought I’d throw mine out in the ether.

After working with LLMs and building Agentic workflows for around year (with various degrees of success) I’ve settled on the opinion that LLMs don’t make you more efficient, rather, they’re a force multiplier.

What I mean by this can be better explained by examples. I hate using the “LLMs are interns” analogy as I think it’s somewhat demeaning to actual human beings but for the sake of this blog I’ll indulge in it once to get my point across.

The entire conversation surrounding LLMs is deceptively nuanced and I’ve struggled with distilling down my thoughts.

Unfortunately, as with most things these days, this topic is approached as “black and white”. People tend to fall into either one of these camps:

• LLMs are the second coming of Jesus

• LLMs are somewhat useless

The reality is very much “gray”-ish.

One of the things that I keep hearing from people is “LLMs have made me 100x times more efficient”. I genuinely don’t understand how that’s possible given that everything that comes out of an LLM needs human review.

That’s just the nature of working with a non-deterministic system. If you’re edging and living that YOLO life I can 100% see how thats possible but realistically almost every automation with an LLM needs a human in the loop.

In a pre-LLM world you’d have a senior developer working on complex tasks while delegating easier tasks to less experienced developers or interns. The senior developer then reviews the suggested code changes and then commits them if the tests pass, there aren’t any security flaws etc..

In a post-LLM world, we’re basically replacing those less experienced developers or interns with LLMs. The workflow hasn’t changed in anyway. You still have to review the code, make sure the tests pass and do the same thing you’d do if the code was produced by humans. There are 0 efficiency gains in the workflow.

What you have gained, however, is a force multiplier. You now have an unlimited amount of less experienced developers in the form of LLMs and you aren’t limited by the amount of humans you can delegate tasks too.

… this seems to be the mantra for any actual workflow efficiency gains when it comes to LLM automations.

I’m not denying there are workflows where you can get the LLM to produce the desired result 95% of the time. And for some low-risk use cases that’s fine. Summarization and search are the first two that come to mind but there are probably many others.

You can “ground” the LLM to a point where the risk of hallucinations is extremely low and couple that with some deterministic guardrails (e.g. programmatic checks) on the output.

If this is your use case for LLMs you have the “goose the laid the golden egg” sorta speak and congrats! You have unlocked efficiency gains.

The second the automation can have a high-risk consequence if the output is wrong you’re back to needing a human in the loop and any efficiency gains are questionable in my mind.

In a previous life I was a Penetration Tester/Red Teamer and an obvious application for LLMs is vulnerability detection (there are a lot of startups tackling this).

You basically have a “swarm” of agents each tasked with detecting a specific vulnerability (e.g. XSS, IDOR, SQLi etc..)

You obviously can follow up the LLM output with programmatic checks to verify the vulnerability exists, however you still need human in the loop to make sure the vulnerability isn’t a false positive.

This doesn’t make you more efficient per se as you still have to review each result, but it allows you an incredible amount of scale and the LLMs are definitely a lot better suited for this work (as they can “intelligently” test a number of attack permutations and adapt to the specific target) as supposed to traditional vulnerability scanners that just throw random strings over the network and inspect the response.

In its current incarnation, my conclusion is undoubtedly yes but mostly for organizations.

Additionally, everyone seems to be confusing efficiency with scale. These are two different things and the difference, in this context, is subtle but important.

LLMs enable companies to not be constrained by the amount humans at their disposal. This doesn’t mean companies who adopt “AI” are more efficient but rather they can scale faster and tackle a theoretically infinite amount of tasks in parallel. And viewed through this lens, it makes sense why the entire industry is pushing “AI” down peoples throats.

It’s unfortunate it’s being marketed to everyone as an “efficiency gainer” and I think it's leaving a sour taste in everyone's mouth.

Now, this might be on purpose in order to justify reducing workforce or to get that 5M dollar seed round, but it’s absolutely counter productive in the long run and I’m worried we will see the so-called “bubble” burst in some fashion as this technology is being sold the wrong way.