Paul Cantrell's Treatise Against Efficiency

On Fediverse, Paul Cantrell, CompSci professor at Macalester in St. Paul Minnesota, posted the following list:

It turns out I do have time to write an article about that, below the fold.

1. Efficiency is asymptotically inefficient: as costs approach zero, the cost of further reducing them approaches infinity.

This is a long-standing axiom of "nines" thinking. Where each "nine" of uptime costs rather more than the previous nine. I won't spend time on this, because this is widely understood. Fighting for better uptime, no matter how good it already was, is a big part of why Site Reliability Engineering came into being: we needed a way to quantify this, and frameworks for deciding when moar uptime! was not actually called for given business realities.

In areas where efficiency is not a proxy for uptime, where we have an entire engineering practice created to manage realities surrounding the topic, you get what we had before SRE was everywhere: managers attempting to score points by driving up efficiencies of operation and damn the costs! Improved efficiency will pay for itself! It wasn't a good time in the tech industry.

2. Efficiency prioritizes the measurable over the difficult-to-measure.

This is an axiom of management overall: you can't manage what you can't measure. Efficiency is a management problem, therefore efficiency is based on what you can measure. QED. Managing through vibes is not based in science.

3. Efficiency prioritizes what those in power see (or imagine) over on-the-ground reality.

This is another emergent axiom out of US tech industry management practices. In any organization with more than three levels on the org chart, the people making the big decisions on costs are at a great remove from the people doing the work. In theory, if your quantization is good enough (point 2) this shouldn't matter. Theory is not reality, though; which is why the big tech orgs often have Staff+ engineers (Staff, Principal, Distinguished Engineers, Architects) partnering with higher level managers to improve the odds of ground-truth being an input to the big decision. Improving the odds is not the same as 'shall.' The management context always wins in a conflict with ground-reality.

4. Following from 2 and 3, efficiency reduces the amount and quality of information flowing into a human system.

By restricting information solely to things that can be quantized (point 2) and with a tall enough org chart that the people who make the big decisions are heavily divorced from the people doing the work (point 3) you are making decisions on heavily summarized data based on a restricted subset of all the data, which makes deriving a high quality solution rather harder. This is a fundamental feature of modern management and data practices.

5. Efficiency foments institutional inflexibility.

Efficiency is a form of optimization.

Optimization requires building your organization and management structure to be fit for the given tasks, with no surplus.

Every time the task changes, you have to reoptimize. Reoptimization has a real cost, so don't change the task for little things.

This leads to inflexibility, and missed innovation opportunities; it simply costs too much to experiment with a new thing.

6. By removing slack, efficiency causes small failures to cascade more readily and increases the risk of catastrophic failure.

Efficiency of management becomes brittle when one failure causes other managers to get overloaded with work even with routine 'failures' like parental leaves. Slack allows you to better buffer these routine failures, and improves the odds of the whole thing avoiding a catastrophic failure when some unplanned failure happens.

Technical efficiency in distributed systems becomes brittle when the ability to scale up to meet demand is compromised, or a changed usage-pattern causes cascading failures due to lack of optimization in the new pattern. Slack, inefficiency, allows better buffering of changed usage patterns, allowing time to reoptimize for the new shape.

7. Following rom 4, 5, and 6, efficiency trades small costs for massive risks: from failures, from missed opportunities, and from inability to adjust.

As efficiency increases, slack decreases.

As slack decreases, tolerance of failure decreases. Even some 'planned' failures can prompt greater incidence-rates of major failure. Unplanned failures are far more likely to result in major failure.

As slack decreases, the cost of innovation increases, leading to missed opportunities. Certain innovations become classed as failures as a result.

8. Efficiency, when pushed, strangles the emergent phenomena that in the long term create all new things of value.

Efficiency by necessity creates brittleness and reduced failure tolerance.

Lack of failure tolerance means lack of tolerance for experimentation, which is the foundation of innovation. Such systems are more likely to wall innovation off into a 'safe' part of the organization, and be chronically underfunded.

9. Thus, although it can be a by-product of evolution, efficiency as a goal in itself strangles evolution.

Efficiency is optimization. While evolution can produce some amazingly optimized builds, such builds do not tolerate changed circumstances. Take the oxygen out of the system, and most of life as we know it has a much harder time working. Efficiency increases change intolerance, which slows evolution.

10. Efficiency as a goal strangles joy.

The joy of making is introducing change. A new, more efficient process. A better way of handling changed usage patterns. A fundamental rethink of how features are planned. A better platform system that's less annoying to keep running.

Efficiency increases change intolerance. Change is joy. Efficiency decreases joy.