AI Is a New Computer

AI is a new kind of computer.

• A traditional computer processes structured data with deterministic instructions.

• AI processes unstructured data with natural-language nondeterministic instructions.

A more formal definition:

• A traditional computer is a universal structured information processor that can simulate any effective procedure (algorithm) given sufficient resources and the proper program.

• AI is a universal unstructured information processor that can simulate any intuitive procedure (reasoning) given sufficient resources and the proper context.

Like a traditional computer, AI will organize storage based on access speed, capacity, and cost.

Primitives of a traditional computer:

1. CPU registers/cache: Fastest access, smallest capacity

2. RAM: Fast access, moderate capacity

3. Disk storage: Slower access, large capacity

4. Network storage: Slowest access, potentially unlimited capacity

Primitives of an AI computer:

1. Transformer attention heads

2. Context window / KV cache

3. Retrieval data

4. Tool use data

To build the great production AI systems of the future - these components must work well together. Memory hierarchy enables software that makes the best tradeoffs of speed, cost and reliability.

This stack is in its infancy today - there is a lot of work to do.

• For example, today, retrieval is modeled more like a tool - whereas in the future it will sit directly next to the GPU in the same cluster (including speaking embeddings directly instead of natural language).

• Today LLMs do retrieval and tool-use one time before the generation - and in the near future (already happening with fine-tunes of Deepseek R1) models will continuously use tools and retrieval.

• Today context is manually managed by the developer - in the future a memory management system (an AI kernel running inside an AI operating system) will manage it and the developer will meta-program the memory management systems (AI compilers).

This all begs the question - if AI is a new kind of computer - then what kind of programs can we create and how do we program them? And how is that different from software as we know it today?