Why do CPUs have cache?
Main memory is so slow relative to a modern processor that without a small, very fast copy of recently used data sitting next to the core, the processor would spend most of its life waiting.
Simple intuition
The plain reason, in everyday words
Imagine you are cooking from a recipe, and every single ingredient is in a warehouse twenty minutes' walk away. You could walk there and back for each ingredient, but you would spend the entire day walking and almost no time cooking. Obviously you would instead carry an armful of things back at once and keep them on the counter next to you. A processor has the same problem. The main memory in a computer, the part usually called RAM, is physically separated from the processor and takes a surprisingly long time to answer. So the processor keeps a tiny counter-top of its own — the cache — holding copies of whatever it has used recently, plus whatever sat next to it. Almost every time the processor needs something, it is already on the counter.
Cache is just a smaller, faster kind of RAM you could scale up if you were willing to pay.
Cache is built from a different circuit — SRAM, which needs roughly six transistors per bit against DRAM's one transistor and one capacitor. It is faster because it is small and close, and making it large would make it slow, so the size limit is physics rather than cost alone.
More cache always means faster.
Larger caches have higher access latency and, past the point where your working set already fits, add nothing. This is why processors use a tiered hierarchy instead of one big cache.
Caching is something the operating system or compiler manages.
The hardware cache is entirely transparent to software — no instruction says 'put this in cache'. Software can only influence it indirectly, by choosing access patterns the hardware happens to handle well.
Cache behaviour is usually the largest single factor in the speed of code that is not doing heavy arithmetic, and it is invisible in the source. Once you know it exists, a whole class of mysterious performance results stops being mysterious: why arrays beat linked lists, why the order of nested loops matters, why adding a thread sometimes makes a program slower, and why two algorithms with the same complexity can differ by an order of magnitude.
Who worked it out
Maurice Wilkes described the idea in 1965 as a 'slave memory', and the IBM System/360 Model 85 shipped the first commercial implementation in 1968.
What problem forced it
Core memory was already the bottleneck, and it was clear that processor logic would keep improving faster than memory access.
How it changed since
The gap Wilkes anticipated widened for forty years, and the response has been more levels rather than bigger ones — from a single cache, to split instruction and data caches, to today's three or four tiers with shared last levels, hardware prefetchers, and increasingly elaborate coherence protocols. The side effects took much longer to surface: the timing-channel vulnerabilities inherent in the design were only widely appreciated after 2018.
How virtual memory works
The layer below cache, which decides what is in RAM at all.
Why SSDs wear out
Continues down the same hierarchy into storage, where the constraints change completely.
Written for Curio rather than collected from a forum — it is part of the curated corpus that ships with the platform. The references it draws on are listed under Sources.