What a chip actually is

The problem

Somebody has to do the sums

Part 2 ended with a loop: for every single token it writes, a model runs one enormous calculation, mostly multiplying and adding numbers. For a large model that is hundreds of billions of small sums per token. No person could do that, and neither could a room of people. A chip can, in a fraction of a second.

So what is on that piece of silicon, and how do switches do arithmetic? The answer builds up in layers, and each layer is simple on its own.

The building block

A switch is either on or off

Think of a light switch. It has exactly two states: on or off. There is no “a bit on”. A chip is built from switches like that, only microscopic and flipped by electricity instead of a finger. One switch can hold one fact: yes or no, 1 or 0. That single yes-or-no is called a bit.

Why only two states, and not ten like our digits? Because on-or-off is very hard to get wrong. A switch that is “mostly on” is still clearly on, so a tiny bit of electrical noise cannot turn a 1 into a 0. Telling ten levels apart reliably, billions of times a second, would be far harder.

Counting with switches

Eight switches can count to 255

One switch can only say 0 or 1. Put several in a row and give each a value, like the places in ordinary numbers. In everyday numbers each place is worth ten times the one to its right: ones, tens, hundreds. With switches each place is worth twice the one to its right: 1, 2, 4, 8, and so on. Add up the values of the switches that are on. Tap the switches:

This way of writing numbers with only 0 and 1 is called binary. Eight bits together are a byte, which can hold any whole number from 0 to 255. The bytes in Part 1, chapter 2, are exactly this: each letter stored as one or more groups of eight switches. Bigger numbers just use more switches: 16 bits reach 65,535, and 32 bits reach about 4.3 billion public.

The real switch

A transistor: a switch flipped by electricity

A light switch needs a finger. A chip’s switches have to flip each other, so each one is flipped by electricity. Picture a water pipe with a tap in the middle, where the tap is opened not by a hand but by water pressure in a third, small pipe. Push on the small pipe and water flows through the big one. Let go and it stops.

That is a transistor: three connections, and a small voltage on one of them (the control, called the gate) decides whether electricity can flow between the other two. Crucially, the output of one transistor can be the control of the next. That is how switches flip switches, with no finger anywhere, and it is the whole trick behind every chip.

Switches that decide

Wire switches together and they make decisions

Put two switches one after the other in a line to a lamp. The lamp lights only if the first and the second are on. Put them side by side instead, as two separate routes, and the lamp lights if the first or the second is on. Flip the inputs:

These little decision circuits are called logic gates. AND and OR are two of them. NOT flips its input: on becomes off. XOR, “one or the other but not both”, lights when the two inputs differ. Each gate takes only a handful of transistors public. Every calculation a computer does, from adding to running an AI model, is built from gates like these, wired together in enormous numbers.

The first sum

Adding two bits takes two gates

Adding single bits has only four cases. 0 + 0 = 0. 0 + 1 = 1. 1 + 0 = 1. And 1 + 1 = 2, which in binary is written “10”: a 0 in the ones place and a 1 carried to the twos place, just like 5 + 5 = 10 carries a 1 to the tens.

Now look at the gates from chapter 5. The ones digit of the answer is 1 exactly when the inputs differ: that is XOR. The carry is 1 exactly when both are 1: that is AND. So two gates add two bits.

This tiny circuit is called a half adder. It is the moment switches stop being switches and start being arithmetic.

Bigger sums

Long addition, done by switches

To add bigger numbers, do what you learned at school: add column by column from the right, carrying when a column overflows. Each column gets a small adder circuit that adds three bits (two digits and the carry coming in) and passes its own carry to the next column. Set two numbers and watch the carries ripple left:

Four columns of adders handle numbers up to 15; real chips chain 32 or 64 columns. A full one-column adder takes about 28 transistors in the common design public, so a 32-column adder is roughly 900 transistors. That sounds like a lot, until chapter 11.

Multiplying

Multiplying is shifting and adding

AI models mostly multiply. In binary, long multiplication is even simpler than in school, because every digit is 0 or 1: you either add a shifted copy of the number, or you add nothing. So a multiplier is just a grid of AND gates choosing what to add, feeding rows of adders.

An 8-bit multiplier built this way takes a couple of thousand transistors example. One multiplication: a couple of thousand switches flipping in a particular pattern, all at nearly the same instant.

Keeping time

The clock: a drumbeat billions of times a second

After the inputs change, the switches inside an adder need a moment to settle into the right answer. If the next step started too early it would read a half-finished result. So every chip has a clock: a signal that ticks on and off at a steady rate, like a drummer keeping a rowing team in time. On each tick, results move one step forward.

Clock speed is measured in gigahertz: billions of ticks per second. A typical laptop chip runs at about 3 to 5 GHz public. At 3 GHz a tick is so short that light itself only gets about 10 centimetres before the next one. That is why distances on a chip matter: signals simply cannot cross a big chip in one tick.

Remembering

Switches that hold their value

Arithmetic is useless if results vanish. A chip also needs memory: switches that keep their state after the input goes away. The classic trick is a loop. Take two NOT gates and connect each one’s output to the other’s input. If the first says 1, the second says 0, which keeps the first at 1. The loop holds itself in place until something forces it to flip.

Memory built this way is called SRAM, and it takes six transistors per bit public. It is very fast but takes a lot of space, so chips keep only a little of it close to the arithmetic. Most memory instead stores each bit as a tiny bucket of electric charge with one transistor as its lid. That is DRAM, the kind in your laptop’s RAM. The buckets slowly leak, so the chip has to top them up every few hundredths of a second public. Part 8 is about why memory, not arithmetic, is now the real bottleneck for AI.

How small

Smaller than a virus

A chip needs billions of these switches, so each one has to be unimaginably small. Slide down the scale, from a human hair to a single atom of silicon:

Neighbouring transistors on a modern chip sit roughly 50 nanometres apart, and their finest parts are only a few nanometres wide: a few tens of silicon atoms public. One gotcha: chip factories name their processes “3 nanometre” or “2 nanometre”, but those names are labels for a generation, not the size of anything on the chip public.

Billions, then more

Twice as many switches, every two years

In 1965 Gordon Moore, who later co-founded Intel, noticed that the number of transistors that fit on a chip was doubling at a steady pace. In 1975 he settled on every two years. The prediction held for about fifty years and became known as Moore’s law public. It is not a law of nature, just a remarkably steady pace of engineering.

Each gridline is ten times the one below, so a straight line means steady doubling. From 2,300 switches in 1971 to 208 billion in a single AI chip package in 2024 is about 90 million times more public. That growth is what made today’s AI possible at all.

Why silicon

A material that can be told when to conduct

Metals always let electricity through. Glass and plastic never do. Silicon sits in between, which is why it is called a semiconductor. On its own it barely conducts. But add a tiny, controlled pinch of other elements, about one atom in millions, and it can be made to conduct in exactly the spots you choose, and switched by a voltage like the tap in chapter 4.

Adding those other elements on purpose is called doping. How a factory puts billions of precisely doped, precisely shaped switches onto one slice of silicon, using light, is a story of its own: it is the last part of this track, part 17.

Connecting them

Switches on the bottom, wires stacked above

Billions of switches are useless unless they are connected the right way. The transistors sit in a single layer at the bottom of the chip. Above them is a stack of wiring layers, like floors in a building. The lowest floors carry tiny, short wires between neighbouring switches; the top floors carry thick wires across the whole chip and bring in power.

Modern chips have more than ten wiring layers, with many kilometres of wire in total, all on a piece of silicon you could cover with a fingernail public. Designing which switch connects to which is done by software, because no team of people could draw billions of connections by hand.

The hidden cost

Every flip makes a little heat

Each time a switch flips, a tiny amount of electricity is used and turned into heat. One flip is nothing. But billions of switches flipping billions of times a second add up to a lot of heat in a very small space.

A big AI chip can draw around 1,000 watts, about half an electric kettle, from a slice of silicon a few centimetres across public. Air alone can barely carry that heat away, so the newest AI servers are often cooled with liquid pumped right over the chips. Heat, not just switch count, limits what a chip can do, which is the gotcha in the next chapter.

The gotcha

Clocks stopped getting faster around 2005

The natural guess is that chips keep getting faster by ticking faster. For decades that was true: clock speeds climbed from about a million ticks a second in the 1970s to over 3 billion in the early 2000s. Then they hit a wall. Faster ticking meant more heat than a chip could get rid of, and clock speeds have stayed around 3 to 5 GHz ever since public.

So the switches kept multiplying, but each tick did not get much shorter. Chips got faster in a different way: doing many things at the same time instead of one thing faster. That single change is the story of the next two parts. A CPU (part 4) uses a few very clever workers. A GPU (part 5) uses thousands of simple ones, and that turned out to be exactly what AI needs.

Zooming out

A chip is a city of blocks

Nobody designs a chip one switch at a time. Switches make gates, gates make adders and multipliers, and those are grouped into larger blocks, each with one job. Every chip, in your phone or in a data centre, is some arrangement of the same four kinds of block:

The big difference between chips is the balance. A laptop’s main chip spends a lot of its space on control, so it can do complicated, unpredictable work quickly. An AI chip spends most of its space on arithmetic and fast memory, because AI work is the same simple sum, over and over. Parts 4 and 5 compare the two directly.

Back to AI

How many sums does one answer take?

Now connect the two tracks. A model uses each of its learned numbers about twice per token: one multiply and one add public. So writing one token takes about two sums per learned number. Pick a model size and an answer length:

A person would need far longer than recorded history; a modern AI chip needs a second or less. That gap, about a thousand trillion to one, is what all those switches buy. The chip track continues with how they are arranged to get there: part 4 (the CPU), part 5 (the GPU), part 8 (memory) and part 10 (AI chips).

The whole picture

From one switch to a chip

Every layer is simple. A transistor is a tap. A gate is a few taps. An adder is a few gates. What makes a chip astonishing is not any single idea, but doing it billions of times over, on a piece of silicon you could lose in your pocket, billions of times a second.