Computing Lineage
A twelve-floor survey of decisions in the history of computation that produced the current architecture of machine learning. Each floor examines a historical fork at which a continuous option was available and a discrete option was chosen. Intended as context for the experimental work in later sections. Read in order.
-
01
The Tower — A Preface
A climb through the history of computation, floor by floor, reading each one as a fork in the road. The road not taken is the one this library is for.
-
02
Floor 1 — Boole & Frege: When Truth Became Binary
The 19th century mathematicians who decided that reasoning was a matter of 1s and 0s — and the continuous logic they walked right past.
-
03
Floor 2 — Turing: The Tape That Forgot It Was Continuous
Turing's machine is the most influential thought experiment of the 20th century. Its discrete tape was a modelling convenience that became a metaphysical commitment.
-
04
Floor 3 — McCulloch & Pitts: The Neuron Made Into a Gate
In 1943, two men wrote down a mathematics of the neuron. They chose the threshold. The biology was already shouting that the neuron was continuous — and they heard it, and chose the threshold anyway.
-
05
Floor 4 — Von Neumann: The Architecture That Chose the Bus
In 1945, a brilliant Hungarian drafted a report that would shape every computer built since. He knew he was compromising. He said so. Nobody listened to that part.
-
06
Floor 5 — Shannon: Information as a Bit, and the Geometry We Forgot to Name
Claude Shannon gave us a mathematics of information that is, strictly speaking, continuous. We simplified it into bits and built an industry. The continuous reading is still sitting there, in the same paper, unused.
-
07
Floor 6 — The Perceptron and the Winter That Didn't Have To Happen
In 1958, Frank Rosenblatt built a machine that could learn. In 1969, two men wrote a book that froze the field for a decade. The mathematics was never the problem.
-
08
Floor 7 — Expert Systems: The Civilisation Built Out of If-Else
For about twenty years, the serious research programme in AI was to build intelligence out of rules — written by hand, in bulk, forever. It ran out of rules before it ran out of world.
-
09
Floor 8 — Backprop Returns, and Brings the One-Hot With It
In 1986, connectionism came back from the cold. It worked. It was also, quietly, wearing a tie. The loss function it carried was a child of Floor 5.
-
10
Floor 9 — Word2Vec: The Field Touches the Manifold and Flinches
In 2013, language became geometric for the first time at industrial scale. The vectors were real. The manifold under them was never quite named.
-
11
Floor 10 — Transformers: The Brilliant Architecture With the Wrong Output
The most impressive machine learning architecture ever built has a continuous heart and a one-hot mouth. Nothing wrong with the heart.
-
12
Floor 11 — RLHF: A Reward Is a Bit With Better Marketing
Reinforcement learning from human feedback is the most expensive alignment technique ever deployed. Underneath the thousand-page eval suites, it is still training a machine to chase a number.
-
13
Floor 12 — You Are Here, and the Other Path Is Still Open
The top of the tower. Where the climb ends and the question begins. The door is held open; it is your turn to decide what walks through it.