Tag: information
-
Portfolio ·
Floor 5 — Shannon: Information as a Bit, and the Geometry We Forgot to Name
Claude Shannon gave us a mathematics of information that is, strictly speaking, continuous. We simplified it into bits and built an industry. The continuous reading is still sitting there, in the same paper, unused.
-
Portfolio ·
Floor 8 — Backprop Returns, and Brings the One-Hot With It
In 1986, connectionism came back from the cold. It worked. It was also, quietly, wearing a tie. The loss function it carried was a child of Floor 5.