First computer company I worked for was still using it in the early 80s. Slow, but it retained state after a power failure.

I knew this was gonna be polymatt! What a great vid.

TL;DW: He makes a memory core from "scratch"

Though not a fan of his reasoning to have it in silicone oil. The computers back then also didn't do that, and they had rougher measuring tooling.
He just wanted a oil-submerged thingy anyway.

Ok that’s uncalled for sir.

It certainly is.

From the wiki:

"It uses ferrite magnetic-core memory in 4096 or 8192 words of 40 bits, comprising 39 bits of data with parity."

So a whooping 39kB of memory on the largest option!

"Tape is read at 500 characters per second and punched at 100 cps."

Compare that with a micro-SD...

"The bit time is 6 microseconds, jumps execute in 288 microseconds and simple arithmetic instructions in 576 microseconds."

And it run and an incredible speed of 1 to 3kHz!

(And this is overselling the computer, it was slower than what the numbers appear.)

Yes, I heard our size and population doubled every two years. It's called Morsel's law iirc.

Seems to have plateaud out now though.

Yeah, not even close.

That's an Elliott 405 computer. It could perform up to 500,000 instructions per second, aka FLoating-point Operations Per Second (FLOPS).

The other is a Raspberry Pi Zero which can perform 250,000,000 FLOPS, 500 times the Elliott 405.

And, the Elliott 405 cost between 140k-350k in 1957, depending on the features and configuration chosen. With inflation to 2015 dollars, that's $1.2-2.9 million ($2.40 per 1 FLOPS)

The Raspberry Pi Zero was their new low-power, low-cost board in 2015. It only cost $5 in 2015 (50 million FLOPS per $1)

And for an extra 30 bucks ($35 total) in 2015, you could have picked up a Raspberry Pi 2 Model B, which is capable of 24,000,000,000 FLOPS. That is 96 times faster than the Pi Zero, and 48,000 times faster than the Elliott 405. (~686 million FLOPS per $1)

Me: I need some new hardware to run my transformers on

Mom: We have transformers at home

The transformers at home:

Joking aside, I think you're right, using discrete floating point math to simulate a transformer architecture will never be able to approach the efficiency of a "native" analog system like actual brains. I think eventually we'll see someone come up with a hardware transformer that doesn't require full synchronized clock signals.

Yes, that's neuromorphic computing. Once we properly figure our how to make semiconductor neurons and how to connect them to each other in a big scale, AI will hopefully be less of an energy waste. The current approach with mapping these problems into GPUs is highly inefficient.