Why Brain-like Computers Are Hard

Computers that run the Von Neumann architecture store their programs and data in the same memory bank.

Since both have to travel the same road to get and from the CPU, we find that the system is ultimately limited not by the CPU or GPU's computational limits but by said road.

This is the famous Von Neumann bottleneck.

In a previous video I talked about in-memory computing as a way to bring a computer’s memory closer to the compute.

But making a computer that thinks like the brain - a neuromorphic system as it is called - entails far more than just memory.

For this video, a look at these brain-inspired systems and the fundamental differences between computer and brain.

Why the Brain

First things, first. The brain.

Computer scientists have long had the desire to replicate the brain. But why? What is so special about the brain? Aren't computers just better?

Computer scientists have long admired the brain's ability to function at very low energy. The brain operates at about 12 to 20 watts of power - 20% of the body's metabolic rate.

The desktop computer on the other hand does about 175 watts. Leading edge AI accelerators like the Nvidia H100 use anything from 300 to 700 watts.

The brain also does not operate at a very fast pace - with a clock frequency of about 10 hertz. Though this varies depending on what the person is doing at the time and their mental state.

So the brain does not use a lot of power and doesn't operate very quickly. And yet it is capable of so much.

Imagine a bee. A bee's brain has less than a million neurons and runs on less than a watt of power. And yet it can fly. It can navigate to flowers and back home. It can socialize and maybe even calculate things.

A bee's brain is just as capable as a 18 billion transistor system on chip, and it can do all that with just a million neurons and virtually no power.

Perhaps we should start using brains and not silicon chips for learning? Oh wait ...

The biological brain's powerful capabilities - achieved at low power - is the single most significant motivation for building neuromorphic hardware today.

Insane Parallelism

A brain accomplishes these with parallelism.

Your brain's 86 billion neurons operate without a central clock. By which I ...