Addressing Objections to the Intelligence Explosion
1 Introduction
My guess is that there will soon be an intelligence explosion.
I think the world will witness extremely rapid economic and technological advancement driven by AI progress. I’d put about 60% odds on the kind of growth depicted variously in AI 2027 and Preparing For The Intelligence Explosion (PREPIE), with GDP growth rates well above 10%, and maybe above 100%. If I’m right, this has very serious implications for how the world should be behaving; a change bigger than the industrial revolution is coming.
In short, the reason I predict an intelligence explosion is that it follows if trends continue at anything like current rates. Specifically, the trends driving AI improvements:
Training compute: this is the amount of raw computing going into AI training. It’s been growing about 5x per year.
Algorithmic efficiency in training: this is how efficiently the algorithms use computational power. Efficiency has been growing 3x per year. For nearly a decade. Combined, these lead to an effective 15x increase in training compute.
Post-training enhancements: these are the improvements in AI capabilities added after training, and it’s been growing about 3x per year, according to Anthropic’s estimates. These three combine to make it as if compute has been going up ~45x per year.
Inference efficiency: this is how cheaply you can run a model at a given level of efficiency. This cost has been dropping 10x per year, on average, since 2022.
Inference compute scaling: This is the amount of physical hardware going into model inference, and it’s been going up 2.5x per year. If these inference trends continue, they could support a growth in the AI population of 25x per year. And both are on track to continue for quite a while.
The authors of PREPIE write “Putting this all together, we can conclude that even if current rates of AI progress slow by a factor of 100x compared to current trends, total cognitive research labour (the combined efforts from humans and AI) will still grow far more rapidly than before.” They note that “total AI cognitive labour is growing more than 500x faster than total human cognitive labour, and this seems likely to remain true up to and beyond the point where the cognitive capabilities of AI surpasses all humans.”
