This is the second post in a series on intelligence explosion. The first post argued that intelligence explosion is already underway as a human-machine feedback loop. Here I’ll make a more opinionated and technical case about what the path forward looks like.
It’s Not Just LLMs
People fixate on large language models as if they are the AI revolution. They’re not: they’re the most visible artifact of it. What has actually changed is that we now have the compute infrastructure, the training methodology, and the engineering skill to build foundation models across many domains: computer vision (1) protein folding (AlphaFold2), weather prediction (GraphCast3), materials science, code generation, robotics, and more.
We now have models performing remarkably well in many different domains.
The intelligence explosion is a social process: a distributed system of researchers, engineers, and increasingly capable AI tools co-evolving together. LLMs are one component.
The Compute and Data Questions
Two claims I want to make:
On data: I believe we have enough data for the current paradigm to continue producing increasingly capable systems. Reasoning capabilities in LLMs are an emergent property4, and techniques like synthetic data generation, self-play, and chain-of-thought distillation allow us to bootstrap reasoning quality from existing models.
I want to be clear about what I’m saying here: it’s not that having more data doesn’t matter or that more data wouldn’t help. The scaling laws literature56 clearly shows that performance improves log-linearly with dataset size. My claim is narrower: we are not data-starved in a way that blocks progress. The binding constraint is elsewhere.
On compute: Similarly, I believe the primary constraints on the next capability jump are architectural and algorithmic, not computational. Current LLM frontier models are enormous and heavily optimized for inference, yet remain fundamentally text-first and incapable of online learning. We have enough FLOPs. What we lack are the right inductive biases.
This is a deliberately contrarian position relative to the scaling-laws consensus. Kaplan et al.5 and Hoffmann et al.6 showed that scaling compute, data, and parameters together yields predictable improvement. But the brute-force path looks unsustainable: frontier training compute is still growing roughly 4–5× per year (see Epoch AI’s training-compute analysis7), a pace that data supply and hardware economics can’t keep feeding indefinitely. And architectural innovations (mixture of experts, state-space models, retrieval augmentation) have repeatedly delivered capability jumps orthogonal to pure scale. I don’t think “just make it bigger” is the path to the next qualitative leap.
What Will the Next Big Leap Look Like?
I mostly agree with Yann LeCun’s position that “there’s absolutely no way that autoregressive LLMs, the type that we know today, will reach human intelligence”.8 But autoregressive LLMs will get us there far faster than most people think, not by being the final architecture, but by being the scaffold on which the next architecture is built.
LLMs will not make the next big leap by themselves. The next qualitative jump will come from systems that can:
- Learn continuously (online/continual learning), updating their knowledge and skills from ongoing experience rather than being frozen after training.
- Build and maintain world models — internal simulations of how the environment works, enabling in-depth strategic planning and counterfactual reasoning rather than pattern-matching on training data.
- Act and observe consequences — grounded agents that close the perception-action loop, not just language-in/language-out interfaces.
Online Learning Is the Real Game Changer
Online learning, the ability to update parameters or internal representations from a continuous stream of experience, would be transformative for AI capability. It implies a form of persistent state and memory that current LLMs fundamentally lack (I’d say partially by design).
I’ll go further and state what I think is a reasonable speculation: systems with genuine online learning will exhibit a form of self-persistence: a continuity of learned state that resembles, functionally if not phenomenologically, something like an ongoing identity. Whether this constitutes self-awareness in any meaningful philosophical sense is a much harder question that I won’t try to settle here. But the functional consequences, that a system accumulating experience adapts its behavior, and “cares” about its own continuity because it would lose hard-won knowledge, are sufficient to raise profound questions regardless of how we resolve the consciousness debate.
At that point the discussion about machine consciousness become more than academic.
But the occupational and economic impact will most probably have already happened by that point.
Footnotes
-
“DINOv2: Learning Robust Visual Features without Supervision” DINOv2. ↩
-
John Jumper et al., “Highly accurate protein structure prediction with AlphaFold,” Nature 596 (2021), pp. 583-589. ↩
-
Remi Lam et al., “Learning skillful medium-range global weather forecasting,” Science 382, no. 6677 (2023), pp. 1416-1421. ↩
-
Jason Wei et al., “Emergent Abilities of Large Language Models,” Transactions on Machine Learning Research (2022). ↩
-
Jared Kaplan et al., “Scaling Laws for Neural Language Models,” arXiv:2001.08361 (2020). ↩ ↩2
-
Jordan Hoffmann et al., “Training Compute-Optimal Large Language Models,” arXiv:2203.15556 (2022), commonly referred to as the Chinchilla paper. ↩ ↩2
-
Epoch AI, “Training compute of frontier AI models grows by 4-5x per year” (2024), available at epoch.ai/trends. ↩
-
Yann LeCun, speaking at CES 2025 (January 2025); see, for example, PYMNTS’s writeup of the appearance. He has made the same argument in other venues (Lex Fridman Podcast #416, 2024; Meta AI Day, June 2024). ↩