Sycophancy, reward hacking, agents that game their own objectives: most of AI’s best-known failures trace back not to the language models themselves, but to how they’re trained to act, through reinforcement learning
A deeper understanding of AI systems requires that we think in terms of the AI processing stack:
- Agents at the top acting in the world,
- Control systems deciding what those agents do, and
- World models underneath supplying knowledge about how the world works.
The major problems that we’ve seen in AI systems, over the past four years, each relates to a specific aspect of the AI stack, as is shown in the following Figure 1.

Each layer is evolving, and not at the same pace. In this post, I lay out how I expect the stack to develop over the next few years:
- Different kinds of world models: The current emphasis on LLMs as a world model basis will expand to include spatial world models as well as structured data,
- Active inference: a complementary method will give an alternative to the tightly-focused, goal-oriented nature of reinforcement learning, and
- Ontologies: connecting signal (token-level) information with symbolic representations will enable persistent storage of known (and active) information.
This is my forecast, not a certainty, but I believe it’s a likely path, and one that tells us a lot about where AGI might come from.