The AI Stack: A Likely Path toward AGI

Sycophancy, reward hacking, agents that game their own objectives: most of AI’s best-known failures trace back not to the language models themselves, but to how they’re trained to act, through reinforcement learning

A deeper understanding of AI systems requires that we think in terms of the AI processing stack:

  • Agents at the top acting in the world,
  • Control systems deciding what those agents do, and
  • World models underneath supplying knowledge about how the world works.

The major problems that we’ve seen in AI systems, over the past four years, each relates to a specific aspect of the AI stack, as is shown in the following Figure 1.

Figure 1. The current AI processing stack – the major problems over the past three years are traceable to using reinforcement learning (RL), both directly and in running agents.

Each layer is evolving, and not at the same pace. In this post, I lay out how I expect the stack to develop over the next few years:

  • Different kinds of world models: The current emphasis on LLMs as a world model basis will expand to include spatial world models as well as structured data,
  • Active inference: a complementary method will give an alternative to the tightly-focused, goal-oriented nature of reinforcement learning, and
  • Ontologies: connecting signal (token-level) information with symbolic representations will enable persistent storage of known (and active) information.

This is my forecast, not a certainty, but I believe it’s a likely path, and one that tells us a lot about where AGI might come from.

Leave a comment

Your email address will not be published. Required fields are marked *

Share via
Copy link
Powered by Social Snap