Why Agentic AI Still Doesn’t Know What It’s Looking At — and What an AMD Acquisition and an Ontology Have in Common
Yesterday morning I was noodling a complex set of thoughts with Claude (my AI associate), and mid-draft, Claude had to “time out’ for a compaction. Claude was able to keep the overall context of what we’d been discussing – but completely lost the more recent and delicate nuances. Not only that, we were just starting to put this complex set of perspectives into our first few paragraphs – and Claude totally lost this comprehensive and subtle “train-of-thought”!
Yikes!
Wowza!
And this was particularly irksome because Claude had been contributing to developing this train of thought – organizing, pulling out connections – being a true “partner” in the work.
But as I was waiting for Claude to complete its compaction and re-engage, I realized this was a superb example of one of the main problems with AI-as-we-know-it-today. That problem is that our AI associates, no matter how much we extend their context windows – or provide other fixes (some of which are very, VERY good!) – are subject to the limitations of the base layer architecture.
The reason that this is so important for us is much deeper, broader, and substantially more significant than the recent attention that we’ve been giving to “rogue agentic swarms.”
AI mis-behaviors, as we like to call them, are a symptom – and not (in themselves) the underlying problem.
The underlying problem, which is very real, and which is the focus of this blogpost is that AI capabilities and behaviors are constrained by their algorithmic cores.
Simply said – if you know the core algorithms, you can predict EXACTLY what sorts of problems will emerge. And the so-called “rogue” agentic behavior is just one kind of problem.
There are others.
Right now, we’re on a threshold.
AI systems are currently built on a fairly straightforward algorithmic stack:
- Base layer: Currently LLM (or transformer-based) models – that is, generative AI methods that generate tokens (text, image elements, or code).
- Control layer: This tells the transformer-core what to do – it guides behavior – and right now, it is almost exclusively RLHF, or reinforcement learning with human feedback.
- Agentic action layer: Agents (singly, or in multiples, or even swarms) carry out the directives of the control layer.
What we’re seeing now are the limitations of this stack, stretched to its extremes. This means that the innate problems are being pushed forward in a way that catches our attention.
From “rogue AI swarms” to Claude needing to compact in the middle of a carefully-developed work project – the problems are inherent to the algorithms and the architectures, and are not readily fixed with either guidelines or guardrails. You can already see it everywhere once you know to look: in a shopping agent that refunds an order twice, in a robot that can’t remember which box it already moved, in a world model that renders a gorgeous scene it has no way of actually understanding.
The Quiet $8.2 Billion Signal
Follow the money.
When the world is going crazy and ricocheting around us, like bullets fired inside a hallway – we need to separate ourselves from the angst and excitement and where everyone’s attention is focused – and follow the real signal.
Money. It’s always the money.
Here’s where it led: on September 28, AMD agreed to acquire Fei-Fei Li’s World Labs for $8.2 billion — in stock, with Li herself joining AMD as Chief Scientist. It got covered, but covered the way acquisitions get covered: a few business-press headlines, a stock-price reaction, then silence. Compare that to the volume of attention any “rogue agentic swarm” story pulls, and you’d think the swarm story was the more consequential one.
I’d argue it’s the opposite. World Labs builds spatial world models — and when a chip company with AMD’s resources spends that kind of money, and hands its Chief Scientist title to the person building them, that’s the real signal about where the actual infrastructure bottleneck sits.
I’ve been tracking this thread since at least November of last year, in a video I called “Five Spatial World Models” — and, fittingly, the conversation Claude and I were having to update that argument is exactly what got lost in yesterday’s compaction. We’re reconstructing it here, in public, as we go.
Here’s that YouTube, and I recommend watching it – because it holds important background & context for this story.
Here’s a key figure – probably the most important summary – from that entire 40-minute YouTube. (And you’re right: it deals with “the money.” – BUT — it’s outdated by a year; both LeCun and Li have received much more money, and the Verses.ai company has surrendered its AI (active inference) initiative completely … and there’s more – so keep reading! )
Figure 1 shows funding for four of the five different “world models.” See the associated blogpost (from a year ago at this time) for in-depth reporting with links to news articles and research sources. (My own CORTECON model is reserved out from this funding layout, as it is not yet commercial.) Let’s start with the biggest and work our way through – because the money tells the story.
Google’s DeepMind and Genie
Google’s investment in their Genie leads, with an overall yearly budget for DeepMind of at least $2Bn, and an unknown portion of that going to their spatial world models such as Genie. Google has the significant advantage of seniority – since Google purchased DeepMind in 2014, they’ve been able to cultivate a strong infrastructure of both knowledge and product development expertise – far outstripping what the newer labs are able to put together. We’re just acknowledging the value of having long-standing teams of top minds who have really learned to work with each other over years, sometimes for longer than a decade.
This cultivation of top minds with long-term collaboration is perhaps Google’s greatest advantage on this playing field. Their greatest weakness is also very likely their greatest strength: Genie is a generative-AI “pure play,” trained with reinforcement learning. But we can look at it as – fundamentally – a strictly generative model. That means that Genie’s world-building is subject to exactly the kinds of “context-window-constraints” that led to Claude having to compact yesterday morning, with a loss of previous experiences.
[Worth noting: my original video is about a year old now, and Google’s generative lineup has moved fast since then — Nano Banana 2 (image generation) and Veo 3.1 (video generation) have both shipped in the meantime. Same underlying bet, more horsepower behind it. The argument hasn’t changed; the stakes have gotten bigger.]
So to sum up the Google/DeepMind approach – huge, in-depth expertise, cultivated for over a decade (the longest-running deep AI lab in the world) – but significantly constrained by DeepMind’s focus on generative AI coupled with reinforcement learning. A strength and a weakness, combined.
A Confession, Before We Go Further
I want to own something here, because it’s exactly the mistake this whole post exists to help you avoid — and I made it myself, in public, on camera.
In January 2025, Google released Veo 2, and I was genuinely excited. The physical plausibility of what it generated had jumped so far past its predecessors that I convinced myself something architectural had changed — that Google had reintroduced real, explicit object-level representation. Separate levels of understanding, genuinely communicating with each other: signal, object, symbolic — the kind of multi-tier architecture that researchers like Grossberg and Marr had worked out decades ago, and that I’d watched get flattened into implicit, undifferentiated neural representations once deep learning took over. I said, on that video, that this was “a breakthrough moment” because it reintroduced the value of having multiple levels of representation and genuine feedback between them.
I was wrong. Veo 2 hadn’t reintroduced an embedded ontology or an explicit object model. It was still a purely generative, signal-level system — just a dramatically better one. The improved physical coherence was an emergent effect of scale and training, not evidence of real understanding underneath. I’d mistaken “looks more physically correct” for “actually has an object-level representation of what physical correctness means” — and those are not the same thing, even when the output makes it very tempting to believe they are.
I never took that video down. I think it’s more useful left up as a record of exactly how easy this mistake is to make — including for someone who’s spent decades studying the difference between signal-level and symbolic-level representation. If I can watch a sufficiently good generative output and convince myself the ontology is already in there, anyone can. That’s not a reason to stop looking for real progress. It’s a reason to look harder at what’s actually underneath the plausible surface, every single time — which is exactly what we’re about to do with the rest of this landscape.
LeCun: Right About the Disease, Still Arguing About the Cure
Before we get to Fei-Fei Li and the $8.2 billion that actually moved, it’s worth understanding why nobody has written a check anywhere near that size for Yann LeCun — because the reason isn’t obscurity, and it isn’t that he’s been wrong. If anything, LeCun has been right for longer than almost anyone else in this field, about the thing most people are only now starting to suspect.
LeCun’s skepticism of generative AI isn’t a recent contrarian pose — it goes back to the lineage itself, and the lineage is more precise than “LeCun reacted against Hinton.” Hinton’s generative thread runs in an unbroken line of its own: Boltzmann Machines (Hinton & Sejnowski, 1983–85), through Restricted Boltzmann Machines — Smolensky’s 1986 “harmonium,” a genuinely useful idea that sat computationally stuck for nearly two decades — until Hinton’s own Contrastive Divergence (2002) finally made RBMs trainable, leading to Deep Belief Networks (2006). That’s Hinton’s bet, start to finish.
The backpropagation lineage runs separately. Paul Werbos described it in his 1974 thesis; Rumelhart, Hinton, and Williams popularized it in 1986 — meaning Hinton himself sits upstream of both threads (generative – in (restricted) Boltzmann machines, and discriminative – in Multilayer Perceptrons trained with backprogagation or any back-prop adjacent methods). This makes Hinton’s later advocacy for the generative path a choice, not an inevitability. LeCun took backprop and did something neither thread had yet done: applied it to convolutional architectures, first for handwritten zip-code recognition at Bell Labs in 1989, then in LeNet-5 (1998). Discriminative, non-generative, by design, from the start — a decade before Contrastive Divergence, over two decades before AlexNet.
AlexNet, in 2012, didn’t invent a new non-generative approach. Sutskever and Krizhevsky, working under Hinton’s supervision, took LeCun’s already-existing CNN architecture and made it work at scale — and the breakthrough only became possible because two more independent contributions arrived at the same moment: Fei-Fei Li’s ImageNet (2009), giving the architecture labeled data at a scale nothing before it had, and NVIDIA’s CUDA, giving it the compute to train on that data — plus simplifications like ReLU replacing sigmoid, and dropout, that made training stable. Three independent threads — LeCun’s architecture, Li’s data, NVIDIA’s hardware — converged under Hinton’s roof to produce a result that was, pointedly, not generative.
LeCun and Li were both standing at that exact inflection point, each supplying a different essential ingredient — and they walked away from it having drawn opposite conclusions. LeCun doubled down on non-generative, discriminative, embedding-based methods, a conviction running in an unbroken line to JEPA (Joint Embedding Predictive Architectures) today. And more than a decade later, Li founded World Labs on an explicitly generative approach to spatial world-building. That’s not incidental biography. It’s the argument each of them is still having, in public, with the field and with each other.
LeCun has kept making his case loudly. In May 2024: “LLMs are useful, but they are an off ramp on the road to human-level AI. If you are a PhD student, don’t work on LLMs. Try to discover methods that would lift the limitations of LLMs.” He’s repeated versions of this since — at VivaTech 2024, and now from his own company, AMI (Advanced Machine Intelligence) Labs, built around VL-JEPA, the latest evolution of a lineage running from the original 2022 JEPA through V-JEPA and I-JEPA.
Here’s the actual technical fork, and I made this exact comparison on camera back in early 2024. This video compares JEPA vs. Action Perception Divergence (APD), which is an outgrowth of Friston’s active inference.
Later that year, Verses.ai (for which Karl Friston was then their Chief Scientist, before they closed their active inference work in Spring, 2026), introduced RGMs (Renormalizing Generative Models) – a “hierarchical” form of active inference.
Around the same time, LeCun and colleagues introduced H-JEPA, or a hierarchical form of JEPA. Both of these were efforts at scaling their respective baseline algorithms – JEPA and active inference.
Now at that time – just two years ago – the only scaling approach available for active inference was RGMs, which were introduced at Verses.ai. RGMs work using factor graphs, which allow for message passing – both spatially and temporally – and they are covered in Namjoshi’s March, 2026 book, Fundamentals of Active Inference.
But at a more basic level, Friston’s active inference is often operates on state-space models: “chunky,” discrete, like moving a piece from square to square on a chessboard.
LeCun’s JEPA requires the opposite: continuous, differentiable spaces, because that’s what gradient-based control demands.
Both active inference and JEPA are trying to solve the same problem — building a model of the representation of the world, not the world itself, and keeping that model dynamically updated — but from structurally incompatible starting assumptions about what kind of space a world model should even live in.
Worth a small aside here, because it’s too good not to mention: Hafner co-authored the 2022 paper establishing Action-Perception Divergence with Friston.
By the time Friston’s 2024 Verses.ai paper on scale-free active inference came out, it cited Hafner’s own more recent work exactly zero times — and Hafner’s own 2022 hierarchical-planning paper returned the favor with zero citations of Friston. Two people who co-authored the foundational paper together, each publishing in the same “pixels to planning” territory two years later, pretending the other doesn’t exist. Make of that what you will.
Both camps, though, share the same blind spot — and LeCun says so more bluntly than anyone: he doesn’t think we need symbolic computing at all. I disagree, and said so at the time: the reason nobody’s built the bridge isn’t that it’s unimportant, it’s that nobody’s figured out how — connecting the flickering, on-off token stream of a neural or state-space representation to the sustained, structured world of ontological meaning. That gap is the actual elephant in the room, for LeCun’s camp and Friston’s camp alike.
Li vs. LeCun – Generative vs. Discriminative (Again!)
So LeCun has been arguing against generative AI since before most of the field existed in its current form, he was right about backprop and discriminative learning a decade before anyone else, and he still believes generative methods are a dead end for real world models. That’s a track record most people in this space would kill for.
And the market has responded — just not the way a simple “who got bought out” comparison would suggest. AMD’s $8.2 billion for World Labs is an all-stock acquisition: no cash changes hands, AMD’s own shares do, and Li joins as Chief Scientist. That’s AMD folding an already-proven capability into its own strategic roadmap — a chip company buying its way into the generative spatial-world business.
AMI Labs took a different kind of money entirely: a seed round — reportedly over $1 billion, among the largest seed rounds ever raised — in cash, from investors betting on an unproven team and an unproven product. Measured against what seed rounds normally look like (tens of millions, not billions), that’s not a smaller vote of confidence than AMD’s acquisition. It may be a more extreme one, dollar for dollar, precisely because it’s speculative capital with no strategic motive attached beyond belief in the thesis.
So this isn’t a story about the market picking a winner. It’s two fundamentally different kinds of money — strategic stock-based acquisition, and record-breaking speculative cash — making enormous, historically unusual bets on two structurally incompatible architectures, within months of each other. Nobody at the top of this market has concluded the base-layer problem is solved. If anything, the size of both bets says the opposite: everyone with real money on the table agrees something has to give, and nobody yet agrees on what.
Fei-Fei Li and the World She’s Building
Before we go further into the landscape, it’s worth coming back to Li herself for a moment, because the funding comparison can make her feel like a data point in someone else’s argument rather than the person actually building something.
Li is the same Fei-Fei Li whose ImageNet — a labeled dataset she and her students built starting in 2009 — was one of the three independent ingredients, alongside LeCun’s CNN architecture and NVIDIA’s CUDA, that made AlexNet possible in 2012. She spent the years since as a leading voice in computer vision and AI policy at Stanford, co-directing the Stanford Institute for Human-Centered AI. In 2024, she founded World Labs, betting that the next frontier wasn’t bigger language models — it was giving AI systems a genuine sense of three-dimensional space.
Her flagship product, Marble, launched commercially in late 2025: type or show it a prompt, and it generates a complete, explorable 3D scene — geometry, lighting, depth — that you can walk through, edit, and export.
That’s the company AMD just spent $8.2 billion in stock to acquire, with Li joining as Chief Scientist. Whatever else is true about the funding structure, the underlying bet is straightforward: spatial understanding, generated and rendered as an actual three-dimensional asset, is worth building a chip company’s AI strategy around.
So now that we know who’s building what — Li generating visible, walkable worlds; LeCun predicting invisible, unrendered representations — it’s worth asking where they each sit relative to the rest of the field, and what each approach actually solves and leaves unsolved.
Four Ways to Build a World
Strip away the company names for a moment and there are really only two questions any of these systems has to answer: is the world being generated, or represented? And if generated, is it a one-shot static asset, or a continuous, unfolding process?
Google DeepMind’s Genie 3 answers both questions the hard way: pure generative, transformer-based, building the world frame by frame, live, at 24fps, from world description and actions. It’s the system most directly exposed to the base-layer problem in spatial form — memory currently holds for “up to a minute,” per the engineers themselves, before the world starts to lose coherence. This is genuinely difficult, frontier engineering, and Google’s superlative execution shows. It’s also, structurally, the same transience problem that makes Claude compact mid-sentence, just rendered in three dimensions instead of text.
World Labs’ Marble answers differently: generative, using Gaussian splats, but static. Marble doesn’t build a world live and keep it running — it generates a complete scene once, as an exportable asset (splat, mesh, or video), that you then load into a game engine. That sidesteps the frame-to-frame forgetting problem entirely, but at a real cost: Marble has, in its own documentation’s terms, no embedded knowledge of object physics. A Roman arch it generates looks structurally correct, but the system has no idea that removing a brick would make it collapse. Objects that need to move or interact have to be imported from outside tools. It isn’t solving object persistence so much as declining to attempt it — a different, narrower bet than Genie’s.
LeCun’s JEPA/AMI Labs answers both questions the opposite way from either: not generative at all. JEPA predicts how abstract latent representations evolve as an agent acts, without ever producing a pixel, an image, or a splat. Worth noting — when I first covered this in January, AMI Labs was reportedly targeting a $3.5B valuation on $587M raised. By March, the actual number came in far larger: a seed round north of $1 billion, among the largest ever recorded. The money kept moving in the direction of “don’t generate, predict” even as the asking price climbed.
VERSES.ai’s AXIOM is the one that doesn’t fit neatly on either side — and that’s exactly what made it interesting. It used active inference to represent scenes as compositions of objects, with dynamics modeled as piecewise-linear trajectories: a semi-symbolic approach, blending explicit object modeling with Friston’s probabilistic framework, released as an open Python package in 2025 and commercially as “Genius.” And on the more technical side of VERSES’s research output: Variational Bayes Gaussian Splatting — using the same splat representation Marble uses, but non-generatively, continually refining a belief distribution instead of regenerating from scratch, explicitly built to solve the catastrophic-forgetting problem that both Genie and Marble either struggle with or avoid by design.
In other words: of the four live approaches, VERSES was arguably the one actually confronting the object-persistence problem head-on, with real architectural teeth — representing objects as objects, not as transient pixel patterns, and updating belief rather than regenerating reality every frame.
It’s also the one that’s dead.
VERSES didn’t fail because the architecture was wrong. It failed the way companies fail — a four-month fundraising effort that didn’t close, extreme liquidity constraints, a president, COO, and CTO walking out the door over unpaid wages. In June 2026, a company that may have had the most structurally sound answer to the spatial-world base-layer problem of anyone on this list ran out of road before the market caught up to why its answer mattered.
But the idea didn’t die with the company. A new team, working from the same active inference foundation — but with a radically different approach to scaling it — picked up almost exactly where VERSES left off, and started posting results on a benchmark everyone else was failing.
We talked about this new approach – based on active inference – in our recent video on scaling in AI systems.
Why This All Comes Back to Ontologies
Here’s the deeper reason none of this is optional dressing on top of “which world-model company wins.”
If spatial-world systems and text-based systems are ever going to operate as part of the same larger AI — and that’s exactly what physically embodied AI requires, an agent that can both understand a language instruction and act correctly in a real three-dimensional space — then those two systems need a way to talk to each other internally. A token stream describing “the red block” and a Gaussian-splat cluster rendering that same block in space aren’t natively comparable representations. Something has to sit above both of them: a shared layer where “block,” “red,” “on the table” are formally defined entities and relations that both sides can be checked against. That’s an ontology, whether anyone building these systems wants to call it that or not.
This isn’t a new problem. It’s the oldest fault line in the field, reopening.
Go back to the mid-1980s, when Paul Werbos’s dissertation work on backpropagation ran straight into Marvin Minsky’s strictly symbolic view of what intelligence computation should look like — what Daniel Dennett later called “High Church Computationalism.” The split that followed shaped the next forty years: after 1986, once backpropagation produced real, working neural networks, the symbolic side lost ground, fast. Connectionism won the argument empirically, even if it never fully won it philosophically. Google built out one of the most serious ontological infrastructures the industry has ever seen — its Knowledge Graph — and then largely let it recede into the background once Gemini could just generate plausible-sounding answers on the fly. Why maintain a structured ontology when a good enough language model can fake having one?
But – the need for that structure never actually went away. It just got quieter, because generative methods got good enough to paper over the gap for most everyday purposes. But “most everyday purposes” doesn’t include a robot that needs to know a box it moved five minutes ago is still the same box, or an agentic system that needs to know “the customer’s account” in one turn means the same thing as “account #4471” in the next. Frank Coyle’s recent conference work on agentic systems is making exactly this case, directly and technically.
And it’s not just Coyle who’s talking — there’s a real uptick in ontology-oriented papers appearing right now, enough that it’s hard to tell anymore whether the field is actually turning back toward this question or whether I’m just primed to notice it because I’m looking for it. Probably some of both. But the volume is real.
And now there’s a federal signal pointing the same direction: NSF’s new X-Labs program, explicitly targeting “AI & Physical Systems” — sensors, robotics, embodied systems, human-robot interfaces, cyber-physical systems — is, whether its authors would phrase it this way or not, a funding call aimed squarely at the grounding problem. You cannot build a reliable physically embodied AI system without solving exactly the signal-to-symbolic bridge this post has been circling the whole way through.
That bridge — what it actually takes to build it, and what Themesis’s own work on it looks like — is where we’re going next.
{To be continued … ]
To your health and well-being — Alianna J. Maren, Ph.D.
Founder & Chief Scientist, Themesis, Inc.
Sunday, Oct. 4, 2026; 12:40 PM, Hawai’i time.