Contrast-and-Compare: Friston et al. (2024) and Hafner et al. (2022)

This blog is in progress. (AJM, Friday, Aug. 30, 2024; 09:00 AM HI Time)

This blogpost accompanies the YouTube on “AGI: Action Perception Divergence (APD) vs. Renormalizing Generative Models (RGMs)”

This Figure will be replaced with the YouTube link once the YouTube is published, which should be about Saturday, Aug. 31, 2024.

This blogpost – and the accompanying YouTube – is in response to a question asked in the Comments section of the prior YouTube.

Here’s that question:

Figure 1. The question that was asked in the Comments section of the prior YouTube on “Big AGI Breakthrough.”

Here’s the link to that PRIOR YouTube:

Maren, Alianna J. 2024. “Big AGI Breakthrough: From Active Inference to Renormalising Generative Models.” Themesis, Inc. YouTube Channel (Aug. 21, 2024). (Accessed Aug. 30, 2024; available online at Themesis Inc. YouTube Channel.)

Quick Review: Action Perception Divergence

“Action Perception Divergence” (APD), introduced by Hafner et al. (2020, 2022) is a more general and abstract evolution of active inference.

Figure 2. “Action Perception Divergence” (APD), introduced by Hafner et al. (2020, rev. 2022), is an evolution of active inference. (See full citation in **Resources and References**.)

We’ve previously discussed the “Action Perception Divergence” (APD) method introduced by Hafner et al. (2020, 2022) (where one of the co-authors is Friston).

Specifically, we’ve put together TWO YouTubes on APD.

The first of these addresses the APD notation.

Maren, Alianna J. 2024. “AGI: Action Perception Divergence (APD) – A Notation Review.” Themesis, Inc. YouTube Channel (July 15, 2024). (Accessed Aug. 30, 2024; available online at Themesis Inc. YouTube Channel.)

The second video walks us through the all-important derivation of Eqn. 3.

Maren, Alianna J. 2024. “AGI: APD (Action Perception Divergence) – Eqn. 3 Derivation.” Themesis, Inc. YouTube Channel (July 17, 2024). (Accessed Aug. 30, 2024; available online at Themesis Inc. YouTube Channel.)

We also summarized some key APD elements in the YouTube that accompanies this blogpost. (See the FIRST YouTube link, at the top of this blogpost. Link will be inserted once the YouTube is published.)

Active Inference: The Early Years

Friston developed active inference, as a step forward from variational inference, beginning about 2010. (For all papers cited, see the Resources and References section at the end of this blogpost.)

Two of the most important early active inference papers are Friston (2013) and Friston et al. (2015a).

A key element of Friston’s early work is that he specifically incorporates the “external reality” (“Psi“) into the mathematical formulation. Friston used notation offered by Matthew Beal in his 2003 dissertation as the basis for his formulation.

One of the most important elements of this exposition is the very specific incorporation of a Markov blanket separating the external reality (“Psi“) from the representation of that reality.

Figure 3. The system conceptualized by Friston in his early works specifically identify the separation of the external reality (“*Psi*“) from the representation of that reality (“r“) via a Markov blanket.

Friston’s early work, from 2010 through 2015, used a consistent notation that captured the notion of a Markov blanket separating the external reality (“Psi“) from the representation of that reality (“r“).

Figure 4. In his early works (2010 – 2015), Friston specifically identified the external reality (“*Psi*“), the representation of that reality (here as “*lambda*“), and distinguished the model (q) from the representation distribution (p). One of the more interesting things in this approach was that, in the model q, the external reality (“*Psi*“) was conditioned upon the representation of that reality (“*lambda*“).

Note that (Figure 4 above) Friston expresses the free energy of the system using a reverse Kullback-Leibler divergence.

For those unfamiliar with how the reverse Kullback-Leibler divergence is used throughout generative AI, Maren’s (2024) KL tutorial provides a useful guide, summarizing notation as used by different researchers, and identifying the difference between the “typical” vs. the “reverse” KL divergence.

Figure 5. Maren (2024) has provided a tutorial on the Kullback-Leibler divergence, emphasizing how the **reverse** KL divergence is typically used throughout generative AI. (See paper citation in **Resources and References**.)

Friston’s conceptualization and notation, based on Matthew Beal’s 2003 dissertation, included explicit incorporation of the external reality “Psi,” and characterized his early active inference work (2010 – 2015).

Figure 6. It is more than a little challenging to trace the notation from Beal (2003) to Friston (2010, 2013, Friston et al. 2015a). Maren’s tutorial on the “Derivation of Variational Bayes” (2019, rev. 2024) elucidates that connection.

Because it is a little challenging to understand Friston’s early conceptualization, and to trace the correspondence with not only Beal (2003) but also with other works on variational inference (e.g., Blei et al. 2018), Maren developed an in-depth tutorial that elucidated the correspondence between the different notations (2019, rev. 2024).

Resources and References

Friston, K. 2010. “The Free-Energy Principle: A Unified Brain Theory?” Nat. Rev. Neurosci. 11:127–138. doi:10.1038/nrn2787.
Friston, K. 2013. “Life as We Know It.” Journal of The Royal Society Interface 10(86).
Friston, K., M. Levin, B. Sengupta, and G. Pezzulo. 2015. “Knowing One’s Place: A Free-Energy Approach to Pattern Regulation.” J. R. Soc. Interface, 12:20141383. doi:10.1098/rsif.2014.1383. (Available online at: http://dx.doi.org/10.1098/rsif.2014.1383.)
Friston, Karl, Conor Heins, Tim Verbelen, Lancelot Da Costa, Tommaso Salvatori, Dimitrije Markovic, Alexander Tschantz, Magnus Koudahl, Christopher Buckley and Thomas Parr. 2024. “From Pixels to Planning: Scale-Free Active Inference.” arXiv:2407.20292v1 [cs.LG] 27 Jul 2024. doi:10.48550/arXiv.2407.20292. (Accessed Aug. 10, 2024; available online at https://arxiv.org/pdf/2407.20292.)
Friston, K., F. Rigoli, D. Ognibene, C. Mathys, T. Fitzgerald, G. Pezzulo. 2015. “Active Inference and Epistemic Value.” Cogn Neurosci, 1-28. (Accessed Aug. 30, 2024; available online at ResearchGate, and also at ChrisMathys.com.)
Hafner, Danijar, Pedro A. Ortega, Jimmy Ba, Thomas Parr, Karl Friston and Nicolas Heess. 2020, rev. 2022. “Action and Perception as Divergence Minimization.” arXiv:2009.01791v3 [cs.AI] (13 Feb 2022). doi:10.48550/arXiv.2009.01791. (Accessed Aug. 10, 2024; available online at https://arxiv.org/pdf/2009.01791.
Maren, Alianna J. 2024. “Minding Your P’s and Q’s: Notational Variations Expressing the Kullback-Leibler Divergence.” Themesis Inc. Technical Note THM TN2024-001v1 (ajm). Technical Note published to Themesis website.
Maren, Alianna J. 2019, rev 2024. “Derivation of the Variational Bayes Equations.” Themesis Inc. Technical Report THM TR2019-001v6 (ajm). arXiv:1906.08804v6 [cs.NE] 18 Aug 2024. abstract; pdf.
Pezzolo, G., Rigoli, and K. Friston. 2015. “Active Inference, Homeostatic Regulation and Adaptive Behavioural Control.” Progr. Neurobiology 134 (Nov. 2015):17-35. doi:10.1016/j.pneurobio.2015.09.001. (Accessed Aug. 30, 2024; available online from Science Direct.)