Steve Grossberg’s book, Conscious Mind, Resonant Brain, summarizes fifty years of neurophysiology-inspired research and inventions in neural networks architectures, most notably the Adaptive Resonance Theory (ART) network, where his colleague Gail Carpenter was first author on two very important papers.
We’ve done something a little different this time … a literal “unboxing” of the physical book!
The primary value of this blogpost is the reference list, please see below.
“Live free or die,” my friend!*
* “Live free or die” – attrib. to U.S. Revolutionary War General John Starck. https://en.wikipedia.org/wiki/Live_Free_or_Die
Alianna J. Maren, Ph.D.
Founder and Chief Scientist
Themesis, Inc.
Adaptive Resonance Theory (ART) Neural Networks
AJM’s Note: This class of bilayer feedforward/feedback (resonating) neural networks originally included one other network (now lost to the mists of time), and is now solely devoted to adaptive resonance theory (ART) networks, invented by Gail Carpenter and Steve Grossberg.
The Book
- Grossberg, Stephen. 2020. Conscious Mind, Resonant Brain: How Each Brain Makes a Mind, 1st Edition. Cambridge, UK: Oxford University Press.
A Good (Semi-Recent) Review Paper
AJM’s Note: If the book is more than you want, this is a good summary. Fairly readable.
- Grossberg, Stephen. 2013. “Adaptive Resonance Theory: How a Brain Learns to Consciously Attend, Learn, and Recognize a Changing World.” Neural Networks 37 (Jan. 2013): 1-47. doi:10.1016/j.neunet.2012.09.017. (Accessed July 25, 2023; available online at https://brainm.com/software/pubs/brain/Grossberg-adaptive-resonance.pdf.)
Important ART Historical Evolution Papers
The Grossberg (1976) article on “Adaptive Pattern Classification …” was a precursor to the 1987(a&b) and 1988 articles below.
By this point, I had already shown in these articles that catastrophic forgetting was possible in models of competitive learning and self-organizing maps. I introduced Adaptive Resonance Theory in Part II of the (1975/76) article(s) to dynamically stabilize learning and memory using attentional focusing by learned top-down expectations.
Stephen Grossberg, describing the evolution of his work. Personal communication.
- Grossberg, S. (1976). Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions. Biological Cybernetics, 23, 187-202. https://sites.bu.edu/steveg/files/2016/06/Gro1976BiolCyb_II.pdf
ART Introduction
AJM’s Note: The following two 1987 Carpenter and Grossberg papers introduce ART.
- Carpenter, Gail A., and Stephen Grossberg. 1987a. “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine.” Computer Vision, Graphics, and Image Processing, 37: 54–115. (Accessed May 22, 2023; available online as http://techlab.bu.edu/files/resources/articles_cns/CarpenterGrossberg1987.pdf.)
- Carpenter, G.A. and Grossberg, S. 1987b. “ART 2: Stable Self-Organization of Pattern Recognition Codes for Analog Input Patterns.” Applied Optics, 26: 4919-4930. doi:10.1364/AO.26.004919. (Accessed May 22, 2023; available online as https://opg.optica.org/ao/fulltext.cfm?uri=ao-26-23-4919&id=30891.)
AJM’s Note: The following article contains a large number of ART applications.
- Grossberg, S. 1988. “Nonlinear Neural Networks: Principles, Mechanisms, and Architectures.” Neural Networks 1: 17-61. (Accessed May 22, 2023; available online as https://sites.bu.edu/steveg/files/2016/06/Gro1988NN.pdf.)
YouTubes: Interviews with Grossberg, Explanations of His Work
AJM’s Note: Interviews with Steve Grossberg; the first one is long, the second is short.
- Grossberg, Stephen. 2022. “Explainable and Reliable AI: Comparing Deep Learning with Adaptive Resonance – Stephen Grossberg.” YouTube video based on Grossberg’s SAI Conference presentation. (Accessed May 25, 2023, available online at https://www.youtube.com/watch?v=RmbtXGp1avk.)
- Interactive Intelligence. 2022. “Dr. Grossberg’s Critical Take on Deep Learning | I2” (Interview with Steve Grossberg). Interactive Intelligence YouTube Channel. (Accessed May 25, 2023, available online at https://www.youtube.com/watch?v=5LgN440VThw.)
AJM’s Note: This appears to be a study group centered on “Grossbergian neuroscience.” The following link is to the first in a SERIES of YouTubes put out by this group.
- NeuroLogos. 2021. “Grossbergian Neuroscience 01: A gentle introduction to Stephen Grossberg’s science of mind and brain.” NeuroLogos YouTube Channel (2021). (Accessed July 25, 2023; available online at https://www.youtube.com/watch?v=HrOxj-3hBiw.)
And More …
The full suite of over 500 of Grossberg archival articles, including the above ones, can be downloaded from Grossberg’s Boston University web page sites.bu.edu/steveg, along with various videos of interviews, keynote lectures, etc.