Welcome in!
You’ve made a wise, well-considered, and profoundly useful choice – you’re going to master just enough of statistical mechanics so that you can work through those classic papers in energy-based neural networks.
You know what we’re talking about:
- The introduction of the Boltzmann machine (Ackley, Hinton, & Sejnowski, 1985),
- The introduction of the contrastive divergence algorithm for Boltzmann machines (Hinton, 2002),
- The original deep learning architecture works, solving the problem of stacking neural network layers (Hinton & Salakhutdinov, 2006, and Salakhutdinov & Hinton, 2012) …
And many, many more.
In fact, we can’t read any of the important works in energy-based neural networks, or any of the equally important works in variational methods, without some understanding of thermodynamics and statistical mechanics.
Case in Point: Salakhutdinov and Hinton (2012)
We know a fancy equation when we see one. (We’ve worked through many of them in our lifetimes.)
However, we also know when enough is enough.
Our desire, goal, and intention is to guide you safely through the metaphorical Sierra Nevada mountains, so that you can reach the “Gold Coast” of AI – the part of AI where all the fun and adventure is happening. (And all the money.)
Getting Through the First Donner Pass
Think of this as going from a (metaphorical) Fort Lauderdale to the “AI Gold Coast” in California.
What You’ll Learn
Introduction to the Top Ten Terms in Statistical Mechanics.
How You Will Transform During this Course
By going through this material, doing the readings, watching the vids, and – most important! – doing the exercises, you will help you transform how you think of yourself.
By way of metaphor – this course gives you the statistical mechanics that you need to get through the mountains. You’ll be able to recognize a stat mech term when you see one. You’ll know what it stands for and why it is there.
In short, you’ll be familiar with the statistical mechanics mountain range that forms one side of the first Donner Pass of AI.
You will understand enough of how the stat mech concepts and equations relate to the neural networks and machine learning methods that you want to learn.
What to Do Today
- Please watch today’s video tutorial. You can find the video – Day 1: What You’ll Learn in This Course – in your Day 1 on the Thinkific course site. Also, download and read the Day 1 Resources PDF.
- Please ensure that you’ve downloaded the key resources that we provided in Day 0: Course Overview and Getting Started.
- Download, read, and DO the exercises in the Day 1 Exercises, which will refer you back to Salahutdinov and Hinton (2012), and to the video and the Resources PDFs that you’ve downloaded for Days 0 & 1.
About the Exercises
We learn the most when we actively engage with and reflect on our course material. So … we will offer you some exercises.
Each exercise is designed to help you confirm that you do, indeed, understand this material.
Today’s exercise is to identify and characterize terms, using Salakhutdinov and Hinton (2012; which we’ll refer to as S&H 2012) as a starting point. It’s available from Day 0: Course Overview and Getting Started.
Use THIS YOUTUBE video to help get a sense of how these terms can be organized:
Related YouTube Vids
This short YouTube vid gives you something to think about – in terms of using statistical mechanics more as a “metaphor” than as a “model.”
This YouTube expands on that notion of statistical mechanics as a metaphor, building out the metaphor of your journey in learning AI as traveling on the “Oregon Trail of Artificial Intelligence.”
Daily Emails
- Check your Email Inbox every day for useful insights and personal stories!
See you tomorrow! – AJM
P.S. – You can jump ahead to tomorrow’s content on Day 2: Thermodynamics.
Use THIS LINK if you’d like to go back to Day 0: Course Overview and Getting Started.