Why Learn Classical AI Models? | 7-03-25

For the last 2 years, I’ve been dedicating my free time to learning AI. I’ve taught myself from open source code, online courses, and books that I could find online. At this point, I feel like I can give some advice on what helped me learn.

While a lot of people want to start by learning LLMs, the huge models like GPT and Claude, they will make absolutely no sense to you on the first run-through unless you have a ridiculous background in mathematics and natural intuition for computer science. Although you’ll eventually work your way up to LLMs and Generative AI, I’m of the opinion that you shouldn’t start with the dial turned to 100 on difficulty.

My preferred starting point is with classical AI models. These encompass the most powerful machine learning models like Support Vector Machines or Decision Trees, in addition to foundational Deep Learning methods like Restricted Boltzmann Machines and Echo-State Models.

Why should you learn these? Well, these earlier models have much more elegant math than the huge models used today. They utilize clever computational tricks and operations to model systems and allow for learning. That’s why I found them so fascinating at first: the return on investment is huge! If you can understand a little bit more of a math-heavy classical model, then you’ll find it much easier to digest the far simpler yet larger Deep Neural Networks used today. Also, they’re useful for verifying that you understand fundamental calculus and linear algebra. If you can design a basic classical AI model, then you can be pretty sure that you’re comfortable with the mathematical concepts underlying it.

That leads me to my next point: it’s far harder to implement a classical AI model from scratch than a Deep Neural Network. In order to code a Restricted Boltzmann Machine in Python from scratch, which I actually did but which took me quite a while, you need to fully understand the complex equations behind them and how to express them in Python. Deep Neural Networks and mainstream AI models are so abstracted that you can create one in barely 2 lines of code, which is definitely not the most effective way to learn the basics. Instead of memorizing a script, you should be grappling with tough concepts in order to learn by trial and error. That’s why I advise learning the basics of classical AI first.

Lastly, classical AI models often have very interesting biological analogues. Restricted Boltzmann Machines function according to the rules of the energy of physical configurations, and Hopfield Networks, another interesting type of foundational AI model that underlies the Deep Neural Network, is inspired by the phenomenon of memory in the brain. By learning these models first, you get a deeper introduction into the biological systems that we are trying to model in the field of AI.

Ultimately, the dilemma is between whether you should kick the rigor ball down the road by learning an easy thing fast, knowing you’ll have to learn the hard foundations later, or diving straight into the foundations to get the most out of your education and time later on. I definitely advise the latter. This is a mistake that I made when first learning to code, as I started with Python because it was thought of as the easiest language to learn. While I found this to be true, when I had to try and apply more complex concepts or techniques, I didn’t know where to start or how to build things from first principles simply because Python was so easy to learn. It was only after I learned Java in AP Computer Science, which doesn’t abstract nearly as much from the programmer as Python does, that I was fluent in the basic understanding of the principles of programming languages, benefitting me immensely when I returned to coding personal projects and researching in AI. This is an identical dilemma, and I say with confidence: learn classical AI first, then dive deeply into Deep Learning with a solid foundation and good basics. You’ll thank me later.