So, you want to learn AI? But you don’t know how or where to get started? To escape tutorial hell and really learn, you have to get hands-on, write algorithms from scratch, implement papers, and do fun side projects using AI to solve problems. This article attempts to craft a free curriculum that follows that philosophy.
Top-down approach
This curriculum adopts a top-down approach, prioritizing hands-on coding experience before delving into theoretical concepts. The individual behind this approach prefers to learn out of necessity, seeking solutions to problems or creating prototypes as a means of driving their learning process. They emphasize the importance of filling knowledge gaps to achieve their goal of becoming proficient in artificial intelligence (AI) engineering, particularly in natural language processing (NLP). This involves mastering skills such as coding transformers from scratch and fine-tuning large language models (LLMs) on graphical processing units (GPUs). Additionally, they specify their focus on NLP and offer assistance to those interested in other AI specializations like computer vision or reinforcement learning. Before sharing a list of recommended resources, they express a desire for having received two crucial pieces of advice prior to embarking on their learning journey.
Learn in Public
There is a vast expanse of knowledge to absorb, particularly in the realm of AI, where groundbreaking papers and concepts emerge with remarkable frequency.
The gravest mistake an individual can make is to confine their learning to solitary endeavours. By doing so, they forego valuable opportunities. Merely completing tasks in isolation yields little beyond personal satisfaction. What holds greater significance is the ability to assimilate information, transform it into actionable knowledge, and share insights with the wider audience. This process often leads to the generation of innovative ideas and solutions.
This entails cultivating a habit of creation, which can manifest in various forms:
- Crafting blogs and tutorials to disseminate knowledge
- Engaging in hackathons and collaborative endeavors
- Participating in Discord communities to seek or provide assistance
- Pursuing passionate side projects
- Sharing intriguing discoveries and insights via social media platforms like Twitter
Mathematics
Machine learning relies heavily on three pillars of mathematics: linear algebra, calculus, and probability. Each plays a unique role in enabling algorithms to function effectively.
Linear Algebra deals with data representation and manipulation using matrices and vectors.
Calculus is essential for optimization, allowing algorithms to learn and improve by understanding gradients and rates of change.
Probability and statistics form the foundation for decision-making under uncertainty, enabling algorithms to predict outcomes and learn from data.
For a code-first approach to linear algebra, check out “Computational Linear Algebra” by the creators of fast.ai. Alternatively, you can explore the “Math for Machine Learning” series by Weights & Biases.
For more traditional learning, consider Imperial College London lectures on linear algebra and multivariate calculus.
Machine learning is a field where you teach computers to learn and make decisions without being explicitly programmed. If you want to learn about it, you can start by reading the “100-page ML book.” While reading, try to understand the concepts deeply by writing the algorithms from scratch.
There are several repositories like eriklindernoren/ML-From-Scratch, JeremyNixon/oracle, and trekhleb/homemade-machine-learning that you can refer to for additional guidance and examples.
If you’re up for a challenge, consider following the “MiniTorch” course, which teaches you how to build PyTorch from scratch. This DIY approach can deepen your understanding of machine learning principles.
To apply what you learn, participate in ML competitions on platforms like bitgrit and Kaggle. Analyze past winning solutions to gain insights into effective strategies.
Engaging in side projects is another way to solidify your understanding. You can explore topics like getting machine learning models into production, as discussed by Vicki Boykis. She also shares her experience building Viberary, a semantic search for books.
For hands-on practice, obtain a dataset, such as NASA Earth data from earthaccess, and build a model. You can enhance your project by creating a user interface with tools like Streamlit. These practical experiences will help reinforce your learning and skills in machine learning.
Deep Learning
If you’re looking for a top-down approach to learning deep learning, fast.ai is a great starting point. They offer comprehensive courses (part 1 and part 2) along with a study group supported by W&B.
If you enjoyed fast.ai and want to explore further, you can check out Full Stack Deep Learning for a deeper dive into the subject.
For those preferring a more traditional course, UNIGE 14×050 — Deep Learning by François Fleuret is recommended.
When you need to delve into theory, several great books are available, such as Dive into Deep Learning, Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Neural networks and deep learning, and Understanding Deep Learning.
While your neural networks are training, you can read The Little Book of Deep Learning on your phone to utilize your time effectively.
Engaging in more competitions like PlantTraits2024 — FGVC11 on Kaggle for computer vision can provide practical experience.
You can also implement papers using resources like labml.ai Annotated PyTorch Paper Implementations and explore explanations of popular models like BERT on Papers with Code.
For specializations within Deep Learning:
- For Computer Vision, consider CS231n: Deep Learning for Computer Vision.
- For Reinforcement Learning, resources like Spinning Up in Deep RL by OpenAI and the Deep Reinforcement Learning Course by Hugging Face are recommended.
- For NLP, explore CS 224N | Natural Language Processing with Deep Learning from Stanford and Hugging Face NLP Course.
Additionally, there are many good articles and breakdowns available, such as BERT Research by Chris McCormick, The Illustrated Word2vec by Jay Alammar, The Illustrated BERT, ELMo, and co. by Jay Alammar, Understanding LSTM Networks by colah’s blog, and PyTorch RNN from Scratch by Jake Tae.
For supplementary reading, consider Natural Language Processing with Transformers Book.
Large Language Models
First, watch an hour-long talk titled “Intro to Large Language Models” by Andrej. Then, delve into “Large Language Models in Five Formulas” by Alexander Rush from Cornell Tech.
Next, check out “Neural Networks: Zero to Hero,” a comprehensive tutorial that starts with explaining and coding backpropagation from scratch and ends with writing GPT from scratch. This series is presented by Andrej Karpathy, who has recently released a new video titled “Let’s build the GPT Tokenizer.”
You can also learn about GPT in a concise format by watching “GPT in 60 Lines of NumPy” by Jay Mody.
For those interested in a more structured learning experience, there’s a free LLM boot camp offered by Full Stack Deep Learning. It covers prompt engineering, LLMOps, UX for LLMs, and how to launch an LLM app in just an hour.
Once you’ve completed the boot camp, you can start building with LLMs. Watch “Application Development using Large Language Models” by Andrew Ng and read resources like “Building LLM applications for production” by Huyen Chip and “Patterns for Building LLM-based Systems & Products” by Eugene Yan. The OpenAI Cookbook and Vercel AI templates can also be useful resources for getting started.
To further deepen your understanding, you can explore the theory behind large language models and understand how everything works. Additionally, consider participating in hackathons, such as those organized by lablab.ai, to apply your knowledge in practical challenges and collaborate with others.
Overall, these resources offer a comprehensive pathway for learning about and applying large language models in various contexts.
