学习资料
浏览视频
返回视频列表
手写 micrograd：理解神经网络与反向传播
学习资料
播放器下方提供英文笔记、测验、闪卡和 PPT 大纲。
PPT 资料为文字大纲，尚非完整演示文稿。
视频概览
Andrej Karpathy《Neural Networks: Zero to Hero》的第一课，用 Python 从零构建小型自动求导引擎 micrograd，逐步解释反向传播与神经网络训练。只需基础 Python 和部分高中微积分知识。
HN 讨论的原因
HN 常推荐它作为深入理解反向传播与神经网络的起点：Karpathy 用直觉解释原理，并通过代码逐步实现。
观看前准备
具备基础 Python、函数和高中求导知识会有帮助。如果想理解梯度如何沿计算图传播，可以从这节课开始。
学习建议
每次反向传播前，先手算一个导数，再与代码结果比较，并解释权重变化如何影响损失。理解标量示例后，再继续学习 makemore。
最新优先
所在讨论： The Illustrated Transformer
If you have 20 hours to spare I highly recommend this youtube playlist from Andrej Karpathy [VIDEO] It starts with the fundamentals of how backpropagation works then advances to building a few simple models and ends with building a GPT-2 clone. It won't taech you everything about AI models but it gives you a solid foundation for branching out.
所在讨论： Deep dive into the small details of micrograd
This blog post analyzes the code with an emphasis on the small details and ideas explored in Andrej Karpathy's awesome micrograd lecture. Link to the original lecture: [VIDEO] I'd love to hear any thougts/reviews about the blog post.
所在讨论： The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]
For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers [LINK] and so on. And of course Yannic Kilcher[4], and also listening in on the paper discussions they do…For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers [LINK] and so on. And of course Yannic Kilcher[4], and also listening in on the paper discussions they do on discord. Practicing a lot with just doing backpropagation by hand and making toy models by hand to get intuition for the signal flow, and building all kinds of smallish systems, e.g. how far can you push whisper, small qwen3, and kokoro to control your computer with voice? People think that deepseek/mistral/meta etc are democratizing AI, but its actually Karpathy who teaches us :) so we can understand them and make our own. [1] [VIDEO] [2] [LINK] [3] [LINK] [4] [LINK]
所在讨论： How linear regression works intuitively and how it leads to gradient descent
In the same vein, Karpathy's video series "Neural Networks from zero to hero"[0] touches upon a lot of this and intuitions as well. One of the best introductory series (even if you ignore the neural net part of it) and brushes on gradients, differentiation and what it means intuitively. [0] [VIDEO]
所在讨论： Understanding Machine Learning: From Theory to Algorithms
From my other comment elsewhere. These resources helped me understand the topics better. If anyone wants to understand fundamentals of machine learning, one of the superb resources I have found is, Stanford's "Probability for computer scientists"[1]. It goes into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. But, this is a primarily a probability course that…From my other comment elsewhere. These resources helped me understand the topics better. If anyone wants to understand fundamentals of machine learning, one of the superb resources I have found is, Stanford's "Probability for computer scientists"[1]. It goes into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. But, this is a primarily a probability course that discusses the fundamentals of machine learning. (Yeah, Andrew Ng is legendary, but his course demands some mathematical familiarity with linear algebra topics) There is a course reader for CS109 [2]. You can download pdf version of this. Caltech's learning from data was really good too, if someone is looking for theoretical understanding of ML topics [3]. There is also book for excellent caltech course[4]. Also, neural networks zero to hero is for understanding how neural networks are built from ground up [5]. [1] [LINK] [2] [LINK] [3] [LINK] [4] [LINK] [5] [VIDEO]
已显示 5 / 20 条评论