学习资料
浏览视频
返回视频列表
深入理解 ChatGPT 等大语言模型
学习资料
这个精选视频的资料正在准备中，可以先观看视频并阅读学习导读。
PPT 资料为文字大纲，尚非完整演示文稿。
视频概览
Andrej Karpathy 面向大众的三小时以上深入讲解，介绍 ChatGPT 类大语言模型如何构建，覆盖从预训练到后训练的完整流程，并解释幻觉等特性及使用方法。
HN 讨论的原因
HN 经常推荐它作为清晰、易懂的大语言模型入门，无需深入的数学基础。
观看前准备
首次建立概念认识无需模型训练经验。这条约三小时半的视频适合想在写代码前全面了解大语言模型的学习者。
学习建议
把视频分成几次观看。每看完一节，区分训练与推理，并写下一个关于模型能学会什么、不能学会什么的问题。如需先建立整体认识，可以先看较短的入门视频。
最新优先
所在讨论： Show HN: How LLMs Work – Interactive visual guide based on Karpathy's lecture
I agreed when I read your comment, but that turned out to be almost directly from the video [VIDEO] From around the 2:20 mark he says: “[…] actually ends up being only about 44 TB of disk space. You can get a USB stick for like a TB very easily, or I think this could fit on a single hard drive almost today” So it’s just slightly altered from what was said in the original video. And the LLM rewritten version of it…I agreed when I read your comment, but that turned out to be almost directly from the video [VIDEO] From around the 2:20 mark he says: “[…] actually ends up being only about 44 TB of disk space. You can get a USB stick for like a TB very easily, or I think this could fit on a single hard drive almost today” So it’s just slightly altered from what was said in the original video. And the LLM rewritten version of it also says “roughly” where he said “almost”, and I guess 44 TB is pretty roughly or pretty almost 32 TB. Although I’d still personally probably put it as “can fit on a pair of decently sized hard drives today” (for example across two 24 TB drives). Regardless, it’s close enough to what was said in the source video that it’s not something the LLM just made up out of nowhere.
所在讨论： Microgpt
Even if you have some basic understanding of how LLMs work, I highly recommend Karpathy’s intro to LLMs videos on YouTube. - [VIDEO] - [LINK]
所在讨论： Ask HN: What would you recommend a vibe coder learn about how all this works?
I highly recommend Andrej Karpathy's Deep Dive into LLMs like ChatGPT: [VIDEO] . It's a 3.5 hour general intro to the topic that I think does a great job of making sense of the tech and how it works under the hood, without needing to build up a foundation of math and programming.
所在讨论： Neural Networks: Zero to Hero
This is a good resource, however for about 99.99% of people, you are most likely to just use a foundation model like ChatGPT, Claude, Gemini etc. so this knowledge/training will get you neither here or there. I would suggest you look into another Karpathy's video -- Deep Dive into LLMs like ChatGPT. [VIDEO]
所在讨论： Using secondary school maths to demystify AI
That's post-training No, it is not. Read the paper. They are discussing an emergent property of the context itself: "For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model." I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th…That's post-training No, it is not. Read the paper. They are discussing an emergent property of the context itself: "For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model." I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th or 6th graders in the US are expected to be able to do with no problem - without using a calculator. So am I. See, for example, Karpathy's discussion of native computation: [VIDEO] (starts at 1:46:56). The first few tokens in the context actually serve as some sort of substrate for general computation. I don't pretend to understand that, and it may still be something of an open research topic, but it's one more unexpected emergent property of transformers. You'd be crazy to trust that property at this stage -- at the time Karpathy was making the video, he needed to explicitly tell the model to "Use code" if he didn't want it to just make up solutions to more complex problems -- but you'd also be crazy to trust answers from a 5th-grader who just learned long division last week. Feeling quite comfortable being classed in the same camp with the greatest living physicist, honestly. Not a great time for you to rest on your intellectual laurels. Same goes for Penrose.
已显示 5 / 18 条评论