Browse videos
Intro to Large Language Models
Study materials
Materials for this selected video are in preparation. Watch the video and use the public learning guide while they are being prepared.
Slides are a PPT outline, not a finished slide deck.
Overview
Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.
Why HN discusses it
HN recommends it as the dense, approachable starting point for understanding LLMs and often cites it for specific claims about scaling and training.
Before you watch
Start here if you want a conceptual introduction to language models. Basic familiarity with computers is enough to begin; this is an overview rather than a coding workshop.
A way to study this video
After watching, describe a language model in your own words and separate its training process from its use in a product. Continue with the deep dive for more context or micrograd for implementation foundations.
Follow the Andrej Karpathy learning guide · Original Zero to Hero course
Newest first
In discussion: The Unbearable Cheapness of Open Weight Models
The weights are just numbers. I don't what technical background you have in other areas of computing, but I think that this is a good, short introduction that doesn't assume too much: [LINK] To quote part of it, Training a model can be thought of as tuning the dials on a really big machine. The way that a language model behaves is entirely determined by these many different continuous values, usually called…The weights are just numbers. I don't what technical background you have in other areas of computing, but I think that this is a good, short introduction that doesn't assume too much: [LINK] To quote part of it, Training a model can be thought of as tuning the dials on a really big machine. The way that a language model behaves is entirely determined by these many different continuous values, usually called parameters or weights. Longer and slightly more technical, "Intro to Large Language Models" by Andrej Karpathy: [VIDEO]
In discussion: The Intuition Behind How Large Language Models Work, Part I
* "[1hr Talk] Intro to Large Language Models" by Andrej Karpathy: [VIDEO] from [LINK] * Andrej Karpathy: [LINK]
In discussion: Andrej Karpathy: Software in the era of AI [video]
I watched Karpathy's Intro to Large Language Models [0] not so long ago and must say that I'm a bit confused by this presentation, and it's a bit unclear to me what it adds. 1,5 years ago he saw all the tool uses in agent systems as the future of LLMs, which seemed reasonable to me. There was (and maybe still is) potential for a lot of business cases to be explored, but every system is defined by its boundaries…I watched Karpathy's Intro to Large Language Models [0] not so long ago and must say that I'm a bit confused by this presentation, and it's a bit unclear to me what it adds. 1,5 years ago he saw all the tool uses in agent systems as the future of LLMs, which seemed reasonable to me. There was (and maybe still is) potential for a lot of business cases to be explored, but every system is defined by its boundaries nonetheless. We still don't know all the challenges we face at that boundaries, whether these could be modelled into a virtual space, handled by software, and therefor also potentially AI and businesses. Now it all just seems to be analogies and what role LLMs could play in our modern landscape. We should treat LLMs as encapsulated systems of their own ...but sometimes an LLM becomes the operating system, sometimes it's the CPU, sometimes it's the mainframe from the 60s with time-sharing, a big fab complex, or even outright electricity itself? He's showing an iOS app, which seems to be, sorry for the dismissive tone, an example for a better looking counter. This demo app was in a presentable state for a demo after a day, and it took him a week to implement Googles OAuth2 stuff. Is that somehow exciting? What was that? The only way I could interpret this is that it just shows a big divide we're currently in. LLMs are a final API product for some, but an unoptimized generative software-model with sophisticated-but-opaque algorithms for others. Both are utterly in need fo
In discussion: TexTube: Chat with any YouTube video transcript in ChatGPT fast
It might be a preview or something because I have YT premium and doesn't show up that anywhere. Can you share a video that works for that? Like this one. [VIDEO]
In discussion: OpenDevin: An Open Platform for AI Software Developers as Generalist Agents
Actually the improvement graphs are still scaling exponentially with training/compute being the bottleneck. So there isn't yet any evidence of diminishing returns. source: [VIDEO]
Showing 5 of 20 comments