Satya Nadella — How Microsoft is preparing for AGI
As part of this interview, Satya Nadella gave Dylan Patel (founder of SemiAnalysis) and me an exclusive first-look at their brand-new Fairwater 2 datacenter.Microsoft is building multiple Fairwaters, each of which has hundreds of thousands of GB200s & GB300s. Between all these interconnected buildings, they’ll have over 2 GW of total capacity. Just to give a frame of reference, even a single one of these Fairwater buildings is more powerful than any other AI datacenter that currently exists.Satya then answered a bunch of questions about how Microsoft is preparing for AGI across all layers of the stack.Watch on YouTube; read the transcript.Sponsors* Labelbox produces high-quality data at massive scale, powering any capability you want your model to have. Whether you’re building a voice agent, a coding assistant, or a robotics model, Labelbox gets you the exact data you need, fast. Reach out at labelbox.com/dwarkesh* CodeRabbit automatically reviews and summarizes PRs so you can understand changes and catch bugs in half the time. This is helpful whether you’re coding solo, collaborating with agents, or leading a full team. To learn how CodeRabbit integrates directly into your workflow, go to coderabbit.aiTo sponsor a future episode, visit dwarkesh.com/advertise.Timestamps(00:00:00) - Tour through Fairwater 2(00:03:20) - Business models for AGI(00:12:48) - Copilot(00:20:02) - Whose margins will expand most?(00:36:17) - MAI(00:47:47) - The hyperscale business(01:02:44) - In-house chip & OpenAI partnership(01:09:35) - The CAPEX explosion(01:15:07) - Will the world trust US companies to lead AI? Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
--------
1:27:47
--------
1:27:47
Sarah Paine – How Russia sabotaged China's rise
In this lecture, military historian Sarah Paine explains how Russia—and specifically Stalin—completely derailed China’s rise, slowing them down for over a century.This lecture was particularly interesting to me because, in my opinion, the Chinese Civil War is 1 of the top 3 most important events of the 20th century. And to understand why it transpired as it did, you need to understand Stalin’s role in the whole thing.Watch on YouTube; read the transcript.SponsorsMercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our cash balance, AR, and AP all in one place. Join us (and over 200,000 other entrepreneurs) at mercury.comLabelbox scrutinizes public benchmarks at the single data-row level to probe what’s really being evaluated. Using this knowledge, they can generate custom training data for hill climbing existing benchmarks, or design new benchmarks from scratch. Learn more at labelbox.com/dwarkeshTo sponsor a future episode, visit dwarkesh.com/advertise.Timestamps(00:00:00) – How Russia took advantage of China’s weakness(00:22:58) – After Stalin, China’s rise(00:33:52) – Russian imperialism(00:45:23) – China’s and Russia’s existential problems(01:04:55) – Q&A: Sino-Soviet Split(01:22:44) – Stalin’s lessons from WW2 Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
--------
1:30:36
--------
1:30:36
Andrej Karpathy — AGI is still a decade away
The Andrej Karpathy episode.During this interview, Andrej explains why reinforcement learning is terrible (but everything else is much worse), why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education.It was a pleasure chatting with him.Watch on YouTube; read the transcript.Sponsors* Labelbox helps you get data that is more detailed, more accurate, and higher signal than you could get by default, no matter your domain or training paradigm. Reach out today at labelbox.com/dwarkesh* Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our accounts, cash flows, AR, and AP all in one place. Apply online in minutes at mercury.com* Google’s Veo 3.1 update is a notable improvement to an already great model. Veo 3.1’s generations are more coherent and the audio is even higher-quality. If you have a Google AI Pro or Ultra plan, you can try it in Gemini today by visiting https://gemini.googleTimestamps(00:00:00) – AGI is still a decade away(00:29:45) – LLM cognitive deficits(00:40:05) – RL is terrible(00:49:38) – How do humans learn?(01:06:25) – AGI will blend into 2% GDP growth(01:17:36) – ASI(01:32:50) – Evolution of intelligence & culture(01:42:55) - Why self driving took so long(01:56:20) - Future of education Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
--------
2:25:19
--------
2:25:19
Nick Lane – Life as we know it is chemically inevitable
Nick Lane has some pretty wild ideas about the evolution of life.He thinks early life was continuous with the spontaneous chemistry of undersea hydrothermal vents.Nick’s story may be wrong, but I find it remarkable that with just that starting point, you can explain so much about why life is the way that it is — the things you’re supposed to just take as givens in biology class:* Why are there two sexes? Why sex at all?* Why are bacteria so simple despite being around for 4 billion years? Why is there so much shared structure between all eukaryotic cells despite the enormous morphological variety between animals, plants, fungi, and protists?* Why did the endosymbiosis event that led to eukaryotes happen only once, and in the particular way that it did?* Why is all life powered by proton gradients? Why does all life on Earth share not only the Krebs Cycle, but even the intermediate molecules like Acetyl-CoA?His theory implies that early life is almost chemically inevitable (potentially blooming on hundreds of millions of planets in the Milky Way alone), and that the real bottleneck is the complex eukaryotic cell.Watch on YouTube; listen on Apple Podcasts or Spotify.Sponsors* Gemini in Sheets lets you turn messy text into structured data. We used it to classify all our episodes by type and topic, no manual tagging required. If you’re a Google Workspace user, you can get started today at docs.google.com/spreadsheets/* Labelbox has a massive network of domain experts (called Alignerrs) who help train AI models in a way that ensures they understand the world deeply, not superficially. These Alignerrs are true experts — one even tutored me in chemistry as I prepped for this episode. Learn more at labelbox.com/dwarkesh* Lighthouse helps frontier technology companies like Cursor and Physical Intelligence navigate the U.S. immigration system and hire top talent from around the world. Lighthouse handles everything, maximizing the probability of visa approval while minimizing the work you have to do. Learn more at lighthousehq.com/employersTo sponsor a future episode, visit dwarkesh.com/advertise.Timestamps(00:00:00) – The singularity that unlocked complex life(00:08:26) – Early life continuous with Earth's geochemistry(00:23:36) – Eukaryotes are the great filter for intelligent life(00:42:16) – Mitochondria are the reason we have sex(01:08:12) – Are bioelectric fields linked to consciousness?Ref: 868329 Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
--------
1:20:08
--------
1:20:08
Some thoughts on the Sutton interview
I have a much better understanding of Sutton’s perspective now. I wanted to reflect on it a bit.(00:00:00) - The steelman(00:02:42) - TLDR of my current thoughts(00:03:22) - Imitation learning is continuous with and complementary to RL(00:08:26) - Continual learning(00:10:31) - Concluding thoughts Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
USA