Episode 2026-06-24↗
AI:AM LIVE — June 24, 2026 — Gradual Disempowerment and the Search for a Stable Post-AGI Equilibrium: David Duvenaud
The opening covered a fast-moving week in AI policy and infrastructure: the first federal lawsuit over the BIS export-control order cutting off Anthropic's Fable 5 for foreign nationals; two papers pointing at latent world-models inside RL agents and a 7M-parameter loop model beating much larger systems on hard reasoning; the $8M super-PAC defeat of New York's frontier-AI-safety lawmaker; and a brisk exchange on Claude Tag's launch as multiplayer AI, GLM 5.2's cost advantage over GPT-5.5 Codex, and Claude's UltraCode orchestration mode. David Duvenaud — ML professor at the University of Toronto, co-creator of neural ODEs, former alignment lead at Anthropic, and co-author of the 'Gradual Disempowerment' paper — then joined for a full hour exploring whether any stable post-AGI equilibrium actually exists where humans keep meaningful control. Nathan pressed every optimist steelman: historical absorption (prior automation shocks were absorbed without permanent disempowerment), comparative advantage (Ricardo says humans keep a niche), constitutional and property anchors (the franchise, rule of law, military command), aligned AIs defending human leverage, and the argument that 'gradual' gives time to correct. Duvenaud's rebuttal to each was consistent: the disempowerment mechanism doesn't require malice or misalignment — it requires only that the systems driving growth stop needing human participation, the way human civilization doesn't need the monkey economy despite occasionally trading bananas with them. He described the 'Earth as a slow zone' scenario — throttled AI growth, bans on recursive self-improvement, no cultural optimization — and argued that when you enumerate everything it requires controlling (research, startups, reproduction, memetics), the list is horrifyingly long, analogous to listing all the mutations that can cause cancer. On timelines, he sketched white-collar automation first, then a decade-plus to build enough robot factories to displace physical labor, putting full human economic irrelevance perhaps 15–20 years out. His two concrete recommendations: restrict frontier compute at the TSMC/fab choke point, and cultivate the temporal coherence of public preferences by chaining the 'is it okay if humanity disappears?' question forward to one's own children and grandchildren until a coherent answer emerges.
Guests·David Duvenaud
Episode 2026-06-23↗
AI:AM LIVE — June 23, 2026 — Self-Improving GPU Kernels and Europe's AI Sovereignty: Bing Xu and Michiel Bakker
The open tracked an unusually quiet news day through a markets lens — rumors that GPT-5.6 was pulled back amid the model-release freeze and that Gemini 3.5 Pro is lagging, a 6% semiconductor selloff as SK Hynix overtook Samsung for the first time in 27 years, Anthropic's first memory-chip deal with Micron, and an extended debate on whether AI's leverage dynamics make the boom a 'too big to fail' bubble. Bing Xu, founder & CEO of INT21 (co-creator of MXNet and AITemplate, co-author of the original GAN paper, founder of NVIDIA-acquired HippoML), then made the case that self-improvement should target the infrastructure, not the model: his PTX Kernel Factory points agent swarms at the GPU ISA below CUDA, matching expert libraries like QuACK on mature kernels and posting up to 59% speedups on newer ones — and, he argued, deepening NVIDIA's moat rather than eroding it, because the evolutionary loop depends on NVIDIA's profiling ecosystem. MIT/DeepMind researcher Michiel Bakker followed on Europe 2031, his viral month-by-month scenario of Europe sleepwalking into AI dependence — a fictional 2028 export-control beat that materialized within a day of publishing when the US restricted Anthropic's models for foreign nationals — laying out why regulation requires capability first, why the nuclear-umbrella analogy fails for an economic technology, and where Europe's real leverage (the ASML/IMEC semiconductor ecosystem, a middle-power coalition) still lies. The hosts closed on the geopolitics of AI data and a tease of David Duvenaud and 'gradual disempowerment' the next morning.
Guests·Bing Xu, Michiel Bakker