When AI Discovers the Next Transformer — Robert Lange
Summary
This video features Robert Lange discussing Sakana AI's *Shinka Evolve* framework, which combines LLMs with evolutionary algorithms for open-ended program search. It highlights the paradigm shift from optimizing fixed problems to co-evolving problems for true scientific progress in AI. This content is valuable for advanced students and educators in AI/ML as it offers a deep dive into cutting-edge research and future directions in AI discovery.
Description
Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves. GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg) In this episode: • Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search. • The *architecture* of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard. • Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks. • Are these systems actually thinking outside the box, or are they parasitic on their starting conditions? When LLMs run autonomously, "nothing interesting happens." Robert pushes back with the stepping-stone argument — evolution doesn't need to extrapolate, just recombine usefully. • The AI Scientist question: can automated research pipelines produce real science, or just workshop-level slop that passes surface-level review? Robert is honest that the current version is more co-pilot than autonomous researcher.
More Videos
1:26:40The Dangerous Illusion of AI Coding? - Jeremy Howard
55:49What If Intelligence Didn't Evolve? It "Was There" From the Start! - Blaise Agüera y Arcas
46:57The Brain Is Just Specialized Agents Talking To Each Other — Dr. Jeff Beck
53:38Why AI Has a Plato Problem — Mazviita Chirimuuta
![Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION]](https://i.ytimg.com/vi/pO0WZsN8Oiw/maxresdefault.jpg)
42:05Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION]
![AutoGrad Changed Everything (Not Transformers) [Dr. Jeff Beck]](https://i.ytimg.com/vi/9suqiofCiwM/maxresdefault.jpg)
1:16:38