Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI

Training Data

There’s a new archetype in Silicon Valley, the AI researcher turned founder. Instead of tinkering in a garage they write papers that earn them the right to collaborate with cutting-edge labs until they break out and start their own.

This is the story of wunderkind Eric Steinberger, the founder and CEO of Magic.dev. Eric came to programming through his obsession with AI and caught the attention of DeepMind researchers as a high school student. In 2022 he realized that AGI was closer than he had previously thought and started Magic to automate the software engineering necessary to get there. Among his counterintuitive ideas are the need to train proprietary large models, that value will not accrue in the application layer and that the best agents will manage themselves. Eric also talks about Magic’s recent 100M token context window model and the HashHop eval they’re open sourcing.

Hosted by: Sonya Huang, Sequoia Capital

Mentioned in this episode:

• David Silver: DeepMind researcher that led the AlphaGo team
• Johannes Heinrich: a PhD student of Silver’s and DeepMind researcher who mentored Eric as a highschooler
• Reinforcement Learning from Self-Play in Imperfect-Information Games: Johannes’s dissertation that inspired Eric
• Noam Brown: DeepMind, Meta and now OpenAI reinforcement learning researcher who eventually collaborated with Eric and brought him to FAIR
• ClimateScience: NGO that Eric co-founded in 2019 while a university student
• Noam Shazeer: One of the original Transformers researchers at Google and founder of Charater.ai
• DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker: the first AI paper Eric ever tried to deeply understand
• LTM-2-mini: Magic’s first 100M token context model, build using the HashHop eval (now available open source)

00:00 - Introduction

01:39 - Vienna-born wunderkind

04:56 - Working with Noam Brown

8:00 - “I can do two things. I cannot do three.”

10:37 - AGI to-do list

13:27 - Advice for young researchers

20:35 - Reading every paper voraciously

23:06 - The army of Noams

26:46 - The leaps still needed in research

29:59 - What is Magic?

36:12 - Competing against the 800-pound gorillas

38:21 - Ideal team size for researchers

40:10 - AI that feels like a colleague

44:30 - Lightning round

47:50 - Bonus round: 200M token context announcement