What if the next AI model could be built from models we already have, instead of being trained from scratch?
In this episode of An Hour of Innovation podcast, Vit Lyoshin speaks with Damian Borth, a professor of artificial intelligence and machine learning at the University of St. Gallen and CEO of Weight Space Labs, about weight-space learning: the idea of treating neural network weights as data. They explore how researchers can read existing AI models, edit parts of them, and work toward generating new, task-specific models. The promise is faster, less expensive AI model development, but Damian is clear that instant generation at scale is still a research goal.
You’ll learn
* Why training neural networks demands so much compute
* What a model’s weights can tell us
* How existing models could become training data
* Where model synthesis differs from training from scratch
* Why small, specialized models matter for robots
* What Damian’s remote-sensing experiments showed about compute
* How one AI model might edit another
* Why reading models matters for safety and provenance
* What limits weight-space learning today
* How AI agents might request models as tools
* Why private data raises new opportunities and risks
* What must improve before on-demand models scale
Damian Borth is a professor of artificial intelligence and machine learning at the University of St. Gallen and CEO of Weight Space Labs. His research and work explore how to learn from the weights of existing neural networks. He describes published work on generating remote-sensing models that performed comparably to models trained from scratch while using less compute. He also discusses work on reading model properties, as well as the limits of current approaches. His team is working toward a foundation model for neural networks at Weight Space Labs.
Timestamps
00:00 Introduction
01:26 Is AI Running Out of Training Data?
02:42 How AI Models Learn and Store Knowledge
04:06 What Is Weight Space Learning?
05:45 Weight Space Learning Use Cases
08:39 20x Less Compute: Efficiency Gains
12:34 Weight Space Learning Limitations
14:47 MIT's Weight Space Programming
18:01 How AI Edits Another Model's Weights
23:45 Model Analysis, Lineage and AI Security
31:41 Risks of Recursive Self-Improving AI
34:23 Skills for Working With Neural Weights
39:07 Synthetic Data and Model Collapse
42:03 Private Data Sharing via Neural Networks
48:36 Getting Started With Weight Space Learning
50:21 Innovation Q&A Round
Connect with Damian
* Website: https://www.weightspacelabs.ai/
* LinkedIn: https://www.linkedin.com/in/damianborth/
Connect with Vit
* Substack: https://anhourofinnovation.substack.com/
* LinkedIn: https://www.linkedin.com/in/vit-lyoshin/
* X: https://x.com/vitlyoshin
* Website: https://vitlyoshin.com/
Support the podcast
Enjoy the podcast? You can help us keep bringing you interesting stories and conversations.
Explore our sponsors and recommended tools:
https://www.anhourofinnovation.com/sponsors/
Some links are affiliate links, which means we may earn a small commission if you make a purchase or sign up, at no extra cost to you.
Interested in sponsoring the podcast?
Reach out to us at iris@anhourofinnovation.com to explore partnership opportunities.
