CLIP Talk - Daniel Khashabi
CLIP Talk - Daniel Khashabi
Hear from Daniel Khashabi (Johns Hopkins University) at CLIP's upcoming talk!
Bets on Scaling Long-Running Agents: The Long and Short of It
- Abstract: Today’s agents can carry out complex tasks, but their work is often fragmented across sessions. The future, in contrast, is about agents that work over weeks or months, carrying forward what they learn and decide as their histories grow. How do we get there? In this talk, I’ll examine few “bets” to getting us to such a future:
Expanding an over-growing context: This bet assumes that progress in AI will continue to bring longer context windows alongside other forms of scaling. I’ll present the Information Abundance Paradox: in our experiments, training on longer contexts does not always improve what a model internalizes in its parameters. This (among other reasons) suggests that scaling context may require more than mere scaling (data and compute).
The remainder of the talk assumes that an agent can work over long periods by selectively preserving what matters from its bounded history: - Continual parametric adaptation: This bet assumes that an agent can carry forward what it learns by updating its parameters over time. I’ll present our study on long-horizon memorization, where models learn a stream of tasks. We find that, while no single continual learning mechanism maintains strong retention, their “composition” substantially reduces forgetting. This suggests that parametric memories may be a viable “bet” to continue to invest in.
- Maintaining a bounded context with a fixed model: This bet focuses on effective use of a bounded window. I'll introduce Self-Compact, a method that lets agents decide when and how to compact their own context. Our results show that this simple, training-free approach improves performance over fixed-interval compaction at similar cost, suggesting that context management may be a useful hill to climb toward long-running agents.
Based on the following works:
Bio: Daniel Khashabi is an Assistant Professor of Computer Science at Johns Hopkins University, and is affiliated with the Data Science and AI Institute and the Center for Language and Speech Processing (CLSP). He is interested in the intersection of reasoning and discovery. He is the recipient of the NSF CAREER award (2026), Ai2’s Lasting Impact Award (2024), multiple industry faculty awards (Amazon, Google, Apple), and several Best Paper Awards (COLM 2024, ACL 2023, NAACL 2022). His teaching has earned him the Joel Dean Award for Excellence in Teaching in the Department of Computer Science in 2025.