CLIP Colloquium: Yonatan Bisk (CMU)

Following Instructions and Asking Questions

Abstract: As we move towards the creation of embodied agents that understand natural language, several new challenges and complexities arise for grounding (e.g. complex state spaces), planning (e.g. long horizons), and social interaction (e.g. asking for help or clarifications).  In this talk, I'll discuss improvements to embodied instruction following within ALFRED and initial steps towards building agents that ask questions or model theory-of-mind.

Bio: Yonatan Bisk is an assistant professor of computer science in Carnegie Mellon's Language Technologies Institute. His group works on grounded and embodied natural language processing, placing perception and interaction as central to how language is learned and understood. Previously, he received his PhD from the University of Illinois at Urbana-Champaign working on unsupervised Bayesian models of syntax, before spending time at USC's ISI (working on grounding), the University of Washington (for commonsense research), and Microsoft Research (for vision+language).