Skip to main content
Skip to main content

VCAI Seminar: Dora Demszky (Stanford)

VCAI Seminar: Dora Demszky (Stanford)

Maryland Language Science Center Thursday, April 18, 2024 2:00 pm - 2:00 pm Brendan Iribe Center, 5137
Large Language Models for Teacher Feedback Click here to attend Zoom Large Language Models show unprecedented potential in scaling many aspects of education — can they also facilitate high-quality instruction by serving as a professional learning tool for teachers? This talk synthesizes findings from three papers that evaluate the ability of LLMs to provide feedback to educators on their discourse. Firstly, we evaluate the ability of GhatGPT to perform instructional coaching tasks, such as using an observational instrument to score instruction and provide teachers with suggestions for improvement. We show that while the model generates feedback that is relevant, it struggled with insightfulness. Secondly, we explore the use of LLMs in identifying punitive and restorative classroom management practices. We demonstrate that the LLM-powered measures can surface fine-grained patterns of escalation and racial disparities in classroom management at scale. Lastly, we examine LLMs ability to use growth mindset supportive language (GMSL) in math teaching moments (responding to student mistakes, introducing/debriefing tasks). Encouragingly, after prompt-engineering, LLMs showcased the ability to reframe unsupportive utterances effectively, even surpassing GMSL-trained teachers in certain evaluations. These collective insights demonstrate the promise of LLMs in complementing existing teacher professional learning processes. However, the current limitations also emphasize the continued need for human expertise in improving and overseeing the model's implementation. Dora Demszky is an Assistant Professor in Education Data Science at Stanford University, and in Computer Science (by courtesy). Her research focuses developing and deploying tools that combine natural language processing, linguistics and input from practitioners to facilitate equitable, student-centered instruction. Her tools analyze educational discourse such as student-teacher interactions, student group work and textbooks, to identify features of of high-quality instruction and suggest areas of improvement for educators. Dr Demszky has received her PhD in Linguistics at Stanford.
Add to Calendar 04/18/24 14:00:00 04/18/24 14:00:00 America/New_York VCAI Seminar: Dora Demszky (Stanford) Large Language Models for Teacher Feedback Click here to attend Zoom Large Language Models show unprecedented potential in scaling many aspects of education — can they also facilitate high-quality instruction by serving as a professional learning tool for teachers? This talk synthesizes findings from three papers that evaluate the ability of LLMs to provide feedback to educators on their discourse. Firstly, we evaluate the ability of GhatGPT to perform instructional coaching tasks, such as using an observational instrument to score instruction and provide teachers with suggestions for improvement. We show that while the model generates feedback that is relevant, it struggled with insightfulness. Secondly, we explore the use of LLMs in identifying punitive and restorative classroom management practices. We demonstrate that the LLM-powered measures can surface fine-grained patterns of escalation and racial disparities in classroom management at scale. Lastly, we examine LLMs ability to use growth mindset supportive language (GMSL) in math teaching moments (responding to student mistakes, introducing/debriefing tasks). Encouragingly, after prompt-engineering, LLMs showcased the ability to reframe unsupportive utterances effectively, even surpassing GMSL-trained teachers in certain evaluations. These collective insights demonstrate the promise of LLMs in complementing existing teacher professional learning processes. However, the current limitations also emphasize the continued need for human expertise in improving and overseeing the model's implementation. Dora Demszky is an Assistant Professor in Education Data Science at Stanford University, and in Computer Science (by courtesy). Her research focuses developing and deploying tools that combine natural language processing, linguistics and input from practitioners to facilitate equitable, student-centered instruction. Her tools analyze educational discourse such as student-teacher interactions, student group work and textbooks, to identify features of of high-quality instruction and suggest areas of improvement for educators. Dr Demszky has received her PhD in Linguistics at Stanford. Brendan Iribe Center false