Machine Learning

Memorization in Large Language Models

Curriculum Learning: Scores, Plans, Dynamics, and NLP

Abstract: Curriculum learning is an effective and natural strategy in human learning. It plays an important role in challenging tasks such as language learning. However, current machine learning (ML) paradigms are mostly built upon repeatedly practicing the same training data/tasks with a random order, which is non-adaptive to the learning process. Moreover, they do not plan multiple learning stages in advance as humans.