Context Language Models: Self-Managing Context to Improve Performance and Reduce Compute Costs

Researchers from Meta, MIT, and the University of Washington introduce Context Language Models (CLMs), a new approach that enables language models to manage and edit their own context rather than relying on predefined mechanisms for summarization, compression, and information retrieval. The work reports substantial gains in both performance and computational efficiency.