Pretraining

OPUS: How to Train LLMs 6x Faster by Choosing the Right Data

OPUS scores every sample by alignment minus redundancy in optimizer update space; on Qwen3-8B it beat full 3B-token training using 0.5B, at 4.7% extra compute.