Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning Paper • 2512.24265 • Published 18 days ago • 4
Does your data spark joy? Performance gains from domain upsampling at the end of training Paper • 2406.03476 • Published Jun 5, 2024 • 4
Running 132 TxT360: Trillion Extracted Text 📖 132 Explore and analyze the TxT360 dataset for LLM pre-training
MAGA: MAssive Genre-Audience Reformulation to Pretraining Corpus Expansion Paper • 2502.04235 • Published Feb 6, 2025 • 23