dakopi/olmo3-7b_data-repetition

This repo is a collection of SFT checkpoints produced by sweeping unique training samples vs epochs, following the setup from the paper:

Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning https://arxiv.org/abs/2602.11149

Default model

The repo root contains the weights and config for the default variant trained with 16 epochs on 800 samples. Calling from_pretrained(repo_id) loads this checkpoint.

Variants

Each subfolder follows:

s{N}_e{M}

where:

• s{N} means N unique samples
• e{M} means M epochs

Example names:

• s3200_e8 means 3200 unique samples trained for 8 epochs
• s12800_e1 means 12800 unique samples trained for 1 epoch

How to load

Load the default model (root):

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "dakopi/olmo3-7b_data-repetition"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

Load a specific variant (subfolder):

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "dakopi/olmo3-7b_data-repetition"
variant = "s6400_e4"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=variant)
model = AutoModelForCausalLM.from_pretrained(repo_id, subfolder=variant)

Reproducibility and code

Official training and evaluation code: https://github.com/dkopi/data-repetition

Citation

@misc{kopiczko2026datarepetitionbeatsdata,
  title        = {Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning},
  author       = {Dawid J. Kopiczko and Sagar Vaze and Tijmen Blankevoort and Yuki M. Asano},
  year         = {2026},
  eprint       = {2602.11149},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL},
  url          = {https://arxiv.org/abs/2602.11149}
}