openeurollm
/

tokenizer-256k-v2

Model card Files Files and versions

tokenizer-256k-v2 / tokenizer.model

Commit History

Ship real HF format: LlamaTokenizerFast + frozen tokenizer.json

c6ca0b9
verified

timpal0l commited on 28 days ago

OpenEuroLLM Tokenizer v2 (256k) — initial upload

a25c453
verified

timpal0l commited on Jun 27