Hugging Face's logo

openeurollm
/
tokenizer-256k-v2

Transformers
tokenizer
sentencepiece
bpe
multilingual
openeurollm
Model card Files
xet
 Community
tokenizer-256k-v2 / tokenizer.model
timpal0l's picture
timpal0l
Ship real HF format: LlamaTokenizerFast + frozen tokenizer.json
c6ca0b9 verified
Download
History Blame Contribute Delete
4.58 MB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
82758f2f31c5b881355e82b9710125122489047f6ae931371dda9cde60b0f7d9
Size of remote file:
4.58 MB
·
SHA256:
a352df5cda48dd73b30bd94bc15b6ca03fe0ca5944bdea4870d47c790f738c47

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.