Model Size
#6
by
bardia79mhd
- opened
Hello,
The model's size on the card indicates 3B. Can you please confirm if this is correct?
Thanks!
Hello, it seems HF doesn't understand the weight packing. The embeddings and lm_head are left a BF16 and account for ~1B parameters. The other ~7B parameters are packed into INT32 tensors (4 8-bit parameters per value), so the HF counts them as ~2B parameters.