Quantized Models (GGUF, IQ, Imatrix)
Collection
Various GGUF quantizations of small models. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 98 items • Updated
• 69
My GGUF-IQ-ARM-Imatrix quants for inflatebot/MN-12B-Mag-Mell-R1.
Mag Mell R1 was tested with Temp 1.25 and MinP 0.2. This was fairly stable up to 10K, but this might be too "hot". If issues with coherency occur, try increasing MinP or decreasing Temperature.
Use the ChatML prompt format.
- Downloads last month
- 278
Hardware compatibility
Log In to add your hardware
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix
Base model
inflatebot/MN-12B-Mag-Mell-R1