Vela-Lumen-31M
A 31M-parameter language model trained from scratch on a single 8GB laptop GPU in under 8 hours.
Downloads
| Format |
File |
Size |
How to Use |
| GGUF F16 |
vela-lumen-31m-f16.gguf |
85 MB |
Works with Ollama, llama.cpp, LM Studio |
| Safetensors FP16 |
model-fp16.safetensors |
84 MB |
Half precision, Python |
| Safetensors FP32 |
model.safetensors |
167 MB |
Full precision, training |
Quick Start (Ollama)
wget https://huggingface.co/ParallaxOpen/Vela-Lumen-31M/resolve/main/vela-lumen-31m-f16.gguf
ollama create vela-lumen-31m -f Modelfile
ollama run vela-lumen-31m
ELO Rating
| Metric |
Value |
| Estimated ELO |
~900-950 |
| GSM8K |
3.0% |
| Params |
31.3M |
Comparison to Known Models
| Model |
Params |
GSM8K |
ELO |
| Claude Fable 5 |
Unknown |
Unknown |
1507 |
| Llama 3.2 1B |
1.26B |
7.6% |
1111 |
| SmolLM2-135M |
135M |
1.4% |
N/A |
| SmolLM2-360M |
360M |
3.2% |
N/A |
| Vela-Lumen-31M |
31.3M |
3.0% |
~900-950 |
What This Means
Vela-Lumen-31M is a proof of concept trained from scratch on consumer hardware:
- Training from scratch on a single GPU is possible
- Small models (31M) can learn some math patterns
- More data and larger models needed for competitive performance
- ELO ~900-950 is below leaderboard models, but expected for 31M from scratch
Model Details
| Property |
Value |
| Developer |
ParallaxOpen |
| Parameters |
31,265,792 |
| Architecture |
Transformer Decoder, GQA 8:4, SwiGLU, RMSNorm, RoPE |
| Context |
512 tokens |
| Vocab |
24,189 |
| License |
CC BY-NC 4.0 |
Training
- Pretrain: 1.5B tokens, 40K steps, loss 4.0, ~4h on RTX 5060
- SFT: 780K ChatML samples, 64K steps, loss 1.54, ~4h on RTX 5060
- Total: 8 hours on a single 8GB laptop GPU
Citation
@software{vela-lumen-31m,
title={Vela-Lumen-31M: A 31M-parameter Language Model Trained from Scratch},
author={ParallaxOpen},
year={2026},
note={Trained on a single RTX 5060 Laptop GPU in 8 hours}
}
