tiiuae/Falcon-H1-Tiny-90M-Instruct-GGUF

Table of Contents

0. TL;DR
1. Model Details
2. Training Details
3. Usage
4. Evaluation
5. Citation

TL;DR

Model Details

Model Description

• Developed by: https://www.tii.ae
• Model type: Causal decoder-only
• Architecture: Hybrid Transformers + Mamba architecture
• Language(s) (NLP): English
• Number of Parameters: 90M
• License: Falcon-LLM License

Training details

For more details about the training protocol of this model, please refer to the Falcon-H1-Tiny technical blogpost.

Usage

Currently to use this model you can either rely on Hugging Face transformers, vLLM, sglang, llama.cpp, ollama or mlx library.

Inference

🤗 transformers

Refer to the snippet below to run H1 models using 🤗 transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-Tiny-90M-Instruct-pre-DPO"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

or

transformers serve tiiuae/Falcon-H1-Tiny-90M-Instruct-pre-DPO

llama.cpp

You can find all GGUF files compatible with llama.cpp under our official collection - an example setup could be:

brew install llama.cpp 
pip install huggingface_hub 
hf download tiiuae/Falcon-H1-Tiny-90M-Instruct-pre-DPO Falcon-H1-Tiny-90M-Instruct-pre-DPO-Q8_0.gguf --local-dir ./ 
llama-cli ./ Falcon-H1-Tiny-90M-Instruct-pre-DPO-Q8_0.gguf -cnv 

ollama

ollama run hf.co/tiiuae/Falcon-H1-Tiny-90M-Instruct-GGUF:Q8_0 

Apple mlx

mlx_lm.chat --model tiiuae/Tiny-H1-SF 

vLLM

For vLLM, simply start a server by executing the command below:

# pip install vllm>=0.9.0
vllm serve tiiuae/Falcon-H1-Tiny-90M-Instruct-pre-DPO --tensor-parallel-size 2 --data-parallel-size 1

sglang

python -m sglang.launch_server \
  --model ttiiuae/Falcon-H1-Tiny-90M-Instruct-pre-DPO \
  --tensor-parallel-size 1 

Evaluation

For detailed evaluation of Tiny-H1 series, please refer to our technical blogpost

Useful links

• View our release blogpost.
• Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

Citation

If the Falcon-H1-Tiny family of models were helpful to your work, feel free to give us a cite.

@misc{falcon_h1_tiny,
  title={Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
  author={Falcon-LLM Team},
  year={2026}, 
}