Instructions to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Use Docker

docker model run hf.co/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Ollama
How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with Ollama:
```
ollama run hf.co/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
```
Unsloth Desktop

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with Docker Model Runner:
```
docker model run hf.co/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M
```

Lemonade

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.GLM-5.3-Flash-Coder-RCO-152E-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash-Coder-RCO-152E-GGUF

New release (October 2026). This repository used to hold REAP144, a 144-expert version calibrated on Python agentic sessions only. REAP144 looped on general questions and on other programming languages and could get simple arithmetic wrong. It is replaced by RCO-152E: 152 of 288 experts per layer, chosen by optimization on a mix of agentic sessions, code in 21 languages, math and general text, then healed. The old REAP144 files were removed; they remain available in the repository history.

GLM-5.3-Flash (321B MoE) with 152 of 288 experts per layer, built for coding and agentic use on a home PC. 321B-A18B becomes ~170B-A17B.

Recipe, mask, patches and evaluation harness: github.com/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E

What changed from REAP144

	REAP144 (old)	RCO-152E (this release)
Loop test: problems in 30 hard instruction-following runs	22	4 (0 repetition loops)
17 x 23, 1234 x 5678, Fibonacci, small executed code task, tool call	arithmetic errors	pass at every quant (Q3_K missed Fibonacci once in one run)
Experts per layer	144	152 (incl. 8 generalists)
Calibration	Python agentic sessions	agentic sessions + code in 21 languages + math + 15% general text

Files

Quant	Size	Fits (GPU VRAM + RAM)	KLD to Q8
Q8_0	180.4 GB (4 parts)	24 + 192, multi-GPU	0
Q4_K_M	102.6 GB (3 parts)	24 + 96, 32 + 96, 128 GB unified	0.024
Q3_K	75.1 GB (2 parts)	24 + 64, 32 + 64	0.045
IQ2_XXS (GSQ)	~47 GB	24 + 32	in progress

KLD: llama.cpp llama-perplexity --kl-divergence against the Q8_0 of this model on held-out code and agentic text. Same top token as Q8: Q4_K_M 94.5%, Q3_K 92.5%.

A 2-bit version for 24 GB VRAM + 32 GB RAM is in progress (GSQ vector quantization of the experts + quantization-aware healing) and will be added here when it passes the same tests.

Results (Q8_0, no special settings)

Against the full GLM-5.3-Flash (Q8), same prompts, default sampling:

Benchmark	RCO-152E	Full model
AIME 2025 (20)	19	20
MATH-500 subset (6)	6	6
LiveCodeBench subset (6)	6	6
IFEval (100)	71	83

Agentic results (51 DeepSWE tasks, same harness as the full model) will be added here when the run completes.

Held-out KL divergence to the full model (top-20 logits, 256 windows): agentic 0.209, code 0.193, math 0.089, general text 0.594.

Recommended settings

Code and agentic use: works without special settings.
Long prose (essays, poems) and non-English prompts: the model can fall into repetition loops; this is where pruning hurts most. A presence penalty of 1.0 reduces it. This model is not meant for creative writing or other languages. Q3_K is more sensitive here than Q8_0 and Q4_K_M; on code and math it behaves like Q8_0.

Run

Split files load directly; point the runtime at the first part.

pip install -U huggingface_hub
hf download lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF --include "*Q3_K*" --local-dir glm53-rco-q3k

llama.cpp: llama-server -m glm53-rco-q3k/GLM-5.3-Flash-Coder-RCO-152E-Q3_K-00001-of-00002.gguf --jinja -c 131072 with expert tensors offloaded to CPU as needed (--n-cpu-moe).
Maya (made for GLM-5.3-Flash, hottest experts on the GPU, the rest in RAM): git clone https://github.com/mw00/project-maya && cd project-maya && ./maya.sh --gguf-dir ../glm53-rco-q3k --no-vision (add --backend hip on AMD).

How it was made

Calibration mix: agentic coding sessions written by the full model, code in 21 languages, math, and 15% general text, all outside the evaluation sets.
Expert selection with RCO (optimization of the expert mask against the full model's outputs), 152 experts per layer. Against a REAP-style mask of the same size: same code and agentic KL, 22% lower KL on general text, and far fewer loops.
Healing: LoRA on attention and the shared expert, distilled from the full model's top-20 logits (96 steps).
imatrix on the same mix, then Q4_K_M and Q3_K.

Limits

Built and tested for code and agentic use. Pruning removes knowledge outside that domain: creative writing and non-English prompts are the weak spots. Text only: no vision projector (mmproj) is included.

Acknowledgements

RCO and GSQ: IST Austria (ISTA-DASLab).
REAP (Lasby et al., 2025) for the expert-saliency score used in earlier versions.
This project was built with the help of Claude (Anthropic) as a coding and research assistant. The decisions are the author's.

License and credits

MIT, like the base model. Base model by Z.ai. Runtimes: llama.cpp, Maya.

Contact: lapadat.bogdanv@gmail.com or LinkedIn

Downloads last month: 23,078

GGUF

Model size

170B params

Architecture

glm5-next

Hardware compatibility

4-bit

8-bit

Model tree for lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF

Base model

zai-org/GLM-5.3-Flash

Quantized

(199)

this model