swiss-ai

/

Apertus-8B-2509

like 153

Follow

Swiss AI Initiative 1.09k

there is a quantization converted to MLX by mlx-community 👉 https://huggingface.co/mlx-community/Apertus-8B-2509-4bit, this can be used in LM Studio

Somehow on my Mac mini (M4, 2024) it doesn't run.

GGUF My Repo don't work too.

could you try again now that is has been merged into MLX-LM officially? https://github.com/ml-explore/mlx-lm/releases/tag/v0.27.1

Tried it since launch and just now again, with the latest update to LMS 0.3.25 (build 2) i still get the same error with the available MLX versions from mlx-community and NexVeridian.

Error when loading model: ValueError: Model type apertus not supported.

yes this seems it's not using mlx 27.1 yet but still 27 internally

in the meantime i guess you have to use MLX-LM directly, or wait till next week's next LMstudio release. i created an issue here: https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/975

The MLX version for LM Studio is throwing this error:

You're welcome to try and convert it, but for me so far no good.

Today when starting my installed LMS 0.3.25 (build 2) came update to the LM Runtime to 0.26.1 with mlx-lm 0.27.1. Now Apertus starts without errors and runs! Great 👍😎😊

Hi,

I tried to load the model Apertus-8B-2509 in LM Studio (Windows, version 0.3.27 build 4), but I get the following error:
Failed to load model
error loading model: error loading model architecture: unknown model architecture: 'apertus'

From what I understand, LM Studio on Windows uses llama.cpp/GGUF under the hood, and it seems the ApertusForCausalLM architecture is not yet supported there. On macOS with MLX Runtime this might work, but Windows doesn’t have MLX.

Questions:

Is there already an official GGUF export or roadmap for Apertus so it can run in LM Studio / llama.cpp on Windows?
If not, would you recommend using transformers + PyTorch/VLLM for now instead of trying LM Studio?
Are there community-converted GGUF weights that you would consider “safe” to use, or is that not recommended yet?

Thanks for the clarification and all your great work on this model!

Recent updates to the llama.cpp library have added needed support for ApertusForCausalLM, and community quantizations run well in various software. My blog post has more details.

Fresh coverage on the use of Apertus in LM Studio with Apple hardware in my blog: https://log.alets.ch/119/