Post
3434
I've written a technical blog post about how we create a multimodal model:
Kairos: a multimodal model built with LFM2.5-2.6B as the LLM, MoonViT-3D (the vision tower of Kimi-K2.6) as the vision encoder, and a custom projector.
The original plan was LLaVA's approach, two stages: first align the projector with the LLM frozen, and then train the projector + LLM together. The first stage worked in terms of loss (ablation with +3.7 nats in favor of the image), but in free generation the image shifted the logits without changing the argmax: the model received the image and ignored it. That's why we jumped directly to early fusion, with a reasoning dataset.
For that, we created Kairos-Multimodal-Reasoning: 116,357 examples with explicit reasoning traces, generated through distillation (60,041 from LLaVA-CC3M, 2,295 from WebSight, and 54,021 from Zebra-CoT), with GPT 5.6 Luna, Inkling, Qwen 3.6 27B, and Qwen 3.7 Plus as teachers.
The training, in two phases:
1. Projector through backbone with 80k image-text pairs (Kairos-Proj-80k).
2. Projector + LoRA (r=16) with 30k examples from the reasoning dataset (Kairos-Alig-30k).
Everything is open source:
- Full blog post with the process: https://aquiles-ai.vercel.app/blog/kairos-a-multimodal-model
- Implementation: https://github.com/Aquiles-ai/Kairos
To be honest: the checkpoints are not a competent model, they are experimental artifacts. But they validated the approach and precisely defined what the next iteration needs.
https://huggingface.co/collections/Aquiles-ai/kairos
Aquiles-ai/MoonViT-3D
LiquidAI/LFM2.5-2.6B
Kairos: a multimodal model built with LFM2.5-2.6B as the LLM, MoonViT-3D (the vision tower of Kimi-K2.6) as the vision encoder, and a custom projector.
The original plan was LLaVA's approach, two stages: first align the projector with the LLM frozen, and then train the projector + LLM together. The first stage worked in terms of loss (ablation with +3.7 nats in favor of the image), but in free generation the image shifted the logits without changing the argmax: the model received the image and ignored it. That's why we jumped directly to early fusion, with a reasoning dataset.
For that, we created Kairos-Multimodal-Reasoning: 116,357 examples with explicit reasoning traces, generated through distillation (60,041 from LLaVA-CC3M, 2,295 from WebSight, and 54,021 from Zebra-CoT), with GPT 5.6 Luna, Inkling, Qwen 3.6 27B, and Qwen 3.7 Plus as teachers.
The training, in two phases:
1. Projector through backbone with 80k image-text pairs (Kairos-Proj-80k).
2. Projector + LoRA (r=16) with 30k examples from the reasoning dataset (Kairos-Alig-30k).
Everything is open source:
- Full blog post with the process: https://aquiles-ai.vercel.app/blog/kairos-a-multimodal-model
- Implementation: https://github.com/Aquiles-ai/Kairos
To be honest: the checkpoints are not a competent model, they are experimental artifacts. But they validated the approach and precisely defined what the next iteration needs.
https://huggingface.co/collections/Aquiles-ai/kairos
Aquiles-ai/MoonViT-3D
LiquidAI/LFM2.5-2.6B