Qwen/Qwen3.8-27B
Image-Text-to-Text • 28B • Updated • 666k • • 11.2k
Tay Buley PRO
buley
AI & ML interests
Affective intelligence, emotional classification, distributed inference, edge runtimes
Recent Activity
View all activity
Organizations
liked a
model 1 day ago
liked 2
models 5 days ago
updated a
model 6 days ago
published a
model 6 days ago
liked a
dataset 8 days ago
liked a
model 8 days ago
updated a
model 8 days ago
published a
model 8 days ago
upvoted a collection 9 days ago
liked a
model 9 days ago
liked 4
models 14 days ago
liked a
Space 28 days ago
liked 2
models about 1 month ago
reacted to sergiopaniego's post with 👍 about 1 month ago
Post
7762
Frontier models use distillation as a step of their post-training pipelines.
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
- 3 replies