digiphyte/fluister-turbo-transformers

Fluister (turbo): Transformers / PyTorch

Fluister is a South African Whisper. ("Fluister" is Afrikaans for "to whisper".) This is the Hugging Face Transformers (PyTorch) serialization of the turbo build: fp16 safetensors plus the standard Whisper configs and processor. It holds the same merged weights as the CTranslate2 / faster-whisper build at digiphyte/fluister-turbo; this repo simply ships them in the native Transformers format.

Use this repo when you need a plain Whisper checkpoint: Core ML / WhisperKit conversion (on-device iOS and macOS), Hugging Face transformers inference, or further fine-tuning. For faster-whisper on a server or desktop, use the CT2 repo instead.

It is a fine-tune of OpenAI whisper-large-v3-turbo for Afrikaans and South African English, including the Afrikaans/English code-switching that is everyday SA speech, merged into the base weights. By DigiPhyte (Pty) Ltd, South Africa.

Use (transformers)

import torch
from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="digiphyte/fluister-turbo-transformers",
    torch_dtype=torch.float16,
    device="cuda",  # or "mps" on Apple silicon, "cpu"
)
print(pipe("audio.wav", generate_kwargs={"language": "af"})["text"])  # or "en"

Tell it the language ("af" or "en") rather than relying on auto-detect. For mixed Afrikaans/English conversations, "af" handles the code-switch well.

Core ML / WhisperKit (iOS and macOS)

This is a standard Transformers Whisper checkpoint, so it feeds straight into whisperkittools to produce a WhisperKit Core ML package for on-device Apple inference.

Evaluation

NCHLT read-speech test sets: Afrikaans WER 0.086, English WER 0.017 (identical weights to the CT2 build). Validated on real SA audio: an Afrikaans physiotherapy intake, an English project meeting, and an Afrikaans/English code-switched conversation. Clean Afrikaans where stock Whisper drifts to Dutch spellings ("gebou" not "gebouw", "mense" not "mensen"), intact code-switching, and accurate SA English.

Limitations

Same as the CT2 build. Fluister narrows specific failures (Whisper spelling Afrikaans as Dutch; degrading SA English); it does not change the base model size. Language auto-detect can still mislabel audio (tell it the language), and proper nouns, numbers, and rare or technical terms can still be wrong. South African place names and surnames in particular are a known gap we are still improving.

Licence and attribution

MIT (see LICENSE). This is a derivative work; the base model (OpenAI Whisper, Apache-2.0) and the training data (andreoosthuizen/afrikaans-30s, CC-BY-4.0; NCHLT afr/eng, CC-BY-3.0) are credited in NOTICE.