File size: 5,911 Bytes
e466e91 9b48181 e466e91 9b48181 e466e91 9b48181 e466e91 9b48181 e466e91 9b48181 e466e91 9b48181 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 e466e91 e9b5f66 fa89d64 e9b5f66 fa89d64 e9b5f66 fa89d64 e9b5f66 fa89d64 e9b5f66 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 |
---
language:
- es
license: apache-2.0
base_model: openai/whisper-large
tags:
- whisper-event
- generated_from_trainer
datasets:
- mozilla-foundation/common_voice_13_0
metrics:
- wer
model-index:
- name: Whisper Large Spanish
results:
- task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: mozilla-foundation/common_voice_13_0 es
type: mozilla-foundation/common_voice_13_0
config: es
split: test
args: es
metrics:
- name: Wer
type: wer
value: 5.126477928109984
---
# Whisper Large Spanish
## Model summary
**Whisper Large Spanish** is a high-accuracy automatic speech recognition (ASR) model for **Spanish (es)**, fine-tuned from [openai/whisper-large] on the **Spanish subset of Mozilla Common Voice 13.0**. It achieves a **Word Error Rate (WER) of 5.1265%** on the evaluation set.
This model is designed for applications that require near state-of-the-art transcription accuracy in Spanish, such as transcription of lectures, podcasts, and other high-quality recordings.
---
## Model description
* **Architecture:** Transformer-based encoder–decoder (Whisper Large)
* **Base model:** openai/whisper-large
* **Language:** Spanish (es)
* **Task:** Automatic Speech Recognition (ASR)
* **Output:** Text transcription in Spanish
* **Decoding:** Autoregressive sequence-to-sequence decoding
Large model offers very high accuracy at the cost of higher computational requirements compared to Medium or Small variants.
---
## Intended use
### Primary use cases
* High-accuracy Spanish speech transcription
* Applications requiring transcription of long-form audio
* Research in Spanish ASR performance and benchmarking
### Limitations
* May underperform in extremely noisy audio or with strong regional accents not well represented in the Common Voice dataset
* High computational cost for real-time inference
* Not suitable for legal, medical, or safety-critical applications without human review
---
## Training and evaluation data
* **Dataset:** Mozilla Common Voice 13.0 (Spanish subset)
* **Data type:** Crowd-sourced read speech
* **Preprocessing:**
* Audio resampled to 16 kHz
* Text tokenized using Whisper tokenizer
* Removal of invalid or corrupted samples
* **Evaluation metric:** Word Error Rate (WER) on held-out evaluation set
---
## Evaluation results
| Metric | Value |
| ---------- | ---------- |
| WER (eval) | **5.1265%** |
---
## Training procedure
### Training hyperparameters
* Learning rate: 1e-5
* Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
* LR scheduler: Linear
* Warmup steps: 500
* Training steps: 20000
* Train batch size: 32 (gradient accumulation 2 → effective batch size 64)
* Eval batch size: 16
* Seed: 42
### Training results (summary)
| Training Loss | Epoch | Step | Validation Loss | WER |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 0.0834 | 2.0 | 1000 | 0.1862 | 6.3852 |
| 0.0871 | 4.0 | 2000 | 0.1777 | 5.9175 |
| 0.039 | 6.0 | 3000 | 0.1780 | 5.7423 |
| 0.0265 | 8.0 | 4000 | 0.2121 | 5.7744 |
| 0.0059 | 10.0 | 5000 | 0.2219 | 5.8097 |
| 0.0855 | 12.01 | 6000 | 0.1839 | 5.9778 |
| 0.0037 | 14.01 | 7000 | 0.2273 | 5.8565 |
| 0.0293 | 16.01 | 8000 | 0.1965 | 5.8078 |
| 0.1174 | 18.01 | 9000 | 0.1984 | 5.8893 |
| 0.0355 | 20.01 | 10000 | 0.2136 | 5.8662 |
| 0.0279 | 22.01 | 11000 | 0.1882 | 5.4960 |
| 0.0043 | 24.01 | 12000 | 0.2444 | 5.3356 |
| 0.0302 | 26.01 | 13000 | 0.2223 | 5.4620 |
| 0.0011 | 28.01 | 14000 | 0.2603 | 5.5608 |
| 0.001 | 30.01 | 15000 | 0.2452 | 5.3087 |
| 0.0003 | 32.01 | 16000 | 0.2573 | 5.3523 |
| 0.0004 | 34.02 | 17000 | 0.2690 | 5.2952 |
| 0.0013 | 36.02 | 18000 | 0.2373 | 5.1438 |
| 0.0004 | 38.02 | 19000 | 0.2618 | 5.1361 |
| 0.0004 | 40.02 | 20000 | 0.2663 | 5.1265 |
---
## Framework versions
- Transformers 4.33.0.dev0
- PyTorch 2.0.1+cu117
- Datasets 2.14.4
- Tokenizers 0.13.3
---
## Example usage
```python
from transformers import pipeline
hf_model = "HiTZ/whisper-large-es" # replace with actual repo ID
device = 0 # -1 for CPU
pipe = pipeline(
task="automatic-speech-recognition",
model=hf_model,
device=device
)
result = pipe("audio.wav")
print(result["text"])
```
---
## Ethical considerations and risks
* This model transcribes speech and may process personal data.
* Users should ensure compliance with applicable data protection laws (e.g., GDPR).
* The model should not be used for surveillance or non-consensual audio processing.
---
## Citation
If you use this model in your research, please cite:
```bibtex
@misc{dezuazo2025whisperlmimprovingasrmodels,
title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
year={2025},
eprint={2503.23542},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
Please, check the related paper preprint in
[arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
for more details.
---
## License
This model is available under the
[Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
You are free to use, modify, and distribute this model as long as you credit
the original creators.
---
## Contact and attribution
* Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
* Base model: OpenAI Whisper
* Dataset: Mozilla Common Voice
For questions or issues, please open an issue in the model repository.
|