File size: 5,911 Bytes
e466e91
9b48181
 
e466e91
 
 
9b48181
e466e91
 
9b48181
e466e91
 
 
9b48181
e466e91
 
 
 
 
9b48181
 
e466e91
 
 
 
 
 
 
 
 
9b48181
e466e91
e9b5f66
 
 
 
 
 
 
e466e91
 
 
e9b5f66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e466e91
e9b5f66
e466e91
e9b5f66
 
 
 
 
e466e91
 
 
e9b5f66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e466e91
 
 
 
 
e9b5f66
 
 
 
 
 
 
 
 
 
 
 
e466e91
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e9b5f66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e466e91
e9b5f66
 
 
 
 
 
 
e466e91
e9b5f66
fa89d64
 
 
e9b5f66
fa89d64
 
 
e9b5f66
 
 
 
 
 
fa89d64
 
 
 
 
 
 
e9b5f66
 
 
fa89d64
 
 
 
e9b5f66
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
---
language:
- es
license: apache-2.0
base_model: openai/whisper-large
tags:
- whisper-event
- generated_from_trainer
datasets:
- mozilla-foundation/common_voice_13_0
metrics:
- wer
model-index:
- name: Whisper Large Spanish
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: mozilla-foundation/common_voice_13_0 es
      type: mozilla-foundation/common_voice_13_0
      config: es
      split: test
      args: es
    metrics:
    - name: Wer
      type: wer
      value: 5.126477928109984
---

# Whisper Large Spanish

## Model summary

**Whisper Large Spanish** is a high-accuracy automatic speech recognition (ASR) model for **Spanish (es)**, fine-tuned from [openai/whisper-large] on the **Spanish subset of Mozilla Common Voice 13.0**. It achieves a **Word Error Rate (WER) of 5.1265%** on the evaluation set.

This model is designed for applications that require near state-of-the-art transcription accuracy in Spanish, such as transcription of lectures, podcasts, and other high-quality recordings.

---

## Model description

* **Architecture:** Transformer-based encoder–decoder (Whisper Large)  
* **Base model:** openai/whisper-large  
* **Language:** Spanish (es)  
* **Task:** Automatic Speech Recognition (ASR)  
* **Output:** Text transcription in Spanish  
* **Decoding:** Autoregressive sequence-to-sequence decoding  

Large model offers very high accuracy at the cost of higher computational requirements compared to Medium or Small variants.

---

## Intended use

### Primary use cases

* High-accuracy Spanish speech transcription  
* Applications requiring transcription of long-form audio  
* Research in Spanish ASR performance and benchmarking  

### Limitations

* May underperform in extremely noisy audio or with strong regional accents not well represented in the Common Voice dataset  
* High computational cost for real-time inference  
* Not suitable for legal, medical, or safety-critical applications without human review  

---

## Training and evaluation data

* **Dataset:** Mozilla Common Voice 13.0 (Spanish subset)  
* **Data type:** Crowd-sourced read speech  
* **Preprocessing:**  
  * Audio resampled to 16 kHz  
  * Text tokenized using Whisper tokenizer  
  * Removal of invalid or corrupted samples  

* **Evaluation metric:** Word Error Rate (WER) on held-out evaluation set  

---

## Evaluation results

| Metric     | Value      |
| ---------- | ---------- |
| WER (eval) | **5.1265%** |

---

## Training procedure

### Training hyperparameters

* Learning rate: 1e-5  
* Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)  
* LR scheduler: Linear  
* Warmup steps: 500  
* Training steps: 20000  
* Train batch size: 32 (gradient accumulation 2 → effective batch size 64)  
* Eval batch size: 16  
* Seed: 42  

### Training results (summary)

| Training Loss | Epoch | Step  | Validation Loss | WER    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 0.0834        | 2.0   | 1000  | 0.1862          | 6.3852 |
| 0.0871        | 4.0   | 2000  | 0.1777          | 5.9175 |
| 0.039         | 6.0   | 3000  | 0.1780          | 5.7423 |
| 0.0265        | 8.0   | 4000  | 0.2121          | 5.7744 |
| 0.0059        | 10.0  | 5000  | 0.2219          | 5.8097 |
| 0.0855        | 12.01 | 6000  | 0.1839          | 5.9778 |
| 0.0037        | 14.01 | 7000  | 0.2273          | 5.8565 |
| 0.0293        | 16.01 | 8000  | 0.1965          | 5.8078 |
| 0.1174        | 18.01 | 9000  | 0.1984          | 5.8893 |
| 0.0355        | 20.01 | 10000 | 0.2136          | 5.8662 |
| 0.0279        | 22.01 | 11000 | 0.1882          | 5.4960 |
| 0.0043        | 24.01 | 12000 | 0.2444          | 5.3356 |
| 0.0302        | 26.01 | 13000 | 0.2223          | 5.4620 |
| 0.0011        | 28.01 | 14000 | 0.2603          | 5.5608 |
| 0.001         | 30.01 | 15000 | 0.2452          | 5.3087 |
| 0.0003        | 32.01 | 16000 | 0.2573          | 5.3523 |
| 0.0004        | 34.02 | 17000 | 0.2690          | 5.2952 |
| 0.0013        | 36.02 | 18000 | 0.2373          | 5.1438 |
| 0.0004        | 38.02 | 19000 | 0.2618          | 5.1361 |
| 0.0004        | 40.02 | 20000 | 0.2663          | 5.1265 |

---

## Framework versions

- Transformers 4.33.0.dev0  
- PyTorch 2.0.1+cu117  
- Datasets 2.14.4  
- Tokenizers 0.13.3  

---

## Example usage

```python
from transformers import pipeline

hf_model = "HiTZ/whisper-large-es"  # replace with actual repo ID
device = 0  # -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])
```

---

## Ethical considerations and risks

* This model transcribes speech and may process personal data.
* Users should ensure compliance with applicable data protection laws (e.g., GDPR).
* The model should not be used for surveillance or non-consensual audio processing.

---

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```

Please, check the related paper preprint in
[arXiv:2503.23542](https://arxiv.org/abs/2503.23542)
for more details.

---

## License

This model is available under the
[Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
You are free to use, modify, and distribute this model as long as you credit
the original creators.

---

## Contact and attribution

* Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
* Base model: OpenAI Whisper
* Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.