Instructions to use BharathYedla/PedzSTAR-AVD-WavLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use BharathYedla/PedzSTAR-AVD-WavLM with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("BharathYedla/PedzSTAR-AVD-WavLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
PedzSTAR-AVD-WavLM (v1.0-paper)
PedzSTAR Lab β Adult Voice Detector, WavLM Base Plus fine-tune.
βΆ In plain words: this model listens to a child's clip and, for every 1/50 of a second (20 ms), says how likely it is that a child, an adult, or a non-voice noise is sounding. Its highest adult score anywhere in the clip ("WavLM adult max") was the main evidence used to decide that a clip is child-only (below 0.2) or contains an adult (0.8 or higher, together with an active examiner microphone or a strong warning sign).
What is in this repository
| Path | Contents |
|---|---|
checkpoints/fold0 β¦ fold4 |
One model per held-out child fold (the models behind the paper's numbers) |
checkpoints/all_data |
Trained on all folds: use this one for new recordings |
| each checkpoint | model.safetensors (WavLM + head, 95.3 M weights, 381 MB), config.json (WavLM config), avd_config.json |
avd_wavlm.py |
Standalone loader + scorer (frame probabilities and clip summaries) |
Converted checkpoints reproduce the study's saved scores exactly (max absolute difference 0.0 on 40 clips per model; GPU, bfloat16 as in the study).
How to use
from avd_wavlm import AVDWavLM
m = AVDWavLM.from_pretrained("checkpoints/all_data") # GPU if available
s = m.score_file("clip.flac") # mono, resampled to 16 kHz if needed
print(s["wl_adult_max"], s["frames"].shape) # frames: (n_frames, 3) = child, adult, noise
CLI: python avd_wavlm.py --model checkpoints/all_data a.flac b.flac > scores.csv
Training data (described; no recordings are included)
All training audio comes from the PedzSTAR GFTA-3 recordings (secured-access research data; sites ATL, BLT, DLS, ISN, LAX, ORL, STL). Children's clips were cut from ~12.7-minute sessions recorded with a room microphone (Zoom H6 XY); the examiner also wore a lavalier microphone.
| Ingredient | Amount |
|---|---|
| Child-only base clips (from the verified first-stage child-only list) | 1,031 |
| Real examiner speech segments, cut from the full sessions between child clips | 6,372 (0.61 h, 116 sessions) |
| Extra adult speech: LibriSpeech dev-clean (40 speakers, CC BY 4.0), room-simulated | 1,768 segments |
| Room noises (taps, bumps) and room tone from the sessions | 7,628 segments |
| Human-labelled clips (two listeners) | 300 (85 with an adult: 81 two-speaker, 4 adult-only) |
| Human-verified child-only clips (blind listening check) | 150 |
Practice mixtures were built from a child base clip plus one of: nothing; a noise; another clip of the same child; adult speech cut off at the end, cut off at the start, a quiet background murmur (β32 to β15 dB), overlapping speech, or a separate adult turn. Added adult speech had to be β₯ 8 dB above the session's noise floor. Children were split into 5 site-balanced folds (177 children); each fold model never saw its held-out children, their examiners' segments or their noises.
Training procedure
| Setting | Value |
|---|---|
| Base | microsoft/wavlm-base-plus (94,381,936 weights; pre-trained on 94k h of unlabelled speech) |
| Trained | 90,181,488 WavLM weights (convolutional front-end frozen; layer-drop 0) + new head 920,336 weights |
| Head | softmax-weighted mix of all 13 hidden layers β bidirectional LSTM (128 per direction) β 3 sigmoid outputs per 20 ms frame |
| Batch | each step: 24 freshly generated mixtures + 8 real clips (labelled or verified), β€ 4 s |
| Steps | 8,000 per model (β192,000 unique mixtures seen) |
| Loss | per-frame binary cross-entropy (weights child 1, adult 3, noise 1) + 0.5 Γ clip-level adult loss (log-sum-exp over frames, sharpness 3; real clips weighted Γ 2) |
| Optimiser | AdamW, weight decay 0.01; learning rate 2e-5 (WavLM) and 1e-3 (head); 300-step warm-up then linear decay to 5 %; gradient clipping 5.0 |
| Precision, hardware | bfloat16 autocast; 1 Γ NVIDIA H100; β8.3 min and 6.6 GB per model |
| Software | torch 2.11.0+cu128, transformers 5.23.0.dev0 |
Evaluation (held-out children; 300 labelled clips, 85 with an adult)
| Measure | Result |
|---|---|
| AUC, WavLM adult max (out-of-fold) | 0.938 (95 % range 0.90β0.97) |
| At adult max β₯ 0.5 | 71 of 85 adult clips caught; 24 of 215 child-only flagged; 11 of 150 verified-clean flagged |
| At adult max β₯ 0.2 (the rescue limit) | 76 of 85 caught; 41 of 215 child-only flagged; 16 of 150 verified-clean flagged |
| Child-only clips kept while catching all 85 adult clips | 33 of 215 (thresholds set on the same labels) |
| Adult-confirmed rule: adult max β₯ 0.8 and either the examiner microphone is active (β₯ 3 strong frames in a session where it works normally) or one of three strong warning signs fires (examiner-style phrase; Nemotron hears a second voiced speaker; Nemotron says 2+ people or an adult) | right on 61 of 63 labelled clips (97 %) |
Adult clips caught at threshold 0.5, by kind:
| Kind | Clips | WavLM | Nemotron | Either |
|---|---|---|---|---|
| At the same time | 33 | 31 | 30 | 32 |
| Both kinds in one clip | 33 | 28 | 31 | 33 |
| One after the other | 15 | 10 | 13 | 13 |
| Adult only (no child) | 4 | 2 | 3 | 3 |
WavLM is strongest when the adult talks at the same time as the child, and weaker for adult-only clips and adults speaking in a separate turn. Its practice mixtures had few such cases. In the study the fine-tuned Nemotron covered that gap. Use in the study: 2,454 of 4,401 files were listed as child-only. Two blind listening checks found no adult in 300 random clips, so at most 2.5 % of the list contains an adult (95 % confidence).
Limitations
- Only 300 labelled clips; one task (GFTA-3), one room-microphone setup, children from 7 US sites.
- Practice mixtures had no adult-only clips and few adult-led ones.
- Frame timing ("at the same time" vs "one after the other") matched listeners in 49 of 58 vs 3 of 13 labelled cases.
Privacy, access and responsible use
- This model was trained on secured-access recordings of young children and their examiners. Fine-tuned models can, in rare cases, reveal traces of their training data. Keep this repository private/gated; share only with people covered by the PedzSTAR data use agreement, and make it public only after PI/IRB/data-governance approval.
- Do not try to recover training audio or transcripts from the weights.
- No audio, transcripts, participant IDs or file names are stored in this repository.
Licences
- WavLM Base Plus is distributed by Microsoft (UniSpeech repository) under Creative Commons Attribution-ShareAlike 3.0 Unported, so this adaptation is shared under the same licence (
LICENSE_WavLM_UniSpeech_CC-BY-SA-3.0.txt). Base model: Chen et al. (2022), WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing. - LibriSpeech segments used in training: CC BY 4.0 (Panayotov et al., 2015).
Citation
PedzSTAR Lab (2026). PedzSTAR-AVD: adult-voice detection for child speech recordings (manuscript in preparation). Please cite the manuscript once published, plus the base model.
Model tree for BharathYedla/PedzSTAR-AVD-WavLM
Base model
microsoft/wavlm-base-plus