Vibrato-ai/vibrato-v1

Model Card: Vibrato

Model Details

• Name: Vibrato
• Version: 1.0 (formerly ToneyMultiTask v2.0)
• Type: Multi-task 1D CNN for vocal analysis
• Framework: PyTorch (training), CoreML (deployment)
• License: Apache-2.0
• Repository: vibrato-ai/vibrato-v1

Project

Vibrato is an open vocal-AI model for the singing community, maintained by Anycompany LLC. It powers Toney, a free iOS vocal-training app (App Store link coming at launch), and is published for anyone to use, study, and improve. The training pipeline (model.py, CoreML export) is included in this repository.

Roadmap: v1 (this release) — VocalSet-trained baseline. v2 — expanded training data (augmentation + additional public datasets). v3 — federated learning: on-device fine-tuning with privacy-preserving aggregated updates (voices never leave the phone).

Intended Use

Real-time vocal analysis for singing practice and coaching. Designed to run on-device on iPhones via CoreML Neural Engine.

Primary use: Analyze vocal recordings to classify voice type, singing technique, vowel production, and vocal quality metrics.

Out of scope: Speech recognition, language identification, speaker identification, medical diagnosis of vocal disorders.

Model Architecture

Input: [1, 4000] raw audio (0.25s @ 16kHz)
  -> SharedEncoder (4x ConvBlock with BatchNorm, stride-2 downsampling, AdaptiveAvgPool)
  -> 256-dim feature vector
  -> Voice Head (FC 256->128->6, logits — apply softmax at inference)
  -> Technique Head (FC 256->128->7, logits — apply softmax at inference)
  -> Vowel Head (FC 256->128->6, logits — apply softmax at inference)
  -> Quality Head (FC 256->128->5, sigmoid applied in-model)

Training Data

Dataset	License	Size	Use
VocalSet	CC BY 4.0	10.1 hrs, 20 singers	All four heads (technique, vowel, voice type; quality labels heuristically derived from the audio)

Planned for v2 (not used in v1): Annotated-VocalSet (F0 annotations for pitch-stability labels), SVQTD (human vocal-quality labels).

Attribution: VocalSet by Wilkins et al., Northwestern University. CC BY 4.0.

Output Labels

Voice Types (6)

soprano, mezzo-soprano, alto, tenor, baritone, bass

Techniques (7)

belt, falsetto, vibrato, straight, breathy, nasal, mixed

Vowels (6)

a, e, i, o, u, schwa

Quality Scores (5, continuous 0-1)

brightness, breathiness, strain, power, stability

Limitations

• Trained on ~10 hrs from 20 professional singers (VocalSet) — limited demographic and stylistic coverage; expect degraded accuracy on voice types, languages, and styles outside that distribution.
• 0.25 s context window — no long-range phrase analysis.
• Quality scores are heuristic-derived labels, not clinician annotations.
• Not for speaker ID, speech recognition, or medical use.

Ethical Considerations

• The model does NOT identify individuals from their voice
• No personally identifiable information is stored or inferred
• All inference runs on-device; no audio data is transmitted
• The model is NOT a medical device and should not be used for vocal health diagnosis