Post
51
Here is the longer version of the demo video of Phase 2 - Voice Input and Output
Actually the voice input and output were fully implemented and even optimized by removing the heavy GPU-utilizing PyTorch and streamlined the pipelines in the beginning of this month Aug 2026:
Phase 2 focus: Ultra-low latency voice I/O with production-grade optimizations. Both ASR and TTS servers with voice cloning are fully functional, and we've completely streamlined the pipeline for real-time performance.
ASR (SenseVoice + SileroVAD-onnx) utllizes about 700MB RAM when in use (on-demand)
TTS (MioTTS 0.4GB Q4 + C++ inference) pre-occupies about 1.2GB RAM (instead of over 2+ GB before with PyTorch)
What's Actually New in Phase 2:
⚡ Production Voice Pipeline: Decoupled LLM + TTS inference with sub-second latency. The ASR→LLM→TTS chain is now optimized end-to-end.
🚀 Zero PyTorch Overhead: Ditched PyTorch dependencies entirely. Now running C++ optimized MioTTS inference + SileroVAD (ONNX) for lightning-fast voice processing—no deep learning framework bloat.
🌍 Seamless Bilingual Voice: English + Japanese in the same sentence, perfect for learning Nihongo alongside your AI companion.
🔧 Full System Optimization: Pre-warmed modules, systemd TTS server loading, and every millisecond shaved off the inference pipeline.
🛠️ Other features include:
Barge-in interruption - Speak in the middle of the voice output of AI to interrupt
Wake word activation - Wake up AI Waifu using special word/phrase defined in YAML
Speaker verification - Supposed to allow only enrolled users' voices for voice input. Need to tighten the model threshold to stop AI from responding to strangers' voices.
👉 GitHub: https://github.com/OppaAI/Aiko-chan
📹Demo: https://youtu.be/VGHBMMFskCQ
Actually the voice input and output were fully implemented and even optimized by removing the heavy GPU-utilizing PyTorch and streamlined the pipelines in the beginning of this month Aug 2026:
Phase 2 focus: Ultra-low latency voice I/O with production-grade optimizations. Both ASR and TTS servers with voice cloning are fully functional, and we've completely streamlined the pipeline for real-time performance.
ASR (SenseVoice + SileroVAD-onnx) utllizes about 700MB RAM when in use (on-demand)
TTS (MioTTS 0.4GB Q4 + C++ inference) pre-occupies about 1.2GB RAM (instead of over 2+ GB before with PyTorch)
What's Actually New in Phase 2:
⚡ Production Voice Pipeline: Decoupled LLM + TTS inference with sub-second latency. The ASR→LLM→TTS chain is now optimized end-to-end.
🚀 Zero PyTorch Overhead: Ditched PyTorch dependencies entirely. Now running C++ optimized MioTTS inference + SileroVAD (ONNX) for lightning-fast voice processing—no deep learning framework bloat.
🌍 Seamless Bilingual Voice: English + Japanese in the same sentence, perfect for learning Nihongo alongside your AI companion.
🔧 Full System Optimization: Pre-warmed modules, systemd TTS server loading, and every millisecond shaved off the inference pipeline.
🛠️ Other features include:
Barge-in interruption - Speak in the middle of the voice output of AI to interrupt
Wake word activation - Wake up AI Waifu using special word/phrase defined in YAML
Speaker verification - Supposed to allow only enrolled users' voices for voice input. Need to tighten the model threshold to stop AI from responding to strangers' voices.
👉 GitHub: https://github.com/OppaAI/Aiko-chan
📹Demo: https://youtu.be/VGHBMMFskCQ