MOSS-TTS-Nano-100M-ONNX-WebGPU

OpenMOSS-Team/MOSS-TTS-Nano-100M with its weights quantized for running in a web browser with ONNX Runtime Web, as used by Quick Inference: the graphs of MOSS-TTS-Nano-100M-ONNX and MOSS-Audio-Tokenizer-Nano-ONNX that plain text-to-speech with a built-in voice needs.

Changes from the original

• All graphs quantized without calibration (round to nearest, symmetric, blocks of 32): matrix multiplications to 4-bit MatMulNBits, embedding lookups to 4-bit GatherBlockQuantized. Activations stay float32.
• onnx/moss_tts_prefill_q4.onnx and onnx/moss_tts_decode_step_q4.onnx (WebGPU) share one weights file, onnx/moss_tts_global_shared_q4.data.
• onnx/moss_tts_local_fixed_sampled_frame_cpu.onnx (the frame sampler) and onnx/moss_audio_tokenizer_decode_step_cpu.onnx (the codec's streaming decoder) run on ONNX Runtime Web's CPU engine, faster than on WebGPU for this small, sequential work. That engine has no 4-bit lookups, so their lookup tables are float16 (the same int4 values, dequantized; tools/export/dequantize_gathers.py).
• tokenizer.json: the SentencePiece tokenizer.model as a Hugging Face BPE tokenizer (byte fallback; ids identical on our test texts once the text is NFKC-normalized and its whitespace collapsed, as SentencePiece does).
• moss.json: from the original browser manifests: prompt templates and special ids, generation defaults, the codec's streaming state, and the prompt codes of ten built-in voices (English: Ava, Bella, Adam, Nathan; Chinese: Xiaoyu, Yuewen, Lingyu, Junhao, Zhiming, Weiguo).

These are modified files: their outputs are close to, but not identical to, the original model's. On eight English test sentences (voice Ava, the original's default sampling), Whisper small.en heard it with an 11.4% word error rate (10.6% over three seeds) against 9.1% (9.8%) for the original float32 graphs; speech comes out about 7% shorter.

Use

As the original's onnx_tts_runtime.py with its defaults ("fixed" sampling, streaming decode, the robust text normalizer on): the text normalized and split into chunks of up to 75 tokens; per chunk, the prompt rows (template, the voice's codes, the text) through the prefill, then per 80 ms frame the sampler (given uniform random numbers and the codes seen so far) and the decode step, until the sampler says stop; frames go through the codec's streaming decoder to 48 kHz stereo audio.

Licence

Apache 2.0, as MOSS-TTS-Nano (LICENSE).