Xiaohongshu Open-Sources Continuous Autoregressive TTS Model dots.tts

dots.tts, open-sourced by the Xiaohongshu dots team in collaboration with the X-LANCE lab at Shanghai Jiao Tong University, is a 2B-parameter, fully continuous, end-to-end autoregressive TTS foundation model. Unlike mainstream discrete token approaches, it directly models autoregressively in a continuous latent space and then reconstructs the waveform, with no reliance on discrete acoustic tokens throughout the pipeline.

Discrete representations map continuous speech to a finite vocabulary, creating an information bottleneck where different acoustic details may fall into the same index, losing information that is hard to recover. Reconstruction evaluations show that four discrete representations achieve PESQ-NB of 2.40-2.92 and SIM of 0.68-0.85, while three continuous representations achieve PESQ-NB of 3.99, 4.23, and 4.09, and SIM of 0.963, 0.950, and 0.969, with continuous representations clearly leading in fidelity.

The challenge of continuous autoregression lies in error accumulation: continuous latent variables lack quantization constraints, so a previous step's deviation enters the subsequent history, potentially causing howling, timbre drift, or text misalignment in long sentences. dots.tts mitigates this through AudioVAE semantic training and a causal Semantic Encoder, which shape a smooth, low-noise latent space and extract stable semantic history, reducing long-range error accumulation.

On the Seed-TTS-Eval zero-shot cloning benchmark, the dots.tts SOAR version achieves WER/CER of 0.94%, 1.30%, and 6.60% on Chinese, English, and Chinese hard sets, respectively, with SIM of 81.0, 77.1, and 79.5; average SIM is 79.2 and average WER/CER is 2.95%, both best.

On the MiniMax-Speech 24-language test, the self-correcting alignment version achieves an average SIM of 83.9, higher than other systems; dots.tts ranks first in SIM for 19 of 24 languages and ties for first in 2 others.

On EmergentTTS-Eval, using gpt-4o-mini-tts as the reference, the dots.tts SOAR version scores 65.7% in syntactic complexity, higher than all closed-source systems; the Base version scores 72.7% in emotional expressiveness, best among open-source systems.

dots.tts natively supports streaming output, taking the full text as prefix input and generating audio blocks incrementally. For real-time LLM broadcasting scenarios, it supports a 1T1A dual-stream mode where the speech side can start generating as soon as the upstream LLM outputs the first text token, minimizing latency.

In 1T1A scenarios, the audio first-packet latency is as low as 54.4 ms. SGLang-Omni already supports dots.tts inference, achieving up to 16-way parallel on a single GPU on the Seed-TTS-Eval EN test set.

These features make dots.tts suitable for real-time voice agents, duplex conversations, and other interactive scenarios, balancing generation quality with engineering practicality.

dots.tts.edit builds on the base model and supports four types of edits—text, emotion, prosody, and pause—which can be combined. It leverages the continuous autoregressive architecture to unify source text, source speech, edit instructions, and target text as conditions, generating edited speech in one pass.

The team built a bilingual evaluation suite, doteBench, containing 2,081 cases (1,841 single edits and 240 combined edits), assessing instruction following, local preservation, and overall audio quality. dots.tts.edit leads overall across five task types.

On Seed-TTS-Eval, dots.tts.edit's recognition error rate and speaker similarity are close to the base model, indicating that the same generator can retain both zero-shot synthesis and precise editing capabilities.

dots.tts open-sources six checkpoints covering base training, self-correcting alignment, MeanFlow four-step/two-step/one-step distillation, and 1T1A dual-stream mode, along with training, inference, fine-tuning, and distillation code under the Apache 2.0 license.

Developers can synthesize and clone via CLI, Python API, or streaming interfaces. The repository provides fine-tuning entry points, training configurations, data manifest formats, and distillation scripts, covering the complete path from invocation to continued training.

This open-source strategy allows researchers to reproduce the continuous autoregressive approach, and product teams to perform domain adaptation and latency optimization, providing a sustainable 'foundation' form for large-scale speech synthesis models.