CosyVoice

CosyVoice is an open-source streaming TTS large model from Alibaba’s Tongyi Lab (FunAudioLLM), and one of the most prominent enterprise-grade open-source solutions in the field of Chinese speech synthesis models. Unlike previous-generation TTS systems, CosyVoice is built around streaming inference as its core design principle. Based on a large-scale pre-trained audio language model, it begins outputting speech with extremely low initial latency, making it ideal for latency-sensitive scenarios such as voice assistants, real-time conversations, and live streaming.

As a “well-rounded contender” in the open-source TTS space, CosyVoice integrates multiple cutting-edge capabilities: zero-shot voice cloning — replicating a speaker’s timbre with just 3 seconds of reference audio; fine-grained emotion control — supporting happy, sad, surprised, and other emotional expressions; and cross-lingual synthesis — speaking fluent English with a Chinese speaker’s voice. The model’s performance in Chinese speech synthesis is particularly outstanding, especially in professional scenarios such as news broadcasting and audiobook narration, where its naturalness and clarity reach industry-leading levels. The project is open-sourced under the Apache 2.0 license and comes with a comprehensive WebUI and API interface.

The release of CosyVoice marked Alibaba’s deep expertise in speech synthesis models being opened to the broader community. The project provides a complete toolchain from model training to inference deployment, supporting multiple hardware platforms including NVIDIA and Ascend NPU. In practice, CosyVoice has been deployed in smart customer service, virtual digital humans, and content creation scenarios. However, like other large-model approaches, CosyVoice requires significant GPU memory (recommended ≥8GB) and offered limited coverage for languages beyond Chinese and English in its early versions.

With the release of CosyVoice 2 (August 2025), the model’s capabilities were comprehensively upgraded. In the open-source TTS benchmark published by Agora in May 2026, CosyVoice 2 achieved a Chinese naturalness MOS of 4.7 (the highest score in the evaluation), Cantonese naturalness of 4.4, and Chinese-English code-mixing capability of 4.5. New features include instruction-controlled speaking rate and intonation (e.g., “Read this in a relaxed tone”), along with support for Cantonese and Shanghainese dialects. Zero-shot cloning requires only 3 seconds of reference audio. CosyVoice 2 is open-sourced under Apache 2.0 and has become the go-to solution for Chinese speech synthesis.

CosyVoice 3.0 (Fun-CosyVoice3-0.5B-2512) was released in December 2025, dramatically expanding multilingual coverage to 9 languages and over 18 dialects, while further improving naturalness, prosody, and cross-lingual consistency. Building on the low-latency advantages of streaming inference, CosyVoice 3 delivers significantly enhanced synthesis quality for low-resource languages and dialectal speech. The project has surpassed 20,000 GitHub stars and continues to see active development.