Instructions to use k2-fsa/TTS_eval_models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use k2-fsa/TTS_eval_models with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="k2-fsa/TTS_eval_models")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("k2-fsa/TTS_eval_models", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,16 +11,16 @@ library_name: transformers
|
|
| 11 |
|
| 12 |
# TTS Evaluation Models
|
| 13 |
|
| 14 |
-
This repository contains models for the objective evaluation of text-to-speech (TTS) models, as presented in the papers [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://huggingface.co/papers/2506.13053)
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
- **Code:** [k2-fsa/ZipVoice](https://github.com/k2-fsa/ZipVoice)
|
| 17 |
-
- **Project Page:** [ZipVoice-Dialog Demos](https://zipvoice-dialog.github.io)
|
| 18 |
|
| 19 |
## Evaluation Metrics
|
| 20 |
|
| 21 |
This repository specifically supports the following evaluation metrics:
|
| 22 |
|
| 23 |
-
- **WER**: Includes [Hubert-based ASR model](https://huggingface.co/facebook/hubert-large-ls960-ft) for LibriSpeech-PC testset, [Paraformer-based ASR model](https://huggingface.co/funasr/paraformer-zh) for Chinese datasets, [Whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) model for general English test sets, [WhisperD](https://huggingface.co/jordand/whisper-d-v1a) model for English dialogue speech.
|
| 24 |
|
| 25 |
- **cpWER**: [WhisperD](https://huggingface.co/jordand/whisper-d-v1a) model is used to compute concatenated minimum permutation word error rate ([cpWER](https://arxiv.org/abs/2507.09318)) for English dialogue speech.
|
| 26 |
|
|
@@ -30,7 +30,7 @@ This repository specifically supports the following evaluation metrics:
|
|
| 30 |
|
| 31 |
- **UTMOS**: The mos prediction model [UTMOS](https://github.com/sarulab-speech/UTMOS22) is used.
|
| 32 |
|
| 33 |
-
For more details, please refer to
|
| 34 |
|
| 35 |
## Citation
|
| 36 |
|
|
@@ -48,4 +48,12 @@ For more details, please refer to the [official repository](https://github.com/k
|
|
| 48 |
journal={arXiv preprint arXiv:2507.09318},
|
| 49 |
year={2025}
|
| 50 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
```
|
|
|
|
| 11 |
|
| 12 |
# TTS Evaluation Models
|
| 13 |
|
| 14 |
+
This repository contains models for the objective evaluation of text-to-speech (TTS) models, as presented in the papers [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://huggingface.co/papers/2506.13053), [ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching](https://huggingface.co/papers/2507.09318), [OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models](https://huggingface.co/papers/2604.00688).
|
| 15 |
+
|
| 16 |
+
- **Code:** [k2-fsa/ZipVoice](https://github.com/k2-fsa/ZipVoice) and [k2-fsa/OmniVoice](https://github.com/k2-fsa/OmniVoice)
|
| 17 |
|
|
|
|
|
|
|
| 18 |
|
| 19 |
## Evaluation Metrics
|
| 20 |
|
| 21 |
This repository specifically supports the following evaluation metrics:
|
| 22 |
|
| 23 |
+
- **WER**: Includes [Hubert-based ASR model](https://huggingface.co/facebook/hubert-large-ls960-ft) for LibriSpeech-PC testset, [Paraformer-based ASR model](https://huggingface.co/funasr/paraformer-zh) for Chinese datasets, [Whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) model for general English and other languages test sets, [WhisperD](https://huggingface.co/jordand/whisper-d-v1a) model for English dialogue speech.
|
| 24 |
|
| 25 |
- **cpWER**: [WhisperD](https://huggingface.co/jordand/whisper-d-v1a) model is used to compute concatenated minimum permutation word error rate ([cpWER](https://arxiv.org/abs/2507.09318)) for English dialogue speech.
|
| 26 |
|
|
|
|
| 30 |
|
| 31 |
- **UTMOS**: The mos prediction model [UTMOS](https://github.com/sarulab-speech/UTMOS22) is used.
|
| 32 |
|
| 33 |
+
For more details, please refer to repositories [ZipVoice](https://github.com/k2-fsa/ZipVoice) and [OmniVoice](https://github.com/k2-fsa/OmniVoice).
|
| 34 |
|
| 35 |
## Citation
|
| 36 |
|
|
|
|
| 48 |
journal={arXiv preprint arXiv:2507.09318},
|
| 49 |
year={2025}
|
| 50 |
}
|
| 51 |
+
|
| 52 |
+
@article{zhu2026omnivoice,
|
| 53 |
+
title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
|
| 54 |
+
author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
|
| 55 |
+
journal={arXiv preprint arXiv:2604.00688},
|
| 56 |
+
year={2026}
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
```
|