Supertonic-TTS-2 - Text-to-Speech Benchmark

Prerequisites

• Ubuntu 20.04 (x86_64)
• Git
• Python 3.10
• Picovoice Console Account

Usage

1. Clone the repository:

git clone https://github.com/Picovoice/text-to-speech-benchmark.git

2. Install the dependencies:

pip3 install -r requirements/requirements-supertonic-tts-2.txt

3. Download the picoLLM model

For each benchmark a picoLLM model is required to generate responses from the LLM. The picoLLM model used in the benchmark is llama-3.2-1b-instruct-385 and can be downloaded from Picovoice Console.

4. Run the benchmark:

Hugging face model download commit hash:

• GitHub:
  • Repo: supertone-inc/supertonic.
  • Commit hash: 6fc89ea89eb29defb0ff2230b77c5a519acfe2a9.
• Hugging face model download commit hash:
  • Repo: Supertone/supertonic-2.
  • Commit hash: 75e6727618a02f323c720cba9478152d4bc16ca4.

For core hour ratio & latency metrics:

python3 benchmark.py \
--picovoice-access-key ${PV_ACCESS_KEY} \
--picollm-model-path ${PICOLLM_MODEL_PATH} \
--engine supertonic_tts_2 \
--supertonictts-repo-dir ${SUPERTONIC_REPO_DIR} \
--supertonictts-onnx-dir ${SUPERTONIC_ONNX_DIR} \
--supertonictts-voice-style-path ${SUPERTONIC_VOICE_STYLE_PATH} \

For peak memory metric:

for i in 1 2 3 4 5 6 7 8 9 10 20 40 80; do
    python3 benchmark.py \
    --picovoice-access-key ${PV_ACCESS_KEY} \
    --picollm-model-path ${PICOLLM_MODEL_PATH} \
    --engine supertonic_tts_2 \
    --supertonictts-repo-dir ${SUPERTONIC_REPO_DIR} \
    --supertonictts-onnx-dir ${SUPERTONIC_ONNX_DIR} \
    --supertonictts-voice-style-path ${SUPERTONIC_VOICE_STYLE_PATH} \
    --test-memory-size-multiple $i \
done

Replace ${SUPERTONIC_REPO_DIR} with the path to Supertonic-TTS-2's repo. E.g. --supertonictts-repo-dir ~/supertonic/. Replace ${SUPERTONIC_ONNX_DIR} with the path to Supertonic-TTS-2's repo. E.g. --supertonictts-onnx-dir ~/supertonic/py/assets/onnx/. Replace ${SUPERTONIC_VOICE_STYLE_PATH} with the path to Supertonic-TTS-2's repo. E.g. --voice-style-path ~/supertonic/py/assets/voice_styles/M1.json. Replace ${PICOLLM_MODEL_PATH} with the path to the model you downloaded. Replace ${PV_ACCESS_KEY} with your AccessKey obtained from Picovoice Console. Everyone who signs up for Picovoice Console receives a unique AccessKey.