Azure Real-Time Speech-to-Text Benchmark

Prerequisites

• Ubuntu 20.04 (x86_64)
• Git
• Python
• PIP
• Azure Account

Usage

Setup

1. Clone the repository:

git clone https://github.com/Picovoice/speech-to-text-benchmark.git

2. Install the dependencies:

pip3 install -r requirements.txt

Word Accuracy Benchmark

1. Run the benchmark:

python3 benchmark.py \
--engine AZURE_SPEECH_TO_TEXT_REAL_TIME \
--dataset ${DATASET} \
--dataset-folder ${DATASET_FOLDER} \
--language ${LANGUAGE} \
--azure-speech-key ${AZURE_SPEECH_KEY} \
--azure-speech-location ${AZURE_SPEECH_LOCATION}

• To measure the Punctuation Error Rate (PER) add --punctuation to the command.

Word Emission Latency Benchmark

1. Follow the instructions here to generate word timing information.

2. Run the benchmark:

python3 benchmark_latency.py \
--engine AZURE_SPEECH_TO_TEXT_REAL_TIME \
--dataset-folder ${DATASET_FOLDER} \
--language ${LANGUAGE} \
--azure-speech-key ${AZURE_SPEECH_KEY} \
--azure-speech-location ${AZURE_SPEECH_LOCATION}