Voice Activity Detection Benchmark
Voice activity detection (VAD) is the recognition of human speech within a stream of audio. Voice activity detection is one of the main building blocks of speech-enabled applications. VAD accuracy has a compounding effect on the system performance as many downstream speech processing blocks depend on it. VoIP, IVR, telemarketing, and security systems incorporate voice activity detection.
Cobra Voice Activity Detection is an enterprise-grade, extremely efficient, and real-time VAD that achieves best-in-class accuracy across platforms. and real-time VAD that achieves best-in-class accuracy. Below is a series of benchmarks to validate the accuracy claims.Methodology
Engines
We compare the accuracy of Cobra with the voice-activity detector used for the WebRTC project, developed by Google and widely regarded as one of the best VADs, and with Silero VAD, another popular open-source VAD released more recently.
Speech Corpus
We use LibriSpeech (test-clean portion) as the speech corpus. It provides a diverse number
of speakers and is gender-balanced.
Noise
The real challenge in building a performant VAD is resilience to noise. To test out the effect of noise, we mix noise with speech data before feeding it to VAD engines. For this purpose, we use the DEMAND dataset that contains noise recordings in diverse environments.
Metric
We use the receiver operator characteristics curve. The ROC curve is a known tool for inspecting the performance of binary classifiers across different decision thresholds. It allows the designer to study the interplay of detection rate vs false positive rate.
To measure performance, we use the real-time factor. This factor measures how much computational time a model needs to process files relative to the duration of the file. These scores were recorded on an Ubuntu 22.04 machine with AMD CPU (AMD Ryzen 9 5900X (12) @ 3.70GHz).
Results
Usage
The data and code used to create this benchmark are available on GitHub under the permissive Apache 2.0 license. Detailed instructions for benchmarking individual engines are in the following documents: