Picovoice Wordmark
Start Building
Introduction
Introduction
AndroidC.NETiOSLinuxmacOSNode.jsPythonRaspberry PiWebWindows
AndroidC.NETiOSNode.jsPythonWeb
SummaryPicovoice picoLLMGPTQ
Introduction
AndroidC.NETFlutteriOSJavaLinuxmacOSNode.jsPythonRaspberry PiReactReact NativeRustWebWindows
AndroidC.NETFlutteriOSJavaNode.jsPythonReactReact NativeRustWeb
SummaryPicovoice LeopardAmazon TranscribeAzure Speech-to-TextGoogle ASRGoogle ASR (Enhanced)IBM Watson Speech-to-TextWhisper Speech-to-Text
FAQ
Introduction
AndroidC.NETFlutteriOSJavaLinuxmacOSNode.jsPythonRaspberry PiReactReact NativeRustWebWindows
AndroidC.NETFlutteriOSJavaNode.jsPythonReactReact NativeRustWeb
SummaryPicovoice Cheetah
FAQ
Introduction
AndroidC.NETiOSLinuxmacOSNode.jsPythonRaspberry PiWebWindows
AndroidC.NETiOSNode.jsPythonWeb
SummaryAmazon PollyAzure TTSElevenLabsOpenAI TTSPicovoice Orca
Introduction
AndroidCiOSLinuxmacOSPythonRaspberry PiWebWindows
AndroidCiOSPythonWeb
SummaryPicovoice KoalaMozilla RNNoise
Introduction
AndroidCiOSLinuxmacOSNode.jsPythonRaspberry PiWebWindows
AndroidCNode.jsPythoniOSWeb
SummaryPicovoice EaglepyannoteSpeechBrainWeSpeaker
Introduction
AndroidCiOSLinuxmacOSPythonRaspberry PiWebWindows
AndroidCiOSPythonWeb
SummaryPicovoice FalconAmazon TranscribeAzure Speech-to-TextGoogle Speech-to-Textpyannote
Introduction
AndroidArduinoCChrome.NETEdgeFirefoxFlutteriOSJavaLinuxmacOSMicrocontrollerNode.jsPythonRaspberry PiReactReact NativeRustSafariUnityWebWindows
AndroidC.NETFlutteriOSJavaMicrocontrollerNode.jsPythonReactReact NativeRustUnityWeb
SummaryPorcupineSnowboyPocketSphinx
Wake Word TipsFAQ
Introduction
AndroidCChrome.NETEdgeFirefoxFlutteriOSJavaLinuxmacOSNode.jsPythonRaspberry PiReactReact NativeRustSafariUnityWebWindows
AndroidC.NETFlutteriOSJavaNode.jsPythonReactReact NativeRustUnityWeb
SummaryPicovoice RhinoGoogle DialogflowAmazon LexIBM WatsonMicrosoft LUIS
Expression SyntaxFAQ
Introduction
AndroidC.NETiOSLinuxmacOSNode.jsPythonRaspberry PiRustWebWindows
AndroidC.NETiOSNode.jsPythonRustWeb
SummaryPicovoice CobraWebRTC VAD
FAQ
Introduction
AndroidC.NETFlutteriOSNode.jsPythonReact NativeRustUnityWeb
AndroidC.NETFlutteriOSNode.jsPythonReact NativeRustUnityWeb
Introduction
C.NETNode.jsPython
C.NETNode.jsPython
FAQGlossary

IBM Watson Benchmark

The IBM Watson Natural Language Understanding service is a cloud offering that is capable of extracting metadata from text. For speech input, Watson Speech to Text can be used to transcribe audio files. For domain-specific contexts, the customization interface can be used to create a custom language model to improve speech recognition performance.

Prerequisites

  • Ubuntu 20.04 (x86_64)
  • Git
  • Python
  • PIP
  • IBM Watson Account

Usage

  1. Clone the repository:
git clone https://github.com/Picovoice/speech-to-intent-benchmark.git
  1. Install the dependencies:
pip3 install -r requirements.txt
  1. Create a NLU service.

  2. Create a standard plan Speech to Text service.

  3. Create a Knowledge Studio service and create a new Workspace.

  4. In your new Workspace, upload the previously created type system data/watson/entity_types.json.

  5. In the Rules page under Rule-based Model, create a class for each entity type.

  6. In the Dictionaries page, import data/watson/barista_dictionaries.zip. Select the corresponding entity type and corresponding rule-class for each dictionary.

  7. In the Versions page, go to the Rule-based Model Type Mapping tab and map each entity type to the corresponding class.

  8. Return to the Rule-based Model page and save for deployment. You should see a model with version number 1.0. Deploy this model to Natural Language Understanding, and take note of your model ID.

  9. Run the benchmark:

python3 src/bench.py \
--engine IBM_WATSON \
--noise cafe \
--ibm_watson_model_id ${YOUR_MODEL_ID} \
--ibm_watson_stt_apikey ${YOUR_STT_API_KEY} \
--ibm_watson_stt_url ${YOUR_STT_URL} \
--ibm_watson_nlu_apikey ${YOUR_NLU_API_KEY} \
--ibm_watson_nlu_url ${YOUR_NLU_URL}
python3 src/bench.py \
--engine IBM_WATSON \
--noise kitchen \
--ibm_watson_model_id ${YOUR_MODEL_ID} \
--ibm_watson_stt_apikey ${YOUR_STT_API_KEY} \
--ibm_watson_stt_url ${YOUR_STT_URL} \
--ibm_watson_nlu_apikey ${YOUR_NLU_API_KEY} \
--ibm_watson_nlu_url ${YOUR_NLU_URL}

Result

IBM Watson voice assistant accuracy across various noise levelsIBM Watson voice assistant accuracy across various noise levels

Was this doc helpful?

Issue with this doc?

Report a GitHub Issue
IBM Watson Benchmark
  • Prerequisites
  • Usage
  • Result
Voice AI
  • Leopard Speech-to-Text
  • Cheetah Streaming Speech-to-Text
  • Orca Text-to-Speech
  • Koala Noise Suppression
  • Eagle Speaker Recognition
  • Falcon Speaker Diarization
  • Porcupine Wake Word
  • Rhino Speech-to-Intent
  • Cobra Voice Activity Detection
Local LLM
  • picoLLM Inference
  • picoLLM Compression
  • picoLLM GYM
Resources
  • Docs
  • Console
  • Blog
  • Use Cases
  • Playground
Sales & Services
  • Consulting
  • Foundation Plan
  • Enterprise Plan
  • Enterprise Support
Company
  • About us
  • Careers
Follow Picovoice
  • LinkedIn
  • GitHub
  • X
  • YouTube
  • AngelList
Subscribe to our newsletter
Terms of Use
Privacy Policy
© 2019-2025 Picovoice Inc.