Offline Voice AI in a Web Browser
July 09, 2020Why offline when you're online?
Offline Voice AI in a browser may seem contradictory. Doesn’t using a web browser mean you’re online? But even if you’re connected to the Internet, running Voice AI offline, in-browser, means eliminating both minimum and variable latency, and that the speech data is intrinsically private: it does not need to leave the device. Local voice AI also unlocks always-listening behavior that is impractical to perform continuously with cloud-based services.
Demo Application: Smart Lighting
Here is a web demonstration application that uses Porcupine (wake word detection) and Rhino (Speech-to-Intent) engines to control smart lights in a home.
Instructions
To start any command, say "Pico Home" to wake the device.
Here are some example phrases:
"Turn the lights in the Living Room on"
"Shut off the lights in the Bathroom"
"Switch on all lights"
Here are some example phrases:
"Turn the lights in the Kitchen Blue"
"Make the closet lights green"
"Switch all lights purple"
Possible Colors:
RedOrangeYellowGreenBluePurplePinkWhite
All of the speech processing is performed privately, offline, in-browser. You can see more web demos here.
How to embed Voice AI in the browser
To achieve offline voice AI, Picovoice technology is available via WebAssembly. Together with the Web Audio API these provided the foundation for accessing microphone data in the browser and processing it in background Web Workers.
These cutting edge web technologies are almost entirely abstracted by the Picovoice SDKs for Web:
| Web | Angular | React | Vue | |
|---|---|---|---|---|
| Porcupine | Quick Start | Quick Start | Quick Start | Quick Start |
| Rhino | Quick Start | Quick Start | Quick Start | Quick Start |
| Picovoice SDK | Quick Start | Quick Start | Quick Start | Quick Start |
Audio processing
Picovoice engines accept industry standard 16kHz audio for speech processing. When accessing microphone audio, we need to downsample it to this format. We also need to ask for permission from the user to access their microphone, and connect the input to the downsampler. To handle downsampling behind the scenes, Picovoice has provided the @picovoice/web-voice-processor package. The Angular, React, and Vue libraries also abstract this detail away. For the "Vanilla" web worker libraries, you simply need to connect the two.
The Angular, React, Angular, and Vue libraries are even simpler, hiding the worker communication completely behind straightforward Services, Hooks, and Components, respectively.
Open Source Demo Applications
There are demo applications available for "Vanilla" JavaScript and HTML ("Web"), Angular, React, and Vue. These are minimal applications that show how to integrate the Picovoice SDKs into each web application.
Porcupine Demo Repositories
Rhino Demo Repositories
Picovoice SDK Demo Repositories
How do I create custom wake words or speech-to-intent contexts for the web?
You can use the Picovoice Console to create wake words and design and train speech-to-intent contexts. Personal use accounts are free and enterprise accounts are available with a 30-day trial.