Offline Voice AI in a Web Browser

  • Microphone
  • NLU
  • Offline Voice AI
  • Privacy
  • Web Audio API
  • Web Workers
  • WebAssembly
  • React
  • Angular
  • Vue
  • TypeScript
  • CDN
July 09, 2020

Why offline when you're online?

Offline Voice AI in a browser may seem contradictory. Doesn’t using a web browser mean you’re online? But even if you’re connected to the Internet, running Voice AI offline, in-browser, means eliminating both minimum and variable latency, and that the speech data is intrinsically private: it does not need to leave the device. Local voice AI also unlocks always-listening behavior that is impractical to perform continuously with cloud-based services.

Demo Application: Smart Lighting

Here is a web demonstration application that uses Porcupine (wake word detection) and Rhino (Speech-to-Intent) engines to control smart lights in a home.

Arrow (pointing at microphone button)
Press the microphone button to activate the demo
LIVING ROOMBEDROOMCLOSETBATHROOMPANTRYHALLWAYKITCHEN

All of the speech processing is performed privately, offline, in-browser. You can see more web demos here.

How to embed Voice AI in the browser

To achieve offline voice AI, Picovoice technology is available via WebAssembly. Together with the Web Audio API these provided the foundation for accessing microphone data in the browser and processing it in background Web Workers.

These cutting edge web technologies are almost entirely abstracted by the Picovoice SDKs for Web:

WebAngularReactVue
PorcupineQuick StartQuick StartQuick StartQuick Start
RhinoQuick StartQuick StartQuick StartQuick Start
Picovoice SDKQuick StartQuick StartQuick StartQuick Start

Audio processing

Picovoice engines accept industry standard 16kHz audio for speech processing. When accessing microphone audio, we need to downsample it to this format. We also need to ask for permission from the user to access their microphone, and connect the input to the downsampler. To handle downsampling behind the scenes, Picovoice has provided the @picovoice/web-voice-processor package. The Angular, React, and Vue libraries also abstract this detail away. For the "Vanilla" web worker libraries, you simply need to connect the two.

The Angular, React, Angular, and Vue libraries are even simpler, hiding the worker communication completely behind straightforward Services, Hooks, and Components, respectively.

Open Source Demo Applications

There are demo applications available for "Vanilla" JavaScript and HTML ("Web"), Angular, React, and Vue. These are minimal applications that show how to integrate the Picovoice SDKs into each web application.

Porcupine Demo Repositories

Rhino Demo Repositories

Picovoice SDK Demo Repositories

How do I create custom wake words or speech-to-intent contexts for the web?

You can use the Picovoice Console to create wake words and design and train speech-to-intent contexts. Personal use accounts are free and enterprise accounts are available with a 30-day trial.