wasmRhino - Web API

  • npm
  • NLU
  • WebAssembly
  • Browser

This document outlines how to integrate the Rhino wake word engine within an application using its Web API.

Requirements

  • yarn (or npm)
  • Secure browser context (i.e. HTTPS or localhost)

Compatibility

  • Chrome, Edge
  • Firefox
  • Safari

The Picovoice SDKs for Web are powered by WebAssembly (WASM), the Web Audio API, and Web Workers. All audio processing is performed in-browser, providing intrinsic privacy and reliability.

All modern browsers are supported, including on mobile. Internet Explorer is not supported.

Using the Web Audio API requires a secure context (HTTPS connection), with the exception of localhost, for local development.

JavaScript Frameworks

Looking to use Rhino with React, Angular, or Vue? There are framework-specific packages available:

The framework-specific packages operate at a higher level of abstraction and are meant to integrate as quickly and easily as possible, following each framework's conventions and best practices. Furthermore, the demo applications show the complete lifecycle of using Rhino in a component, including setup and teardown. Interacting with Web Workers is hidden behind a facade, and the Web Voice Processor is also coordinated behind the scenes to automatically setup the microphone.

Otherwise, this doc will provide instruction on how to use Rhino with "Vanilla" JavaScript and HTML, connecting it to the Web Voice Processor for microphone use.

Introduction

Rhino for Web is available in two flavors: Worker and Factory:

  • The Worker packages are all-in-one Web Workers which wrap Rhino instances that will work with the web-voice-processor (and the Angular, React, and Vue packages).
  • The Factory packages give you access to instances directly. This is useful if you wish to build your own worker/worklet, or perhaps use Rhino in some other custom scenario.

Structure

The Rhino SDK for Web is provided in several npm packages, due to the logistics and size of shipping ~3-4MB voice models.

Workers

For typical cases, use the worker packages. Worker packages create complete self-contained Rhino Web Worker instances that can be immediately used with @picovoice/web-voice-processor and with the Angular, React, and Vue packages.

Factories

Factory packages allow you to create instances of Rhino directly. Useful for building your own custom Worker/Worklet, or some other bespoke purpose.

Installation & Usage

Worker: Using modern JavaScript, ES Modules, Bundlers (e.g. Webpack)

To obtain a Rhino Worker, we can use the static create factory method from the RhinoWorkerFactory. Here is a complete example that:

  1. Obtains a Worker from the RhinoWorkerFactory (in this case, English) to listen for speech in the
  2. Handles to the inference event by setting the worker's onmessage event handler and looking for messages with a data property set to "rhn-inference".
  3. Starts up the WebVoiceProcessor to forward microphone audio to the Rhino Worker
  4. Sets up a button to trigger the push-to-talk functionality for Rhino and begin a voice interaction. This consists of sending the Rhino worker a "resume" and "pause" message.

E.g.:

yarn add @picovoice/web-voice-processor @picovoice/rhino-web-en-worker
import { WebVoiceProcessor } from "@picovoice/web-voice-processor"
import { RhinoWorkerFactory } from "@picovoice/rhino-web-en-worker";
const RHN_CONTEXT_64 = /* Base64 representation of a .rhn context */
async startRhino()
// Create a Rhino Worker (English language) to listen for
// commands in the specified context
const rhinoWorker = await RhinoWorkerFactory.create(
{context: RHN_CONTEXT_64 }
);
// The worker will send a message with data.command = "rhn-inference" upon concluding
// Here we tell it to log it to the console
rhinoWorker.onmessage = (msg) => {
switch (msg.data.command) {
case 'rhn-inference':
// Log the event
console.log("Rhino inference: " + msg.data.inference);
// Pause Rhino processing until the push-to-talk button is pressed again
rhinoWorker.postMessage({command: "pause"})
break;
default:
break;
}
};
// Start up the web voice processor. It will request microphone permission
// It downsamples the audio to voice recognition standard format (16-bit 16kHz linear PCM, single-channel)
// The incoming microphone audio frames will then be forwarded to the Rhino Worker
// n.b. This promise will reject if the user refuses permission! Make sure you handle that possibility.
const webVp = await WebVoiceProcessor.init({
engines: [rhinoWorker],
start: true,
});
}
// Rhino is push-to-talk. We need to to tell it that we
// are starting a voice interaction:
function pushToTalk() {
rhinoWorker.postMessage({command: "resume"})
}
}
startRhino()
...
// Finished with Rhino? Release the WebVoiceProcessor and the worker.
if (done) {
webVp.release()
rhinoWorker.sendMessage({command: "release"})
}

Worker: Script Tag / IIFE / CDN

Rhino's worker and factory packages are also available in IIFE format, intended for direct inclusion into HTML instead of a bundler. You can use local node modules, or use the CDN unpkg version for rapid prototyping.

These will add RhinoWebXxWorker and WebVoiceProcessor as global variables on window:

<!DOCTYPE html>
<html lang="en">
<head>
<script src="https://unpkg.com/@picovoice/rhino-web-en-worker/dist/iife/index.js"></script>
<script src="https://unpkg.com/@picovoice/web-voice-processor/dist/iife/index.js"></script>
<script type="application/javascript">
const CLOCK_CONTEXT_64 =
"cmhpbm8xLjYuMNEaAAAJAAAAcGljb3ZvaWNl//////////+UMAAAAGNvbnRleHQ6CiAgZXhwcmVzc2lvbnM6CiAgICB0aW1lcjoKICAgICAgLSAkdGltZXJBY3Rpb246YWN0aW9uIHRpbWVyCiAgICBzdG9wd2F0Y2g6CiAgICAgIC0gJHN0b3B3YXRjaEFjdGlvbjphY3Rpb24gc3RvcHdhdGNoCiAgICBjbG9jazoKICAgICAgLSAkY2xvY2tBY3Rpb246YWN0aW9uIChtZSkgKHRoZSkgY2xvY2sKICAgICAgLSAkY2xvY2tBY3Rpb246YWN0aW9uIChtZSkgKHRoZSkgdGltZQogICAgc2V0VGltZXI6CiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICBzZXRBbGFybToKICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VyICRwdi5Ud29EaWdpdEludGVnZXI6bWludXRlCiAgICAgICAgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkZGF5OmRheSAoYXQpICRwdi5Ud29EaWdpdEludGVnZXI6aG91cgogICAgICAgICRwdi5Ud29EaWdpdEludGVnZXI6bWludXRlICRhbVBtOmFtUG0KICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJGRheTpkYXkgKGF0KSAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGUKICAgICAgICAkYW1QbTphbVBtIChvbikgJGRheTpkYXkKICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VyICRhbVBtOmFtUG0gKG9uKSAkZGF5OmRheQogICAgYWxhcm06CiAgICAgIC0gJGFsYXJtQWN0aW9uOmFjdGlvbiBhbGFybQogICAgYXZhaWxhYmxlQ29tbWFuZHM6CiAgICAgIC0gd2hhdCBjYW4gSSBzYXkKICBzbG90czoKICAgIHRpbWVyQWN0aW9uOgogICAgICAtIHN0YXJ0CiAgICAgIC0gcGF1c2UKICAgICAgLSBzdG9wCiAgICAgIC0gcmVzZXQKICAgICAgLSBzaG93CiAgICAgIC0gcmVzdGFydAogICAgICAtIGNsZWFyCiAgICBjbG9ja0FjdGlvbjoKICAgICAgLSBzaG93CiAgICBzdG9wd2F0Y2hBY3Rpb246CiAgICAgIC0gc3RhcnQKICAgICAgLSBzdG9wCiAgICAgIC0gcGF1c2UKICAgICAgLSByZXNldAogICAgICAtIHJlc3RhcnQKICAgICAgLSBzaG93CiAgICBhbVBtOgogICAgICAtIGEgbQogICAgICAtIHAgbQogICAgZGF5OgogICAgICAtIHRvZGF5CiAgICAgIC0gdG9tb3Jyb3cKICAgICAgLSBtb25kYXkKICAgICAgLSB0dWVzZGF5CiAgICAgIC0gd2VkbmVzZGF5CiAgICAgIC0gdGh1cnNkYXkKICAgICAgLSBmcmlkYXkKICAgICAgLSBzYXR1cmRheQogICAgICAtIHN1bmRheQogICAgYWxhcm1BY3Rpb246CiAgICAgIC0gZGVsZXRlCgAAjwAAAAAAAAACAAAABgAAAAwAAAAPAAAAEwAAABYAAAAaAAAAIAAAACYAAAAtAAAAMwAAADwAAABDAAAAUAAAAFwAAABoAAAAdAAAAH8AAACMAAAAlwAAAKQAAACvAAAAtgAAAL4AAADEAAAA0AAAANsAAADmAAAA8QAAAPsAAAAHAQAAEQEAAB0BAAAnAQAALAEAADABAAA2AQAAQgEAAE0BAABYAQAAYwEAAG0BAAB5AQAAgwEAAI8BAACZAQAAngEAAKcBAACuAQAAswEAALkBAAC7AQAAvgEAAMUBAADNAQAA1AEAANkBAADiAQAA6QEAAPYBAAACAgAADgIAABoCAAAlAgAAMgIAAD0CAABKAgAAVQIAAFgCAABcAgAAYAIAAGYCAABsAgAAdAIAAH0CAACBAgAAiAIAAJACAACUAgAAmgIAAKQCAACsAgAAugIAAMcCAADUAgAA4QIAAO0CAAD7AgAABwMAABUDAAAhAwAAJgMAACoDAAAyAwAAOAMAAEQDAABPAwAAWgMAAGUDAABvAwAAewMAAIUDAACRAwAAmwMAAKEDAACmAwAAsAMAALcDAAC7AwAAvwMAAMgDAADPAwAA3AMAAOgDAAD0AwAAAAQAAAsEAAAYBAAAIwQAADAEAAA7BAAAQQQAAEoEAABPBAAAVQQAAFsEAABkBAAAbAQAAHMEAAB6BAAAhwQAAJMEAACfBAAAqwQAALYEAADDBAAAzgQAANsEAADmBAAA6gQAAPQEAAD5BAAA/gQAAGEAYSBtAGFsYXJtAGFuAGFuZABhdABjYW4AY2xlYXIAY2xvY2sAZGVsZXRlAGVpZ2h0AGVpZ2h0ZWVuAGVpZ2h0eQBlaWdodHkgZWlnaHQAZWlnaHR5IGZpdmUAZWlnaHR5IGZvdXIAZWlnaHR5IG5pbmUAZWlnaHR5IG9uZQBlaWdodHkgc2V2ZW4AZWlnaHR5IHNpeABlaWdodHkgdGhyZWUAZWlnaHR5IHR3bwBlbGV2ZW4AZmlmdGVlbgBmaWZ0eQBmaWZ0eSBlaWdodABmaWZ0eSBmaXZlAGZpZnR5IGZvdXIAZmlmdHkgbmluZQBmaWZ0eSBvbmUAZmlmdHkgc2V2ZW4AZmlmdHkgc2l4AGZpZnR5IHRocmVlAGZpZnR5IHR3bwBmaXZlAGZvcgBmb3J0eQBmb3J0eSBlaWdodABmb3J0eSBmaXZlAGZvcnR5IGZvdXIAZm9ydHkgbmluZQBmb3J0eSBvbmUAZm9ydHkgc2V2ZW4AZm9ydHkgc2l4AGZvcnR5IHRocmVlAGZvcnR5IHR3bwBmb3VyAGZvdXJ0ZWVuAGZyaWRheQBob3VyAGhvdXJzAGkAbWUAbWludXRlAG1pbnV0ZXMAbW9uZGF5AG5pbmUAbmluZXRlZW4AbmluZXR5AG5pbmV0eSBlaWdodABuaW5ldHkgZml2ZQBuaW5ldHkgZm91cgBuaW5ldHkgbmluZQBuaW5ldHkgb25lAG5pbmV0eSBzZXZlbgBuaW5ldHkgc2l4AG5pbmV0eSB0aHJlZQBuaW5ldHkgdHdvAG9uAG9uZQBwIG0AcGF1c2UAcmVzZXQAcmVzdGFydABzYXR1cmRheQBzYXkAc2Vjb25kAHNlY29uZHMAc2V0AHNldmVuAHNldmVudGVlbgBzZXZlbnR5AHNldmVudHkgZWlnaHQAc2V2ZW50eSBmaXZlAHNldmVudHkgZm91cgBzZXZlbnR5IG5pbmUAc2V2ZW50eSBvbmUAc2V2ZW50eSBzZXZlbgBzZXZlbnR5IHNpeABzZXZlbnR5IHRocmVlAHNldmVudHkgdHdvAHNob3cAc2l4AHNpeHRlZW4Ac2l4dHkAc2l4dHkgZWlnaHQAc2l4dHkgZml2ZQBzaXh0eSBmb3VyAHNpeHR5IG5pbmUAc2l4dHkgb25lAHNpeHR5IHNldmVuAHNpeHR5IHNpeABzaXh0eSB0aHJlZQBzaXh0eSB0d28Ac3RhcnQAc3RvcABzdG9wd2F0Y2gAc3VuZGF5AHRlbgB0aGUAdGhpcnRlZW4AdGhpcnR5AHRoaXJ0eSBlaWdodAB0aGlydHkgZml2ZQB0aGlydHkgZm91cgB0aGlydHkgbmluZQB0aGlydHkgb25lAHRoaXJ0eSBzZXZlbgB0aGlydHkgc2l4AHRoaXJ0eSB0aHJlZQB0aGlydHkgdHdvAHRocmVlAHRodXJzZGF5AHRpbWUAdGltZXIAdG9kYXkAdG9tb3Jyb3cAdHVlc2RheQB0d2VsdmUAdHdlbnR5AHR3ZW50eSBlaWdodAB0d2VudHkgZml2ZQB0d2VudHkgZm91cgB0d2VudHkgbmluZQB0d2VudHkgb25lAHR3ZW50eSBzZXZlbgB0d2VudHkgc2l4AHR3ZW50eSB0aHJlZQB0d2VudHkgdHdvAHR3bwB3ZWRuZXNkYXkAd2hhdAB6ZXJvAAAAAAAAAAIAAAAEAAAABQAAAAcAAAAJAAAACgAAAAwAAAANAAAADgAAAA8AAAAQAAAAEQAAABIAAAATAAAAFAAAABUAAAAWAAAAGAAAABkAAAAaAAAAGwAAABwAAAAeAAAAHwAAACAAAAAhAAAAIgAAACMAAAAkAAAAJgAAACcAAAAoAAAAKQAAACoAAAArAAAALgAAAC8AAAAwAAAAMQAAADIAAAAzAAAANQAAADYAAAA3AAAAOAAAADkAAAA6AAAAOwAAAD0AAAA/AAAAQQAAAEIAAABDAAAARgAAAEcAAABJAAAASgAAAEsAAABMAAAATQAAAE4AAABPAAAAUAAAAFIAAABTAAAAVAAAAFUAAABWAAAAWAAAAFoAAABbAAAAXAAAAF0AAABeAAAAYAAAAGEAAABjAAAAZQAAAGYAAABnAAAAaAAAAGoAAABsAAAAbgAAAHAAAAByAAAAdgAAAHgAAAB6AAAAfAAAAH4AAAB/AAAAgAAAAIEAAACCAAAAgwAAAIQAAACFAAAAhgAAAIgAAACJAAAAigAAAIsAAACMAAAAjQAAAI4AAACPAAAAkQAAAJIAAACUAAAAlQAAAJcAAACZAAAAmwAAAJ0AAACfAAAAowAAAKUAAACnAAAAqQAAAKsAAACsAAAArgAAAK8AAACwAAAAsgAAALQAAAC3AAAAuAAAALoAAAC8AAAAvgAAAMAAAADCAAAAxgAAAMgAAADKAAAAzAAAAM4AAADPAAAA0QAAANMAAADVAAAAAAAAAAEAAAACAAAABQAAAAgAAAANAAAADwAAABEAAAAUAAAAFwAAABkAAAAcAAAAHwAAACMAAAAnAAAALAAAAC4AAAAyAAAANQAAADoAAABAAAAARgAAAEwAAABSAAAAWQAAAGEAAABoAAAAbgAAAHMAAAB5AAAAfwAAAIUAAACKAAAAkQAAAJkAAAChAAAAqQAAALEAAAC6AAAAxAAAAM0AAADVAAAA3AAAAN8AAADiAAAA5AAAAOcAAADsAAAA8wAAAPsAAAADAQAACwEAABMBAAAcAQAAJgEAAC8BAAA3AQAAPgEAAEEBAABHAQAATAEAAFEBAABTAQAAVQEAAFgBAABbAQAAXAEAAF4BAABjAQAAaAEAAG4BAAB0AQAAeQEAAH4BAACBAQAAhwEAAIwBAACTAQAAmwEAAKMBAACrAQAAswEAALwBAADGAQAAzwEAANcBAADeAQAA4AEAAOIBAADlAQAA6QEAAO0BAADwAQAA9QEAAPwBAAACAgAACAIAAAoCAAAQAgAAFQIAABwCAAAiAgAAJQIAACoCAAAyAgAAOQIAAD8CAABIAgAAUAIAAFoCAABjAgAAbQIAAHYCAACAAgAAiQIAAJMCAACeAgAApwIAALECAAC9AgAAyAIAANMCAADdAgAA5wIAAPACAAD5AgAAAQMAAAMDAAAHAwAADgMAABQDAAAcAwAAJQMAAC4DAAA3AwAAQAMAAEoDAABVAwAAXwMAAGgDAABwAwAAdQMAAHkDAACAAwAAhQMAAIoDAACNAwAAjwMAAJEDAACWAwAAmgMAAJ4DAACkAwAAqgMAALEDAAC4AwAAvwMAAMYDAADNAwAA1AMAANsDAADjAwAA6gMAAPIDAAD7AwAABAQAAAwEAAAUBAAAGwQAACIEAAAoBAAALgQAADEEAAA2BAAAOwQAAD4EAABCBAAARgQAAEoEAABQBAAAVgQAAFsEAABgBAAAZgQAAGsEAABxBAAAdgQAAH4EAACFBAAAjgQAAJYEAACfBAAApwQAALAEAAC4BAAAwQQAAMsEAADTBAAA3AQAAOcEAADxBAAA+wQAAAQFAAANBQAAFQUAAB0FAAAkBQAAJgUAACwFAAAyBQAANQUAADkFAAA9BQAAQQUAAAMNAwsWDQsWAxUBHBYCFwMXAxcJAhcJAh8UAhcUAxcUFREcFBUBFAkRFRIfDR8NHxIXDR8SDR8SDR8NHxIOBiMNHxIOBBwNHxIXBhcNHxIkAxcNHxIQJAMXDR8SHQsjAxcNHxIdERQdDR8SIBwSDR8SHyIRFQsjAxcSFQsjAxcOEQ4fEhcOEQ4fEg4RDh8SDR8OEQ4fEg4GIw4RDh8SDgQcDhEOHxIXBhcOEQ4fEiQDFw4RDh8SECQDFw4RDh8SHQsjAxcOEQ4fEh0RFB0OEQ4fEiAcEg4RDh8SHyIOBiMOBBwODA4cDA4EHB8SDgQcHxINHw4EHB8SDgYjDgQcHxIOBBwOBBwfEhcGFw4EHB8SJAMXDgQcHxIQJAMXDgQcHxIdCyMDFw4EHB8SHREUHQ4EHB8SIBwSDgQcHxIfIg4EHA4EHB8SFw4cBgkSDhwGCQ0FDAUcBQwmBRwmBhYSFhEXAx8WBhciHxYGFyUiHxYRFwMfHRYDFwkSFgMXCQ0XBhcXBhcfEhcXBhcfEhcGFx8SDR8XBhcfEg4GIxcGFx8SDgQcFwYXHxIXBhcXBhcfEiQDFxcGFx8SECQDFxcGFx8SHQsjAxcXBhcfEh0RFB0XBhcfEiAcEhcGFx8SHyIBFwQXJAMXECQDFxsSCxYbBCYcEh0LHxwSHR8BHB8dAh8MCRIdAh8RCQ0dDR0LFAMXCR0LFAMXHQsUAxcJJh0LFAMXJh0LHx0LIwMXHQsjAxcfEhcdCyMDFx8SHQsjAxcSHQsjAxcfEg0fHQsjAxcSDR8dCyMDFx8SDgYjHQsjAxcSDgYjHQsjAxcfEg4EHB0LIwMXEg4EHB0LIwMXHxIXBhcdCyMDFxIXBhcdCyMDFx8SJAMXHQsjAxcfEhAkAxcdCyMDFxIkAxcdCyMDFxIQJAMXHQsjAxcfEh0LIwMXHQsjAxcSHQsjAxcdCyMDFx8SHREUHR0LIwMXEh0RFB0dCyMDFx8SIBwSHQsjAxcSIBwSHQsjAxcfEh8iHQsjAxcSHyIeGR0RFB0dERQdHxIXHREUHR8SHREUHR8SDR8dERQdHxIOBiMdERQdHxIOBBwdERQdHxIXBhcdERQdHxIkAxcdERQdHxIQJAMXHREUHR8SHQsjAxcdERQdHxIdERQdHREUHR8SIBwSHREUHR8SHyIdHwEcHx0fARsdHwEbJAEIHQMXCQ0dAxcJEh8LFwoDChIgDB8SFyAMCRIgDB8SIAwJEg0fIAwfEg0fIAwJEg4GIyAMHxIOBiMgDAkSDgQcIAwfEg4EHCAMCRIXBhcgDB8SFwYXIAwJEiQDFyAMCRIQJAMXIAwfEiQDFyAMHxIQJAMXIAwJEh0LIwMXIAwfEh0LIwMXIAwJEh0RFB0gDB8SHREUHSAMCRIgHBIgDB8SIBwSIAwJEh8iIAwfEh8iIBwSIAwmCQ0gDCYJEh8GFh8GFgwfAwkNHyIJDR8DFgEcGR8iFgEcGR8iJgkSHyImCQ0fJSImCQ0fJAsVIx8kCxcfEh8kCxcSHyQLFx8SDR8fJAsXEg0fHyQLFx8SDgYjHyQLFxIOBiMfJAsXHxIOBBwfJAsXEg4EHB8kCxcfEhcGFx8kCxcSFwYXHyQLFx8SJAMXHyQLFx8SECQDFx8kCxcSJAMXHyQLFxIQJAMXHyQLFx8SHQsjAxcfJAsXEh0LIwMXHyQLFx8SHREUHR8kCxcSHREUHR8kCxcfEiAcEh8kCxcSIBwSHyQLFx8SHyIfJAsXEh8iHyIkCxcmCRIkCxcmCQ0kAx8QJAMfJhEcGSYSHBkAAAAgAAAAAAAAAAEAAAADAAAABAAAAA0AAABxAAAA1QAAADkBAACdAQAAAQIAAAcCAAAOAgAAEAIAABICAAAUAgAAFQIAABYCAAAXAgAAGAIAABkCAAAaAgAAGwIAABwCAAAdAgAAHgIAAB8CAAAgAgAAIQIAACICAAAjAgAAJAIAACUCAAAmAgAACQAAAAEAAABGAAAAWwAAAH0AAAB+AAAANwAAAH8AAACMAAAAegAAADAAAABKAAAAawAAAI4AAABFAAAAiwAAAHkAAAAuAAAAIgAAAFwAAABPAAAACgAAADgAAABsAAAAFgAAAIAAAABuAAAALwAAABcAAABdAAAAUAAAAAsAAAA5AAAAgQAAAIYAAACKAAAAiQAAAIQAAACDAAAAiAAAAIcAAACCAAAAhQAAAG8AAAB0AAAAeAAAAHcAAAByAAAAcQAAAHYAAAB1AAAAcAAAAHMAAAAkAAAAKQAAAC0AAAAsAAAAJwAAACYAAAArAAAAKgAAACUAAAAoAAAAGAAAAB0AAAAhAAAAIAAAABsAAAAaAAAAHwAAAB4AAAAZAAAAHAAAAF4AAABjAAAAZwAAAGYAAABhAAAAYAAAAGUAAABkAAAAXwAAAGIAAABRAAAAVgAAAFoAAABZAAAAVAAAAFMAAABYAAAAVwAAAFIAAABVAAAADAAAABEAAAAVAAAAFAAAAA8AAAAOAAAAEwAAABIAAAANAAAAEAAAADoAAAA/AAAAQwAAAEIAAAA9AAAAPAAAAEEAAABAAAAAOwAAAD4AAACOAAAARQAAAIsAAAB5AAAALgAAACIAAABcAAAATwAAAAoAAAA4AAAAbAAAABYAAACAAAAAbgAAAC8AAAAXAAAAXQAAAFAAAAALAAAAOQAAAIEAAACGAAAAigAAAIkAAACEAAAAgwAAAIgAAACHAAAAggAAAIUAAABvAAAAdAAAAHgAAAB3AAAAcgAAAHEAAAB2AAAAdQAAAHAAAABzAAAAJAAAACkAAAAtAAAALAAAACcAAAAmAAAAKwAAACoAAAAlAAAAKAAAABgAAAAdAAAAIQAAACAAAAAbAAAAGgAAAB8AAAAeAAAAGQAAABwAAABeAAAAYwAAAGcAAABmAAAAYQAAAGAAAABlAAAAZAAAAF8AAABiAAAAUQAAAFYAAABaAAAAWQAAAFQAAABTAAAAWAAAAFcAAABSAAAAVQAAAAwAAAARAAAAFQAAABQAAAAPAAAADgAAABMAAAASAAAADQAAABAAAAA6AAAAPwAAAEMAAABCAAAAPQAAADwAAABBAAAAQAAAADsAAAA+AAAAjgAAAEUAAACLAAAAeQAAAC4AAAAiAAAAXAAAAE8AAAAKAAAAOAAAAGwAAAAWAAAAgAAAAG4AAAAvAAAAFwAAAF0AAABQAAAACwAAADkAAACBAAAAhgAAAIoAAACJAAAAhAAAAIMAAACIAAAAhwAAAIIAAACFAAAAbwAAAHQAAAB4AAAAdwAAAHIAAABxAAAAdgAAAHUAAABwAAAAcwAAACQAAAApAAAALQAAACwAAAAnAAAAJgAAACsAAAAqAAAAJQAAACgAAAAYAAAAHQAAACEAAAAgAAAAGwAAABoAAAAfAAAAHgAAABkAAAAcAAAAXgAAAGMAAABnAAAAZgAAAGEAAABgAAAAZQAAAGQAAABfAAAAYgAAAFEAAABWAAAAWgAAAFkAAABUAAAAUwAAAFgAAABXAAAAUgAAAFUAAAAMAAAAEQAAABUAAAAUAAAADwAAAA4AAAATAAAAEgAAAA0AAAAQAAAAOgAAAD8AAABDAAAAQgAAAD0AAAA8AAAAQQAAAEAAAAA7AAAAPgAAAI4AAABFAAAAiwAAAHkAAAAuAAAAIgAAAFwAAABPAAAACgAAADgAAABsAAAAFgAAAIAAAABuAAAALwAAABcAAABdAAAAUAAAAAsAAAA5AAAAgQAAAIYAAACKAAAAiQAAAIQAAACDAAAAiAAAAIcAAACCAAAAhQAAAG8AAAB0AAAAeAAAAHcAAAByAAAAcQAAAHYAAAB1AAAAcAAAAHMAAAAkAAAAKQAAAC0AAAAsAAAAJwAAACYAAAArAAAAKgAAACUAAAAoAAAAGAAAAB0AAAAhAAAAIAAAABsAAAAaAAAAHwAAAB4AAAAZAAAAHAAAAF4AAABjAAAAZwAAAGYAAABhAAAAYAAAAGUAAABkAAAAXwAAAGIAAABRAAAAVgAAAFoAAABZAAAAVAAAAFMAAABYAAAAVwAAAFIAAABVAAAADAAAABEAAAAVAAAAFAAAAA8AAAAOAAAAEwAAABIAAAANAAAAEAAAADoAAAA/AAAAQwAAAEIAAAA9AAAAPAAAAEEAAABAAAAAOwAAAD4AAACOAAAARQAAAIsAAAB5AAAALgAAACIAAABcAAAATwAAAAoAAAA4AAAAbAAAABYAAACAAAAAbgAAAC8AAAAXAAAAXQAAAFAAAAALAAAAOQAAAIEAAACGAAAAigAAAIkAAACEAAAAgwAAAIgAAACHAAAAggAAAIUAAABvAAAAdAAAAHgAAAB3AAAAcgAAAHEAAAB2AAAAdQAAAHAAAABzAAAAJAAAACkAAAAtAAAALAAAACcAAAAmAAAAKwAAACoAAAAlAAAAKAAAABgAAAAdAAAAIQAAACAAAAAbAAAAGgAAAB8AAAAeAAAAGQAAABwAAABeAAAAYwAAAGcAAABmAAAAYQAAAGAAAABlAAAAZAAAAF8AAABiAAAAUQAAAFYAAABaAAAAWQAAAFQAAABTAAAAWAAAAFcAAABSAAAAVQAAAAwAAAARAAAAFQAAABQAAAAPAAAADgAAABMAAAASAAAADQAAABAAAAA6AAAAPwAAAEMAAABCAAAAPQAAADwAAABBAAAAQAAAADsAAAA+AAAAaAAAAGkAAABHAAAASAAAAEkAAABbAAAAaAAAAEcAAABpAAAASAAAAFsAAABJAAAABwAAADEAAAAyAAAANQAAADYAAABMAAAATQAAAAAAAAACAAAAAwAAAAQAAAAFAAAABgAAAAgAAAAjAAAAMwAAADQAAABEAAAASwAAAE4AAABqAAAAbQAAAHsAAAB8AAAAjQAAAAAAAAAMAAAAEQAAAB0AAAAhAAAANAAAAEcAAABaAAAAbQAAAIAAAACQAAAAnAAAAJ0AAACeAAAAnwAAAKAAAAChAAAAogAAAKMAAACkAAAApQAAAKYAAACnAAAAqAAAAKkAAACqAAAAqwAAAKwAAACtAAAArgAAAK8AAACwAAAAsQAAAGFsYXJtQWN0aW9uAGFtUG0AY2xvY2tBY3Rpb24AZGF5AHB2LlR3b0RpZ2l0SW50ZWdlcgBwdi5Ud29EaWdpdEludGVnZXIAcHYuVHdvRGlnaXRJbnRlZ2VyAHB2LlR3b0RpZ2l0SW50ZWdlcgBwdi5Ud29EaWdpdEludGVnZXIAc3RvcHdhdGNoQWN0aW9uAHRpbWVyQWN0aW9uAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHAAAADAAAABMAAAAXAAAAHAAAACIAAAApAAAAMQAAADkAAABAAAAARwAAAEgAAABJAAAASgAAAEsAAABMAAAATQAAAE4AAABPAAAAUAAAAFEAAABSAAAAUwAAAFQAAABVAAAAVgAAAFcAAABYAAAAWQAAAFoAAABbAAAAXAAAAGFjdGlvbgBhbVBtAGFjdGlvbgBkYXkAaG91cgBob3VycwBtaW51dGUAbWludXRlcwBzZWNvbmRzAGFjdGlvbgBhY3Rpb24AAAAAAAAAAAAAAAAAAAAAAAAAAAAABwAAAAAAAAAGAAAAGAAAAB4AAAAnAAAAMAAAADoAAABAAAAAYWxhcm0AYXZhaWxhYmxlQ29tbWFuZHMAY2xvY2sAc2V0QWxhcm0Ac2V0VGltZXIAc3RvcHdhdGNoAHRpbWVyAAQAAAAAAAAA//////////8GAAAAKAAAAEwAAABwAAAAXAEAAAQKAAAoCgAAAAAAAAoAAAD/////AQAAADwAAAAoAAAAHgAAAAYAAAAAAAAAAAAAAAkAAAD/////AQAAAGAAAABMAAAAGwAAAAUAAAAAAAAAAAAAAAIAAAD/////BAAAAJAAAAAEAQAAPAEAAEwBAABwAAAAFwAAAP////8DAAAArAAAAOQAAAD0AAAAkAAAABwAAAD/////AgAAAMQAAADUAAAArAAAABQAAAACAAAAAAAAAKwAAAAdAAAAAgAAAAAAAACQAAAAFAAAAAIAAAAAAAAAkAAAAB0AAAACAAAAAAAAAHAAAAAcAAAA/////wIAAAAcAQAALAEAAAQBAAAUAAAAAgAAAAAAAAAEAQAAHQAAAAIAAAAAAAAAcAAAABQAAAACAAAAAAAAAHAAAAAdAAAAAgAAAAAAAAAAAAAAGgAAAP////8EAAAAfAEAAAwEAACIBgAAUAgAAFwBAAAOAAAA/////wEAAACQAQAAfAEAAB4AAAD/////AQAAAKQBAACQAQAAFQAAAP////8DAAAAwAEAAGADAADoAwAApAEAAAUAAAD/////AQAAANQBAADAAQAACwAAAAQAAAADAAAA8AEAALQCAAA8AwAA1AEAABEAAAD/////AgAAAAgCAACQAgAA8AEAAAcAAAD/////AQAAABwCAAAIAgAADAAAAAQAAAACAAAANAIAAGwCAAAcAgAAEQAAAP////8BAAAASAIAADQCAAAIAAAA/////wEAAABcAgAASAIAAA0AAAAEAAAAAAAAABwCAAAIAAAA/////wEAAACAAgAAbAIAAA0AAAAEAAAAAAAAAPABAAAIAAAA/////wEAAACkAgAAkAIAAA0AAAAEAAAAAAAAANQBAAAHAAAA/////wEAAADIAgAAtAIAAAwAAAAEAAAAAgAAAOACAAAYAwAAyAIAABEAAAD/////AQAAAPQCAADgAgAACAAAAP////8BAAAACAMAAPQCAAANAAAABAAAAAAAAADIAgAACAAAAP////8BAAAALAMAABgDAAANAAAABAAAAAAAAADUAQAACAAAAP////8BAAAAUAMAADwDAAANAAAABAAAAAAAAACkAQAABwAAAP////8BAAAAdAMAAGADAAAMAAAABAAAAAIAAACMAwAAxAMAAHQDAAARAAAA/////wEAAACgAwAAjAMAAAgAAAD/////AQAAALQDAACgAwAADQAAAAQAAAAAAAAAdAMAAAgAAAD/////AQAAANgDAADEAwAADQAAAAQAAAAAAAAApAEAAAgAAAD/////AQAAAPwDAADoAwAADQAAAAQAAAAAAAAAXAEAAB4AAAD/////AQAAACAEAAAMBAAAFQAAAP////8DAAAAPAQAANwFAABkBgAAIAQAAAUAAAD/////AQAAAFAEAAA8BAAACwAAAAQAAAADAAAAbAQAADAFAAC4BQAAUAQAABEAAAD/////AgAAAIQEAAAMBQAAbAQAAAcAAAD/////AQAAAJgEAACEBAAADAAAAAQAAAACAAAAsAQAAOgEAACYBAAAEQAAAP////8BAAAAxAQAALAEAAAIAAAA/////wEAAADYBAAAxAQAAA0AAAAEAAAAAAAAAJgEAAAIAAAA/////wEAAAD8BAAA6AQAAA0AAAAEAAAAAAAAAGwEAAAIAAAA/////wEAAAAgBQAADAUAAA0AAAAEAAAAAAAAAFAEAAAHAAAA/////wEAAABEBQAAMAUAAAwAAAAEAAAAAgAAAFwFAACUBQAARAUAABEAAAD/////AQAAAHAFAABcBQAACAAAAP////8BAAAAhAUAAHAFAAANAAAABAAAAAAAAABEBQAACAAAAP////8BAAAAqAUAAJQFAAANAAAABAAAAAAAAABQBAAACAAAAP////8BAAAAzAUAALgFAAANAAAABAAAAAAAAAAgBAAABwAAAP////8BAAAA8AUAANwFAAAMAAAABAAAAAIAAAAIBgAAQAYAAPAFAAARAAAA/////wEAAAAcBgAACAYAAAgAAAD/////AQAAADAGAAAcBgAADQAAAAQAAAAAAAAA8AUAAAgAAAD/////AQAAAFQGAABABgAADQAAAAQAAAAAAAAAIAQAAAgAAAD/////AQAAAHgGAABkBgAADQAAAAQAAAAAAAAAXAEAABAAAAD/////AQAAAJwGAACIBgAADwAAAP////8BAAAAsAYAAJwGAAAVAAAA/////wIAAADIBgAAjAcAALAGAAAEAAAA/////wIAAADgBgAAQAcAAMgGAAAGAAAA/////wEAAAD0BgAA4AYAAAEAAAADAAAAAgAAAAwHAAAwBwAA9AYAABgAAAD/////AQAAACAHAAAMBwAAAwAAAAMAAAAAAAAA9AYAAAMAAAADAAAAAAAAAMgGAAABAAAAAwAAAAIAAABYBwAAfAcAAEAHAAAYAAAA/////wEAAABsBwAAWAcAAAMAAAADAAAAAAAAAEAHAAADAAAAAwAAAAAAAACwBgAAAwAAAP////8CAAAApAcAAAQIAACMBwAAEgAAAP////8BAAAAuAcAAKQHAAAEAAAA/////wIAAADQBwAA9AcAALgHAAAGAAAA/////wEAAADkBwAA0AcAAAEAAAADAAAAAAAAALgHAAABAAAAAwAAAAAAAACMBwAABAAAAP////8CAAAAHAgAAEAIAAAECAAABgAAAP////8BAAAAMAgAABwIAAABAAAAAwAAAAAAAAAECAAAAQAAAAMAAAAAAAAAXAEAAA8AAAD/////AQAAAGQIAABQCAAAFQAAAP////8CAAAAfAgAAEAJAABkCAAABAAAAP////8CAAAAlAgAAPQIAAB8CAAABgAAAP////8BAAAAqAgAAJQIAAABAAAAAwAAAAIAAADACAAA5AgAAKgIAAAYAAAA/////wEAAADUCAAAwAgAAAMAAAADAAAAAAAAAKgIAAADAAAAAwAAAAAAAAB8CAAAAQAAAAMAAAACAAAADAkAADAJAAD0CAAAGAAAAP////8BAAAAIAkAAAwJAAADAAAAAwAAAAAAAAD0CAAAAwAAAAMAAAAAAAAAZAgAAAMAAAD/////AgAAAFgJAAC4CQAAQAkAABIAAAD/////AQAAAGwJAABYCQAABAAAAP////8CAAAAhAkAAKgJAABsCQAABgAAAP////8BAAAAmAkAAIQJAAABAAAAAwAAAAAAAABsCQAAAQAAAAMAAAAAAAAAQAkAAAQAAAD/////AgAAANAJAAD0CQAAuAkAAAYAAAD/////AQAAAOQJAADQCQAAAQAAAAMAAAAAAAAAuAkAAAEAAAADAAAAAAAAAAAAAAAAAAAA/////wEAAAAYCgAABAoAAA8AAAAAAAAAAAAAAAAAAAAfAAAA/////wEAAAA8CgAAKAoAABMAAAD/////AQAAAFAKAAA8CgAAFgAAAP////8BAAAAZAoAAFAKAAAZAAAAAQAAAAAAAAA="
function writeMessage(message) {
console.log(message)
let p = document.createElement("p")
let text = document.createTextNode(message)
p.appendChild(text)
document.body.appendChild(p)
}
async function startRhino() {
writeMessage("Rhino is loading. Please wait...")
window.rhinoClockWorker = await RhinoWebEnWorker.RhinoWorkerFactory.create(
{
context: {
base64: CLOCK_CONTEXT_64,
sensitivity: 0.5,
},
start: false,
}
)
writeMessage("Rhino worker ready!")
window.rhinoClockWorker.onmessage = msg => {
if (msg.data.command === "rhn-inference") {
writeMessage(
"Inference detected: " + JSON.stringify(msg.data.inference)
)
window.rhinoClockWorker.postMessage({ command: "pause" })
document.getElementById("push-to-talk").disabled = false
writeMessage(
"Rhino is paused. Press the 'Push to Talk' button to speak again."
)
}
}
writeMessage(
"WebVoiceProcessor initializing. Microphone permissions requested ..."
)
try {
let webVp = await WebVoiceProcessor.WebVoiceProcessor.init({
engines: [window.rhinoClockWorker],
})
writeMessage(
"WebVoiceProcessor ready! Press the 'Push to Talk' button to talk."
)
} catch (e) {
writeMessage("WebVoiceProcessor failed to initialize: " + e)
}
}
document.addEventListener("DOMContentLoaded", function () {
startRhino()
document.getElementById("push-to-talk").onclick = function (event) {
writeMessage("Rhino is listening for your commands ...")
this.disabled = true
window.rhinoClockWorker.postMessage({ command: "resume" })
}
})
</script>
</head>
<body>
<h1>Rhino Web Demo</h1>
<p>This demo uses Rhino for Web and the WebVoiceProcessor to:</p>
<ol>
<li>
Create an English instance of Rhino that understands commands in the
"Pico Clock" context;
</li>
<li>
Acquire microphone (& ask permission) data stream and convert to voice
processing format (16kHz 16-bit linear PCM). The downsampled audio is
forwarded to the Rhino engines. The audio <i>does not</i> leave the
browser: all processing is occurring via the Rhino WebAssembly code.
</li>
<li>
Await inference events from the Rhino engine and output them to the
page. When the inference is concluded, the push-to-talk button is
enabled again.
</li>
</ol>
<hr />
<button id="push-to-talk">Push to Talk</button>
</body>
</html>

Factory

If you wish to build your own worker/worklet, or perhaps not use workers at all, use the factory packages. This will let you instantiate Rhino engine instances directly.

The audio passed to the worker must be of the correct format. The WebVoiceProcessor handles downsampling in the examples above. If you are not using that, you must ensure you do it yourself.

E.g.:

import { Rhino } from "@picovoice/rhino-web-en-worker"
async function startRhino() {
const handle = await Rhino.create(
{ context: /* Base64 representation of a .rhn context */ },
)
// Send Rhino frames of audio (check handle.frameLength for size of array)
const audioFrames = new Int16Array(/* Provide data with correct format and size */)
const rhinoInference = handle.process(audioFrames)
// rhinoInference: isFinalized = true when
// Rhino has concluded
}
startRhino()

The Rhino factory returns instances with the RhinoEngine interface:

export interface RhinoEngine {
release(): void
process(frames: Int16Array): RhinoInference
version: string
sampleRate: number
frameLength: number
}

Results

Ultimately, when Rhino produces a conclusion, it will return a RhinoInference object:

export type RhinoInference = {
/** Rhino has concluded the inference (isUnderstood is now set) */
isFinalized: boolean
/** The intent was understood (it matched an expression in the context) */
isUnderstood?: boolean
/** The name of the intent */
intent?: string
/** Map of the slot variables and values extracted from the utterance */
slots?: Record<string, string>
}

Issue with this doc? Please let us know.