Rhino - Web API

npm
NLU
WebAssembly
Browser

This document outlines how to integrate the Rhino wake word engine within an application using its Web API.

Requirements

yarn (or npm)
Secure browser context (i.e. HTTPS or localhost)

Compatibility

Chrome, Edge
Firefox
Safari

The Picovoice SDKs for Web are powered by WebAssembly (WASM), the Web Audio API, and Web Workers. All audio processing is performed in-browser, providing intrinsic privacy and reliability.

All modern browsers are supported, including on mobile. Internet Explorer is not supported.

Using the Web Audio API requires a secure context (HTTPS connection), with the exception of localhost, for local development.

JavaScript Frameworks

Looking to use Rhino with React, Angular, or Vue? There are framework-specific packages available:

The framework-specific packages operate at a higher level of abstraction and are meant to integrate as quickly and easily as possible, following each framework's conventions and best practices. Furthermore, the demo applications show the complete lifecycle of using Rhino in a component, including setup and teardown. Interacting with Web Workers is hidden behind a facade, and the Web Voice Processor is also coordinated behind the scenes to automatically setup the microphone.

Otherwise, this doc will provide instruction on how to use Rhino with "Vanilla" JavaScript and HTML, connecting it to the Web Voice Processor for microphone use.

Introduction

Rhino for Web is available in two flavors: Worker and Factory:

The Worker packages are all-in-one Web Workers which wrap Rhino instances that will work with the web-voice-processor (and the Angular, React, and Vue packages).
The Factory packages give you access to instances directly. This is useful if you wish to build your own worker/worklet, or perhaps use Rhino in some other custom scenario.

Structure

The Rhino SDK for Web is provided in several npm packages, due to the logistics and size of shipping ~3-4MB voice models.

Workers

For typical cases, use the worker packages. Worker packages create complete self-contained Rhino Web Worker instances that can be immediately used with @picovoice/web-voice-processor and with the Angular, React, and Vue packages.

Factories

Factory packages allow you to create instances of Rhino directly. Useful for building your own custom Worker/Worklet, or some other bespoke purpose.

Installation & Usage

Worker: Using modern JavaScript, ES Modules, Bundlers (e.g. Webpack)

To obtain a Rhino Worker, we can use the static create factory method from the RhinoWorkerFactory. Here is a complete example that:

Obtains a Worker from the RhinoWorkerFactory (in this case, English) to listen for speech in the
Handles to the inference event by setting the worker's onmessage event handler and looking for messages with a data property set to "rhn-inference".
Starts up the WebVoiceProcessor to forward microphone audio to the Rhino Worker
Sets up a button to trigger the push-to-talk functionality for Rhino and begin a voice interaction. This consists of sending the Rhino worker a "resume" and "pause" message.

E.g.:

yarn add @picovoice/web-voice-processor @picovoice/rhino-web-en-worker

import { WebVoiceProcessor } from "@picovoice/web-voice-processor"
import { RhinoWorkerFactory } from "@picovoice/rhino-web-en-worker";

const RHN_CONTEXT_64 = /* Base64 representation of a .rhn context */

async startRhino()
  // Create a Rhino Worker (English language) to listen for
  // commands in the specified context
  const rhinoWorker = await RhinoWorkerFactory.create(
    {context: RHN_CONTEXT_64 }
  );

  // The worker will send a message with data.command = "rhn-inference" upon concluding
  // Here we tell it to log it to the console
  rhinoWorker.onmessage = (msg) => {
    switch (msg.data.command) {
      case 'rhn-inference':
        // Log the event
        console.log("Rhino inference: " + msg.data.inference);
        // Pause Rhino processing until the push-to-talk button is pressed again
        rhinoWorker.postMessage({command: "pause"})
        break;
      default:
        break;
    }
  };

  // Start up the web voice processor. It will request microphone permission
  // It downsamples the audio to voice recognition standard format (16-bit 16kHz linear PCM, single-channel)
  // The incoming microphone audio frames will then be forwarded to the Rhino Worker
  // n.b. This promise will reject if the user refuses permission! Make sure you handle that possibility.
  const webVp = await WebVoiceProcessor.init({
    engines: [rhinoWorker],
    start: true,
  });
  }

  // Rhino is push-to-talk. We need to to tell it that we
  // are starting a voice interaction:
  function pushToTalk() {
    rhinoWorker.postMessage({command: "resume"})
  }

}
startRhino()

...

// Finished with Rhino? Release the WebVoiceProcessor and the worker.
if (done) {
  webVp.release()
  rhinoWorker.sendMessage({command: "release"})
}

Worker: Script Tag / IIFE / CDN

Rhino's worker and factory packages are also available in IIFE format, intended for direct inclusion into HTML instead of a bundler. You can use local node modules, or use the CDN unpkg version for rapid prototyping.

These will add RhinoWebXxWorker and WebVoiceProcessor as global variables on window:

<!DOCTYPE html>
<html lang="en">
  <head>
    <script src="https://unpkg.com/@picovoice/rhino-web-en-worker/dist/iife/index.js"></script>
    <script src="https://unpkg.com/@picovoice/web-voice-processor/dist/iife/index.js"></script>
    <script type="application/javascript">
      const CLOCK_CONTEXT_64 =
        "cmhpbm8xLjYuMNEaAAAJAAAAcGljb3ZvaWNl//////////+UMAAAAGNvbnRleHQ6CiAgZXhwcmVzc2lvbnM6CiAgICB0aW1lcjoKICAgICAgLSAkdGltZXJBY3Rpb246YWN0aW9uIHRpbWVyCiAgICBzdG9wd2F0Y2g6CiAgICAgIC0gJHN0b3B3YXRjaEFjdGlvbjphY3Rpb24gc3RvcHdhdGNoCiAgICBjbG9jazoKICAgICAgLSAkY2xvY2tBY3Rpb246YWN0aW9uIChtZSkgKHRoZSkgY2xvY2sKICAgICAgLSAkY2xvY2tBY3Rpb246YWN0aW9uIChtZSkgKHRoZSkgdGltZQogICAgc2V0VGltZXI6CiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGVzIFttaW51dGUsIG1pbnV0ZXNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICAgIC0gc2V0IChhKSB0aW1lciBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VycyBbaG91ciwgaG91cnNdIChhbmQpCiAgICAgICAgJHB2LlR3b0RpZ2l0SW50ZWdlcjpzZWNvbmRzIFtzZWNvbmQsIHNlY29uZHNdCiAgICBzZXRBbGFybToKICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VyICRwdi5Ud29EaWdpdEludGVnZXI6bWludXRlCiAgICAgICAgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkZGF5OmRheSAoYXQpICRwdi5Ud29EaWdpdEludGVnZXI6aG91cgogICAgICAgICRwdi5Ud29EaWdpdEludGVnZXI6bWludXRlICRhbVBtOmFtUG0KICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJGRheTpkYXkgKGF0KSAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJGFtUG06YW1QbQogICAgICAtIHNldCAoYW4pIGFsYXJtIGZvciAkcHYuVHdvRGlnaXRJbnRlZ2VyOmhvdXIgJHB2LlR3b0RpZ2l0SW50ZWdlcjptaW51dGUKICAgICAgICAkYW1QbTphbVBtIChvbikgJGRheTpkYXkKICAgICAgLSBzZXQgKGFuKSBhbGFybSBmb3IgJHB2LlR3b0RpZ2l0SW50ZWdlcjpob3VyICRhbVBtOmFtUG0gKG9uKSAkZGF5OmRheQogICAgYWxhcm06CiAgICAgIC0gJGFsYXJtQWN0aW9uOmFjdGlvbiBhbGFybQogICAgYXZhaWxhYmxlQ29tbWFuZHM6CiAgICAgIC0gd2hhdCBjYW4gSSBzYXkKICBzbG90czoKICAgIHRpbWVyQWN0aW9uOgogICAgICAtIHN0YXJ0CiAgICAgIC0gcGF1c2UKICAgICAgLSBzdG9wCiAgICAgIC0gcmVzZXQKICAgICAgLSBzaG93CiAgICAgIC0gcmVzdGFydAogICAgICAtIGNsZWFyCiAgICBjbG9ja0FjdGlvbjoKICAgICAgLSBzaG93CiAgICBzdG9wd2F0Y2hBY3Rpb246CiAgICAgIC0gc3RhcnQKICAgICAgLSBzdG9wCiAgICAgIC0gcGF1c2UKICAgICAgLSByZXNldAogICAgICAtIHJlc3RhcnQKICAgICAgLSBzaG93CiAgICBhbVBtOgogICAgICAtIGEgbQogICAgICAtIHAgbQogICAgZGF5OgogICAgICAtIHRvZGF5CiAgICAgIC0gdG9tb3Jyb3cKICAgICAgLSBtb25kYXkKICAgICAgLSB0dWVzZGF5CiAgICAgIC0gd2VkbmVzZGF5CiAgICAgIC0gdGh1cnNkYXkKICAgICAgLSBmcmlkYXkKICAgICAgLSBzYXR1cmRheQogICAgICAtIHN1bmRheQogICAgYWxhcm1BY3Rpb246CiAgICAgIC0gZGVsZXRlCgAAjwAAAAAAAAACAAAABgAAAAwAAAAPAAAAEwAAABYAAAAaAAAAIAAAACYAAAAtAAAAMwAAADwAAABDAAAAUAAAAFwAAABoAAAAdAAAAH8AAACMAAAAlwAAAKQAAACvAAAAtgAAAL4AAADEAAAA0AAAANsAAADmAAAA8QAAAPsAAAAHAQAAEQEAAB0BAAAnAQAALAEAADABAAA2AQAAQgEAAE0BAABYAQAAYwEAAG0BAAB5AQAAgwEAAI8BAACZAQAAngEAAKcBAACuAQAAswEAALkBAAC7AQAAvgEAAMUBAADNAQAA1AEAANkBAADiAQAA6QEAAPYBAAACAgAADgIAABoCAAAlAgAAMgIAAD0CAABKAgAAVQIAAFgCAABcAgAAYAIAAGYCAABsAgAAdAIAAH0CAACBAgAAiAIAAJACAACUAgAAmgIAAKQCAACsAgAAugIAAMcCAADUAgAA4QIAAO0CAAD7AgAABwMAABUDAAAhAwAAJgMAACoDAAAyAwAAOAMAAEQDAABPAwAAWgMAAGUDAABvAwAAewMAAIUDAACRAwAAmwMAAKEDAACmAwAAsAMAALcDAAC7AwAAvwMAAMgDAADPAwAA3AMAAOgDAAD0AwAAAAQAAAsEAAAYBAAAIwQAADAEAAA7BAAAQQQAAEoEAABPBAAAVQQAAFsEAABkBAAAbAQAAHMEAAB6BAAAhwQAAJMEAACfBAAAqwQAALYEAADDBAAAzgQAANsEAADmBAAA6gQAAPQEAAD5BAAA/gQAAGEAYSBtAGFsYXJtAGFuAGFuZABhdABjYW4AY2xlYXIAY2xvY2sAZGVsZXRlAGVpZ2h0AGVpZ2h0ZWVuAGVpZ2h0eQBlaWdodHkgZWlnaHQAZWlnaHR5IGZpdmUAZWlnaHR5IGZvdXIAZWlnaHR5IG5pbmUAZWlnaHR5IG9uZQBlaWdodHkgc2V2ZW4AZWlnaHR5IHNpeABlaWdodHkgdGhyZWUAZWlnaHR5IHR3bwBlbGV2ZW4AZmlmdGVlbgBmaWZ0eQBmaWZ0eSBlaWdodABmaWZ0eSBmaXZlAGZpZnR5IGZvdXIAZmlmdHkgbmluZQBmaWZ0eSBvbmUAZmlmdHkgc2V2ZW4AZmlmdHkgc2l4AGZpZnR5IHRocmVlAGZpZnR5IHR3bwBmaXZlAGZvcgBmb3J0eQBmb3J0eSBlaWdodABmb3J0eSBmaXZlAGZvcnR5IGZvdXIAZm9ydHkgbmluZQBmb3J0eSBvbmUAZm9ydHkgc2V2ZW4AZm9ydHkgc2l4AGZvcnR5IHRocmVlAGZvcnR5IHR3bwBmb3VyAGZvdXJ0ZWVuAGZyaWRheQBob3VyAGhvdXJzAGkAbWUAbWludXRlAG1pbnV0ZXMAbW9uZGF5AG5pbmUAbmluZXRlZW4AbmluZXR5AG5pbmV0eSBlaWdodABuaW5ldHkgZml2ZQBuaW5ldHkgZm91cgBuaW5ldHkgbmluZQBuaW5ldHkgb25lAG5pbmV0eSBzZXZlbgBuaW5ldHkgc2l4AG5pbmV0eSB0aHJlZQBuaW5ldHkgdHdvAG9uAG9uZQBwIG0AcGF1c2UAcmVzZXQAcmVzdGFydABzYXR1cmRheQBzYXkAc2Vjb25kAHNlY29uZHMAc2V0AHNldmVuAHNldmVudGVlbgBzZXZlbnR5AHNldmVudHkgZWlnaHQAc2V2ZW50eSBmaXZlAHNldmVudHkgZm91cgBzZXZlbnR5IG5pbmUAc2V2ZW50eSBvbmUAc2V2ZW50eSBzZXZlbgBzZXZlbnR5IHNpeABzZXZlbnR5IHRocmVlAHNldmVudHkgdHdvAHNob3cAc2l4AHNpeHRlZW4Ac2l4dHkAc2l4dHkgZWlnaHQAc2l4dHkgZml2ZQBzaXh0eSBmb3VyAHNpeHR5IG5pbmUAc2l4dHkgb25lAHNpeHR5IHNldmVuAHNpeHR5IHNpeABzaXh0eSB0aHJlZQBzaXh0eSB0d28Ac3RhcnQAc3RvcABzdG9wd2F0Y2gAc3VuZGF5AHRlbgB0aGUAdGhpcnRlZW4AdGhpcnR5AHRoaXJ0eSBlaWdodAB0aGlydHkgZml2ZQB0aGlydHkgZm91cgB0aGlydHkgbmluZQB0aGlydHkgb25lAHRoaXJ0eSBzZXZlbgB0aGlydHkgc2l4AHRoaXJ0eSB0aHJlZQB0aGlydHkgdHdvAHRocmVlAHRodXJzZGF5AHRpbWUAdGltZXIAdG9kYXkAdG9tb3Jyb3cAdHVlc2RheQB0d2VsdmUAdHdlbnR5AHR3ZW50eSBlaWdodAB0d2VudHkgZml2ZQB0d2VudHkgZm91cgB0d2VudHkgbmluZQB0d2VudHkgb25lAHR3ZW50eSBzZXZlbgB0d2VudHkgc2l4AHR3ZW50eSB0aHJlZQB0d2VudHkgdHdvAHR3bwB3ZWRuZXNkYXkAd2hhdAB6ZXJvAAAAAAAAAAIAAAAEAAAABQAAAAcAAAAJAAAACgAAAAwAAAANAAAADgAAAA8AAAAQAAAAEQAAABIAAAATAAAAFAAAABUAAAAWAAAAGAAAABkAAAAaAAAAGwAAABwAAAAeAAAAHwAAACAAAAAhAAAAIgAAACMAAAAkAAAAJgAAACcAAAAoAAAAKQAAACoAAAArAAAALgAAAC8AAAAwAAAAMQAAADIAAAAzAAAANQAAADYAAAA3AAAAOAAAADkAAAA6AAAAOwAAAD0AAAA/AAAAQQAAAEIAAABDAAAARgAAAEcAAABJAAAASgAAAEsAAABMAAAATQAAAE4AAABPAAAAUAAAAFIAAABTAAAAVAAAAFUAAABWAAAAWAAAAFoAAABbAAAAXAAAAF0AAABeAAAAYAAAAGEAAABjAAAAZQAAAGYAAABnAAAAaAAAAGoAAABsAAAAbgAAAHAAAAByAAAAdgAAAHgAAAB6AAAAfAAAAH4AAAB/AAAAgAAAAIEAAACCAAAAgwAAAIQAAACFAAAAhgAAAIgAAACJAAAAigAAAIsAAACMAAAAjQAAAI4AAACPAAAAkQAAAJIAAACUAAAAlQAAAJcAAACZAAAAmwAAAJ0AAACfAAAAowAAAKUAAACnAAAAqQAAAKsAAACsAAAArgAAAK8AAACwAAAAsgAAALQAAAC3AAAAuAAAALoAAAC8AAAAvgAAAMAAAADCAAAAxgAAAMgAAADKAAAAzAAAAM4AAADPAAAA0QAAANMAAADVAAAAAAAAAAEAAAACAAAABQAAAAgAAAANAAAADwAAABEAAAAUAAAAFwAAABkAAAAcAAAAHwAAACMAAAAnAAAALAAAAC4AAAAyAAAANQAAADoAAABAAAAARgAAAEwAAABSAAAAWQAAAGEAAABoAAAAbgAAAHMAAAB5AAAAfwAAAIUAAACKAAAAkQAAAJkAAAChAAAAqQAAALEAAAC6AAAAxAAAAM0AAADVAAAA3AAAAN8AAADiAAAA5AAAAOcAAADsAAAA8wAAAPsAAAADAQAACwEAABMBAAAcAQAAJgEAAC8BAAA3AQAAPgEAAEEBAABHAQAATAEAAFEBAABTAQAAVQEAAFgBAABbAQAAXAEAAF4BAABjAQAAaAEAAG4BAAB0AQAAeQEAAH4BAACBAQAAhwEAAIwBAACTAQAAmwEAAKMBAACrAQAAswEAALwBAADGAQAAzwEAANcBAADeAQAA4AEAAOIBAADlAQAA6QEAAO0BAADwAQAA9QEAAPwBAAACAgAACAIAAAoCAAAQAgAAFQIAABwCAAAiAgAAJQIAACoCAAAyAgAAOQIAAD8CAABIAgAAUAIAAFoCAABjAgAAbQIAAHYCAACAAgAAiQIAAJMCAACeAgAApwIAALECAAC9AgAAyAIAANMCAADdAgAA5wIAAPACAAD5AgAAAQMAAAMDAAAHAwAADgMAABQDAAAcAwAAJQMAAC4DAAA3AwAAQAMAAEoDAABVAwAAXwMAAGgDAABwAwAAdQMAAHkDAACAAwAAhQMAAIoDAACNAwAAjwMAAJEDAACWAwAAmgMAAJ4DAACkAwAAqgMAALEDAAC4AwAAvwMAAMYDAADNAwAA1AMAANsDAADjAwAA6gMAAPIDAAD7AwAABAQAAAwEAAAUBAAAGwQAACIEAAAoBAAALgQAADEEAAA2BAAAOwQAAD4EAABCBAAARgQAAEoEAABQBAAAVgQAAFsEAABgBAAAZgQAAGsEAABxBAAAdgQAAH4EAACFBAAAjgQAAJYEAACfBAAApwQAALAEAAC4BAAAwQQAAMsEAADTBAAA3AQAAOcEAADxBAAA+wQAAAQFAAANBQAAFQUAAB0FAAAkBQAAJgUAACwFAAAyBQAANQUAADkFAAA9BQAAQQUAAAMNAwsWDQsWAxUBHBYCFwMXAxcJAhcJAh8UAhcUAxcUFREcFBUBFAkRFRIfDR8NHxIXDR8SDR8SDR8NHxIOBiMNHxIOBBwNHxIXBhcNHxIkAxcNHxIQJAMXDR8SHQsjAxcNHxIdERQdDR8SIBwSDR8SHyIRFQsjAxcSFQsjAxcOEQ4fEhcOEQ4fEg4RDh8SDR8OEQ4fEg4GIw4RDh8SDgQcDhEOHxIXBhcOEQ4fEiQDFw4RDh8SECQDFw4RDh8SHQsjAxcOEQ4fEh0RFB0OEQ4fEiAcEg4RDh8SHyIOBiMOBBwODA4cDA4EHB8SDgQcHxINHw4EHB8SDgYjDgQcHxIOBBwOBBwfEhcGFw4EHB8SJAMXDgQcHxIQJAMXDgQcHxIdCyMDFw4EHB8SHREUHQ4EHB8SIBwSDgQcHxIfIg4EHA4EHB8SFw4cBgkSDhwGCQ0FDAUcBQwmBRwmBhYSFhEXAx8WBhciHxYGFyUiHxYRFwMfHRYDFwkSFgMXCQ0XBhcXBhcfEhcXBhcfEhcGFx8SDR8XBhcfEg4GIxcGFx8SDgQcFwYXHxIXBhcXBhcfEiQDFxcGFx8SECQDFxcGFx8SHQsjAxcXBhcfEh0RFB0XBhcfEiAcEhcGFx8SHyIBFwQXJAMXECQDFxsSCxYbBCYcEh0LHxwSHR8BHB8dAh8MCRIdAh8RCQ0dDR0LFAMXCR0LFAMXHQsUAxcJJh0LFAMXJh0LHx0LIwMXHQsjAxcfEhcdCyMDFx8SHQsjAxcSHQsjAxcfEg0fHQsjAxcSDR8dCyMDFx8SDgYjHQsjAxcSDgYjHQsjAxcfEg4EHB0LIwMXEg4EHB0LIwMXHxIXBhcdCyMDFxIXBhcdCyMDFx8SJAMXHQsjAxcfEhAkAxcdCyMDFxIkAxcdCyMDFxIQJAMXHQsjAxcfEh0LIwMXHQsjAxcSHQsjAxcdCyMDFx8SHREUHR0LIwMXEh0RFB0dCyMDFx8SIBwSHQsjAxcSIBwSHQsjAxcfEh8iHQsjAxcSHyIeGR0RFB0dERQdHxIXHREUHR8SHREUHR8SDR8dERQdHxIOBiMdERQdHxIOBBwdERQdHxIXBhcdERQdHxIkAxcdERQdHxIQJAMXHREUHR8SHQsjAxcdERQdHxIdERQdHREUHR8SIBwSHREUHR8SHyIdHwEcHx0fARsdHwEbJAEIHQMXCQ0dAxcJEh8LFwoDChIgDB8SFyAMCRIgDB8SIAwJEg0fIAwfEg0fIAwJEg4GIyAMHxIOBiMgDAkSDgQcIAwfEg4EHCAMCRIXBhcgDB8SFwYXIAwJEiQDFyAMCRIQJAMXIAwfEiQDFyAMHxIQJAMXIAwJEh0LIwMXIAwfEh0LIwMXIAwJEh0RFB0gDB8SHREUHSAMCRIgHBIgDB8SIBwSIAwJEh8iIAwfEh8iIBwSIAwmCQ0gDCYJEh8GFh8GFgwfAwkNHyIJDR8DFgEcGR8iFgEcGR8iJgkSHyImCQ0fJSImCQ0fJAsVIx8kCxcfEh8kCxcSHyQLFx8SDR8fJAsXEg0fHyQLFx8SDgYjHyQLFxIOBiMfJAsXHxIOBBwfJAsXEg4EHB8kCxcfEhcGFx8kCxcSFwYXHyQLFx8SJAMXHyQLFx8SECQDFx8kCxcSJAMXHyQLFxIQJAMXHyQLFx8SHQsjAxcfJAsXEh0LIwMXHyQLFx8SHREUHR8kCxcSHREUHR8kCxcfEiAcEh8kCxcSIBwSHyQLFx8SHyIfJAsXEh8iHyIkCxcmCRIkCxcmCQ0kAx8QJAMfJhEcGSYSHBkAAAAgAAAAAAAAAAEAAAADAAAABAAAAA0AAABxAAAA1QAAADkBAACdAQAAAQIAAAcCAAAOAgAAEAIAABICAAAUAgAAFQIAABYCAAAXAgAAGAIAABkCAAAaAgAAGwIAABwCAAAdAgAAHgIAAB8CAAAgAgAAIQIAACICAAAjAgAAJAIAACUCAAAmAgAACQAAAAEAAABGAAAAWwAAAH0AAAB+AAAANwAAAH8AAACMAAAAegAAADAAAABKAAAAawAAAI4AAABFAAAAiwAAAHkAAAAuAAAAIgAAAFwAAABPAAAACgAAADgAAABsAAAAFgAAAIAAAABuAAAALwAAABcAAABdAAAAUAAAAAsAAAA5AAAAgQAAAIYAAACKAAAAiQAAAIQAAACDAAAAiAAAAIcAAACCAAAAhQAAAG8AAAB0AAAAeAAAAHcAAAByAAAAcQAAAHYAAAB1AAAAcAAAAHMAAAAkAAAAKQAAAC0AAAAsAAAAJwAAACYAAAArAAAAKgAAACUAAAAoAAAAGAAAAB0AAAAhAAAAIAAAABsAAAAaAAAAHwAAAB4AAAAZAAAAHAAAAF4AAABjAAAAZwAAAGYAAABhAAAAYAAAAGUAAABkAAAAXwAAAGIAAABRAAAAVgAAAFoAAABZAAAAVAAAAFMAAABYAAAAVwAAAFIAAABVAAAADAAAABEAAAAVAAAAFAAAAA8AAAAOAAAAEwAAABIAAAANAAAAEAAAADoAAAA/AAAAQwAAAEIAAAA9AAAAPAAAAEEAAABAAAAAOwAAAD4AAACOAAAARQAAAIsAAAB5AAAALgAAACIAAABcAAAATwAAAAoAAAA4AAAAbAAAABYAAACAAAAAbgAAAC8AAAAXAAAAXQAAAFAAAAALAAAAOQAAAIEAAACGAAAAigAAAIkAAACEAAAAgwAAAIgAAACHAAAAggAAAIUAAABvAAAAdAAAAHgAAAB3AAAAcgAAAHEAAAB2AAAAdQAAAHAAAABzAAAAJAAAACkAAAAtAAAALAAAACcAAAAmAAAAKwAAACoAAAAlAAAAKAAAABgAAAAdAAAAIQAAACAAAAAbAAAAGgAAAB8AAAAeAAAAGQAAABwAAABeAAAAYwAAAGcAAABmAAAAYQAAAGAAAABlAAAAZAAAAF8AAABiAAAAUQAAAFYAAABaAAAAWQAAAFQAAABTAAAAWAAAAFcAAABSAAAAVQAAAAwAAAARAAAAFQAAABQAAAAPAAAADgAAABMAAAASAAAADQAAABAAAAA6AAAAPwAAAEMAAABCAAAAPQAAADwAAABBAAAAQAAAADsAAAA+AAAAjgAAAEUAAACLAAAAeQAAAC4AAAAiAAAAXAAAAE8AAAAKAAAAOAAAAGwAAAAWAAAAgAAAAG4AAAAvAAAAFwAAAF0AAABQAAAACwAAADkAAACBAAAAhgAAAIoAAACJAAAAhAAAAIMAAACIAAAAhwAAAIIAAACFAAAAbwAAAHQAAAB4AAAAdwAAAHIAAABxAAAAdgAAAHUAAABwAAAAcwAAACQAAAApAAAALQAAACwAAAAnAAAAJgAAACsAAAAqAAAAJQAAACgAAAAYAAAAHQAAACEAAAAgAAAAGwAAABoAAAAfAAAAHgAAABkAAAAcAAAAXgAAAGMAAABnAAAAZgAAAGEAAABgAAAAZQAAAGQAAABfAAAAYgAAAFEAAABWAAAAWgAAAFkAAABUAAAAUwAAAFgAAABXAAAAUgAAAFUAAAAMAAAAEQAAABUAAAAUAAAADwAAAA4AAAATAAAAEgAAAA0AAAAQAAAAOgAAAD8AAABDAAAAQgAAAD0AAAA8AAAAQQAAAEAAAAA7AAAAPgAAAI4AAABFAAAAiwAAAHkAAAAuAAAAIgAAAFwAAABPAAAACgAAADgAAABsAAAAFgAAAIAAAABuAAAALwAAABcAAABdAAAAUAAAAAsAAAA5AAAAgQAAAIYAAACKAAAAiQAAAIQAAACDAAAAiAAAAIcAAACCAAAAhQAAAG8AAAB0AAAAeAAAAHcAAAByAAAAcQAAAHYAAAB1AAAAcAAAAHMAAAAkAAAAKQAAAC0AAAAsAAAAJwAAACYAAAArAAAAKgAAACUAAAAoAAAAGAAAAB0AAAAhAAAAIAAAABsAAAAaAAAAHwAAAB4AAAAZAAAAHAAAAF4AAABjAAAAZwAAAGYAAABhAAAAYAAAAGUAAABkAAAAXwAAAGIAAABRAAAAVgAAAFoAAABZAAAAVAAAAFMAAABYAAAAVwAAAFIAAABVAAAADAAAABEAAAAVAAAAFAAAAA8AAAAOAAAAEwAAABIAAAANAAAAEAAAADoAAAA/AAAAQwAAAEIAAAA9AAAAPAAAAEEAAABAAAAAOwAAAD4AAACOAAAARQAAAIsAAAB5AAAALgAAACIAAABcAAAATwAAAAoAAAA4AAAAbAAAABYAAACAAAAAbgAAAC8AAAAXAAAAXQAAAFAAAAALAAAAOQAAAIEAAACGAAAAigAAAIkAAACEAAAAgwAAAIgAAACHAAAAggAAAIUAAABvAAAAdAAAAHgAAAB3AAAAcgAAAHEAAAB2AAAAdQAAAHAAAABzAAAAJAAAACkAAAAtAAAALAAAACcAAAAmAAAAKwAAACoAAAAlAAAAKAAAABgAAAAdAAAAIQAAACAAAAAbAAAAGgAAAB8AAAAeAAAAGQAAABwAAABeAAAAYwAAAGcAAABmAAAAYQAAAGAAAABlAAAAZAAAAF8AAABiAAAAUQAAAFYAAABaAAAAWQAAAFQAAABTAAAAWAAAAFcAAABSAAAAVQAAAAwAAAARAAAAFQAAABQAAAAPAAAADgAAABMAAAASAAAADQAAABAAAAA6AAAAPwAAAEMAAABCAAAAPQAAADwAAABBAAAAQAAAADsAAAA+AAAAaAAAAGkAAABHAAAASAAAAEkAAABbAAAAaAAAAEcAAABpAAAASAAAAFsAAABJAAAABwAAADEAAAAyAAAANQAAADYAAABMAAAATQAAAAAAAAACAAAAAwAAAAQAAAAFAAAABgAAAAgAAAAjAAAAMwAAADQAAABEAAAASwAAAE4AAABqAAAAbQAAAHsAAAB8AAAAjQAAAAAAAAAMAAAAEQAAAB0AAAAhAAAANAAAAEcAAABaAAAAbQAAAIAAAACQAAAAnAAAAJ0AAACeAAAAnwAAAKAAAAChAAAAogAAAKMAAACkAAAApQAAAKYAAACnAAAAqAAAAKkAAACqAAAAqwAAAKwAAACtAAAArgAAAK8AAACwAAAAsQAAAGFsYXJtQWN0aW9uAGFtUG0AY2xvY2tBY3Rpb24AZGF5AHB2LlR3b0RpZ2l0SW50ZWdlcgBwdi5Ud29EaWdpdEludGVnZXIAcHYuVHdvRGlnaXRJbnRlZ2VyAHB2LlR3b0RpZ2l0SW50ZWdlcgBwdi5Ud29EaWdpdEludGVnZXIAc3RvcHdhdGNoQWN0aW9uAHRpbWVyQWN0aW9uAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHAAAADAAAABMAAAAXAAAAHAAAACIAAAApAAAAMQAAADkAAABAAAAARwAAAEgAAABJAAAASgAAAEsAAABMAAAATQAAAE4AAABPAAAAUAAAAFEAAABSAAAAUwAAAFQAAABVAAAAVgAAAFcAAABYAAAAWQAAAFoAAABbAAAAXAAAAGFjdGlvbgBhbVBtAGFjdGlvbgBkYXkAaG91cgBob3VycwBtaW51dGUAbWludXRlcwBzZWNvbmRzAGFjdGlvbgBhY3Rpb24AAAAAAAAAAAAAAAAAAAAAAAAAAAAABwAAAAAAAAAGAAAAGAAAAB4AAAAnAAAAMAAAADoAAABAAAAAYWxhcm0AYXZhaWxhYmxlQ29tbWFuZHMAY2xvY2sAc2V0QWxhcm0Ac2V0VGltZXIAc3RvcHdhdGNoAHRpbWVyAAQAAAAAAAAA//////////8GAAAAKAAAAEwAAABwAAAAXAEAAAQKAAAoCgAAAAAAAAoAAAD/////AQAAADwAAAAoAAAAHgAAAAYAAAAAAAAAAAAAAAkAAAD/////AQAAAGAAAABMAAAAGwAAAAUAAAAAAAAAAAAAAAIAAAD/////BAAAAJAAAAAEAQAAPAEAAEwBAABwAAAAFwAAAP////8DAAAArAAAAOQAAAD0AAAAkAAAABwAAAD/////AgAAAMQAAADUAAAArAAAABQAAAACAAAAAAAAAKwAAAAdAAAAAgAAAAAAAACQAAAAFAAAAAIAAAAAAAAAkAAAAB0AAAACAAAAAAAAAHAAAAAcAAAA/////wIAAAAcAQAALAEAAAQBAAAUAAAAAgAAAAAAAAAEAQAAHQAAAAIAAAAAAAAAcAAAABQAAAACAAAAAAAAAHAAAAAdAAAAAgAAAAAAAAAAAAAAGgAAAP////8EAAAAfAEAAAwEAACIBgAAUAgAAFwBAAAOAAAA/////wEAAACQAQAAfAEAAB4AAAD/////AQAAAKQBAACQAQAAFQAAAP////8DAAAAwAEAAGADAADoAwAApAEAAAUAAAD/////AQAAANQBAADAAQAACwAAAAQAAAADAAAA8AEAALQCAAA8AwAA1AEAABEAAAD/////AgAAAAgCAACQAgAA8AEAAAcAAAD/////AQAAABwCAAAIAgAADAAAAAQAAAACAAAANAIAAGwCAAAcAgAAEQAAAP////8BAAAASAIAADQCAAAIAAAA/////wEAAABcAgAASAIAAA0AAAAEAAAAAAAAABwCAAAIAAAA/////wEAAACAAgAAbAIAAA0AAAAEAAAAAAAAAPABAAAIAAAA/////wEAAACkAgAAkAIAAA0AAAAEAAAAAAAAANQBAAAHAAAA/////wEAAADIAgAAtAIAAAwAAAAEAAAAAgAAAOACAAAYAwAAyAIAABEAAAD/////AQAAAPQCAADgAgAACAAAAP////8BAAAACAMAAPQCAAANAAAABAAAAAAAAADIAgAACAAAAP////8BAAAALAMAABgDAAANAAAABAAAAAAAAADUAQAACAAAAP////8BAAAAUAMAADwDAAANAAAABAAAAAAAAACkAQAABwAAAP////8BAAAAdAMAAGADAAAMAAAABAAAAAIAAACMAwAAxAMAAHQDAAARAAAA/////wEAAACgAwAAjAMAAAgAAAD/////AQAAALQDAACgAwAADQAAAAQAAAAAAAAAdAMAAAgAAAD/////AQAAANgDAADEAwAADQAAAAQAAAAAAAAApAEAAAgAAAD/////AQAAAPwDAADoAwAADQAAAAQAAAAAAAAAXAEAAB4AAAD/////AQAAACAEAAAMBAAAFQAAAP////8DAAAAPAQAANwFAABkBgAAIAQAAAUAAAD/////AQAAAFAEAAA8BAAACwAAAAQAAAADAAAAbAQAADAFAAC4BQAAUAQAABEAAAD/////AgAAAIQEAAAMBQAAbAQAAAcAAAD/////AQAAAJgEAACEBAAADAAAAAQAAAACAAAAsAQAAOgEAACYBAAAEQAAAP////8BAAAAxAQAALAEAAAIAAAA/////wEAAADYBAAAxAQAAA0AAAAEAAAAAAAAAJgEAAAIAAAA/////wEAAAD8BAAA6AQAAA0AAAAEAAAAAAAAAGwEAAAIAAAA/////wEAAAAgBQAADAUAAA0AAAAEAAAAAAAAAFAEAAAHAAAA/////wEAAABEBQAAMAUAAAwAAAAEAAAAAgAAAFwFAACUBQAARAUAABEAAAD/////AQAAAHAFAABcBQAACAAAAP////8BAAAAhAUAAHAFAAANAAAABAAAAAAAAABEBQAACAAAAP////8BAAAAqAUAAJQFAAANAAAABAAAAAAAAABQBAAACAAAAP////8BAAAAzAUAALgFAAANAAAABAAAAAAAAAAgBAAABwAAAP////8BAAAA8AUAANwFAAAMAAAABAAAAAIAAAAIBgAAQAYAAPAFAAARAAAA/////wEAAAAcBgAACAYAAAgAAAD/////AQAAADAGAAAcBgAADQAAAAQAAAAAAAAA8AUAAAgAAAD/////AQAAAFQGAABABgAADQAAAAQAAAAAAAAAIAQAAAgAAAD/////AQAAAHgGAABkBgAADQAAAAQAAAAAAAAAXAEAABAAAAD/////AQAAAJwGAACIBgAADwAAAP////8BAAAAsAYAAJwGAAAVAAAA/////wIAAADIBgAAjAcAALAGAAAEAAAA/////wIAAADgBgAAQAcAAMgGAAAGAAAA/////wEAAAD0BgAA4AYAAAEAAAADAAAAAgAAAAwHAAAwBwAA9AYAABgAAAD/////AQAAACAHAAAMBwAAAwAAAAMAAAAAAAAA9AYAAAMAAAADAAAAAAAAAMgGAAABAAAAAwAAAAIAAABYBwAAfAcAAEAHAAAYAAAA/////wEAAABsBwAAWAcAAAMAAAADAAAAAAAAAEAHAAADAAAAAwAAAAAAAACwBgAAAwAAAP////8CAAAApAcAAAQIAACMBwAAEgAAAP////8BAAAAuAcAAKQHAAAEAAAA/////wIAAADQBwAA9AcAALgHAAAGAAAA/////wEAAADkBwAA0AcAAAEAAAADAAAAAAAAALgHAAABAAAAAwAAAAAAAACMBwAABAAAAP////8CAAAAHAgAAEAIAAAECAAABgAAAP////8BAAAAMAgAABwIAAABAAAAAwAAAAAAAAAECAAAAQAAAAMAAAAAAAAAXAEAAA8AAAD/////AQAAAGQIAABQCAAAFQAAAP////8CAAAAfAgAAEAJAABkCAAABAAAAP////8CAAAAlAgAAPQIAAB8CAAABgAAAP////8BAAAAqAgAAJQIAAABAAAAAwAAAAIAAADACAAA5AgAAKgIAAAYAAAA/////wEAAADUCAAAwAgAAAMAAAADAAAAAAAAAKgIAAADAAAAAwAAAAAAAAB8CAAAAQAAAAMAAAACAAAADAkAADAJAAD0CAAAGAAAAP////8BAAAAIAkAAAwJAAADAAAAAwAAAAAAAAD0CAAAAwAAAAMAAAAAAAAAZAgAAAMAAAD/////AgAAAFgJAAC4CQAAQAkAABIAAAD/////AQAAAGwJAABYCQAABAAAAP////8CAAAAhAkAAKgJAABsCQAABgAAAP////8BAAAAmAkAAIQJAAABAAAAAwAAAAAAAABsCQAAAQAAAAMAAAAAAAAAQAkAAAQAAAD/////AgAAANAJAAD0CQAAuAkAAAYAAAD/////AQAAAOQJAADQCQAAAQAAAAMAAAAAAAAAuAkAAAEAAAADAAAAAAAAAAAAAAAAAAAA/////wEAAAAYCgAABAoAAA8AAAAAAAAAAAAAAAAAAAAfAAAA/////wEAAAA8CgAAKAoAABMAAAD/////AQAAAFAKAAA8CgAAFgAAAP////8BAAAAZAoAAFAKAAAZAAAAAQAAAAAAAAA="

      function writeMessage(message) {
        console.log(message)
        let p = document.createElement("p")
        let text = document.createTextNode(message)
        p.appendChild(text)
        document.body.appendChild(p)
      }

      async function startRhino() {
        writeMessage("Rhino is loading. Please wait...")
        window.rhinoClockWorker = await RhinoWebEnWorker.RhinoWorkerFactory.create(
          {
            context: {
              base64: CLOCK_CONTEXT_64,
              sensitivity: 0.5,
            },
            start: false,
          }
        )

        writeMessage("Rhino worker ready!")

        window.rhinoClockWorker.onmessage = msg => {
          if (msg.data.command === "rhn-inference") {
            writeMessage(
              "Inference detected: " + JSON.stringify(msg.data.inference)
            )
            window.rhinoClockWorker.postMessage({ command: "pause" })
            document.getElementById("push-to-talk").disabled = false
            writeMessage(
              "Rhino is paused. Press the 'Push to Talk' button to speak again."
            )
          }
        }

        writeMessage(
          "WebVoiceProcessor initializing. Microphone permissions requested ..."
        )

        try {
          let webVp = await WebVoiceProcessor.WebVoiceProcessor.init({
            engines: [window.rhinoClockWorker],
          })
          writeMessage(
            "WebVoiceProcessor ready! Press the 'Push to Talk' button to talk."
          )
        } catch (e) {
          writeMessage("WebVoiceProcessor failed to initialize: " + e)
        }
      }

      document.addEventListener("DOMContentLoaded", function () {
        startRhino()
        document.getElementById("push-to-talk").onclick = function (event) {
          writeMessage("Rhino is listening for your commands ...")
          this.disabled = true
          window.rhinoClockWorker.postMessage({ command: "resume" })
        }
      })
    </script>
  </head>
  <body>
    <h1>Rhino Web Demo</h1>
    <p>This demo uses Rhino for Web and the WebVoiceProcessor to:</p>
    <ol>
      <li>
        Create an English instance of Rhino that understands commands in the
        "Pico Clock" context;
      </li>
      <li>
        Acquire microphone (& ask permission) data stream and convert to voice
        processing format (16kHz 16-bit linear PCM). The downsampled audio is
        forwarded to the Rhino engines. The audio <i>does not</i> leave the
        browser: all processing is occurring via the Rhino WebAssembly code.
      </li>
      <li>
        Await inference events from the Rhino engine and output them to the
        page. When the inference is concluded, the push-to-talk button is
        enabled again.
      </li>
    </ol>
    <hr />
    <button id="push-to-talk">Push to Talk</button>
  </body>
</html>

Factory

If you wish to build your own worker/worklet, or perhaps not use workers at all, use the factory packages. This will let you instantiate Rhino engine instances directly.

Picovoice engines require 16-bit 16kHz linearly-encoded PCM (single-channel) audio.

The audio passed to the worker must be of the correct format. The WebVoiceProcessor handles downsampling in the examples above. If you are not using that, you must ensure you do it yourself.

E.g.:

import { Rhino } from "@picovoice/rhino-web-en-worker"

async function startRhino() {
  const handle = await Rhino.create(
    { context: /* Base64 representation of a .rhn context */ },
  )

  // Send Rhino frames of audio (check handle.frameLength for size of array)
  const audioFrames = new Int16Array(/* Provide data with correct format and size */)
  const rhinoInference = handle.process(audioFrames)
  // rhinoInference: isFinalized = true when
  // Rhino has concluded
}

startRhino()

The Rhino factory returns instances with the RhinoEngine interface:

export interface RhinoEngine {
  release(): void
  process(frames: Int16Array): RhinoInference
  version: string
  sampleRate: number
  frameLength: number
}

Results

Ultimately, when Rhino produces a conclusion, it will return a RhinoInference object:

export type RhinoInference = {
  /** Rhino has concluded the inference (isUnderstood is now set) */
  isFinalized: boolean
  /** The intent was understood (it matched an expression in the context) */
  isUnderstood?: boolean
  /** The name of the intent */
  intent?: string
  /** Map of the slot variables and values extracted from the utterance */
  slots?: Record<string, string>
}