Picovoice SDK - React Web Quick Start

Platforms

Chrome & Chromium-based browsers
Edge
Firefox
Safari

Requirements

Picovoice Account and AccessKey
Node.js 12+
React 16.8+
npm

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

Install Node.js.
Install the npm packages:

npm install @picovoice/picovoice-web-react @picovoice/picovoice-web-en-worker @picovoice/web-voice-processor

Usage

Create a usePicovoice Hook that detects the built-in keyword Picovoice and infers intent from spoken commands within a given context :

import React, { useState } from 'react';
import { PicovoiceWorkerFactory } from '@picovoice/picovoice-web-en-worker';
import { usePicovoice } from '@picovoice/picovoice-web-react';

const ACCESS_KEY = ${ACCESS_KEY};
const RHN_CONTEXT_CLOCK_64 = ${RHN_MODEL_B64}; /* Base64 representation of English-language `clock_wasm.rhn`, omitted for brevity */

export default function VoiceWidget() {
  const [keywordDetections, setKeywordDetections] = useState([]);
  const [inference, setInference] = useState(null);

  const inferenceEventHandler = (rhinoInference) => {
    console.log(rhinoInference);
    setInference(rhinoInference);
  };

  const keywordEventHandler = (porcupineKeywordLabel) => {
    console.log(porcupineKeywordLabel);
    setKeywordDetections((x) => [...x, porcupineKeywordLabel]);
  };

  const {
    contextInfo,
    isLoaded,
    isListening,
    isError,
    errorMessage,
    start,
    pause,
    engine,
  } = usePicovoice(
    PicovoiceWorkerFactory,
    {
      accessKey: ACCESS_KEY, 
      porcupineKeyword: "Picovoice",
      rhinoContext: { base64: RHN_CONTEXT_CLOCK_64 },
      start: true,
    },
    keywordEventHandler,
    inferenceEventHandler
  );

  return (
    <div className="voice-widget">
        <h3>Engine: {engine}</h3>
        <h3>Keyword Detections:</h3>
        {keywordDetections.length > 0 && (
        <ul>
            {keywordDetections.map((label, index) => (
            <li key={index}>{label}</li>
            ))}
        </ul>
        )}
        <h3>Latest Inference:</h3>
        {JSON.stringify(inference)}
    </div>
  )
}

The Hook will take care of microphone access and audio downsampling ( via @picovoice/web-voice-processor).

Custom Models

Create custom keywords and contexts using the Picovoice Console. Train the Porcupine and Rhino models for the target platform WebAssembly (WASM). Download the custom model files. Inside the .zip files, there is a _b64.txt file which contains the binary model encoded with Base64 for both Porcupine and Rhino. Copy the base64 and provide them as arguments to Picovoice:

const DEEP_SKY_BLUE_PPN_64 = ${PPN_BASE64_MODEL}; /* Base64 representation of deep_sky_blue.ppn */
const CUSTOM_CONTEXT_RHN_64 = ${RHN_BASE64_MODEL}; /* Base64 representation of custom_context.rhn */

const [keywords] = useState([
  { base64: DEEP_SKY_BLUE_PPN_64, custom: "Deep Sky Blue"},
]);


const {
  contextInfo,
  isLoaded,
  isListening,
  isError,
  errorMessage,
  start,
  pause,
  engine,
} = usePicovoice(
  PicovoiceWorkerFactory,
  {
    accessKey: ACCESS_KEY,
    porcupineKeyword: keywords,
    rhinoContext: { base64: RHN_CONTEXT_CLOCK_64 },
    start: true,
  },
  keywordEventHandler,
  inferenceEventHandler
);

Non-English Languages

Import { PicovoiceWorkerFactory } from the @picovoice/picovoice-web-xx-worker series of packages, where xx is the two-letter language code:

import { PicovoiceWorkerFactory } from "@picovoice/picovoice-web-xx-worker";

These worker packages are compatible with the React SDK:

German: @picovoice/picovoice-web-de-worker
Spanish: @picovoice/picovoice-web-es-worker
French: @picovoice/picovoice-web-fr-worker

Demo

For the Picovoice React SDK, there is a React demo project available on the Picovoice GitHub repository.

Setup

Clone the Picovoice repository from GitHub:

git clone --recurse-submodules https://github.com/Picovoice/picovoice.git

Usage

Install dependencies and run:

cd picovoice/demo/react
npm install
npm run start

Picovoice SDK - React Web Quick Start

Platforms

Requirements

Picovoice Account & AccessKey

Quick Start

Setup

Usage

Custom Models

Non-English Languages

Demo

Setup

Usage

Resources

Packages

API

GitHub

Benchmark

Further Reading