Porcupine - React API

This document outlines how to integrate the Porcupine wake word engine within an application using its React API.

Requirements

• React 16.8+
• yarn (or npm)

Compatibility

• Chrome, Edge
• Firefox
• Safari

The Picovoice SDKs for Web are powered by WebAssembly (WASM), the Web Audio API, and Web Workers.

All modern browsers (Chrome/Edge/Opera, Firefox, Safari) are supported, including on mobile. Internet Explorer is not supported.

Using the Web Audio API requires a secure context (HTTPS connection), with the exception of localhost, for local development.

Installation

Use npm or yarn to install the package and its peer dependencies. Each spoken language (e.g. 'en', 'de') is a separate package. For this example we'll use English:

yarn add @picovoice/porcupine-web-react @picovoice/porcupine-web-en-worker @picovoice/web-voice-processor

(or)

npm install @picovoice/porcupine-web-react @picovoice/porcupine-web-en-worker @picovoice/web-voice-processor

Language-specific Porcupine web worker packages

These worker packages are compatible with the React SDK:

• @picovoice/porcupine-web-de-worker
• @picovoice/porcupine-web-en-worker
• @picovoice/porcupine-web-es-worker
• @picovoice/porcupine-web-fr-worker

Usage

The Porcupine SDK for React is based on the Porcupine SDK for Web. The library provides a React hook: usePorcupine. The hook will take care of microphone access and audio downsampling (via @picovoice/web-voice-processor) and provide a wake word detection event to which your application can subscribe.

The usePorcupine hook provides a collection of fields and methods shown below. You can pass the keywordEventHandler to respond to Porcupine detection events.

The usePorcupine hook accepts three arguments:

1. The worker factory (language specific, imported as { PorcupineWorkerFactory } from the @picovoice/porcupine-web-xx-worker series of packages, where xx is the two-letter language code. (The Porcupine WebAssembly code, wrapped in a Web worker. Dependency injection is needed due to the size of the package)
2. The PorcupineHookArgs (what we want Porcupine to listen for; see below for details)
3. The callback function with signature (keywordLabel: string) => void (what to do when Porcupine detects our keyword)

Make sure you handle the possibility of errors with the isError and errorMessage fields. Users may not have a working microphone, and they can always decline (and revoke) permissions; your application code should anticipate these scenarios.

usePorcupine Hook Parameters

Porcupine accepts an array of (or a single) PorcupineKeyword object argument(s). If you provide null instead of PorcupineHookArgs, this will (deliberately) prevent startup (if it was already active, it will terminate Porcupine). You can use this to dynamically change the wake words and/or defer starting up at runtime.

export type PorcupineHookArgs = {
  /** Immediately start the microphone upon initialization? */
  start: boolean;
  /** Keywords to listen for */
  keywords: Array<PorcupineKeyword | string> | PorcupineKeyword | string;
}

PorcupineKeyword can be either PorcupineKeywordBuiltin or PorcupineKeywordCustom:

export type PorcupineKeywordBuiltin = {
  /** Name of a builtin keyword for the specific language (e.g. "Grasshopper" for English, or "Ananas" for German) */
  builtin: string
  /** Value in range [0,1] that trades off miss rate for false alarm */
  sensitivity?: number
}

If you simply pass a string of a builtin keyword instead of an object, that will also work.

Use PorcupineKeywordCustom for custom keywords:

export type PorcupineKeywordCustom = {
  /** Base64 representation of a trained Porcupine keyword (`.ppn` file) */
  base64: string
  /** An arbitrary label that you want Porcupine to report when the detection occurs */
  custom: string
  /** Value in range [0,1] that trades off miss rate for false alarm */
  sensitivity?: number
}

Imports

You can use Porcupine by importing the worker package statically or dynamically. Static is more straightforward to implement, but will impact your initial bundle size with an additional ~2MB. Depending on your requirements, this may or may not be feasible. If you require a small bundle size, see dynamic importing below.

Static Import

import React, { useState } from "react"
import { PorcupineWorkerFactory } from "@picovoice/porcupine-web-en-worker"
import { usePorcupine } from "@picovoice/porcupine-web-react"

const keywords = [{ builtin: "Picovoice", sensitivity: 0.65 }]

function VoiceWidget(props) {
  const keywordEventHandler = keywordLabel => {
    console.log(`Porcupine detected ${keywordLabel}`)
  }

  const {
    isLoaded,
    isListening,
    isError,
    errorMessage,
    start,
    resume,
    pause,
  } = usePorcupine(
    PorcupineWorkerFactory,
    { keywords, start: true },
    keywordEventHandler
  )
}

The keywordEventHandler will log the keyword detections to the browser's JavaScript console.

Important Note: Internally, usePorcupine performs work asynchronously to initialize, as well as asking for microphone permissions. Not until the asynchronous tasks are done and permission given will Porcupine actually be running. Therefore, it makes sense to use the isLoaded state to update your UI to let users know your application is actually ready to process voice (and isError in case something went wrong). Otherwise, they may start speaking and their audio data will not be processed, leading to a poor/inconsistent experience.

Dynamic Import / Code Splitting

If you are shipping Porcupine for Web and wish to avoid adding its ~2MB to your application's initial bundle, you can use dynamic imports. These will split off the porcupine-web-xx-worker packages into separate bundles and load them asynchronously. This means we need additional logic.

We add a useEffect hook to kick off the dynamic import. We store the result of the dynamically loaded worker chunk into a useState hook. When usePorcupine receives a non-null/undefined value for the worker factory, it will start up Porcupine.

See the Webpack docs for more information about Code Splitting.

import { useEffect, useState } from "react"
import { usePorcupine } from "@picovoice/porcupine-web-react"

const DEEP_SKY_BLUE_PPN_64 = /* base64 representation of "deep_sky_blue.ppn", omitted for brevity */

export default function VoiceWidget() {
  const [keywordDetections, setKeywordDetections] = useState([])
  const [workerChunk, setWorkerChunk] = useState({ workerFactory: null })
  const [keywords] = useState([
    { builtin: "Alexa", sensitivity: 0.7 },// Built-in keyword
    "Picovoice",// Built-in keyword
    { custom: "Deep Sky Blue", base64: DEEP_SKY_BLUE_PPN_64}// Custom keyword
  ])

  useEffect(() => {
    async function loadPorcupineWorkerChunk() {
      const ppnWorkerFactory = (
        await import("@picovoice/porcupine-web-en-worker")
      ).PorcupineWorkerFactory // <-- Dynamically import the worker
      console.log("Porcupine worker chunk is loaded.")
      return ppnWorkerFactory
    }

    if (workerChunk.factory === null) {
      // <-- We only want to load once!
      loadPorcupineWorkerChunk().then(ppnWorkerFactory => {
        setWorkerChunk({ workerFactory: ppnWorkerFactory })
      })
    }
  }, [workerChunk])

  const keywordEventHandler = porcupineKeywordLabel => {
    setKeywordDetections(x => [...x, porcupineKeywordLabel])
  }

  const {
    isLoaded,
    isListening,
    isError,
    errorMessage,
    start,
    resume,
    pause,
  } = usePorcupine(
    workerChunk.workerFactory, // <-- When this is null/undefined, it's ignored. Otherwise, usePorcupine will start.
    { keywords, start: true },
    keywordEventHandler
  )
}

Custom Wake Words

You can create custom wake word models using Picovoice Console. Making Porcupine wake word models for the Web platform requires an Enterprise account.