unityPorcupine - Unity API

  • Wake Word Detection
  • Local Voice Commands
  • Offline Keyword Spotting
  • Always Listening
  • Voice Activation
  • Unity
  • C#
  • Desktop
  • Mobile
  • Windows
  • macOS
  • Linux
  • Android
  • iOS

This document outlines how to integrate the Porcupine wake word engine within an application using its Unity API.

Requirements

  • Unity 2017.4+

To deploy to iOS or Android, ensure you install the relevant Unity build support modules using Unity Hub.

Compatibility

  • Android 4.1+ (API 16+) (ARM only)
  • iOS 9.0+
  • Windows (x86_64)
  • macOS (x86_64)
  • Linux (x86_64)

Installation

The easiest way to install the Porcupine Unity SDK is to import porcupine.unitypackage into your Unity project, by either dropping it into the Unity editor or going to Assets>Import Package>Custom Package...

Packaging

To build the package from source, you have first have to clone the repo with submodules:

git clone --recurse-submodules [email protected]:Picovoice/porcupine.git
# or
git clone --recurse-submodules https://github.com/Picovoice/porcupine.git

Next, run the copy.sh file to copy the package resources from various locations in the repo to the Unity project located at /binding/unity.

Finally, open the Unity project, right click the Assets folder and select "Export Package". The resulting Unity package can be imported into other Unity projects as desired.

Usage

The module provides you with two levels of API to choose from depending on your needs.

High-Level API

PorcupineManager provides a high-level API that takes care of audio recording. This class is the quickest way to get started.

Using the constructor PorcupineManager.FromKeywords will create an instance of the PorcupineManager using one or more of the built-in keywords:

using Pv.Unity;
try {
List<string> keywords = new List<string>() { "picovoice", "porcupine" };
PorcupineManager _porcupineManager = PorcupineManager.FromKeywords(
keywords,
OnWakeWordDetected);
}
catch (Exception ex)
{
// handle porcupine init error
}

The wakeWordCallback parameter is a function that will be excecuted when Porcupine has detected one of the keywords. The function should accept a single integer, keywordIndex, which specifies which wake word has been detected.

private void OnWakeWordDetected(int keywordIndex) {
if (keywordIndex == 0) {
// picovoice detected
}
else if (keywordIndex === 1) {
// porcupine detected
}
}

Available built-in keywords are stored in the constants PorcupineManager.BUILT_IN_KEYWORDS and Porcupine.BUILT_IN_KEYWORDS.

To create an instance of PorcupineManager that detects custom keywords, you can use the PorcupineManager.FromKeywordPaths static constructor and provide the paths to the .ppn file(s).

List<string> keywordPaths = new List<string>(){ "/path/to/keyword.ppn" };
PorcupineManager _porcupineManager = PorcupineManager.FromKeywordPaths(
keywordPaths,
OnWakeWordDetected);

In addition to custom keywords, you can override the default Porcupine model file and/or keyword sensitivities, as well as add an error callback you want to trigger if there's a problem encountered while Porcupine is processing frames:

List<string> keywordPaths = new List<string>()
{
"/path/to/keyword/file/one.ppn",
"/path/to/keyword/file/two.ppn"
};
string modelPath = "path/to/model/file.pv";
List<float> sensitivites = new List<float>(){ 0.25f, 0.6f };
PorcupineManager _porcupineManager = PorcupineManager.FromKeywordPaths(
keywordPaths,
OnWakWordDetected,
modelPath: modelPath,
sensitivities: sensitivities,
errorCallback: OnError);
void OnError(Exception ex){
Debug.LogError(ex.ToString());
}

Once you have instantiated a PorcupineManager, you can start audio capture and wake word detection by calling:

_porcupineManager.Start();

Stop it by calling:

_porcupineManager.Stop();

Once the app is done using an instance of PorcupineManager, you can explicitly release the audio resources and the resources allocated to Porcupine:

_porcupineManager.Delete();

There is no need to deal with audio capture to enable wake word detection with PorcupineManager. This is because it uses our unity-voice-processor Unity package to capture frames of audio and automatically pass it to the wake word engine.

Low-Level API

Porcupine provides low-level access to the wake word engine for those who want to incorporate wake word detection into a different audio processing pipeline.

To create an instance of Porcupine, use the .Create static constructor. You can pass a list of built-in keywords as its keywords argument or a list or paths to custom keywords using its keywordPaths argument.

using Pv.Unity;
try
{
List<string> keywords = new List<string>(){ "porcupine", "picovoice" };
Porcupine _porcupine = Porcupine.Create(keywords: keywords);
}
catch (Exception ex)
{
// handle porcupine init error
}

To search for a keyword in audio, you must pass frames of audio to Porcupine using the Process function. The keywordIndex returned will either be -1 if no detection was made, or an integer specifying which keyword was detected.

short[] frame = getAudioFrame();
try
{
int keywordIndex = _porcupine.Process(frame);
if (keywordIndex >= 0)
{
// detection made!
}
}
catch (Exception ex)
{
Debug.LogError(ex.ToString());
}

For Process to work correctly, the audio data must be in the audio format required by Picovoice. The required sample rate is specified by the SampleRate property and the required number of audio samples in each frame is specified by the FrameLength property.

Porcupine implements the IDisposable interface, so you can use Porcupine in a using block. If you don't use a using block, resources will be released by the garbage collector automatically, or you can explicitly release the resources like so:

_porcupine.Dispose();

Custom Wake Words

You can create custom Porcupine wake word models using Picovoice Console.


Issue with this doc? Please let us know.