Rhino - NodeJS Quick Start

Requirements

• NodeJS 10+
• yarn (or npm)

Compatibility

• Linux (x86_64)
• macOS (x86_64)
• Raspberry Pi (2,3,4)

Web Browsers

Looking to run Picovoice in-browser? See the JavaScript WebAssembly demo instead.

Setup

Cloning the Repository

If using SSH, clone the repository with:

git clone [email protected]:Picovoice/rhino.git

If using HTTPS, then type

git clone https://github.com/Picovoice/rhino.git

Microphone

macOS

See the documentation for node-record-lpm16 for instructions on installing SoX.

Linux & Raspberry Pi

Connect the microphone and get the list of available input audio devices:

arecord -L

The output will be similar to below

null
    Discard all samples (playback) or generate zero samples (capture)
default
mic
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions

In this case, we pick plughw:CARD=Device,DEV=0. Note that this device comes with software conversions which are handy for resampling. In what follows we note this value as ${INPUT_AUDIO_DEVICE}.

create ~/.asoundrc

pcm.!default {
  type asym
  capture.pcm "mic"
}
pcm.mic {
  type plug
  slave {
    pcm ${INPUT_AUDIO_DEVICE}
  }
}

If you have a speaker add a section for that to ~/.asoundrc as well.

Check if the microphone works properly by recording audio into a file:

arecord --format=S16_LE --duration=5 --rate=16000 --file-type=wav ~/test.wav

If the command above executes without any errors, then the microphone is functioning as expected. We recommend inspecting the recorded file for recording side effects such as clipping.

Demo Applications

The Rhino NodeJS demo package provides two demonstration command-line applications: a file based demo, which scans a compatible WAV file for speech commands, and a microphone demo, which listens for commands and and outputs detections live.

Install the demo package using the global argument, so that will be available on the command line:

yarn global add @picovoice/rhino-node-demo

or

npm install -g @picovoice/rhino-node-demo

Microphone Demo

The microphone demo allows you monitor microphone input for commands using Rhino. For microphone access, the node-record-lpm16 package is used. Please follow that documentation for troubleshooting. The node-record-lpm16 library spawns a different microphone recording process depending on the OS used. The microphone program (SoX or Arecord) must be setup manually and is not included with yarn/npm.

Using the 'global' install methods above should add rhn-mic-demo to your system path, which we can use to run the mic demo. Specify the Speech-to-Intent context (.rhn file) with --context_path:

Here is an example which will understand commands from the "Smart Lighting" demo from the Rhino GitHub repository. Note that context files are platform-dependent. Choose the appropriate one for the platform you are using. This demo uses the "mac" version. From the root of repository execute:

rhn-mic-demo --context_path resources/contexts/mac/smart_lighting_mac.rhn

The context source in YAML format will be output to show you the grammar that the context supports:

Context info:
-------------
context:
  expressions:
    changeColor:
      - (please) [change, set, switch] (the) $location:location (to) $color:color
      - (please) [change, set, switch] (the) $location:location color (to) $color:color
      - (please) [change, set, switch] (the) $location:location lights (to) $color:color
      ...

The demo will listen for a phrase that the contexts understands, and upon reaching a conclusion (or timeout), it will output the results.

Platform: 'mac'; attempting to use 'sox' to access microphone ...
Listening for speech within the context of 'smart_lighting_mac'. Please speak your phrase into the microphone.
 
# (say e.g. "please turn on the lights in the kitchen")
 
...
 
Inference result:
{
    "isUnderstood": true,
    "intent": "changeLightState",
    "slots": {
        "state": "on",
        "location": "kitchen"
    }
}

Try running the mic demo again, but this time say something that it is not designed to understand, like "tell me a joke":

Platform: 'mac'; attempting to use 'sox' to access microphone ...
Listening for speech within the context of 'smart_lighting_mac'. Please speak your phrase into the microphone.
 
# (say e.g. "tell me a joke")
 
Inference result:
{
    "isUnderstood": false
}

File Demo

The file-based demo allows you to scan a compatible wave file with Rhino. To run the file-based demo, we need to provide a Speech-to-Intent context along with a path to a compatible wave file.

We can use a couple of test wave files that are bundled in the Rhino GitHub repository. These are intended to be used with the sample "Coffee Maker" context, also available in the Rhino GitHub repository. Note that context files are platform-dependent. Choose the appropriate one for the platform you are using. This demo uses the "mac" version.

Run the file demo and the successful inference with the intent "orderDrink" along with the specific details are returned. Run this from the root of the repository:

rhn-file-demo \
--context_path resources/contexts/mac/coffee_maker_mac.rhn \
--input_audio_file_path resources/audio_samples/test_within_context.wav

Inference result of 'test_within_context.wav' using context 'coffee':
{
    "isUnderstood": true,
    "intent": "orderDrink",
    "slots": {
        "size": "medium",
        "numberOfShots": "double shot",
        "coffeeDrink": "americano",
        "milkAmount": "lots of milk",
        "sugarAmount": "some sugar"
    }
}

Custom Context

You can create custom Rhino context models using Picovoice Console.