Rhino - NodeJS Quick Start
Requirements
- NodeJS 10+
- yarn (or npm)
Compatibility
- Linux (x86_64)
- macOS (x86_64)
- Raspberry Pi (2,3,4)
Web Browsers
Looking to run Picovoice in-browser? See the JavaScript WebAssembly demo instead.
Setup
Cloning the Repository
If using SSH, clone the repository with:
git clone [email protected]:Picovoice/rhino.git
If using HTTPS, then type
git clone https://github.com/Picovoice/rhino.git
Microphone
macOS
See the documentation for node-record-lpm16 for instructions on installing SoX.
Linux & Raspberry Pi
Connect the microphone and get the list of available input audio devices:
arecord -L
The output will be similar to below
nullDiscard all samples (playback) or generate zero samples (capture)defaultmicsysdefault:CARD=DeviceUSB PnP Sound Device, USB AudioDefault Audio Devicehw:CARD=Device,DEV=0USB PnP Sound Device, USB AudioDirect hardware device without any conversionsplughw:CARD=Device,DEV=0USB PnP Sound Device, USB AudioHardware device with all software conversions
In this case, we pick plughw:CARD=Device,DEV=0
. Note that this device comes with software conversions which are handy
for resampling. In what follows we note this value as ${INPUT_AUDIO_DEVICE}
.
create ~/.asoundrc
pcm.!default {type asymcapture.pcm "mic"}pcm.mic {type plugslave {pcm ${INPUT_AUDIO_DEVICE}}}
If you have a speaker add a section for that to ~/.asoundrc
as well.
Check if the microphone works properly by recording audio into a file:
arecord --format=S16_LE --duration=5 --rate=16000 --file-type=wav ~/test.wav
If the command above executes without any errors, then the microphone is functioning as expected. We recommend inspecting the recorded file for recording side effects such as clipping.
Demo Applications
The Rhino NodeJS demo package provides two demonstration command-line applications: a file based demo, which scans a compatible WAV file for speech commands, and a microphone demo, which listens for commands and and outputs detections live.
Install the demo package using the global
argument, so that will be available on the command line:
yarn global add @picovoice/rhino-node-demo
or
npm install -g @picovoice/rhino-node-demo
Microphone Demo
The microphone demo allows you monitor microphone input for commands using Rhino. For microphone access, the node-record-lpm16 package is used. Please follow that documentation for troubleshooting. The node-record-lpm16 library spawns a different microphone recording process depending on the OS used. The microphone program (SoX or Arecord) must be setup manually and is not included with yarn/npm.
Using the 'global' install methods above should add rhn-mic-demo
to your system path, which we can use to run the mic demo. Specify the Speech-to-Intent context (.rhn file) with --context_path
:
Here is an example which will understand commands from the "Smart Lighting" demo from the Rhino GitHub repository. Note that context files are platform-dependent. Choose the appropriate one for the platform you are using. This demo uses the "mac" version. From the root of repository execute:
rhn-mic-demo --context_path resources/contexts/mac/smart_lighting_mac.rhn
The context source in YAML format will be output to show you the grammar that the context supports:
Context info:-------------context:expressions:changeColor:- (please) [change, set, switch] (the) $location:location (to) $color:color- (please) [change, set, switch] (the) $location:location color (to) $color:color- (please) [change, set, switch] (the) $location:location lights (to) $color:color...
The demo will listen for a phrase that the contexts understands, and upon reaching a conclusion (or timeout), it will output the results.
Platform: 'mac'; attempting to use 'sox' to access microphone ...Listening for speech within the context of 'smart_lighting_mac'. Please speak your phrase into the microphone.# (say e.g. "please turn on the lights in the kitchen")...Inference result:{"isUnderstood": true,"intent": "changeLightState","slots": {"state": "on","location": "kitchen"}}
Try running the mic demo again, but this time say something that it is not designed to understand, like "tell me a joke":
Platform: 'mac'; attempting to use 'sox' to access microphone ...Listening for speech within the context of 'smart_lighting_mac'. Please speak your phrase into the microphone.# (say e.g. "tell me a joke")Inference result:{"isUnderstood": false}
File Demo
The file-based demo allows you to scan a compatible wave file with Rhino. To run the file-based demo, we need to provide a Speech-to-Intent context along with a path to a compatible wave file.
We can use a couple of test wave files that are bundled in the Rhino GitHub repository. These are intended to be used with the sample "Coffee Maker" context, also available in the Rhino GitHub repository. Note that context files are platform-dependent. Choose the appropriate one for the platform you are using. This demo uses the "mac" version.
Run the file demo and the successful inference with the intent "orderDrink" along with the specific details are returned. Run this from the root of the repository:
rhn-file-demo \--context_path resources/contexts/mac/coffee_maker_mac.rhn \--input_audio_file_path resources/audio_samples/test_within_context.wav
Inference result of 'test_within_context.wav' using context 'coffee':{"isUnderstood": true,"intent": "orderDrink","slots": {"size": "medium","numberOfShots": "double shot","coffeeDrink": "americano","milkAmount": "lots of milk","sugarAmount": "some sugar"}}
Custom Context
You can create custom Rhino context models using Picovoice Console.