Rhino - NVIDIA Jetson Quick Start

Speech to Intent
NLU
Voice Recognition
Speech Recognition
Voice Commands
NVIDIA Jetson
Python
C

Requirements

Python 3

Compatibility

NVIDIA Jetson nano

Setup

Cloning the Repository

We need the Rhino repository to obtain pre-trained context models for showcasing the functionality. If you have a custom context file trained using Picovoice Console, you may skip this step.

If using SSH, clone the repository with:

git clone --recurse-submodules [email protected]:Picovoice/rhino.git

If using HTTPS, then type

git clone --recurse-submodules https://github.com/Picovoice/rhino.git

Microphone

Connect the microphone and get the list of available input audio devices:

arecord -L

The output will be similar to below

null
    Discard all samples (playback) or generate zero samples (capture)
default
mic
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions

In this case, we pick plughw:CARD=Device,DEV=0. Note that this device comes with software conversions which are handy for resampling. In what follows we note this value as ${INPUT_AUDIO_DEVICE}.

create ~/.asoundrc

pcm.!default {
  type asym
  capture.pcm "mic"
}
pcm.mic {
  type plug
  slave {
    pcm "${INPUT_AUDIO_DEVICE}"
  }
}

If you have a speaker add a section for that to ~/.asoundrc as well. You may need to reboot the system for these settings to take effect.

Check if the microphone works properly by recording audio into a file:

arecord --format=S16_LE --duration=5 --rate=16000 --file-type=wav ~/test.wav

If the command above executes without any errors, then the microphone is functioning as expected. We recommend inspecting the recorded file for recording side effects such as clipping.

Installation

The core of the Speech-to-Intent engine is shipped as a pre-compiled ANSI C library. Hence, it can be used within a C/C++ application directory or in a high-level language such as Python via its bindings.

Python

Install PyAudio, Portable audio I/O, and FFI library packages:

sudo apt-get install python3-pyaudio portaudio19-dev libffi-dev

Then install the demo package:

sudo pip3 install pvrhinodemo

The demo package will be installed in /user/local/bin, which should be in the PATH variable to run the demo from anywhere within a terminal. From the root of the repository, run the microphone demo application. It opens an input audio stream, monitors it using Picovoice wake word detection engine, and when the wake phrase ("Picovoice") is detected it will extract the intent within the follow-up spoken command using Speech-to-Intent engine.

rhino_demo_mic --context_path resources/contexts/jetson/smart_lighting_jetson.rhn

Now you can say something like "turn on the lights in the kitchen" and it outputs the result of inference into terminal

{
  intent : 'changeLightState'
  slots : {
    state : 'on'
    location : 'kitchen'
  }
}

C

Install ALSA development library

sudo apt-get install libasound-dev

Change the current directory to the root of the repository and compile the C demo application.

gcc -O3 -o demo/c/rhino_demo_mic -I include demo/c/rhino_demo_mic.c -ldl -lasound -std=c99

Then run the demo. It opens an input audio stream and extracts the intent within the spoken command using Speech-to-Intent engine.

demo/c/rhino_demo_mic \
lib/jetson/cortex57-aarch64/libpv_rhino.so \
lib/common/rhino_params.pv \
resources/contexts/jetson/smart_lighting_jetson.rhn \
${INPUT_AUDIO_DEVICE}

Now you can say something like "turn on the lights in the kitchen" and it outputs the result of inference into terminal

{
  intent : 'changeLightState'
  slots : {
    state : 'on'
    location : 'kitchen'
  }
}

Custom Context

You can create custom Rhino context models using Picovoice Console.