Rhino - NVIDIA Jetson Quick Start

Requirements

Python:

• Python 3

C:

• C99-compatible compiler
• CMake 3.4 or higher

Compatibility

• NVIDIA Jetson nano

Setup

Cloning the Repository

If using SSH, clone the repository with:

git clone --recurse-submodules [email protected]:Picovoice/rhino.git

If using HTTPS, then type

git clone --recurse-submodules https://github.com/Picovoice/rhino.git

Microphone

Connect the microphone and get the list of available input audio devices:

arecord -L

The output will be similar to below

null
    Discard all samples (playback) or generate zero samples (capture)
default
mic
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions

In this case, we pick plughw:CARD=Device,DEV=0. Note that this device comes with software conversions which are handy for resampling. In what follows we note this value as ${INPUT_AUDIO_DEVICE}.

create ~/.asoundrc

pcm.!default {
  type asym
  capture.pcm "mic"
}
pcm.mic {
  type plug
  slave {
    pcm "${INPUT_AUDIO_DEVICE}"
  }
}

If you have a speaker add a section for that to ~/.asoundrc as well. You may need to reboot the system for these settings to take effect.

Check if the microphone works properly by recording audio into a file:

arecord --format=S16_LE --duration=5 --rate=16000 --file-type=wav ~/test.wav

If the command above executes without any errors, then the microphone is functioning as expected. We recommend inspecting the recorded file for recording side effects such as clipping.

Demo Applications

The core of the Speech-to-Intent engine is shipped as a pre-compiled ANSI C library. Hence, it can be used within a C/C++ application directory or in a high-level language such as Python via its bindings.

Python

Install PyAudio, Portable audio I/O, and FFI library packages:

sudo apt-get install python3-pyaudio portaudio19-dev libffi-dev

Then install the demo package:

sudo pip3 install pvrhinodemo

The demo package will be installed in /user/local/bin, which should be in the PATH variable to run the demo from anywhere within a terminal. From the root of the repository, run the microphone demo application. It opens an input audio stream, monitors it using Picovoice wake word detection engine, and when the wake phrase ("Picovoice") is detected it will extract the intent within the follow-up spoken command using Speech-to-Intent engine.

rhino_demo_mic --context_path resources/contexts/jetson/smart_lighting_jetson.rhn

Now you can say something like "turn on the lights in the kitchen" and it outputs the result of inference into terminal

{
  intent : 'changeLightState'
  slots : {
    state : 'on'
    location : 'kitchen'
  }
}

C

Go to the root of the directory:

cd rhino

Compile the C demo application with CMake:

cmake -S demo/c/. -B demo/c/build && cmake --build demo/c/build --target rhino_demo_mic

List input audio devices with:

./demo/c/build/rhino_demo_mic --show_audio_devices

Replace ${AUDIO_DEVICE_INDEX} in the command below with the index of the audio device selected based on the previous command and run the demo:

./demo/c/build/rhino_demo_mic \
lib/jetson/cortex57-aarch64/libpv_rhino.so \
lib/common/rhino_params.pv \
resources/contexts/jetson/smart_lighting_jetson.rhn \
${AUDIO_DEVICE_INDEX}

It opens an input audio stream and extracts the intent within the spoken command using the Speech-to-Intent engine. Now you can say something like "turn on the lights in the kitchen" and it outputs the inference results into the terminal.

{
  intent : 'changeLightState'
  slots : {
    state : 'on'
    location : 'kitchen'
  }
}

Custom Context

You can create custom Rhino context models using Picovoice Console.