Picovoice Console — Rhino Speech-to-Intent Engine

Rhino is Picovoice's Speech-to-Intent engine. Rhino directly infers intent from spoken commands within a given context of interest, in real-time. For example, given a spoken command such as:

"Set the lights in the living room to blue."

Rhino infers the user's intent from the spoken command:

{
  "intent": "changeLightColor",
  "slots": {
    "location": "living room",
    "color": "blue"
  }
}

Why Rhino?

• Accuracy: Resilient to noise and reverberation. Outperforms cloud-based solutions with high margins.
• Latency: Edge-first architecture eliminates unpredictable network delay.
• Efficiency: Compact and computationally efficient. Can run even on tiny microcontrollers.
• Cost-Effectiveness: Process locally on commodity hardware. Avoid unbounded cloud fees.

Create a context

A context represents the set of expressions (spoken commands), intents, and intent arguments (slots) within a domain of interest. Let's create a context for a "smart lighting system". This context will understand voice commands for controlling lights in a home, what the various rooms are called, and colors available.

First, navigate to the Rhino Speech-to-Intent console from the Picovoice Console's landing page:

Create a new Rhino context named PicoSmartLighting from an "Empty" template:

Once created, it appears in your list of contexts. Click on the context name to go to context editor.

Create an intent

At the top level, a context is a collection of intents. For example, "turning lights on and off" can be a user intent in the context of PicoSmartLighting. Create an intent called turnLight:

Adding expressions to an intent

A user can utter an intent in several ways. Each variation of a spoken command is called an expression. For example, "turn off all lights" and "turn the lights off" both map to the turnLight intent. Add these expressions to the turnLight intent:

Use the Microphone to test the context

A unique feature of the Picovoice Console is that one can test the context at design time within the browser. Click on the microphone icon on the right. Picovoice Console then saves the context into your account, trains a Rhino model for it, and then loads it into your browser. When the microphone turns red, say:

"Turn off all lights."

Use slots to capture variables within utterances

If a user can turn the lights off by saying "turn all lights off", it makes sense to allow turning them on using a command such as "turn all lights on". The only portion of the command that has changed if the state variable "on/off". We can capture this variable using a slot in Rhino contexts. Create a slot named lightState:

Modify the existing expressions to include the newly-created slot:

When entering a slot, use $ to instruct the editor that you are adding a slot. The autocomplete dropdown shows the list of available slot types. Pick the desired slot type and assign it a name. Click on the microphone and say:

Turn all lights on.

Built-in slots

Some frequently-used slots are included in the Picovoice Console Rhino editor by default. The built-in slot types start with pv.. They appear in the autocomplete of the expression editor when you type $ (see also: Rhino Expression Syntax Cheat Sheet).

Let's say we want to set the intensity of the lights. For example:

Set the lights to 72%.

We can use the built-in slot pv.Percent to achieve this:

Optionals

Often there are words within a sentence that can be omitted without changing the intent and meaning of the utterance:

• turn $lightState:state lights
• turn $lightState:state the lights
• turn $lightState:state all lights
• turn $lightState:state all the lights
• please turn $lightState:state lights
• please turn $lightState:state all lights
• please turn $lightState:state all the lights

We can handle these variations using optional phrase syntax of Rhino, capturing all the above expressions with single entry:

(please) turn $lightState:state (all) (the) lights

Phrases in parentheses can be omitted in utterances, and will still match the intent. This allows the context to handle a large amount of phrase variations without the number of expressions becoming unwieldy.

Choice

Synonyms and alternate phrasing can be captured using Rhino choice syntax:

(please) [switch, turn] $lightState:state (all) (the) lights

The square brackets indicate a logical "OR", where one of the phrases can be spoken to match the expression.

Additional syntax

See the Rhino Expression Syntax Cheat Sheet for the complete list of supported syntax in Rhino expressions.

Training a context

When done with the design of the context, click on the train button. Read through the terms, select a target platform, and click train. The Picovoice Console schedules a job to train a Rhino model for the active context to run on the target platform. The training can take up to a few hours.

Downloading a context

Navigate back to the context editor and click on Models. From the middle pane, select the desired model and click on "Download". You can now run the trained model on the target platform using the Picovoice SDK.