Build AI-Powered Voice Assistant for Smart Kitchen Appliances

🎯 Voice AI Consulting

Get dedicated support and consultation to ensure your specific needs are met.

Embedding voice AI into kitchen appliances shouldn’t mean handing data and control to external ecosystems. Cloud assistants can introduce external dependencies and tie products to rigid platforms that dictate UX and certification requirements. They require device cloud accounts, stream customer audio to external servers, and create latency and reliability risks.

Picovoice takes a different approach: AI-powered voice technology that runs entirely on-device. In this tutorial, we use Python with Raspberry Pi to show how voice powered appliance actions, such as turning on devices, adjusting oven temperatures, and setting timers, can be mapped directly into your existing tech stack. The result is a customizable embedded voice layer that is responsive, operates offline, and scales across product lines.

This demo can run on a similar low-cost computer, mobile phone, laptop, desktop, or even within web browsers. Check out Picovoice's voice AI SDKs to add voice AI to your product.

Why Choose On-Device Voice AI for Smart Kitchen Appliances?

For smart kitchen appliances, Picovoice enables:

Customizable experiences: Define your own wake words, intents, and voice responses for a personalized experience rather than adapting to rigid platforms.
Seamless Integration: Embed voice AI directly into existing appliance stacks without relying on cloud APIs or re-architecting device firmware.
Low Latency: Handle speech processing locally, eliminating network latency and keeping performance consistent.
Privacy by default: Keep all audio processing on-device, to meet strict compliance standards like GDPR.

The Voice AI Stack for Smart Appliances

Wake Word Detection: The kitchen appliance continuously listens for a predefined wake phrase using a wake word engine. Porcupine Wake Word detects the custom phrase and activates the rest of the voice pipeline.
Speech-to-Intent Recognition: Once the wake word is detected, the system identifies what action the user wants the smart appliance to perform. Rhino Speech-to-Intent processes the user's speech and directly maps the user's speech to a predefined intent and its parameters (or "slots"). This enables real-time voice-powered appliance tasks and is 6x more accurate than Big Tech alternatives.
Text-to-Speech: Orca Text-to-Speech provides natural-sounding voice feedback, allowing the appliance to confirm appliance actions such as “Turning oven on”

Step 1: Create a Custom Wake Word for the Kitchen Assistant

To train the custom wake word model, create an account on the Picovoice Console
In the Porcupine section of the Console, train the custom wake phrase, such as "Hey Chef".
Download the keyword file, choosing Raspberry Pi as the target platform.

You will now find a zipped folder in your downloads containing the .ppn file for the custom wake word.

For tips on choosing an effective wake word, refer to this guide: Choosing a wake word

Step 2: Create a Custom Speech-to-Intent model for the AI Kitchen Assistant

In the Rhino section of the Picovoice console, create an empty Rhino context for the AI voice assistant.
Click the "Import YAML" button in the top-right corner of the Console. Paste the YAML provided below to add the intents and expressions for the voice assistant.

Download the context file, choosing Raspberry Pi as the target platform.

You will now find a zipped folder in your downloads containing the .rhn file for the custom context.

You can refer to the Rhino Syntax Cheat Sheet for more details on how to build your custom context.

YAML Context for the Kitchen Assistant:

context:
  expressions:
    TimerControl:
      - (@politeness) @set_on (a) timer for $pv.TwoDigitInteger:minutes
        minutes
      - "@set_on $appliance:appliance timer for
        $pv.TwoDigitInteger:minutes minutes"
      - (@politeness) [switch, power] $power:power (the) timer for $appliance:appliance
      - (@politeness) [switch, power] (the) $appliance:appliance timer
        $power:power
    TemperatureControl:
      - (@politeness) @heat (oven) to $temperature:temperature (degrees)
    ApplianceControl:
      - (@politeness) turn $appliance:appliance $power:power
      - (@politeness) switch $power:power (the) $appliance:appliance
      - (@politeness) power $power:power (the) $appliance:appliance
    AskState:
      - what is (the) $query_type:query_type (information)
      - (@politeness) tell me (the) $query_type:query_type (information)
      - (@politeness) check (the) $query_type:query_type (information)
  slots:
    appliance:
      - oven
      - stove
    temperature:
      - three fifty
      - three hundred fifty
      - four twenty five
      - four hundred twenty five
      - four hundred
    query_type:
      - time left
      - timer
      - temperature
    power:
      - on
      - off
  macros:
    politeness:
      - please
      - can you
      - could you
    set_on:
      - set
      - start
      - create
    heat:
      - set
      - preheat
      - decrease
      - increase

This context defines four main intents:

TimerControl: Set and manage kitchen timers
TemperatureControl: Adjust oven temperature
ApplianceControl: Turn kitchen devices on/off
AskState: Ask current appliance status

With these intents, the AI voice assistant can take on appliance-focused tasks. This context lays the groundwork for seamless integration with IoT platforms or embedded systems, enabling smart workflows where multiple devices coordinate together in real time.

To see how Rhino Speech-to-Intent can be applied beyond appliances, take a look at the AI Cooking Companion in Python

Step 3: Build the Smart Kitchen Voice Control Pipeline in Python

With the wake word and Rhino context ready, let's write the Python code that brings everything together.

Initialize the Voice AI Engines and Audio I/O

porcupine = pvporcupine.create(
    access_key=args.ACCESS_KEY,
    keyword_paths=[args.KEYWORD_FILE_PATH]
)
rhino = pvrhino.create(
    access_key=args.ACCESS_KEY,
    context_path=args.CONTEXT_FILE_PATH
)
orca = pvorca.create(access_key=args.ACCESS_KEY)

recorder = PvRecorder(device_index=args.AUDIO_DEVICE_INDEX_INPUT,
                      frame_length=porcupine.frame_length)
speaker = PvSpeaker(sample_rate=orca.sample_rate,
                    bits_per_sample=16,
                    device_index=args.AUDIO_DEVICE_INDEX_OUTPUT)

Detect the Custom Wake Word

Porcupine Wake Word continuously processes the microphone input to detect the predefined wake phrase. When the phrase is recognized, the system triggers the inference pipeline and generates an acknowledgment through Orca Text-to-Speech.

pcm = recorder.read()
if porcupine.process(pcm) >= 0:
    speak_text(orca, speaker, "Yes?")
    listening_for_intent = True

Run On-Device Intent Recognition

Once activated, Rhino Speech-to-Intent processes the incoming audio in real time. When the user finishes speaking, it returns a structured inference containing the intent and any relevant slots.

is_finalized = rhino.process(pcm)
if is_finalized:
    inference = rhino.get_inference()

    if inference.is_understood:
        intent = inference.intent
        slots = inference.slots
        print(f"[UNDERSTOOD] intent={intent}, slots={slots}")
        response_text = generate_response(intent, slots)
    else:
        print("[NOT UNDERSTOOD]")
        response_text = "Sorry, I did not understand."

Execute Appliance Actions from User Intents

The execute_appliance_action function converts Rhino’s inference output into actions for the target device and generates a text response for playback.

def execute_appliance_action(action):
    """
    Executes appliance commands based on the action dictionary.
    In production, this is where you'd integrate with the IoT stack.
    """
    action_type = action.get("action")
    
    if action_type == "start_timer":
        minutes = action.get("minutes")
        appliance = action.get("appliance")
        # Logic to start timer on appliance
        if appliance:
            return f"Starting {appliance} timer for {minutes} minutes."
        return f"Timer set for {minutes} minutes."
    
    elif action_type == "toggle_timer":
        power = action.get("power")
        appliance = action.get("appliance", "timer")
        # Logic to toggle timer on/off
        return f"Turning {appliance} {power}."
        # Add more appliance actions 
    return "Command received."

Generate Voice Feedback

Orca Text-to-Speech synthesizes the response text into natural-sounding audio, completing the embedded conversational loop.

def speak_text(orca, speaker, text: str, label: str = ""):
    try:
        print(f"Orca speaking: {label or text}")
        pcm_out, _ = orca.synthesize(text)
        pcm_out = [int(s) for s in pcm_out]  # ensure ints
        speaker.start()
        speaker.write(pcm_out)
        speaker.flush()
        speaker.stop()
    except Exception as e:
        print(f"[FAIL] Orca failed to respond ({label}): {e}")

Complete Python Script

Here's the complete Python script integrating all components into an embedded voice AI stack for smart kitchen appliances:

# Kitchen Assistant: Porcupine + Rhino + Orca
import argparse
import sys
import pvporcupine
import pvrhino
import pvorca
from pvrecorder import PvRecorder
from pvspeaker import PvSpeaker


def speak_text(orca, speaker, text: str, label: str = ""):
    try:
        print(f"Orca speaking: {label or text}")
        pcm_out, _ = orca.synthesize(text)
        pcm_out = [int(s) for s in pcm_out]  # ensure ints
        speaker.start()
        speaker.write(pcm_out)
        speaker.flush()
        speaker.stop()
    except Exception as e:
        print(f"[FAIL] Orca failed to respond ({label}): {e}")


def execute_appliance_action(intent, slots): 
    if intent == "TimerControl":
      # Add the appliance logic here
        minutes = slots.get("minutes")
        appliance = slots.get("appliance")
        power = slots.get("power")
        if minutes:
            response_text = f"Starting a timer for {minutes} minutes."
            if appliance:
                response_text = (
                    f"Starting a {appliance} timer for {minutes} minutes."
                )
        elif power:
            response_text = f"Turning {appliance or 'the'} timer {power}."
        else:
            response_text = "Timer command received."

    elif intent == "TemperatureControl":
      # Add the appliance logic here 
        temp = slots.get("temperature")
        response_text = (
            f"Preheating oven to {temp} degrees."
            if temp else "Temperature control command received."
        )

    elif intent == "ApplianceControl":
      # Add the appliance logic here 
        appliance = slots.get("appliance")
        power = slots.get("power")
        if appliance and power:
            response_text = f"Turning {appliance} {power}."
        else:
            response_text = "Appliance control command received."

    elif intent == "AskState":
      # Add the appliance logic here 
        query = slots.get("query_type")
        response_text = (
            f"Checking {query}." if query else "Query command received."
        )

    else:
        response_text = f"Command received. (raw intent={intent})"

    return response_text


def main():
    parser = argparse.ArgumentParser(
        description="Kitchen Assistant using Porcupine, Rhino, and Orca"
    )
    parser.add_argument(
        "--access_key",
        required=True,
        help="Picovoice access key"
    )
    parser.add_argument(
        "--keyword_file_path",
        required=True,
        help="Path to keyword file for wake word detection"
    )
    parser.add_argument(
        "--context_file_path",
        required=True,
        help="Path to context file for intent recognition"
    )
    parser.add_argument(
        "--audio_device_index_input",
        type=int,
        default=-1,
        help="Input audio device index (default: -1)"
    )
    parser.add_argument(
        "--audio_device_index_output",
        type=int,
        default=0,
        help="Output audio device index (default: 0)"
    )

    args = parser.parse_args()

    print("Initializing Kitchen Assistant")

    # Use command line arguments
    access_key = args.access_key
    keyword_path = args.keyword_file_path
    context_path = args.context_file_path
    input_device_index = args.audio_device_index_input
    output_device_index = args.audio_device_index_output

    # Initialize engines
    porcupine = pvporcupine.create(
        access_key=access_key,
        keyword_paths=[keyword_path]
    )
    rhino = pvrhino.create(
        access_key=access_key,
        context_path=context_path
    )
    orca = pvorca.create(access_key=access_key)

    recorder = PvRecorder(
        device_index=input_device_index,
        frame_length=porcupine.frame_length
    )
    speaker = PvSpeaker(
        sample_rate=orca.sample_rate,
        bits_per_sample=16,
        device_index=output_device_index
    )

    print("[OK] Engines initialized")
    print(
        f"Using input device index={input_device_index}, "
        f"output={output_device_index}"
    )
    print("Say the wake word… (Ctrl+C to stop)")

    recorder.start()
    try:
        listening_for_intent = False

        while True:
            pcm = recorder.read()

            if not listening_for_intent:
                # Wake word detection
                keyword_index = porcupine.process(pcm)
                if keyword_index >= 0:
                    print("[EVENT] Wake word detected!")
                    speak_text(
                        orca,
                        speaker,
                        "Yes?",
                        label="wake word confirmation"
                    )
                    listening_for_intent = True
            else:
                # Rhino intent processing
                is_finalized = rhino.process(pcm)
                if is_finalized:
                    inference = rhino.get_inference()

                    if inference.is_understood:
                        intent = inference.intent
                        slots = inference.slots
                        print(
                            f"[UNDERSTOOD] intent={intent}, "
                            f"slots={slots}"
                        )

                        response_text = generate_response(intent, slots)

                    else:
                        print("[NOT UNDERSTOOD]")
                        response_text = "Sorry, I did not understand."

                    # Speak the response
                    speak_text(
                        orca,
                        speaker,
                        response_text,
                        label="intent response"
                    )

                    # Reset Rhino for next turn
                    rhino.reset()
                    listening_for_intent = False

    except KeyboardInterrupt:
        print("\n[EXIT] Stopping assistant...")
    finally:
        recorder.stop()
        recorder.delete()
        speaker.delete()
        porcupine.delete()
        rhino.delete()
        orca.delete()
        print("[CLEANUP] Resources released.")


if __name__ == "__main__":
    main()

Step 4: Run the Voice Stack on Raspberry Pi

With the script and model files ready, it’s time to deploy the embedded voice AI pipeline on a Raspberry Pi. For this demo, we used a Raspberry Pi 4 with a USB microphone and Bluetooth speaker for input and output. The same setup can run on laptops, web, or other single-board computers.

Transfer your project to the Raspberry Pi

Transfer your script and model files to the Raspberry Pi with scp. Replace <your-pi-ip-address> with your Pi’s address and add the path to your project folder:

scp -r /path/to/your/folder pi@<your-pi-ip-address>:/home/pi/

Connect and prepare the Raspberry Pi

Make sure your Raspberry Pi is connected to a microphone and speaker so it can capture voice and play spoken responses. You can log in and launch the application. Run the following on the Raspberry Pi’s terminal:

ssh pi@<your-pi-ip-address>

Install Dependencies on Raspberry Pi

Install all required Python SDKs and supporting libraries:

Porcupine Wake Word Python SDK pvporcupine,
Rhino Speech-to-Intent Python SDK pvrhino,
Orca Text-to-Speech Python SDK pvorca,
Picovoice Python Recorder library pvrecorder,
Picovoice Python Speaker library pvspeaker.

pip install pvporcupine pvrhino pvorca pvrecorder pvspeaker

Run the Smart Appliance Stack

You will need the Picovoice AccessKey to use the SDKs. Copy it from the Picovoice Console.

Run the following commands in the Raspberry Pi’s terminal. Replace the placeholder values with your actual ACCESS_KEY, and the file paths to your models. The audio device indices are optional and will use defaults if not specified.

python3 smart-kitchen-appliance.py \
    --access_key="${ACCESS_KEY}" \
    --keyword_file_path="${KEYWORD_FILE_PATH}" \
    --context_file_path="${CONTEXT_FILE_PATH}" \
    --audio_device_index_input="${AUDIO_DEVICE_INDEX_INPUT}" \
    --audio_device_index_output="${AUDIO_DEVICE_INDEX_OUTPUT}"

The smart kitchen appliance is now ready and listening for a wake word.

Extending the Smart Kitchen Voice Assistant

Once the on-device voice assistant is running, here are some ways you can expand it:

Integrate with Smart Kitchen IoT Devices: Move beyond the Raspberry Pi by connecting the AI voice assistant to real-world smart kitchen appliances.
Multiple Languages: Picovoice engines support a wide range of languages, allowing you to adapt the voice assistant for different regions.
Advanced Conversational Flows: Implement more complex dialogue management for multi-turn interactions.

Start Free