picoLLM Inference Engine
Node.js Quick Start

Platforms

• Linux (x86_64)
• macOS (x86_64, arm64)
• Windows (x86_64, arm64)
• Raspberry Pi (4, 5)

Requirements

• Picovoice Account & AccessKey
• Node.js 18+
• npm

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

1. Install Node.js.

2. Install the picollm-node npm package:

npm install @picovoice/picollm-node

3. Download a picoLLM model file (.pllm) from Picovoice Console.

Usage

1. Create an instance of the engine:

const { PicoLLM } = require("@picovoice/picollm-node");

const pllm = new PicoLLM(
    "${ACCESS_KEY}",
    "${MODEL_PATH}");

2. Generate a prompt completion:

const res = await pllm.generate("${PROMPT}");
console.log(res.completion);

3. To interrupt completion generation before it has finished:

pllm.interrupt();

4. When done, be sure to release the resources explicitly:

pllm.release()

Vision models

To run a VLM such as qwen3-vl-2b-it:

const image = {
  width: ${IMAGE_NUM_PIXELS_WIDTH},
  height: ${IMAGE_NUM_PIXELS_HEIGHT},
  data: ${IMAGE_DATA},
};

const res = await pllm.generateWithImage("${PROMPT}", image);
console.log(res.completion);

Replace ${PROMPT} with a text prompt. For the image, you will need to get image height and width in number of pixels and the raw pixel values of the image in 8-bit, RGB format.

OCR models

To run an OCR model such as deepseek-ocr-2:

const image = {
  width: ${IMAGE_NUM_PIXELS_WIDTH},
  height: ${IMAGE_NUM_PIXELS_HEIGHT},
  data: ${IMAGE_DATA},
};

const res = await pllm.generateOCR(image);
console.log(res.completion);

For the image, you will need to get image height and width in number of pixels and the raw pixel values of the image in 8-bit, RGB format.

Embedding models

To run an embedding model such as embeddinggemma-300m:

const embeddings = await pllm.generateEmbeddings("${PROMPT}");
embeddings.forEach(embedding => {
  console.log(embedding);
});

Replace ${PROMPT} with a text prompt that you want to generate embeddings for.

Demo

For the picoLLM Node.js SDK, we offer a demo application that demonstrates how to use it to generate text from a prompt or in a chat-based environment.

Setup

Install the picoLLM demo package:

npm install -g @picovoice/picollm-node-demo

This package installs command-line utilities for the picoLLM Node.js demos.

Usage

Use the --help flag to see the usage options for the completion demo:

picollm-completion-demo --help

Run the following command to generate text:

picollm-completion-demo --access_key ${ACCESS_KEY} --model_path ${MODEL_PATH} --prompt ${PROMPT}

For more information on our picoLLM demos for Node.js or to see a chat-based demo, head over to our GitHub repository.

Resources