picoLLM Inference Engine
Web Quick Start

Platforms

• Chrome & Chromium-based browsers
• Edge
• Firefox
• Safari

Requirements

• Picovoice Account and AccessKey
• Node.js 16+
• npm

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

1. Install Node.js.

2. Install the picoLLM Web package:

npm install @picovoice/picollm-web

3. Download a picoLLM model file (.pllm) from Picovoice Console.

Usage

1. Either create an HTML input tag that accepts the .pllm model file:

<input id="modelFile" type="file" accept="pllm" />

<script>
  const modelFile = document.getElementById("modelFile").files;
</script>

or put the model file in a web server or public directory:

const modelFile = `${FULL_URL_TO_MODEL_FILE}`;
// or
const modelFile = `${MODEL_FILE_PATH_RELATIVE_TO_PUBLIC_DIRECTORY}`;

2. Create a picoLLM instance using PicoLLMWorker and the model file from above:

import { PicoLLMWorker } from "@picovoice/picollm-web"

const pllm = await PicoLLMWorker.create(
  "${ACCESS_KEY}",
  modelFile
);

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

3. Generate prompt completion:

const res = await pllm.generate("${PROMPT}");
console.log(res.completion);

4. To interrupt completion generation before it has finished:

await pllm.interrupt();

5. Release resources explicitly when done with picoLLM:

await pllm.release()

Vision models

To run a VLM such as qwen3-vl-2b-it:

const image: PicoLLMImage = {
  width: ${IMAGE_NUM_PIXELS_WIDTH},
  height: ${IMAGE_NUM_PIXELS_HEIGHT},
  data: ${IMAGE_DATA},
};

const res = await picoLLM.generateWithImage("${PROMPT}", image);
console.log(res.completion);

Replace ${PROMPT} with a text prompt. For the image, you will need to get image height and width in number of pixels and the raw pixel values of the image in 8-bit, RGB format.

OCR models

To run an OCR model such as deepseek-ocr-2:

const image: PicoLLMImage = {
  width: ${IMAGE_NUM_PIXELS_WIDTH},
  height: ${IMAGE_NUM_PIXELS_HEIGHT},
  data: ${IMAGE_DATA},
};

const res = await picoLLM.generateOCR(image);
console.log(res.completion);

For the image, you will need to get image height and width in number of pixels and the raw pixel values of the image in 8-bit, RGB format.

Embedding models

To run an embedding model such as embeddinggemma-300m:

const res = await picoLLM.generateEmbeddings("${PROMPT}");
res.embeddings.forEach(embedding => {
  console.log(embedding);
});

Replace ${PROMPT} with a text prompt that you want to generate embeddings for.

Demo

For the picoLLM Web SDK, we offer demo applications that demonstrate how to use it to generate text from a prompt or in a chat-based environment.

Setup

Clone the picoLLM repository from GitHub:

git clone https://github.com/Picovoice/picollm.git

Usage

1. Install dependencies and run:

cd picollm/demo/web/completion
npm install
npm run start

2. Open http://localhost:5000 to view it in the browser.

For more information on our picoLLM demos for Web or to see a chat-based demo, head over to our GitHub repository.

Resources