picoLLM Inference Engine
Web Quick Start

Platforms

• Chrome & Chromium-based browsers
• Edge
• Firefox
• Safari

Requirements

• Picovoice Account and AccessKey
• Node.js 16+
• npm

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

1. Install Node.js.

2. Install the picoLLM Web package:

npm install @picovoice/picollm-web

3. Download a picoLLM model file (.pllm) from Picovoice Console.

Usage

1. Either create an HTML input tag that accepts the .pllm model file:

<input id="modelFile" type="file" accept="pllm" />

<script>
  const modelFile = document.getElementById("modelFile").files;
</script>

or put the model file in a web server or public directory:

const modelFile = `${FULL_URL_TO_MODEL_FILE}`;
// or
const modelFile = `${MODEL_FILE_PATH_RELATIVE_TO_PUBLIC_DIRECTORY}`;

2. Create a picoLLM instance using PicoLLMWorker and the model file from above:

import { PicoLLMWorker } from "@picovoice/picollm-web"

const pllm = await PicoLLMWorker.create(
  "${ACCESS_KEY}",
  modelFile
);

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

3. Generate prompt completion:

const res = await pllm.generate("${PROMPT}");
console.log(res.completion);

4. To interrupt completion generation before it has finished:

await pllm.interrupt();

5. Release resources explicitly when done with picoLLM:

await pllm.release()

Demo

For the picoLLM Web SDK, we offer demo applications that demonstrate how to use it to generate text from a prompt or in a chat-based environment.

Setup

Clone the picoLLM repository from GitHub:

git clone https://github.com/Picovoice/picollm.git

Usage

1. Install dependencies and run:

cd picollm/demo/web/completion
npm install
npm run start

2. Open http://localhost:5000 to view it in the browser.

For more information on our picoLLM demos for Web or to see a chat-based demo, head over to our GitHub repository.

Resources