picoLLM Inference Engine
.NET Quick Start

Platforms

• Linux (x86_64)
• macOS (x86_64, arm64)
• Windows (x86_64, arm64)
• Raspberry Pi (4, 5)

Requirements

• Picovoice Account & AccessKey

.NET Framework 4.6.1+ / .NET Standard 2.0+ / .NET Core 3.0+:

• Windows (x86_64)

.NET Standard 2.0+ / .NET Core 3.0+:

• macOS (x86_64)

.NET 6.0+:

• macOS (arm64)
• Windows (arm64)
• Linux (x86_64)
• Raspberry Pi (4, 5)

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

1. Install .NET.

2. Install the picoLLM Inference Engine NuGet package in Visual Studio or using the .NET CLI:

dotnet add package PicoLLM

3. Download a picoLLM model file (.pllm) from Picovoice Console.

Usage

1. Create an instance of the inference engine:

using Pv;

PicoLLM pllm = PicoLLM.Create(
    "${ACCESS_KEY}",
    "${MODEL_PATH}");

2. Generate a prompt completion:

PicoLLMCompletion res = pllm.Generate('${PROMPT}');
Console.WriteLine(res.Completion);

3. To interrupt completion generation before it has finished:

pllm.Interrupt()

4. PicoLLM will have its resources freed by the garbage collector, but to have resources freed immediately after use, wrap it in a using statement or call .Dispose() directly:

using(PicoLLM pllm = PicoLLM.Create(accessKey, modelPath))
{
    // .. picoLLM usage here
}

Demo

For the picoLLM .NET SDK, we offer demo applications that demonstrate how to use it to generate text from a prompt or in a chat-based environment.

Setup

1. Clone the picoLLM Inference Engine repository from GitHub:

git clone https://github.com/Picovoice/picollm.git

2. Build the demo:

cd picollm/demo/dotnet/PicoLLM
dotnet build -c CompletionDemo.Release

Usage

Run the demo by entering the following in the terminal:

dotnet run -c CompletionDemo.Release -- --access_key ${ACCESS_KEY} --model_path ${MODEL_PATH} --prompt ${PROMPT}

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console, ${MODEL_PATH} with the path to a model file downloaded from Picovoice Console, and ${PROMPT} with a prompt string.

To get information about all the available options in the demo, run the following:

dotnet run -c CompletionDemo.Release -- --help

For more information on our picoLLM demos for .NET or to see a chat-based demo, head over to our GitHub repository.

Resources