picoLLM Inference Engine
Android Quick Start

Platforms

• Android (5.0+, API 21+)

Requirements

• Picovoice Account and AccessKey
• Android Studio
• Android device with USB debugging enabled or Android simulator

Picovoice Account & AccessKey

Signup or Login to Picovoice Console to get your AccessKey. Make sure to keep your AccessKey secret.

Quick Start

Setup

1. Install Android Studio.

2. Include mavenCentral() repository in the top-level build.gradle. Then add the following to the app's build.gradle:

dependencies {
    // ...
    implementation 'ai.picovoice:picollm-android:${LATEST_VERSION}' // replace with latest version
}

3. Add the following to the app's AndroidManifest.xml file to enable AccessKey validation:

<uses-permission android:name="android.permission.INTERNET" />

Model File Deployment

picoLLM Inference Engine supports a variety of open-weight models The models are available for download on the Picovoice Console.

Android APKs have a size limit which does not allow for the direct inclusion of a picoLLM model file (.pllm) as a resource. To deploy a model file as part of an Android app, there are a few alternative options:

1. Include in App Bundle:

   • Utilize Google Play's Dynamic Delivery feature to include the model file in your app bundle.
   • Model file will be hosted on Google Play's servers and downloaded on-demand.

2. APK Expansion File (OBB File):

   • Store the model file as an expansion file (OBB file) and upload it alongside your APK.
   • Google Play will handle downloading the expansion file along with the APK.

3. Host Externally:

   • Host the model file on a server or cloud storage service.
   • Download the file from within the app.

4. ADB Push (for testing or manual installation):

   • Use the Android Debug Bridge (ADB) command adb push to transfer the model file directly to a connected device.
   • Access the file programmatically within your app.

Usage

1. Create an instance of the inference engine:

import ai.picovoice.picollm.*;

final String accessKey = "${ACCESS_KEY}"; // AccessKey provided by Picovoice Console (https://console.picovoice.ai/)
final String modelPath = "${MODEL_PATH}"; // absolute path to `.pllm` file on-device

try {
    PicoLLM pllm = new PicoLLM.Builder()
        .setAccessKey(accessKey)
        .setModelPath(modelPath)
        .build();
} catch (PicoLLMException e) { }

NOTE: Android 11+ cannot directly open files from external storage. To load model files from external storage, copy the model file to the application's directory and load it from there. For an example, take a look at Android Completion Demo.

2. Generate a prompt completion:

final String prompt = "${PROMPT}"; // a text prompt

try {
    PicoLLMCompletion res = pllm.generate(
        prompt,
        new PicoLLMGenerateParams.Builder().build());
} catch (PicoLLMException e) { }

3. To interrupt completion generation before it has finished:

try {
    pllm.interrupt();
} catch (PicoLLMException e) { }

4. When done, be sure to release the resources explicitly:

pllm.delete()

Demo

For the picoLLM Android SDK, we offer demo applications that demonstrate how to use it to generate text from a prompt or in a chat-based environment.

Setup

1. Clone the picoLLM repository from GitHub using HTTPS:

git clone https://github.com/Picovoice/picollm.git

2. Connect an Android device via USB or launch an Android device simulator.

Usage

1. Open the Android Completion demo using Android Studio.

2. Copy your AccessKey from Picovoice Console into the ACCESS_KEY variable in MainActivity.java.

3. Upload the picoLLM model file (.pllm) to your device using Android Studio's Device Explorer or using adb push:

adb push ~/model.pllm /sdcard/Downloads/

4. Build and run the app.

For more information on our picoLLM demo for Android or to see a chat-based demo, head over to our GitHub repository.

Resources