picoLLM Inference Engine
Android API

API Reference for the picoLLM Android SDK (Maven Central)

PicoLLM

public class PicoLLM { }

Class for the picoLLM Inference Engine.

PicoLLM must be initialized using the Builder() Class. Resources should be cleaned when you are done using the delete() function.

PicoLLM.getVersion()

public static String getVersion() throws PicoLLMException

Retrieves the version of the picoLLM library.

Returns

• String : Version of the picoLLM library.

Throws

• PicoLLMException: If getting the version fails.

PicoLLM.getMaxTopChoices()

public static int getMaxTopChoices()

Retrieves the maximum number of top choices allowed during generation.

Returns

• int : Maximum number of top choices allowed during generation.

PicoLLM.getAvailableDevices()

public static String[] getAvailableDevices() throws PicoLLMException

Lists all available devices that picoLLM can use for inference.

Returns

• String[] : Array of all available devices that picoLLM can be used for inference.

Throws

• PicoLLMException: If getting available devices fails.

PicoLLM.delete()

public void delete()

Deletes the resources acquired by picoLLM.

PicoLLM.generate()

public PicoLLMCompletion generate(
    String prompt,
    PicoLLMGenerateParams generateParams) throws PicoLLMException

Generates completion text and relevant metadata given a text prompt and a set of generation parameters.

Parameters

• prompt String : Text prompt.
• generateParams PicoLLMGenerateParams: Generation parameters.

Returns

• PicoLLMCompletion: Completion result.

Throws

• PicoLLMException: If generation fails.

PicoLLM.interrupt()

public void interrupt() throws PicoLLMException

Interrupts .generate() if generation is in progress. Otherwise, it has no effect.

Throws

• PicoLLMException: If interrupt fails.

PicoLLM.tokenize()

public int[] tokenize(
    String text,
    boolean bos,
    boolean eos) throws PicoLLMException

Tokenizes a given text using the model's tokenizer.

Parameters

• text String : Text.
• bos boolean : If set to true, the tokenizer prepends the beginning of the sentence token to the result.
• eos boolean : If set to true, the tokenizer appends the end of the sentence token to the result.

Returns

• int[] : Tokens representing the input text.

Throws

• PicoLLMException: If tokenization fails.

PicoLLM.forward()

public float[] forward(int token) throws PicoLLMException

Performs a single forward pass given a token and returns the logits.

Parameters

• token int : Input token.

Returns

• float[] : Logits.

Throws

• PicoLLMException: If the forward pass fails.

PicoLLM.reset()

public void reset() throws PicoLLMException

Resets the internal state of LLM.

Throws

• PicoLLMException: If resetting fails.

PicoLLM.getModel()

public String getModel()

Getter for model's name.

Returns

• String : Model's name.

PicoLLM.getContextLength()

public int getContextLength()

Getter for model's context length.

Returns

• int : Model's context length.

PicoLLM.getDialogBuilder()

public DialogBuilder getDialogBuilder()

Retrieves a new instance of DialogBuilder for constructing dialog objects from the currently loaded picoLLM model.

Returns

• DialogBuilder : A new instance of DialogBuilder.

PicoLLM.Builder

public static class Builder { }

Builder for creating an instance of PicoLLM with a mixture of default arguments.

PicoLLM.Builder.setAccessKey()

public Builder setAccessKey(String accessKey)

Sets the AccessKey.

Parameters

• accessKey String : AccessKey obtained from Picovoice Console.

Returns

• Builder : Builder instance.

PicoLLM.Builder.setModelPath()

public Builder setModelPath(String modelPath)

Sets the model path.

Parameters

• modelPath String : Absolute path to the file containing LLM parameters.

Returns

• Builder : Builder instance.

PicoLLM.Builder.setDevice()

public Builder setDevice(String device)

Sets the device to use for inference.

Parameters

• device String : String representation of the device (e.g., CPU or GPU) to use for inference.

Returns

• Builder : Builder instance.

PicoLLM.Builder.build()

public PicoLLM build() throws PicoLLMException

Creates an instance of PicoLLM Inference Engine.

Returns

• PicoLLM: An instance of picoLLM Inference Engine.

Throws

• PicoLLMException: If an error occurs while creating an instance of picoLLM Inference Engine.

PicoLLM.DialogBuilder

public class DialogBuilder { }

Builder for creating an instance of PicoLLMDialog with a mixture of default arguments.

PicoLLM.DialogBuilder.setMode()

public DialogBuilder setMode(String mode)

Sets the mode for the dialog builder.

Parameters

• mode String : The mode to set.

Returns

• DialogBuilder : DialogBuilder instance.\

PicoLLM.DialogBuilder.setHistory()

public DialogBuilder setHistory(int history)

Sets the history length for the dialog builder.

Parameters

• history int : The history length to set.

Returns

• DialogBuilder : DialogBuilder instance.

PicoLLM.DialogBuilder.setSystem()

public DialogBuilder setSystem(String system)

Sets the system instruction for the dialog builder.

Parameters

• system String : The system instruction to set.

Returns

• DialogBuilder : DialogBuilder instance.

PicoLLM.DialogBuilder.build()

public PicoLLMDialog build() throws PicoLLMException

Constructs a PicoLLMDialog instance based on the configured settings.

Returns

• PicoLLMDialog : A new instance of PicoLLMDialog.

Throws

• PicoLLMException: If there's an issue constructing the dialog.

PicoLLMException

public class PicoLLMException extends Exception { }

Exception thrown if an error occurs within picoLLM Inference Engine.

Exceptions:

public class PicoLLMActivationException           extends PicoLLMException { }
public class PicoLLMActivationLimitException      extends PicoLLMException { }
public class PicoLLMActivationRefusedException    extends PicoLLMException { }
public class PicoLLMActivationThrottledException  extends PicoLLMException { }
public class PicoLLMIOException                   extends PicoLLMException { }
public class PicoLLMInvalidArgumentException      extends PicoLLMException { }
public class PicoLLMInvalidStateException         extends PicoLLMException { }
public class PicoLLMKeyException                  extends PicoLLMException { }
public class PicoLLMMemoryException               extends PicoLLMException { }
public class PicoLLMRuntimeException              extends PicoLLMException { }
public class PicoLLMStopIterationException        extends PicoLLMException { }

PicoLLMGenerateParams

public class PicoLLMGenerateParams { }

Class for configuring generation parameters when calling PicoLLM.generate().

PicoLLMGenerateParams.getCompletionTokenLimit()

public Integer getCompletionTokenLimit()

Getter for the maximum number of tokens allowed in the completion.

Returns

• Integer : The completion token limit.

PicoLLMGenerateParams.setCompletionTokenLimit()

public void setCompletionTokenLimit(Integer completionTokenLimit)

Setter for the maximum number of tokens allowed in the completion.

Parameters

• completionTokenLimit Integer : The completion token limit to set.

PicoLLMGenerateParams.getStopPhrases()

public String[] getStopPhrases()

Getter for phrases that trigger early completion termination.

Returns

• String[] : Phrases that trigger early completion termination.

PicoLLMGenerateParams.setStopPhrases()

public void setStopPhrases(String[] stopPhrases)

Setter for phrases that trigger early completion termination.

Parameters

• stopPhrases String[] : Phrases that trigger early completion termination.

PicoLLMGenerateParams.getSeed()

public Integer getSeed()

Getter for the seed value for the internal random number generator.

Returns

• Integer : The seed value for the internal random number generator.

PicoLLMGenerateParams.setSeed()

public void setSeed(Integer seed)

Setter for the seed value for the internal random number generator.

Parameters

• seed Integer : The seed value for the internal random number generator.

PicoLLMGenerateParams.getPresencePenalty()

public float getPresencePenalty()

Getter for the presence penalty.

Returns

• float : The presence penalty.

PicoLLMGenerateParams.setPresencePenalty()

public void setPresencePenalty(float presencePenalty)

Setter for the presence penalty.

Parameters

• presencePenalty float : The presence penalty.

PicoLLMGenerateParams.getFrequencyPenalty()

public float getFrequencyPenalty()

Getter for the frequency penalty.

Returns

• float : The frequency penalty.

PicoLLMGenerateParams.setFrequencyPenalty()

public void setFrequencyPenalty(float frequencyPenalty)

Setter for the frequency penalty.

Parameters

• frequencyPenalty float : The frequency penalty.

PicoLLMGenerateParams.getTemperature()

public float getTemperature()

Getter for the temperature parameter.

Returns

• float : The temperature parameter.

PicoLLMGenerateParams.setTemperature()

public void setTemperature(float temperature)

Setter for the temperature parameter.

Parameters

• temperature float : The temperature parameter.

PicoLLMGenerateParams.getTopP()

public float getTopP()

Getter for the top-p parameter.

Returns

• float : The top-p parameter.

PicoLLMGenerateParams.setTopP()

public void setTopP(float topP)

Setter for the top-p parameter.

Parameters

• topP float : The top-p parameter.

PicoLLMGenerateParams.getNumTopChoices()

public int getNumTopChoices()

Getter for the number of top choices.

Returns

• int : The number of top choices.

PicoLLMGenerateParams.setNumTopChoices()

public void setNumTopChoices(int numTopChoices)

Setter for the number of top choices.

Parameters

• numTopChoices int : The number of top choices.

PicoLLMGenerateParams.getStreamCallback()

public PicoLLMStreamCallback getStreamCallback()

Getter for the completion stream callback.

Returns

• PicoLLMStreamCallback : The stream callback.

PicoLLMGenerateParams.setStreamCallback()

public void setStreamCallback(PicoLLMStreamCallback streamCallback)

Setter for the completion stream callback.

Parameters

• streamCallback PicoLLMStreamCallback : The stream callback.

PicoLLMGenerateParams.Builder

public static class Builder { }

Builder class for creating a PicoLLMGenerateParams instance.

PicoLLMGenerateParams.Builder.setCompletionTokenLimit()

public Builder setCompletionTokenLimit(Integer completionTokenLimit)

Sets the maximum number of tokens allowed in the completion.

Parameters

• completionTokenLimit Integer : The maximum number of tokens allowed in the completion. If not set, there will be no limit imposed.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setStopPhrases()

public Builder setStopPhrases(String[] stopPhrases)

Sets the phrases that trigger early completion termination.

Parameters

• stopPhrases String[] : Phrases that trigger early completion termination. Set to null to turn off this feature.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setSeed()

public Builder setSeed(Integer seed)

Sets the seed value for the internal random number generator.

Parameters

• seed Integer : The seed value for the internal random number generator. If not set, randomized outputs will be generated for a given prompt.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setPresencePenalty()

public Builder setPresencePenalty(float presencePenalty)

Sets the presence penalty.

Parameters

• presencePenalty float : The presence penalty.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setFrequencyPenalty()

public Builder setFrequencyPenalty(float frequencyPenalty)

Sets the frequency penalty.

Parameters

• frequencyPenalty float : The frequency penalty.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setTemperature()

public Builder setTemperature(float temperature)

Sets the temperature parameter.

Parameters

• temperature float : The temperature parameter.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setTopP()

public Builder setTopP(float topP)

Sets the top-p parameter.

Parameters

• topP float : The top-p parameter.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setNumTopChoices()

public Builder setNumTopChoices(int numTopChoices)

Sets the number of top choices.

Parameters

• numTopChoices int : The number of top choices.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.setStreamCallback()

public Builder setStreamCallback(PicoLLMStreamCallback streamCallback)

Sets the completion stream callback.

Parameters

• streamCallback PicoLLMStreamCallback : The stream callback.

Returns

• Builder : The Builder instance.

PicoLLMGenerateParams.Builder.build()

public PicoLLMGenerateParams build()

Constructs a new PicoLLMGenerateParams object.

Returns

• PicoLLMGenerateParams : A new PicoLLMGenerateParams object with the parameters set in the builder.

PicoLLMStreamCallback

public interface PicoLLMStreamCallback { }

Interface for receiving streaming result callbacks from picoLLM's .generate() method.

PicoLLMCompletion

public class PicoLLMCompletion { }

Represents a picoLLM completion result.

PicoLLMCompletion.getUsage()

public Usage getUsage()

Getter for usage information.

Returns

• Usage : The usage information.

PicoLLMCompletion.getEndpoint()

public Endpoint getEndpoint()

Getter for the reason for ending the generation process.

Returns

• Endpoint : The reason for ending the generation process.

PicoLLMCompletion.getCompletionTokens()

public CompletionToken[] getCompletionTokens()

Getter for the generated tokens within completion and top alternative tokens.

Returns

• CompletionToken[] : The generated tokens within completion and top alternative tokens.

PicoLLMCompletion.getCompletion()

public String getCompletion()

Getter for the completion string.

Returns

• String : The completion string.

PicoLLMCompletion.Usage

public static class Usage { }

Represents usage information.

PicoLLMCompletion.Usage.getPromptTokens()

public int getPromptTokens()

Getter for the number of tokens in the prompt.

Returns

• int : The number of tokens in the prompt.

PicoLLMCompletion.Usage.getCompletionTokens()

public int getCompletionTokens()

Getter for the number of tokens in the completion.

Returns

• int : The number of tokens in the completion.

PicoLLMCompletion.Endpoint

public enum Endpoint {
    END_OF_SENTENCE,
    COMPLETION_TOKEN_LIMIT_REACHED,
    STOP_PHRASE_ENCOUNTERED
}

Reasons for ending the generation process.

PicoLLMCompletion.Token

public static class Token { }

Represents a generated token.

PicoLLMCompletion.Token.getToken

public String getToken()

Getter for the token.

Returns

• String : The token.

PicoLLMCompletion.Token.getLogProb

public float getLogProb()

Getter for the log probability.

Returns

• float : The log probability.

PicoLLMCompletion.CompletionToken

public static class CompletionToken { }

Represents a generated token within completion and top alternative tokens.

PicoLLMCompletion.CompletionToken.getToken

public Token getToken()

Getter for the token.

Returns

• Token : The token.

PicoLLMCompletion.CompletionToken.getTopChoices

public Token[] getTopChoices()

Getter for the top alternative tokens.

Returns

• Token[] : The top alternative tokens.

PicoLLMDialog

public class PicoLLMDialog { }

PicoLLMDialog is a helper class that stores a chat dialog and formats it according to an instruction-tuned LLM's chat template. PicoLLMDialog is the base class. Each supported instruction-tuned LLM has an accompanying concrete subclass.

PicoLLMDialog.addHumanRequest()

public void addHumanRequest(String content) throws PicoLLMInvalidStateException

Adds a human request to the dialog.

Parameters

• content String : The human request to add.

Throws

• PicoLLMInvalidStateException : if adding a human request without a corresponding LLM response.

PicoLLMDialog.addLLMResponse()

public void addLLMResponse(String content) throws PicoLLMInvalidStateException

Adds an LLM response to the dialog.

Parameters

• content String : The LLM response to add.

Throws

• PicoLLMInvalidStateException : if adding an LLM response without a corresponding human request.

PicoLLMDialog.getPrompt()

public String getPrompt() throws PicoLLMException

Creates a prompt string given parameters passed the constructor and dialog's content.

Returns

• String : Formatted prompt.

Throws

• PicoLLMException : if called from the base PicoLLMDialog class.

PicoLLMDialog.Builder

public static class Builder { }

Builder class for constructing instances of PicoLLMDialog.

PicoLLMDialog.Builder.setHistory()

public Builder setHistory(Integer history)

Sets the history length for the builder.

Parameters

• history Integer : The history length to set. History refers to the number of latest back-and-forths to include in the prompt. Setting history to null will embed the entire dialog in the prompt.

Returns

• Builder : Builder instance.

PicoLLMDialog.Builder.setSystem()

public Builder setSystem(String system)

Sets the system instruction for the builder.

Parameters

• system String : The system instruction to set. System instruction to embed in the prompt for configuring the model's responses.

Returns

• Builder : Builder instance.

PicoLLMDialog.Builder.build()

public PicoLLMDialog build()

Builds a new instance of PicoLLMDialog based on the configured settings.

Returns

• PicoLLMDialog : A new instance of PicoLLMDialog.

GemmaChatDialog

public class GemmaChatDialog { }

Represents a dialog helper specific for gemma-2b-it and gemma-7b-it models.

Llama2ChatDialog

public class Llama2ChatDialog { }

Represents a dialog helper specific for llama-2-7b-chat, llama-2-13b-chat, and llama-2-70b-chat models.

Llama3ChatDialog

public class Llama3ChatDialog { }

Represents a dialog helper specific for llama-3-8b-instruct and llama-3-70b-instruct models.

MistralChatDialog

public class MistralChatDialog { }

Represents a dialog helper specific for mistral-7b-instruct-v0.1 and mistral-7b-instruct-v0.2 models.

MixtralChatDialog

public class MixtralChatDialog { }

Represents a dialog helper specific for the mixtral-8x7b-instruct-v0.1 model.

Phi2ChatDialog

public class Phi2ChatDialog { }

Represents a dialog helper specific for phi-2 models in chat mode.

Phi2Dialog

public class Phi2Dialog { }

Represents a dialog helper for the phi-2 model within PicoLLM. This is a base class that provides functionalities common to both chat and qa modes

Phi2QADialog

public class Phi2QADialog { }

Represents a dialog helper specific for phi-2 models in qa mode.