Picovoice Wordmark
Picovoice Wordmark
Products
Technologies
Log inStart Free
picoCompression Model Compression Algorithm

AI model quantization without the accuracy loss.

A novel AI model compression and quantization algorithm that makes sub-4-bit compression with no or minimal accuracy loss possible, even when GPTQ, GGUF, or SpinQuant fails. No ML team required.

Start Free
Loved by developers, trusted by enterprises
What it is
picoCompression is a novel AI model compression and quantization algorithm that learns the optimal bit allocation across and within a model's weights, producing the smallest models with minimal accuracy loss.
What it replaces
picoCompression replaces popular quantization schemes like GPTQ and enables sub-4-bit quantization with minimal accuracy loss, recovering most of their accuracy degradation at the same model size.
What you get
Compressed model files that meet the model size target with the highest possible accuracy and run on CPU and GPU across embedded, mobile, desktop, and browser environments.
WHAT IS PICOCOMPRESSION MODEL COMPRESSION ALGORITHM?

X-bit quantization that maintains accuracy where GPTQ, GGUF, and SpinQuant collapse for any Transformer model

Most AI model quantization methods rely on uniform bit allocation: every weight gets the same precision, regardless of its impact on model output. GPTQ, GGUF, AWQ, and even rotation-based methods like SpinQuant share this constraint. At 4-bit precision, this is acceptable. Below 4 bits, it collapses. Some weights matter more than others.

picoCompression takes a different approach. Given a target size and a task-specific cost function, the algorithm learns the optimal bit allocation both across model components (inter-functional allocation) and within each weight (intra-functional allocation). The result is a compressed model that consistently outperforms fixed-precision quantization at the same final size.

X-Bit QuantizationLLM QuantizationAI Model QuantizationOptimal Bit AllocationLanguage Model QuantizationVision Model QuantizationHardware-Aware QuantizationAir-gapped CompatibleVision Language Model QuantizationOCR QuantizationLLM CompressionAI Model CompressionEdge AI CompatibleLanguage Model CompressionVision Model CompressionVision Language Model CompressionOCR CompressionSub-4-bit QuantizationTransformer CompressionEdge AI DeploymentX-Bit QuantizationLLM QuantizationAI Model QuantizationOptimal Bit AllocationLanguage Model QuantizationVision Model QuantizationHardware-Aware QuantizationAir-gapped CompatibleVision Language Model QuantizationOCR QuantizationLLM CompressionAI Model CompressionEdge AI CompatibleLanguage Model CompressionVision Model CompressionVision Language Model CompressionOCR CompressionSub-4-bit QuantizationTransformer CompressionEdge AI Deployment
AI Model Compression Architecture

What makes picoCompression different from other AI model quantization algorithms

The number of weights in a modern language model is in the hundreds, and the number of columns within each weight is in the thousands. Treating all of them with the same bit precision wastes capacity on unimportant components and starves the salient ones.

picoCompression uses gradient descent to learn the bit budget at both levels. Across components, it allocates more bits to functions that contribute most to task accuracy. Within each weight, it allocates more bits to columns whose quantization error has the largest impact on the output. The bit distribution is not chosen by hand. It emerges from the data.

Why enterprises choose picoCompression

picoCompression is the AI model compression algorithm chosen by enterprises that can't accept the accuracy collapse of fixed-precision methods at sub-4-bit. By learning bit allocation across and within weights, it delivers the smallest models with the highest accuracy on any target platform and integrates without a dedicated ML team.

01Highest AccuracyCompared against the most popular and cited quantization framework, GPTQ, picoCompression recovers accuracy loss up to 100% at sub-4-bit-compression.
Llama-3-8b Accuracy Recovery
Measured Against float16 Baseline. Higher is better.
picoCompression - 2-bit94.5%
GPTQ Quantization - 2-bit38.7%
picoCompression - 3-bit99%
GPTQ Quantization - 3-bit83.1%
picoCompression - 4-bit100%
GPTQ Quantization - 4-bit99%
02Inter-Functional AllocationpicoCompression treats the model as a chain of functions and learns how many bits each function should receive, given a global size budget. Some weight matrices need more precision than others. The algorithm finds out which ones, automatically, using gradient descent over a few hundred component-level allocations.
Three-row bar chart titled "Optimal Bit Allocation across Weights of Llama-2-7b" comparing nine weight components — att.k, att.o, att.q, att.v, emb, ff.1, ff.2, ff.3, and out — at three compression ratios. At 3× compression, values range from 4.25 to 6.48 bits, with the output projection receiving the most. At 5× compression, values drop to 2.26–4.72 bits, with the output projection still highest. At 7× compression, attention key, output, and query layers each receive only 1.65 bits, while output projection retains 3.10 and embedding 3.00. The optimal distribution shifts non-linearly with compression ratio.
03Intra-Functional AllocationWithin each weight, picoCompression learns the bit allocation across columns, while other methods use hard thresholds to separate salient and non-salient weights. picoCompression treats salience as a continuous spectrum and lets gradient descent decide. The number of columns to allocate over runs into the thousands per weight.
Three-row logarithmic bar chart titled "Optimal Bit Allocation within Weights of Llama-2-7b" showing density of weight columns at each bit depth from 1-bit to 8-bit, across three compression ratios. At 3× compression, density peaks at 5-bit (53%) and 4-bit (38%). At 5× compression, the peak shifts to 3-bit (61%) and 2-bit (33%). At 7× compression, the distribution shifts further toward fewer bits, with 2-bit at 68%, 1-bit at 19%, and 3-bit at 12%. As compression increases, bit allocation concentrates at lower precision levels.
04X-bit Inference EngineX-bit quantization breaks compatibility with off-the-shelf inference engines. A model averaging 2.56 bits doesn't conform to any standard depth, so it can't run on frameworks built around fixed 4-bit or 8-bit assumptions. picoInference is purpose-built for this: it implements optimized kernels for every bit depth from 1 to 8 across x86, ARM, CUDA, Metal, WebGPU, DirectX, and Web Workers, with runtime detection to select the right kernel per platform.
05No ML Team RequiredMost quantization methods require ML engineers to choose bit depths, configure calibration, and validate accuracy against held-out tasks. picoCompression learns these decisions automatically. You define the target size; Picovoice Deep Learning researchers oversee the algorithm and ensure it produces a model that's already calibrated, evaluated, and ready for deployment at scale.
06Enterprise ReadyEnterprises with in-house AI models can compress them using picoCompression for local deployment. Available through enterprise engagement with the NDA-protected model handling.
07Compliance by architecturepicoCompression makes models smaller to run on a server, desktop, mobile, and embedded, so data never leaves the device. For healthcare practices, legal teams, financial institutions, and defense applications, picoCompression makes intuitive compliance possible.
Get Started

Quantize custom models. Deploy them on CPU, GPU, mobile, embedded, and web.

Open-weight language and vision models are ready-to-use or test. Custom model compression available through enterprise engagements with NDA-protected model handling.

Start Free
FAQ
+
What is X-bit quantization?

X-bit quantization automatically assigns a different number of bits to each weight in a model based on its importance to model outputs, rather than applying a uniform bit depth across all parameters. This produces a model with a fractional average bit rate, e.g., 2.56 bits, that cannot be matched by any standard fixed-depth format. picoCompression uses X-bit quantization and achieves near-float16 accuracy at sub-4-bit levels even when uniform methods like GPTQ result in catastrophic accuracy losses.

+
What is the difference between bit depth and bit rate in LLM quantization?

Bit depth refers to a fixed, uniform number of bits assigned to every weight in a model — for example, exactly 8 bits per parameter in LLM.int8() quantization . Bit rate refers to the average bits per weight when allocation varies across parameters. X-bit quantization methods like picoCompressionassign different bit depths to different weights based on their importance, achieving a target bit rate — such as 2.56 bits — that no standard fixed depth can match. This distinction is the core reason X-bit quantized models require purpose-built inference engines.

+
What is sub-4-bit LLM quantization?

Sub-4-bit LLM quantization compresses large language model weights to fewer than 4 bits per parameter, reducing memory requirements enough to enable deployment on laptops, phones, and browsers. Below 4 bits, uniform allocation methods cannot maintain accuracy: 4-bit precision gives only 16 representable values, 3-bit gives 8, 2-bit gives 4. Sub-4-bit compression therefore requires non-uniform bit allocation, assigning more bits to high-importance weights and fewer to less critical ones, to avoid significant accuracy loss.

+
What is the difference between GPTQ and GGUF?

GPTQ is a quantization algorithm that minimizes quantization error layer by layer, adjusting remaining weights sequentially to compensate for errors introduced at each step. It requires calibration data to estimate input feature statistics. GGUF is a file format, not an algorithm, that packages model weights and metadata into a single binary with built-in support for quantization levels from Q2 to Q8 using block-wise quantization, applying individual scale factors to blocks of weights to handle outliers. The two are not direct alternatives: GPTQ produces quantized weights that can be stored in various formats, while GGUF defines how quantized models are packaged and distributed.

+
How does picoCompression compare to GPTQ, GGUF, and SpinQuant?

GPTQ uses sequential layer-by-layer quantization with fixed bit depth. GGUF is a file format with block-wise quantization. SpinQuant uses learned Hadamard rotations before quantization to eliminate outliers and is integrated into Meta's ExecuTorch. All three use uniform bit allocation. picoCompression learns the bit allocation as part of the quantization process, both across model components and within each weight. On Llama-3-8b MMLU, picoCompression maintains 94.5%, 99.9%, and 100% of float16 baseline accuracy vs. GPTQ maintains 38.7%, 83.1% and 99% of float16 baseline accuracy at 2-bit, 3-bit, and 4-bit settings, respectively.

+
Why is inference harder for X-bit quantized LLMs than 4-bit-quantized LLMs?

Standard inference engines are built around fixed bit depths, mainly for 4 and 8 bits. Bit rates — for example, 2.56 bits per weight on average — breaks compatibility with existing frameworks. Building an inference engine for X-bit quantized models requires implementing SIMD operations for every bit depth from 1 to 8 across multiple instruction set architectures. For x86 alone, supporting five SIMD variants across eight bit depths requires 80+ specialized functions, with additional separate implementations needed for CUDA, Metal, WebGPU, and mobile platforms. That's why picoLLM Inference Engine implements all of these.

+
Which LLM quantization method is best for on-device deployment?

It depends on target hardware, required compression ratio, and accuracy tolerance. For 4-bit deployment with broad community support, GGUF is the most practical choice given its 125,000+ available models on Hugging Face. For full end-to-end 4-bit quantization, including activations and KV cache,SpinQuant maintains near-float16 accuracy where GPTQ collapses. For sub-4-bit deployment, where memory constraints require 2-bit or 3-bit compression,picoCompression's X-bit allocation maintains near-float16 MMLU scores across model families where GPTQ drops to near-random performance.

+
What models does picoCompression support?

picoCompression supports any transformer-based language and vision model. The algorithm itself is architecture-agnostic and applies to any neural network with quantizable weights.