TTS Audio Formats: WAV, MP3, PCM, and Opus Compared

TLDR: Text-to-speech engines internally generate raw PCM (Pulse-Code Modulation) audio, then encode it into a container or compressed format for output. WAV wraps PCM with a 44-byte header and preserves full quality at roughly 768 kbps for standard 16-bit, 48 kHz mono files. MP3 compresses speech to 32-128 kbps with lossy encoding, adding 20-50 ms of encoder latency. Opus (RFC 6716) compresses speech to 6-64 kbps at higher quality than MP3 at equivalent bitrates, with an algorithmic latency of approximately 25 ms at the default 20 ms frame size in speech mode. For real-time TTS applications such as voice assistants and telephony, PCM or Opus minimizes latency. For stored audio files served over the web, MP3 provides the broadest playback compatibility. WAV is the reference format for quality evaluation and post-processing pipelines.

What Is PCM Audio?

PCM (Pulse-Code Modulation) is the raw, uncompressed audio baseline produced by all TTS neural vocoders before encoding. It is the standard method used to digitally represent sampled analog signals. An analog-to-digital converter (or, in TTS, a neural vocoder) samples the waveform at a fixed rate and quantizes each sample to a fixed bit depth (typically 16-bit signed integers or 32-bit floats). The result is a stream of numbers, one per sample, with no compression, no headers, and no metadata.

Every TTS engine produces PCM internally. The vocoder (WaveRNN, HiFi-GAN, or equivalent) outputs a PCM waveform, which is then optionally encoded into WAV, MP3, Opus, or another format before being returned to the caller. Requesting PCM output from a TTS API means receiving the vocoder's raw output with zero encoding overhead.

PCM parameters that matter for TTS:

• Sample rate: 8 kHz is used for traditional telephony, while 16 kHz is standard for wideband telephony and modern IVR. 22.05 kHz is common for on-device neural TTS; 24 kHz and 48 kHz are used for high-fidelity cloud TTS

• Bit depth: 16-bit is standard; 32-bit float is used in processing pipelines

• Channel count: Mono for speech; stereo is unnecessary for single-speaker TTS

WAV: PCM with a Header

WAV (Waveform Audio File Format) is a container that wraps PCM data with a 44-byte RIFF header specifying sample rate, bit depth, channel count, and data length. While the RIFF container specification permits various codecs, WAV files in TTS pipelines almost exclusively utilize uncompressed PCM to guarantee lossless audio. A WAV file at 16-bit, 24 kHz mono produces 48 KB per second of audio, or roughly 2.9 MB per minute.

WAV is the default output format for quality evaluation because it preserves the exact output of the TTS vocoder with no lossy compression artifacts. Any post-processing pipeline (noise reduction, normalization, equalization, concatenation) should operate on WAV or raw PCM to avoid compounding lossy encoding artifacts across processing stages.

The tradeoff is file size. For web delivery, a 30-second mono 16-bit PCM WAV at 24 kHz is about 1.44 MB, versus about 240 KB for an MP3 encoded at 64 kbps. For stored prompt libraries, WAV files therefore use roughly 6× more storage than a 64 kbps MP3, and often 6–10× more than common compressed speech formats depending on bitrate.

MP3: Lossy Compression with Universal Playback

MP3 (MPEG-1 Audio Layer III) uses psychoacoustic modeling to discard audio information that human hearing is less sensitive to, then applies Huffman coding to compress the result. For speech content, 64 kbps mono MP3 is perceptually transparent to most listeners; 32 kbps is acceptable for telephony-grade applications.

MP3's ubiquitous native support across browsers, mobile operating systems, and hardware decoders makes it the de facto standard for HTTP delivery, entirely eliminating the need to ship client-side decoder libraries. This makes MP3 the default choice for audio files served over HTTP, embedded in web pages, or distributed as downloadable content.

However, MP3's reliance on a 1,152-sample frame structure and a bit reservoir inherently introduces 20-50 ms of algorithmic latency, rendering it sub-optimal for latency-sensitive streaming pipelines. For streaming text-to-speech (where audio must begin playing as the first tokens are generated), this encoding latency adds to the overall time-to-first-audio. At equivalent perceptual quality, MP3 also requires roughly 2x the bitrate of Opus for speech content.

Opus: Low-Latency Compression for Streaming

Opus is an open, royalty-free audio codec standardized by the IETF as RFC 6716. It was designed for interactive speech and music transmission. Independent listening tests on the Opus comparison page show Opus at 64 kbps matching or exceeding MP3 at 128 kbps. The codec supports frame sizes from 2.5 ms to 60 ms, with the overall algorithmic latency ranging from 5 to 66.5 ms depending on mode and frame size. At the default 20 ms frame size in speech mode (SILK), latency is approximately 25 ms.

Opus operates in two internal modes: SILK (optimized for speech, effective at 6-40 kbps) and CELT (optimized for music and full-bandwidth audio). The encoder automatically selects the appropriate mode based on the content and target bitrate, or a hybrid of both for intermediate bitrates. For TTS output, which is pure speech, the SILK mode activates at typical bitrates and delivers high intelligibility at low bandwidth.

Browser support for Opus decoding is universal in modern browsers (Chrome, Firefox, Safari 11+, Edge). However, Opus is not natively playable by all legacy systems: older Android media players, some IVR platforms, and hardware audio decoders in IoT devices may lack Opus support. For these environments, MP3 or raw PCM remains necessary.

Audio Format Comparison for TTS

Requesting raw PCM directly from an on-device/on-prem TTS engine completely eliminates host CPU encoding overhead, maximizing server instance density for high-concurrency environments, whereas encoding to Opus minimizes egress bandwidth at the cost of slight algorithmic latency. Opus serves as the optimal codec for real-time TTS streaming due to its superior compression and low algorithmic latency, whereas MP3 remains the pragmatic fallback for static audio libraries demanding universal client compatibility. For quality evaluation and audio processing pipelines, WAV/PCM preserves the vocoder output without artifacts.

TTS Provider Output Format Support

Each TTS provider supports a different subset of output formats. The format choice is specified in the API request, not in SSML.

Sources: Google Cloud TTS API, Amazon Polly API, Azure Speech REST API, ElevenLabs API, Orca API. Checked July 2026.

How to Choose an Audio Format for TTS Applications

• Real-time voice assistants and conversational AI: Minimize time-to-first-audio by using raw PCM if the client can play uncompressed audio natively. Use Opus over WebRTC or WebSocket if bandwidth is constrained. Avoid MP3, as the encoder latency adds to an already latency-sensitive pipeline.

• IVR and telephony: Request mu-law PCM directly from the TTS API to avoid unnecessary sample rate conversion for telephony systems operating at 8 kHz. For pre-rendered IVR prompt libraries, WAV at 8 kHz/mu-law is the standard storage format.

• Web audio and podcast generation: Use MP3 at 64 kbps mono for maximum compatibility, or Opus at 32 kbps for smaller files on modern browsers. If post-processing audio, generate WAV first, process it, then encode to the distribution format as the final step.

• On-device TTS: Use output raw PCM directly to the audio playback buffer with no encoding or decoding step, as the audio never traverses a network, eliminating encoding latency and bandwidth constraints.

Build Low-Latency TTS Applications

Orca Streaming Text-to-Speech outputs raw PCM directly to the audio playback buffer. No encoding or decoding step is needed because the audio never traverses a network. This eliminates both encoding latency and bandwidth constraints. Orca outputs 16-bit mono PCM at 22050 Hz with a 7 MB model and 29 MB peak memory, achieving 128 ms first-token-to-speech (FTTS) on CPU.

FAQ