Learn how to add subtitles to any video using the Picovoice Leopard speech-to-text Python SDK.

Setup

Extract Audio

First, extract the audio from your video content. You can accomplish this using a tool such as FFmpeg. Leopard ASR engine supports almost any audio format including FLAC, MP3, MP4, m4a, Ogg, WAV, and WebM.

Install Speech Recognition SDK

Install Leopard STT Python SDK:

Log in to (Sign up for) Picovoice Console. It is free. Grab your AccessKey and initialize Leopard:

Implementation

Convert Voice to Text

Process the audio file:

Leopard returns the transcription as an str. It also returns a sequence of word-level metadata including timestamps and confidence. For example:

Convert to SRT Format

SRT (SubRip subtitle) is a file format for storing subtitles. The transcript is organized in sections and each section is accompanied by a start and end timecode. Below is a snippet of a given .srt file:

We need to break the transcription into sections. We use two criteria:

  • If there is a certain duration of silence between two words we consider it an endpoint (i.e. breakpoint). The user is most likely done talking and they or someone else will start a new sentence later.
  • If there is a certain number of words already in the section we should contain it. Otherwise, it makes the screen too crowded.

to_srt method below implements these two logics:

Once we get the content of an SRT file we can simply save it:

YouTube Demo

Let's create subtitles for a YouTube video. First, install PyTube:

Grab the URL for a video you wish to generate captions for (e.g. https://www.youtube.com/watch?v=L1CK9bE3H_s) and download it using PyTube:

Go to the previous section and generate subtitles for it.

Explore

Transcription Confidence

No speech recognition technology is 100% accurate. If you want to assure zero errors a manual check is required at the end. But this can take time. Leopard outputs word-level confidence metrics (i.e. a number within 0 and 1). An interesting avenue to explore is highlighting only low-confidence words for manual review to save time.

Custom Vocabulary and Boosting Keywords

Leopard allows adding custom vocabulary and also boosting keywords via Picovoice Console. We can use this capability to increase the accuracy of Leopard for certain jargon (e.g. when transcribing a React tutorial).

The code is available on GitHub.