Subtitles are essential for accessibility, global reach, and viewer engagement. But generating and burning subtitles manually is tedious and time‑consuming. With AI‑powered speech‑to‑text, you can automate the entire process — from audio transcription to subtitle burn‑in — making your content accessible to a global audience.
This guide shows you how to build a fully automated subtitle generation pipeline that:
- Extracts audio from uploaded videos
- Transcribes speech using OpenAI Whisper (state‑of‑the‑art multilingual model)
- Generates SRT subtitle files with timestamps
- Burns subtitles into the video using FFmpeg
- Delivers the final video with embedded subtitles
Key takeaways
- One SQL script — everything is set up with a single execution against your Supabase database.
- Whisper AI — state‑of‑the‑art speech‑to‑text supporting 99 languages.
- Zero manual intervention — upload a video, get a subtitled version automatically.
- Scalable and reliable — pgmq provides durable, transaction‑safe job queuing.
- Configurable — adjust language, model size, and subtitle styling.
The Gap: Accessibility at Scale
Subtitles are no longer optional. They're essential for:
- Accessibility — deaf and hard‑of‑hearing viewers
- Global reach — viewers who speak different languages
- Engagement — videos with subtitles have higher watch time
- Compliance — accessibility regulations (ADA, WCAG, etc.)
But generating subtitles manually is expensive and doesn't scale. What if the pipeline could be fully automated — triggered by the upload itself, transcribing and burning subtitles in the background?
This guide shows you exactly how to build that pipeline.
Architecture Overview
User Uploads Video
→
Supabase Storage
→
PostgreSQL Trigger
→
pgmq Queue
→
ffmpeglab-runner
Extract Audio → Whisper AI → Generate SRT → Burn Subtitles
Subtitled Video → Public Folder → pg_notify → User Notified
Extract Audio → Whisper AI → Generate SRT → Burn Subtitles
Subtitled Video → Public Folder → pg_notify → User Notified
The pipeline consists of:
- Supabase Storage — two buckets:
private-uploads(per‑user) andpublic-processed(per‑user). - RLS policies — restrict access to each user's own folders.
- PostgreSQL trigger — fires on
INSERTintostorage.objects. - pgmq — message queue for job processing.
- ffmpeglab-runner — extracts audio, runs Whisper, burns subtitles.
- pg_notify — real‑time status updates.
What the Pipeline Delivers
| Output | Format | Location |
|---|---|---|
| Subtitled Video | MP4 (H.264) with embedded subtitles | public-processed/{userId}/subtitled/ |
| SRT Subtitle File | .srt (standalone subtitles) | public-processed/{userId}/subtitles/ |
| Real‑time notifications | pg_notify channels | N/A |
| Job tracking | render table | Existing FFmpegLab table |
| Logs | logpiece table | Existing FFmpegLab table |
Prerequisites
- A Supabase project (cloud or self‑hosted).
- ffmpeglab-server and ffmpeglab-runner deployed.
- Whisper.cpp installed on the runner (see Setting Up Whisper).
renderandlogpiecetables must exist (from FFmpegLab server migrations).- Access to your Supabase database (psql or the Supabase SQL Editor).
Quick Start – The Setup Script
Step 1
Save the SQL script
Copy the complete SQL script from the section below and save it as
setup_subtitle_pipeline.sql.Step 2
Run the script
Execute the script against your Supabase database.
# Via psql
psql -U postgres -d your_database -f setup_subtitle_pipeline.sql
# Or via the Supabase SQL Editor # Copy and paste the entire script into the SQL Editor and run it.
# Or via the Supabase SQL Editor # Copy and paste the entire script into the SQL Editor and run it.
Step 3
Configure the runner
Add the subtitle queue to your runner's environment.
# Add to your .env or docker-compose.yml
SUBTITLE_QUEUE_NAME=subtitle_queue
# Whisper model path WHISPER_MODEL_PATH=/app/models/whisper/ggml-base.bin
# Whisper language (auto, en, es, fr, etc.) WHISPER_LANGUAGE=auto
# Whisper model path WHISPER_MODEL_PATH=/app/models/whisper/ggml-base.bin
# Whisper language (auto, en, es, fr, etc.) WHISPER_LANGUAGE=auto
Step 4
Restart the runner
Restart the runner to pick up the new queue.
docker compose restart ffmpeglab-runner
The Complete SQL Setup Script
-- ============================================================
-- AUTOMATED SUBTITLE GENERATION PIPELINE
-- Complete SQL Setup Script
-- ============================================================
-- This script adds the subtitle generation pipeline components.
-- It assumes the render and logpiece tables already exist.
-- ============================================================
-- ============================================================
-- 1. Create storage buckets
-- ============================================================
INSERT INTO storage.buckets (id, name, public, avif_autodetection, file_size_limit, allowed_mime_types)
VALUES
('private-uploads', 'private-uploads', false, false, 5368709120, ARRAY['video/mp4', 'video/quicktime', 'video/x-msvideo', 'video/webm', 'video/mpeg']),
('public-processed', 'public-processed', true, false, 5368709120, ARRAY['video/mp4', 'text/plain'])
ON CONFLICT (id) DO NOTHING;
-- ============================================================
-- 2. RLS policies for private-uploads bucket
-- ============================================================
DROP POLICY IF EXISTS "Users can upload to their own folder" ON storage.objects;
CREATE POLICY "Users can upload to their own folder"
ON storage.objects
FOR INSERT
TO authenticated
WITH CHECK (
bucket_id = 'private-uploads' AND
(storage.foldername(name))[1] = auth.uid()::text
);
DROP POLICY IF EXISTS "Users can download from their own folder" ON storage.objects;
CREATE POLICY "Users can download from their own folder"
ON storage.objects
FOR SELECT
TO authenticated
USING (
bucket_id = 'private-uploads' AND
(storage.foldername(name))[1] = auth.uid()::text
);
-- ============================================================
-- 3. RLS policies for public-processed bucket
-- ============================================================
DROP POLICY IF EXISTS "Public read access to processed media" ON storage.objects;
CREATE POLICY "Public read access to processed media"
ON storage.objects
FOR SELECT
USING (bucket_id = 'public-processed');
DROP POLICY IF EXISTS "Service role can manage processed media" ON storage.objects;
CREATE POLICY "Service role can manage processed media"
ON storage.objects
FOR ALL
TO service_role
USING (bucket_id = 'public-processed');
DROP POLICY IF EXISTS "Users can read their own processed media" ON storage.objects;
CREATE POLICY "Users can read their own processed media"
ON storage.objects
FOR SELECT
TO authenticated
USING (
bucket_id = 'public-processed' AND
(storage.foldername(name))[1] = auth.uid()::text
);
-- ============================================================
-- 4. Enable pgmq extension and create the queue
-- ============================================================
CREATE EXTENSION IF NOT EXISTS pgmq;
SELECT pgmq.create('subtitle_queue');
-- ============================================================
-- 5. Create the trigger function for subtitle generation
-- ============================================================
DROP FUNCTION IF EXISTS handle_subtitle_generation() CASCADE;
CREATE OR REPLACE FUNCTION handle_subtitle_generation()
RETURNS TRIGGER AS $$
DECLARE
user_id text;
file_path text;
file_name text;
file_extension text;
base_filename text;
msg jsonb;
commands jsonb := '[]';
video_mime_types text[] := ARRAY['video/mp4', 'video/quicktime', 'video/x-msvideo', 'video/webm', 'video/mpeg'];
BEGIN
IF NEW.bucket_id = 'private-uploads' AND NEW.metadata->>'mimetype' = ANY(video_mime_types) THEN
user_id := (storage.foldername(NEW.name))[1];
file_path := NEW.name;
file_name := (storage.filename(NEW.name));
file_extension := split_part(file_name, '.', array_length(string_to_array(file_name, '.'), 1));
base_filename := replace(file_name, '.' || file_extension, '');
-- Command 1: Extract audio
commands := commands || jsonb_build_object(
'type', 'extract_audio',
'output_path', user_id || '/temp/' || base_filename || '.wav',
'command', 'ffmpeg -i "INPUT_FILE" -ac 1 -ar 16000 -vn "AUDIO_OUTPUT"'
);
-- Command 2: Run Whisper transcription
commands := commands || jsonb_build_object(
'type', 'transcribe',
'output_path', user_id || '/subtitles/' || base_filename || '.srt',
'command', 'whisper --model "WHISPER_MODEL" --language "WHISPER_LANG" --output-srt "AUDIO_INPUT"'
);
-- Command 3: Burn subtitles into video
commands := commands || jsonb_build_object(
'type', 'burn_subtitles',
'output_path', user_id || '/subtitled/' || base_filename || '_subtitled.mp4',
'command', 'ffmpeg -i "INPUT_FILE" -vf "subtitles=SRT_FILE:force_style=\'FontName=Arial,FontSize=24,PrimaryColour=&HFFFFFF,OutlineColour=&H000000,Outline=2\'" -c:a copy "OUTPUT_FILE"'
);
msg := jsonb_build_object(
'userId', user_id,
'inputPath', file_path,
'inputBucket', NEW.bucket_id,
'outputBucket', 'public-processed',
'fileName', file_name,
'baseFilename', base_filename,
'commands', commands,
'timestamp', NOW()
);
PERFORM pgmq.send('subtitle_queue', msg::jsonb);
INSERT INTO "render" (id, title, project, status, public, user_id, data)
VALUES (
gen_random_uuid(),
file_name,
'subtitle-generation',
'queued',
false,
user_id::uuid,
msg
);
PERFORM pg_notify(
'subtitle_generation_channel',
jsonb_build_object(
'userId', user_id,
'filePath', file_path,
'status', 'queued',
'timestamp', NOW()
)::text
);
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
-- ============================================================
-- 6. Attach the trigger to storage.objects
-- ============================================================
DROP TRIGGER IF EXISTS subtitle_upload_trigger ON storage.objects;
CREATE TRIGGER subtitle_upload_trigger
AFTER INSERT ON storage.objects
FOR EACH ROW
EXECUTE FUNCTION handle_subtitle_generation();
-- ============================================================
-- 7. Helper views for monitoring
-- ============================================================
DROP VIEW IF EXISTS subtitle_queue_view;
CREATE OR REPLACE VIEW subtitle_queue_view AS
SELECT
msg_id,
read_ct,
enqueued_at,
vt,
message::jsonb as payload,
(message::jsonb->>'userId') as user_id,
(message::jsonb->>'fileName') as file_name
FROM pgmq.q_subtitle_queue
ORDER BY msg_id DESC;
-- ============================================================
-- 8. Initialize notification channels
-- ============================================================
DO $$
BEGIN
PERFORM pg_notify('subtitle_generation_channel', '{"init": true}');
PERFORM pg_notify('render_status_channel', '{"init": true}');
PERFORM pg_notify('log_channel', '{"init": true}');
END $$;
FFmpeg Commands
The trigger function generates the following commands:
1. Extract Audio
ffmpeg -i "INPUT_FILE" -ac 1 -ar 16000 -vn "AUDIO_OUTPUT"
💡
Audio Extraction Parameters Explained
-ac 1— Convert to mono (Whisper works best with mono audio)-ar 16000— Resample to 16kHz (Whisper's optimal sample rate)-vn— Disable video output (audio only)
2. Whisper Transcription
whisper --model /app/models/whisper/ggml-base.bin --language auto --output-srt "AUDIO_INPUT"
💡
Whisper Parameters Explained
--model— Path to the Whisper model (base, small, medium, large)--language— Language code (auto, en, es, fr, etc.)--output-srt— Output SRT subtitle file
3. Burn Subtitles into Video
ffmpeg -i "INPUT_FILE" -vf "subtitles=SRT_FILE:force_style='FontName=Arial,FontSize=24,PrimaryColour=&HFFFFFF,OutlineColour=&H000000,Outline=2'" -c:a copy "OUTPUT_FILE"
💡
Subtitle Burn-in Parameters Explained
subtitles=SRT_FILE— Input SRT fileforce_style— Styling options:FontName=Arial— Font familyFontSize=24— Font size in pointsPrimaryColour=&HFFFFFF— White text (HEX: &HBBGGRR)OutlineColour=&H000000— Black outlineOutline=2— Outline width-c:a copy— Copy audio stream without re-encoding
Setting Up Whisper
The runner needs Whisper installed. You can use the Python version or whisper.cpp for better performance.
Option 1: whisper.cpp (Recommended for CPU)
# Clone whisper.cpp
git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp
# Build
make -j
# Download models
bash models/download-ggml-model.sh base
bash models/download-ggml-model.sh small
bash models/download-ggml-model.sh medium
# Move models to models directory
cp models/ggml-*.bin /app/models/whisper/
Option 2: Python Whisper (More Features)
# Install whisper
pip install openai-whisper
# Download models
whisper --model base.en --download_only
whisper --model small.en --download_only
Docker Integration
# docker-compose.yml addition
services:
ffmpeglab-runner:
volumes:
- ./models/whisper:/app/models/whisper
environment:
- WHISPER_MODEL_PATH=/app/models/whisper/ggml-base.bin
- WHISPER_LANGUAGE=auto
Configure ffmpeglab-runner
Step 1
Add the queue to your environment
Add the following to your
.env file or Docker Compose configuration.
# Subtitle generation queue
SUBTITLE_QUEUE_NAME=subtitle_queue
# Whisper settings
WHISPER_MODEL_PATH=/app/models/whisper/ggml-base.bin
WHISPER_LANGUAGE=auto
Step 2
Implement the processing loop
The runner should execute the following steps:
# 1. Connect to Supabase and listen to the queue
# 2. For each job:
# a. Download the input file from private-uploads
# b. Parse the 'commands' array from the job payload
# c. Execute each command:
# - Extract audio (ffmpeg)
# - Run Whisper transcription
# - Burn subtitles (ffmpeg)
# d. Upload outputs to public-processed
# e. Update render table with progress and logs
# f. Mark job complete and delete from queue
# 2. For each job:
# a. Download the input file from private-uploads
# b. Parse the 'commands' array from the job payload
# c. Execute each command:
# - Extract audio (ffmpeg)
# - Run Whisper transcription
# - Burn subtitles (ffmpeg)
# d. Upload outputs to public-processed
# e. Update render table with progress and logs
# f. Mark job complete and delete from queue
Step 3
Restart the runner
After updating the environment, restart the runner service.
docker compose restart ffmpeglab-runner
Monitor the Pipeline
Step 1
Check queued jobs
Use the helper view to see all queued jobs.
SELECT * FROM subtitle_queue_view;
Step 2
Check render status
Query the existing
render table for job status.
SELECT id, title, status, progress, data
FROM "render"
WHERE project = 'subtitle-generation'
ORDER BY created_at DESC;
Step 3
Listen to notifications
In your application, listen for real‑time updates.
-- In your PostgreSQL client:
LISTEN subtitle_generation_channel;
LISTEN render_status_channel;
LISTEN log_channel;
Frequently Asked Questions (FAQ)
How does the automated subtitle pipeline work?
When a video is uploaded, a PostgreSQL trigger fires and pushes a job to pgmq. The ffmpeglab-runner extracts the audio, transcribes it using Whisper AI, generates an SRT subtitle file, and burns it into the video using FFmpeg's subtitles filter. The result is a video with embedded subtitles, stored in a public bucket.
What AI model is used for transcription?
The pipeline uses OpenAI's Whisper model (whisper.cpp implementation) for speech-to-text transcription. Whisper is a state-of-the-art multilingual model that supports 99 languages and runs efficiently on CPU, GPU, or Apple Silicon.
What languages are supported?
Whisper supports 99 languages including English, Spanish, French, German, Chinese, Japanese, Hindi, and many more. You can specify the language in the pipeline configuration or let Whisper auto-detect it.
Can I customize the subtitle styling?
Yes. The FFmpeg subtitles filter supports styling options including font, size, color, shadow, outline, and positioning. You can customize these in the runner's FFmpeg command.
How accurate is the transcription?
Whisper achieves state-of-the-art accuracy, with word error rates (WER) as low as 2-5% for English in clean audio. Accuracy depends on audio quality, background noise, and accents. Using a larger model (medium or large) improves accuracy.
Final Word
You now have a fully automated subtitle generation pipeline that uses Whisper AI to transcribe speech and burn subtitles into videos. With PostgreSQL triggers, pgmq, and Supabase Storage, you get:
- AI-powered transcription — state-of-the-art Whisper model
- 99 language support — global reach
- Real‑time notifications — know when processing is complete
- Full observability — monitoring views and logs
- Scalable architecture — handles any volume of videos
The pipeline is production‑ready, scalable, and configurable. It uses the existing render and logpiece tables from the FFmpegLab server, so there are no table conflicts — just pure, automated subtitle generation.