Scene detection is one of FFmpeg's most powerful—and most overlooked—features. It can automatically find every cut in a video, split it into individual scenes, extract representative thumbnails, and even build highlight reels. Yet most users don't know these filters exist, and those who do struggle to find working examples.
As one frustrated user wrote on the FFmpeg mailing list back in 2011: "I have been looking for >1 year on the www, from top to bottom and could not find anything"[reference:0]. A decade later, the situation has improved—but only slightly.
This guide changes that. You'll learn the three main approaches to scene detection, how to tune the parameters for different content types, and how to build automated editing pipelines that save hours of manual work.
Key takeaways
- Three scene detection methods:
select(simple),scdet(metadata‑based), andblackdetect(black frames). - Threshold tuning is the key—lower values detect more cuts, higher values detect fewer.
select='gt(scene, X)'is the simplest way to detect scene changes.scdetsets frame metadata (lavfi.scd.score,lavfi.scd.time) that other filters can use.blackdetectdetects black frames—perfect for finding commercials or fades.- Automated editing workflows can split videos, extract thumbnails, and build highlight reels with a single command.
The Gap: A Hidden Superpower
The FFmpeg documentation for scene detection is scattered across three different filter pages. The select filter has a scene parameter buried in its expression documentation. The scdet filter was added in 2019 but is still poorly documented[reference:1]. The blackdetect filter is mentioned in passing.
Worse, the examples online are often incomplete or contradictory. Some users recommend select='gt(scene,0.4)', others say 0.3 or 0.5. Few explain why these values work or how to tune them for different content.
This guide bridges that gap with clear explanations, working examples, and practical workflows.
The Symptom: "I Spent Hours Finding the Good Parts"
You've got hours of footage. You need to find the best moments, split the video into scenes, or extract thumbnails for a storyboard. Manually scrubbing through the timeline is tedious and error‑prone.
What if FFmpeg could do it for you? It can.
Core Concepts: What Is Scene Detection?
Scene detection (also called shot boundary detection) identifies the points in a video where the content changes significantly. These are typically:
- Hard cuts — an abrupt transition from one shot to another
- Fades — a gradual transition to/from black or white
- Dissolves — a gradual transition where one shot blends into another
FFmpeg provides three filters for detecting these boundaries:
| Filter | Method | Best for |
|---|---|---|
select with scene | Pixel‑difference analysis | Quick scene detection, thumbnail extraction |
scdet | MAFD (Mean Absolute Frame Difference) | Detailed scene detection with metadata |
blackdetect | Black frame detection | Fades, commercials, title cards |
Method 1: The select Filter with scene
The select filter is the simplest way to detect scene changes. The scene variable returns a value between 0 and 1 indicating the probability of a scene change[reference:2].
Syntax:
select='gt(scene, THRESHOLD)'THRESHOLD— A value between 0 and 1. Lower values detect more cuts (including subtle ones); higher values detect fewer[reference:3].- Sane values are typically in the 0.3 to 0.5 range[reference:4].
To detect scene changes and print the frame numbers:
# Detect scene changes and print frame numbers (threshold 0.4) ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',metadata=print:file=-" -an -f null - 2>&1 | grep scene
As the FFmpeg mailing list confirms: "Check the select filter and the scene parameter"[reference:5]. The select filter is the most widely used method for scene detection[reference:6].
Method 2: The scdet Filter
The scdet (scene detection) filter was added in 2019[reference:7]. It's more sophisticated than the select filter approach because it sets frame metadata that other filters can use[reference:8].
Syntax:
scdet=threshold=VALUE[:sc_pass=FLAG]threshold(ort) — Scene change threshold as a percentage of maximum change. Good values are in the[8.0, 14.0]range[reference:9].sc_pass(ors) — If set to1, only scene change frames are passed to the next filter[reference:10].
The filter sets three metadata keys on every frame[reference:11]:
lavfi.scd.mafd— Mean Absolute Frame Differencelavfi.scd.score— Scene change scorelavfi.scd.time— Timestamp of the frame
# Detect scene changes with scdet (threshold 10%) ffmpeg -i input.mp4 -vf "scdet=t=10" -an -f null - 2>&1 | grep "lavfi.scd"
Pass‑through mode: When sc_pass=1, only scene change frames are passed through[reference:12]:
# Pass only scene change frames (for thumbnail extraction) ffmpeg -i input.mp4 -vf "scdet=s=1,t=10" -vsync vfr thumb_%03d.jpg
Note: As one developer noted, "scdet filter doesn't choose a mode. The default has to be current 'Legacy' for backward compatibility"[reference:13]. The filter forwards all frames but sets metadata on each one[reference:14].
Method 3: The blackdetect Filter
The blackdetect filter detects frames that are (almost) black[reference:15]. This is useful for finding fades, commercial breaks, title cards, and scene transitions that use black frames[reference:16].
Syntax:
blackdetect=d=DURATION[:pix_th=PIXEL_THRESHOLD]d(duration) — Minimum duration of detected black in seconds[reference:17]pix_th(pixel threshold) — Pixel luminance threshold (default 0.10)[reference:18]
# Detect black frames (0.05 seconds minimum, 0.10 pixel threshold) ffmpeg -i input.mp4 -vf "blackdetect=d=0.05:pix_th=0.10" -an -f null - 2>&1 | grep blackdetect
As noted in a real‑world example: "On a 64 core system it takes around 31 seconds to process an hour‑long CNN broadcast"[reference:19]. The blackdetect filter is efficient even on long videos.
For structured output, use ffprobe instead of parsing the raw log[reference:20]:
# Get blackdetect output as JSON via ffprobe ffprobe -v quiet -show_entries frame_tags=lavfi.blackdetect.start,lavfi.blackdetect.end -of json input.mp4
Practical Example 1: Split Video by Scene
One of the most powerful applications is automatically splitting a video into individual scenes. This can be done using the select filter with the segment muxer[reference:21].
# Split video into scenes using scene detection + segment muxer ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',setpts=N/FRAME_RATE/TB" -vsync vfr -f segment -segment_time 0.001 -reset_timestamps 1 scene_%03d.mp4
As one Gist example shows, you can also use the scdet filter directly with the segment muxer[reference:22]:
# Split using scdet with segment muxer ffmpeg -i input.mp4 -vf "scdet" -map 0 -f segment -segment_format mp4 scene_%07d.mp4
Warning: Splitting using scdet with the segment muxer can be slow on long videos. One user reported: "I didnt bother to wait for the scene detector to finish parsing the 40 minutes"[reference:23]. For faster results, use keyframe‑based splitting[reference:24].
Practical Example 2: Extract Thumbnails
Extract one thumbnail per scene—perfect for video previews or storyboards.
# Extract one thumbnail per scene ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',scale=320:-1" -vsync vfr thumb_%03d.jpg
For a more comprehensive storyboard, you can extract thumbnails and arrange them in a grid[reference:25].
Practical Example 3: Create a Storyboard
A storyboard displays multiple scene thumbnails in a single image, arranged in a grid[reference:26].
# Create a storyboard grid (6 columns, 80 rows) ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',scale=160:-1,tile=6x80" -frames:v 1 -q:v 3 storyboard.jpg
This command selects frames where scene changes occur, scales them to 160px wide, and arranges them in a grid of 6 columns and up to 80 rows[reference:27].
Practical Example 4: Detect Commercials
The blackdetect filter is commonly used to identify commercial breaks. Many TV broadcasts use black frames as markers between content and commercials[reference:28].
# Detect commercial breaks (black periods >= 0.5 seconds) ffmpeg -i broadcast.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -an -f null - 2>&1 | grep blackdetect
As documented in a real‑world example: "we use FFMPEG to detect all of the blackframe transition periods in the video where the programming transitions either to or from a commercial break"[reference:29].
Practical Example 5: Build a Highlight Reel
Combine scene detection with other filters to build automated highlight reels. For example, detect the "busiest" parts of a video by using a low threshold[reference:30].
# Detect high-activity segments (low threshold = more "busy" frames detected) ffmpeg -i input.mp4 -vf "select='gt(scene,0.05)'" -vsync vfr -f concat -safe 0 highlight_reel.txt
As noted by one user: "If you set a low threshold, every frame is regarded as a scene change when there is sufficient 'activity'"[reference:31]. This can be used to identify the most dynamic parts of a video.
Debugging Common Scene Detection Issues
Here are the most common issues and how to fix them.
| Problem | Likely Cause | Fix |
|---|---|---|
| Too few scenes detected | Threshold is too high | Lower the threshold (e.g., 0.3 instead of 0.5)[reference:32] |
| Too many false positives | Threshold is too low | Raise the threshold (e.g., 0.5 instead of 0.3) |
| No output from scdet | Filter isn't logging or metadata isn't being printed | Use -loglevel debug or check metadata with ffprobe[reference:33] |
| Blackdetect misses fades | Duration threshold is too high or pixel threshold is too low | Lower d (e.g., 0.05) or adjust pix_th[reference:34] |
| Splitting is slow | Scene detection on long videos is CPU‑intensive | Use keyframe‑based splitting for speed[reference:35] |
| Thumbnails are blurry | Not scaling or using wrong format | Add scale filter before output[reference:36] |
Visualize Scene Changes with the FFmpegLab IDE
Scene detection is much easier to debug with visual feedback. The FFmpegLab IDE lets you:
- Visualize detected scenes on a timeline—see exactly where every cut occurs
- Adjust the threshold with a slider and see which scenes appear/disappear in real time
- Preview scene thumbnails in a storyboard view
- Compare different detection methods (
select,scdet,blackdetect) side by side - Export scene timestamps as CSV, JSON, or EDL[reference:37]
Here's how scene detection visualization looks in the FFmpegLab IDE.
# Visual scene timeline with detected cuts # Adjust threshold slider → scenes update in real time
The IDE shows you exactly what's happening—a visual timeline of detected scenes, color‑coded by type (scene changes, fades, commercials). You can adjust the threshold with a slider and see which scenes appear or disappear in real time.
Frequently Asked Questions (FAQ)
What's the difference between scdet and select with scene?
select='gt(scene, X)' is simpler and works well for most use cases. scdet is more advanced—it sets frame metadata (lavfi.scd.score, lavfi.scd.time) that other filters can use[reference:38]. For basic scene detection, select is sufficient. For complex pipelines where you need the scene score, use scdet.
What threshold should I use for scene detection?
For select='gt(scene, X)', values between 0.3 and 0.5 work well for most content[reference:39][reference:40]. For scdet, good values are in the [8.0, 14.0] range[reference:41]. Lower values detect more cuts (including subtle ones); higher values detect fewer.
How do I get the timestamps of scene changes?
Use the metadata filter with select: ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',metadata=print:file=-" -an -f null - 2>&1 | grep scene. For scdet, the lavfi.scd.time metadata key provides the timestamp[reference:42].
Can I split a video into scenes without re-encoding?
Yes—but with caveats. Use -c copy with the segment muxer. However, keyframe‑accurate splitting requires re‑encoding. For fastest results, use keyframe‑based splitting: ffmpeg -i input.mp4 -c copy -map 0 -segment_time 0.01 -f segment scene_%07d.mp4[reference:43]. The segment muxer can split by scene changes[reference:44], but accuracy depends on keyframe placement.
How do I detect fades and dissolves?
Use blackdetect for fades to/from black[reference:45]. For dissolves (where one shot blends into another), the select and scdet filters can detect them as scene changes, but the threshold may need tuning. Dissolves typically produce lower scene change scores than hard cuts.
Is scene detection available in hardware‑accelerated encoding?
Scene detection filters are CPU‑based. However, you can use hardware‑accelerated encoding (-c:v h264_nvenc or hevc_nvenc) after scene detection is complete. For real‑time applications, consider using scdet_vulkan[reference:46] for GPU‑accelerated scene detection.
Final Word
Scene detection is one of FFmpeg's most powerful—and most underutilized—features. With the select, scdet, and blackdetect filters, you can automatically split videos, extract thumbnails, detect commercials, and build highlight reels.
The key is understanding the three methods and knowing when to use each one:
select='gt(scene, X)'— Simple, fast, works for most use casesscdet— Advanced, sets metadata for complex pipelinesblackdetect— Specialized for fades, commercials, and title cards
With the FFmpegLab IDE's visual timeline and real‑time threshold adjustment, you can experiment, debug, and perfect your scene detection without the guesswork.
Next time you're facing hours of footage, let FFmpeg do the heavy lifting. Your eyes—and your schedule—will thank you.