cubicibo
22nd July 2024, 22:00
The Blu-ray format supports a feature named Picture-in-Picture, where a secondary video is overlaid on top of the main program. To not display gross video framing in the overlay process, the Blu-ray format can luma-key the secondary video: pixels below a specified luma are made transparent, and the main program shall appear in these regions.
This luma-key approach is simply a binary mask: a pixel either display the primary or secondary video: no alpha blending is ever performed between the two. However, we can outsmart the process and achieve smooth alpha blending!
PIPelette is a VapourSynth script that, given:
a main program (primary video)
a video track to blend smoothly with the main program.
an alpha mask tied to the overlay.
will prepare a secondary video that can be encoded with your favorite AVC encoder (x264) and imported in your authoring software as a secondary video track. In the professional authoring software, you will need to:
Enable the picture-in-picture luma-key feature, with a cut-off luma at Y=0 (or 1, if incl.)
Set the PIP scale to be 1x (primary video = PIP video scale).
Then in your BD menu you may allow the user to enable or disable the secondary video.
Python script (https://github.com/cubicibo/PIPelette). You need to update the inputs and plugins paths in the script, but it works.
The Principle
A picture is worth a thousand words. Of course, the authorer has another plane on-top of those two to display arbitrary subtitles (Presentation Graphics).
https://i.postimg.cc/kVs13thp/mpv-shot0005.png (https://postimg.cc/kVs13thp)
Q&A
What for?
Complex ASS typesetting is probably going to be the primary usage. The ASS format is the perfect example as it provides both the overlay and alpha clips.
Why? Hardsubbing works.
Burning signs is a terrible approach and prevents the watcher to experience a motion picture the way it was originally made. With PIP, signs can be made optional, and numerous languages may be supported.
Can't I just use SUPer?
Animated typesetting will encode better as a video overlay, and require fewer bits.
What about x264 smearing with non-mod16 edges?
The script ensures all of the overlay edges are aligned to mod16. PIPelette targets smooth, high quality overlaying.
I don't have a professional authoring suite. Can I still benefit from it?
Yes! tsMuxer supports secondary PIP tracks. You just need to set the appropriate PIP parameters (0, 0, top left...) and luma-key threshold.
Any known limit?
BDAV 48 Mbps. In any case, most overlays will compress extremely well, as most of the secondary video is black and constant. I.e don't expect a bitrate higher than 1 Mbps in most case.
The overlay luma range has one less value as 0 is reserved. However it doesn't matter anyway as luma display is 16-235.
This luma-key approach is simply a binary mask: a pixel either display the primary or secondary video: no alpha blending is ever performed between the two. However, we can outsmart the process and achieve smooth alpha blending!
PIPelette is a VapourSynth script that, given:
a main program (primary video)
a video track to blend smoothly with the main program.
an alpha mask tied to the overlay.
will prepare a secondary video that can be encoded with your favorite AVC encoder (x264) and imported in your authoring software as a secondary video track. In the professional authoring software, you will need to:
Enable the picture-in-picture luma-key feature, with a cut-off luma at Y=0 (or 1, if incl.)
Set the PIP scale to be 1x (primary video = PIP video scale).
Then in your BD menu you may allow the user to enable or disable the secondary video.
Python script (https://github.com/cubicibo/PIPelette). You need to update the inputs and plugins paths in the script, but it works.
The Principle
A picture is worth a thousand words. Of course, the authorer has another plane on-top of those two to display arbitrary subtitles (Presentation Graphics).
https://i.postimg.cc/kVs13thp/mpv-shot0005.png (https://postimg.cc/kVs13thp)
Q&A
What for?
Complex ASS typesetting is probably going to be the primary usage. The ASS format is the perfect example as it provides both the overlay and alpha clips.
Why? Hardsubbing works.
Burning signs is a terrible approach and prevents the watcher to experience a motion picture the way it was originally made. With PIP, signs can be made optional, and numerous languages may be supported.
Can't I just use SUPer?
Animated typesetting will encode better as a video overlay, and require fewer bits.
What about x264 smearing with non-mod16 edges?
The script ensures all of the overlay edges are aligned to mod16. PIPelette targets smooth, high quality overlaying.
I don't have a professional authoring suite. Can I still benefit from it?
Yes! tsMuxer supports secondary PIP tracks. You just need to set the appropriate PIP parameters (0, 0, top left...) and luma-key threshold.
Any known limit?
BDAV 48 Mbps. In any case, most overlays will compress extremely well, as most of the secondary video is black and constant. I.e don't expect a bitrate higher than 1 Mbps in most case.
The overlay luma range has one less value as 0 is reserved. However it doesn't matter anyway as luma display is 16-235.