[AviSynth+] Need help with rendering subtitles on YUV420P10

[Atak_Snajpera]

VSFilter does not work with 10 bit so I'm trying to overlay subtitles manually.

My code
SubtitleColor=BlankClip(video,color=$FFFF00,pixel_type="RGB32")
SubtitleMask=BlankClip(video,color=$000000,pixel_type="RGB32")

Loadplugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\VSFilter\VSFilter.dll")
SubtitleMask=VobSub(SubtitleMask,"C:\Temp\RipBot264temp\job1\5_subtitles_English_1080.sub")

SubtitleColor=ConvertToYUV420(SubtitleColor,matrix="Rec2020").ConvertBits(10)
SubtitleMask=ConvertToYUV420(SubtitleMask,matrix="Rec2020").ConvertBits(10)

video=Overlay(video,SubtitleColor,mask=SubtitleMask,opacity=1)

Result
http://i.cubeupload.com/f4uP9g.png

Original
http://i.cubeupload.com/piKuHH.png

As you can see there is problem with mask because yellow color is not entirely removed. I reckon that it's a problem with 255 (8bit) vs 1023 scale (10bit) after conversion from RGB.
Can some expert help me to correct this issue?

[raffriff42]

Yes, I've had this experience (https://forum.doom9.org/showthread.php?p=1806888#post1806888). Use mt_inpand.

[Yanak]

Can't try for now but what version of VSfilter is used, r4 ?

The R5 version might help maybe, if it's not already the one used
https://github.com/sorayuki/VSFilterMod/releases
binaries here : https://www.nmm-hd.org/newbbs/viewtopic.php?f=17&t=1989

Edit : Nevermind, YUV420P10 is supported only for vapoursynth it seems , sorry.

[Atak_Snajpera]

I think I figured out how to overlay subtitles correctly on YUV420P10!


#Subtitles
Loadplugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\VSFilter\VSFilter.dll")
SubtitleColor=BlankClip(video,color=$FFFF00).Tweak(sat=0.5)
SubtitleMask=BlankClip(video,color=$000000,pixel_type="YV12")
SubtitleMask=VobSub(SubtitleMask,"C:\Temp\RipBot264temp\job1\5_subtitles_English_1080.sub")
SubtitleMask=Grayscale(SubtitleMask).ConvertBits(10)
SubtitleMask=Levels(SubtitleMask,0, 0.25, 768, 0,1023, coring=false)
video=Overlay(video,SubtitleColor,mask=SubtitleMask,opacity=1)



Result
http://i.cubeupload.com/nGwsQO.png

[jpsdr]

Looking at the text in the subtitles make me curious...:p What movie is that ?

[Sparktank]

Looking at the text in the subtitles make me curious...:p What movie is that ?

I think that's Passengers? With Chris Pratt?

[jpsdr]

Now that you said it... Indeed, it must be.

[Atak_Snajpera]

Do you know guys a good and fast anti-aliasing filter? Something similar to FXAA? I would like to smooth a little bit mask without using simple blur.
zoom 200%
http://i.cubeupload.com/4zjrdS.png

[raffriff42]

> a good and fast anti-aliasing filter? Something similar to FXAA?

Don't know if it's fast enough for you, butUpscale with eedi3 (sclip=nnedi3()) and then scale it back
Repeat it for a few times
My implementation of feisty2's idea, called pow2Qt, here (https://forum.doom9.org/showthread.php?p=1737201#post1737201).

[Atak_Snajpera]

When I see eedi3 I know that It will be too slow. For now I'm using Sharpen(-1). To bad that after all those years we still do not have FXAA/SMAA in avisynth.

[raffriff42]

Don't think FXAA can work on a bitmap, but SMAA is a post-process shader AA (https://www.google.com/search?q=SweetFX+SMAA).
Maybe MysteryX's AviSynth Shader (http://forum.doom9.org/showthread.php?t=172698) can help. (I'd be interested in that, myself)

[Atak_Snajpera]

Both FXAA and SMAA work as post-process AA.

[nautilus7]

I used to smooth my subtitles using one of the 2 methods:

1. GaussianBlur()
2. Render the subs in 2x resolution and then doing a GaussResize() at half resolution.

It worked fine, but I wouldn't say it was fast.

[`Orum]

Do you know guys a good and fast anti-aliasing filter? Something similar to FXAA?
While I totally agree about the need for FXAA/SMAA and friends for AviSynth (perhaps I'll port them over at some point), for what you're doing they seem unnecessary. A better approach for subtitles is to OCR them and include them in a text-based format, e.g. SRT or ASS formats. The only reasons not to do so that I can think of is laziness (you can't be bothered to OCR), or playback speed/support on devices.

[kolak]

I used to smooth my subtitles using one of the 2 methods:

1. GaussianBlur()
2. Render the subs in 2x resolution and then doing a GaussResize() at half resolution.

It worked fine, but I wouldn't say it was fast.

This is quite effective way. I use to do the same.

[Atak_Snajpera]

In case you are interested here is my final function

function TextSubHDR(clip video, string "SubPath", int "SubColor", int "SubOutlineColor", int "SubShadowColor" )
{
SubtitleColor=BlankClip(video,color=SubColor,pixel_type="YUV420P10")
OutlineColor=BlankClip(video,color=SubOutlineColor,pixel_type="YUV420P10")
ShadowColor=BlankClip(video,color=SubShadowColor,pixel_type="YUV420P10")

SubtitleOutlineColor=BlankClip(video,color=$FFFFFF,pixel_type="YV12")
SubtitleOutlineMask=BlankClip(video,color=$000000,pixel_type="YV12")

SubtitleMask=BlankClip(video,color=$000000,pixel_type="YV12")
SubtitleMask=TextSub(SubtitleMask,SubPath,-1)
SubtitleMask=Grayscale(SubtitleMask).ConvertToYV24

OutlineMask=AddBorders(SubtitleMask,1,0,0,0).Crop(0,0,-1,0)
SubtitleOutlineMask=Overlay(SubtitleOutlineMask,SubtitleOutlineColor,mask=OutlineMask,opacity=1)

OutlineMask=AddBorders(SubtitleMask,0,1,0,0).Crop(0,0,0,-1)
SubtitleOutlineMask=Overlay(SubtitleOutlineMask,SubtitleOutlineColor,mask=OutlineMask,opacity=1)

OutlineMask=Crop(SubtitleMask,1,0,0,0).AddBorders(0,0,1,0)
SubtitleOutlineMask=Overlay(SubtitleOutlineMask,SubtitleOutlineColor,mask=OutlineMask,opacity=1)

OutlineMask=Crop(SubtitleMask,0,1,0,0).AddBorders(0,0,0,1)
SubtitleOutlineMask=Overlay(SubtitleOutlineMask,SubtitleOutlineColor,mask=OutlineMask,opacity=1)

SubtitleOutlineMask=ConvertBits(SubtitleOutlineMask,10)
SubtitleOutlineMask=Levels(SubtitleOutlineMask,128, 1, 896, 0,1023, coring=false)

SubtitleMask=ConvertToYV12(SubtitleMask).ConvertBits(10)
SubtitleMask=Levels(SubtitleMask,128, 1, 896, 0,1023, coring=false)

ShadowMask=AddBorders(SubtitleMask,2,2,0,0).Crop(0,0,-2,-2)

video=Overlay(video,ShadowColor,mask=ShadowMask,opacity=0.5)
video=Overlay(video,OutlineColor,mask=SubtitleOutlineMask,opacity=1)
video=Overlay(video,SubtitleColor,mask=SubtitleMask,opacity=1)

return video
}

usage
Loadplugin("VSFilter.dll")
Import("TextSubHDR.avs")
video=TextSubHDR(video,"Passengers.srt",$C0C0C0)

Examples (.srt)
http://i.cubeupload.com/FpTOdM.png

(.ass)
http://i.cubeupload.com/FU7aRE.png