Anime encode has artifacts around subtitles

[danjx]

I'm encoding an Anime. The audio is foriegn and I'm using permanent subtitles saved in the AVI. The encode is good except the subs sometimes have artifacts around them. The subs have a black outline and yellow filler. If the background is white, the black outline is enhanced with extra pixels. If the background is flesh tone, I sometimes get a green outline.

Can anyone suggest a method to reduce this effect?

Here are my AVS settings:

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\ivtc.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\WINDOWS\SYSTEM\vobsub.dll")
mpeg2source("D:\VIDEO_TS\MyAnime.d2v")
ivtc(44,11,95)
FieldDeinterlace()
crop(3,0,715,477)
BicubicResize(512,384,0,0.5)
VobSub("D:\VIDEO_TS\IMMORAL_SISTERS_1\VIDEO_TS\VTS_01_0")
Import("D:\VIDEO_TS\VDub_AVS\vdub_filters.avs")
ConvertToRGB()
VD_Logo(alpha=254,transparent=true, tolerance=30, xr=255, xg=255, xb=255, filename="D:\VIDEO_TS\VDub_AVS\HZ-Logo_Black.bmp",start=2,duration=150)
ConvertToYUY2()

[manono]

Hi-

One obvious answer is to keep the subs separate. But if you're sure they're not on the DVD like that, and you still want the subs burned into the video, then if I were you, I'd make sure my crop values were all even numbers, maybe like this:

crop(4,0,714,476),

or whatever it takes to maintain AR. I know that GKnot gave you those values, but there's nothing to prevent you from using the Pixel Crop to adjust them a little bit. Odd crop values can result in artifacts of the type you describe.

[danjx]

Thanks for your suggestion, Manono, but I have not been able to remove the artifacts by changing the crop parameters to even numbers. I tried several combinations and used Gordian Knot to generate some others.

I cannot check the original subs on the DVD, but the subs show up fine in Subrip. Is there anything else I can check? Another filter I should be using?

If you can point me to a guide for snipping the VOB, I'll provide a short clip of the original.

[danjx]

I've tried several things, including the new decomb 4 dll, but nothing helps. I've attached a still image of the most serious problem. The AVS script looks like this:

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\WINDOWS\SYSTEM\vobsub.dll")
mpeg2source("D:\VIDEO_TS\SISTERS\Sisters.d2v")
Telecide(post=false)
Decimate(cycle=5)
crop(6,2,712,476)
BicubicResize(512,384,0,0.5)
VobSub("D:\VIDEO_TS\SISTERS\VTS_01_0")
TemporalSmoother(2)

[DJ Bobo]

All seems to be right in your AVS, except that the subtitle line must come at the very last!

Another thing, which was already suggested by manono, is to keep the subtitles separate. This is the *VERY BEST* thing you can do: perfectly readable subtitles, you can change their position as you want, and most important, the video quality will be much better.
I personally *ALWAYS* keep the subtitles switchable.

[manono]

Hi-

OK, the attachment is there now. Another thing, did you use either GMC or Q-PEL? If so, most likely the GMC caused it (or maybe, as D J Bobo implied, putting the TempSmooth after the subs). Try using only B-Frames and see if that fixes it.

Edit many hours later-I meant to say "...TempSmooth before the subs". Darn.

[danjx]

Moving the Subtitles to the very end did the trick. Thanks to all for their suggestions. Here is my current AVS script:

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\Avisynth_Spatial.dll")
LoadPlugin("C:\WINDOWS\SYSTEM\vobsub.dll")
mpeg2source("D:\VIDEO_TS\ANIME\DIVX5\Sisters.d2v")
Telecide(post=false)
Decimate(cycle=5)
crop(4,1,714,476)
BilinearResize(512,384)
TemporalSmoother(2,1)
Import("D:\VIDEO_TS\VDub_AVS\vdub_filters.avs")
ConvertToRGB()
VD_Logo(alpha=254,transparent=true, tolerance=30, xr=255, xg=255, xb=255, filename="D:\VIDEO_TS\VDub_AVS\MyLogo.bmp",start=2,duration=150)
VD_Logo(alpha=254,transparent=true, tolerance=30, xr=255, xg=255, xb=255, filename="D:\VIDEO_TS\VDub_AVS\MyLogo.bmp",start=21846,duration=150)
ConvertToYUY2()
VobSub("D:\VIDEO_TS\ANIME\DIVX5\VTS_01_0")

[danjx]

To DJ Bobo:
I agree with you that separate subtitles are better, and that is what I normally do. This was a special request to make the subs permanent.

[Scary]

AHHHHHHHH!!! he's ENCODING hentai!!!

sorry i couldn't offer any help.

[Angrychair]

The problem is the subtitle files are really just little bitmaps. You need to either find the setting to make vobsub filter them when it resizes them (because it IS resizing them from their native resolution they'd be at when the movie is played back at dvd native res 720x480) or do your frame resize after your subtitles are added so that they are properly resampled along with the rest of the frame. The artifacts you're seeing around them, I'm pretty sure, are just a result of bad nearest neighbor resampling.