Interlacing/blending & rainbows

[Codex0nz]

Well im going through and rebacking up all of my dvd's to x264 (Need more hdd space, and i enjoy learning these skills :) )

and i've come across what i think is field blending

I'm not positive though, but heres a sample:
http://www.megaupload.com/?d=SS61ENRK
It's 50 megs but has all of what im trying to fix in it, the blending, and the rainbowing -_-

I can upload a smaller sample if you want

here is the script i am using:
LoadPlugin("D:\Convert\Plugins\checkmate.dll")
LoadPlugin("D:\Convert\Plugins\TIVTCv105\TIVTC\TIVTC.dll")
LoadPlugin("D:\Convert\Plugins\masktools-v2.0a35\mt_masktools-26.dll")

Import("D:\Convert\Plugins\R_pack\mrestore.avs")

# Set DAR in encoder to 4 : 3. The following line is for automatic signalling
global MeGUI_darx = 4
global MeGUI_dary = 3
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
Checkmate()
ColorMatrix(hints=true)

#deinterlace
crop( 12, 2, -10, -2)

Bob() # fast & ugly. LeakKernelDeint, TDeint(mode=1), Yadif(mode=1) are more sophisticated
mrestore()
crop( 12, 2, -10, -2)

BicubicResize(704,576,0,0.5) # Bicubic (Neutral)
#denoise

problem is is that im not sure if i am doing it right and wanted to ask the pro's, also the script isn't removing any rainbowing at all

I looked around the forums but found mixed answers, any help is highly appreciated :)

Oh, and i am using x264 & megui

EDIT: Maybe i should be using http://forum.doom9.org/showthread.php?t=138305 ?

[Nightshiver]

1: You are using a PAL source for anime.
2: checkmate is for removing dot crawl

animeivtc(mode=2, aa=0, iregion="PAL")
derainbow(10)

And it's Trigun, which is a little notorious for having crappy transfers.

[Codex0nz]

1: You are using a PAL source for anime.
2: checkmate is for removing dot crawl

animeivtc(mode=2, aa=0, iregion="PAL")
derainbow(10)

And it's Trigun, which is a little notorious for having crappy transfers.

Thank you very much for your reply :)

I will have to try that out, so i guess it isnt necessary for me to keep checkmate there then is it?

Can you please explain what mode 2 does? Trying to learn the ropes so i can backup my collection again.

Does derainbow come as a part of animeivtcs plug-in package?

Yeah that may be why i am having trouble with it then

but then again madman anime are a bunch of evil bastards and field blend most things :(

[Nightshiver]

No, derainbow is a separate script:

function DeRainbow(clip org, int "thresh")
{
assert(org.isYV12(),"DeRainbow() requires YV12 input!")
thresh = default(thresh, 10)

org_u = utoy(org)
org_v = vtoy(org)

msharpen(org, threshold = thresh, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)
filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)

u_final = mt_merge(org_u, filtered_u, uv)
v_final = mt_merge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}

Copy that to notepad and save as "DeRainbow.avsi"

Mode 2 in animeivtc is for Double hard telecine // Field-blended norm conversion, as from the help file.

[Codex0nz]

Ok this weekend i will be trying out these great suggestions on my sample clip :)

Thank you very much for the help, i will post my results then!

Is there anything else i am missing? and i need to derainbow before touching the field blending right?

[Adub]

Also, make sure that you are using Colormatrix right after your DGDecode_mpeg2source call, as it is looking for hints, and I would guess that Checkmate will destroy these hints. Also be sure to use the "interlaced=true" flag.

Ex:
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
ColorMatrix(hints=true, interlaced=true)
Checkmate()

[Codex0nz]

will checkmate still be necessary do you think?

Thanks i didn't know about the interlaced tag, is there a specific tag to say the content is blended?

[martino]

It tends to remove rainbows, or at least it did for me. The best thing to do is try and see for yourself.

[Codex0nz]

i did run checkmate through but i dont think it worked / didnt make any noticeable difference

[thetoof]

So, don't use it!

About the tags, knowing how to use your eyes is the best option to know which type of processing your source needs regarding IVTC/Deinterlacing/Deblending.

Oh, and for your question, mode=2 bobs the clip, removes the blends and decimates to the desired framerate. It can all be done externally, so you could see AnimeIVTC as a "wrap-up" function. Actually, the first reason why I wrote it was to simplify the whole process into a single code line.

[Nightshiver]

As before, derainbow(10) works just fine.

[Codex0nz]

right i think it's right now...
plugins scripts etc

# Set DAR in encoder to 4 : 3. The following line is for automatic signalling
global MeGUI_darx = 4
global MeGUI_dary = 3
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
ColorMatrix(hints=true, interlaced=true)

animeivtc(mode=2, aa=0, iregion="PAL")
derainbow(10)

crop( 12, 2, -10, -2)

BicubicResize(704,576,0,0.5) # Bicubic (Neutral)
#denoise

i saw the aa option there i assume this is anti alias, should i use it? also should i derainbow before removing the blending?

Also if derainbowing desaturates the image a bit what command is it to increase the saturation again?

[Nightshiver]

tweak(sat=1.0)

increase the 1.0 to 1.1 or whatever you want to increase the saturation, but I don't think you'll need to. As for aa, take the aa out of the animeivtc script and that will force you to look at the different aa options. Pick whichever one you like best.

[Leak]

i did run checkmate through but i dont think it worked / didnt make any noticeable difference
Try putting it before any deinterlacing/IVTCing in your script - usually dot crawl as well as rainbowing are "added" after telecining, but if the video has been normconverted/blended with those artifacts in there the filters probably won't help anymore...

[Codex0nz]

:/ fail1:
http://img155.imageshack.us/img155/7636/fail1mt0.jpg
when using:
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ReduceFlickerSSE3.dll")
# Set DAR in encoder to 88 : 65. The following line is for automatic signalling
global MeGUI_darx = 88
global MeGUI_dary = 65
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
ColorMatrix(hints=true, interlaced=true)

animeivtc(mode=2, aa=0, iregion="PAL")

crop( 12, 10, -10, -2)

BicubicResize(704,576,0,0.5) # Bicubic (Neutral)

but if i remove the first line and let it attempt to load it automatically it also fails :/
so if i use this:
# Set DAR in encoder to 88 : 65. The following line is for automatic signalling
global MeGUI_darx = 88
global MeGUI_dary = 65
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
ColorMatrix(hints=true, interlaced=true)

animeivtc(mode=2, aa=0, iregion="PAL")

crop( 12, 10, -10, -2)

BicubicResize(704,576,0,0.5) # Bicubic (Neutral)

either way derainbow came back as invalid as well :(

http://img219.imageshack.us/img219/3649/fail2xb3.jpg



EDIT: Ok after manually reinstalling all of the plugins etc i got it sorted:
http://www.megaupload.com/?d=QK13WOEX

all i need to sort now is the derainbow

oh and it takes forever to pass now, took 40 minutes to do just that small sample o_O must be doing something wrong

using:
program --pass 2 --bitrate 700 --stats ".stats" --ref 4 --mixed-refs --no-fast-pskip --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock 1:1 --trellis 1 --partitions all --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads 2 --thread-input --progress --no-psnr --no-ssim --output "output" "input"

do you all think i am doing it right now?

[Nightshiver]

Nothings wrong, it's just that animeivtc is slow. Get used to it. As for the rainbows, the derainbow script I provided works. If you want the encoding to go somewhat faster, do a rendering pass in MeGUI. Also, why do you only have 3 bframes? In anime, it's always good to have a large number of bframes. If you are using MeGUI, I recommend using sharktooths anime profiles.

[thetoof]

As said in the guide, you can lower precision to lower quality and increase processing speed.

[Codex0nz]

i would wait days for the best quality :D

looks great

cant get derainbow to work, the script you provided comes up with "there is no function named msharpen"

Guess im missing a plugin, but cant find it, can you please link me to the plugin required to gain that function?

[Comatose]

You should look at other areas with issues too, and check a lot of them.
I had tried AnimeIVTC for an old problematic source before and I just happened to land on good frames when testing, but in motion it looked horrible... so I wasted 4 days just to change my settings.

[Codex0nz]

i found all of the plugins after googling for ages -_-

but look at the results :D although looks a little over sharp :S or is it just me?

avisynth script used:
# Set DAR in encoder to 88 : 65. The following line is for automatic signalling
global MeGUI_darx = 88
global MeGUI_dary = 65
DGDecode_mpeg2source("D:\Convert\Trigun\Copy of Copy of Clip\vts_04_1.d2v",info=3)
ColorMatrix(hints=true, interlaced=true)

animeivtc(mode=2, aa=0, iregion="PAL")
derainbow(10)

crop( 12, 10, -10, -2)

BicubicResize(704,576,0,0.5) # Bicubic (Neutral)

Megui x264 settings used:
program --pass 2 --bitrate 1000 --stats ".stats" --ref 8 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --direct auto --deblock 1:1 --subme 7 --trellis 2 --psy-rd 0.6:0 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --aq-mode 0 --progress --no-psnr --no-ssim --output "output" "input"
(I wonder if the bitrate is too high :S)

Produced clip:
http://www.megaupload.com/?d=3XD4RISJ

I am very VERY grateful for all of your guys help :)

[thetoof]

I had tried AnimeIVTC for an old problematic source before and I just happened to land on good frames when testing, but in motion it looked horrible... so I wasted 4 days just to change my settings.
I'm wondering what looked so horrible in motion... aliasing? Also, which version did you use, which settings, which type of source, etc etc...

Sorry for the slight OT (well, since i't about interlacing/blending, I guess it's still kinda related), and good to know everything turned out fine for you, Codex0nz!

[Comatose]

Well, it was an old source and lines flickered badly. NNEDI coupled with that flickering resulted in jumping lines of doom.
It's really just bad luck. It was rather obvious even when there wasn't any motion, but when previewing the script I somehow landed ONLY on good frames.

I ended up using Vinverse to "build" a comb mask (using Subtract) and only using nnedi on those areas, since vinverse never touched the flickering lines.

It was a pretty weird source... it had chroma field blending but the luma was fine. I just used MergeChroma(nnedi()) which probably isn't the way I should've solved it, but it worked, and after wasting 4 days due to bad luck I just wanted to get it over with already :P

[Codex0nz]

I'm wondering what looked so horrible in motion... aliasing? Also, which version did you use, which settings, which type of source, etc etc...

Sorry for the slight OT (well, since i't about interlacing/blending, I guess it's still kinda related), and good to know everything turned out fine for you, Codex0nz!

yeah turned out well, im glad that this topic can help other people in my situation also

i was hoping that one of you pro people could review my converted sample and let me know their thoughts on it please?

[thetoof]

@Comatose
Chroma_rep, written by MOmonster, could be the answer of you chroma field blending... even though it seems you won't touch that source anymore :p

@Codex0nz
will have a look at it in a few hours

[Codex0nz]

@Comatose
Chroma_rep, written by MOmonster, could be the answer of you chroma field blending... even though it seems you won't touch that source anymore :p

@Codex0nz
will have a look at it in a few hours

Lucky, with the way the exchange rates are going i have no choice but to source it within my country :/


Thanks :D you're a legend.

EDIT: After reviewing the clip a couple times there are still some rainbows remaining :(
http://img224.imageshack.us/img224/3003/81092475oz5.jpg
in the eyes -_-

[thetoof]

Then try higher settings... or use

block=32 #size of the analysis block, try 32 or 16 for optimal results
overlap=block/2 #Sets overlap to half-block size (optimal quality)
plane=3 #Will filter only the chroma planes
bt=3 #amount of frames used for calculation. You can try 5 (more aggressive, slower, but I can sometimes lead to weird things, such parts of frames appearing in others)
High= #tweak this value. It'll filter high frequencies (small details)
Medium= #same, on lower frequencies
Mid-low= #even lower (bigger "sploches")
low= #lowest (biggest)
#Usually, I set 50 for all the values except "High" until I find the appropriate value that derainbows perfectly without screwing up the details, then use that "high" value (with 50 for mid-low and low) to tweak medium.... and so on)
Sat= #Tweak at the end to add up some saturation to repair washed out colors
coring= #true for TV-scale and false for PC-scale

FFT3DFilter(bw=block, bh=block,oh=overlap, ow=overlap,plane=plane,bt=bt,sigma=High, sigma2=medium, sigma3=mid-low, sigma4=low)
Tweak(sat=sat,coring=coring)
You should also look at some dehaloing filters for your source.

[Codex0nz]

I tried a higher setting with the original derainbow script and it didnt appear to have any effect :S

i raised it to derainbow(20)

How high should i raise it? If you think i should try that script you just provided i will, but im not quite sure how to use that -_- sorry for being n00b at this

[thetoof]

I tried to make my comments n00b friendly, but heh :p

How to use the script:
Copy and paste after animeivtc (and remove derainbow)
Fill the blanks with the comments' help. If something is not clear, please post!
High=
Medium=
Mid-low=
low=
Sat=
coring=

Maybe something like "contra-saturation" could be added by calculating the difference between "original" and "chroma denoised" and then applying it as saturation to prevent oversaturation by the user, but heh, don't have the time to look into that now :p