A new deinterlacing technique for anime?

[bilu]

#OLD THREAD# MOVED HERE

@mf

Would like your opinion on this. Looked better to me than SSTMC IMHO on my Neo Genesis Evangelion clip.

And it's YV12 already :D

#MPEG-2 sources only
function FOrder(clip c) {
fo= GetParity(c)? 1:0
return fo
}

function TMC(clip input, int parity) {
input.TomsMoComp(parity, -1, 0)
TomsMoComp(parity, 5, 0)
SeparateFields()
SelectOdd()
}

#Usage: TMC(1) for TFF, TMC(0) for BFF,
#TMC(FOrder()) for autodetection on MPEG-2 sources

TMC(FOrder())


Bilu

[mf]

Actually, on my clip, your function looks worse than sstmc :confused:.

Original (http://mf.creations.nl/avs/t6-orig.png)
TomsMoComp (http://mf.creations.nl/avs/t6-tmc.png)
SSTMC (http://mf.creations.nl/avs/t6-sstmc.png)
Bilu's TMC (http://mf.creations.nl/avs/t6-bilutmc.png)

Edit: remote linking seems to be disabled without me knowing. I'll investigate, but for now, open in a new window (about:blank) and then paste the url in there.
Edit: my lovely admin enabled remote linking again :D.

[bilu]

@mf

Try this one. It looks very good to me, although this sounds to me like a glitch in the Matrix: http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?t=130

function BiluTMC(clip input, int par) {
input.TomsMoComp(par, -1, 0)
#TomsMoComp(par, 5, 0)
SeparateFields()
(par==1)? SelectOdd():SelectEven()
}

As you see the third line is remarked. It still does wonders in my Neo Genesis Evangelion clip this way, without ANY post-processing! :eek:

I think the secret here is that TomsMoComp(x,-1,0) treats the original frames as progressive and creates a new field. This new field is created using the parity field only and interpolates with the static areas of the other original field. It's the only logical explanation I can find for getting such great results in a strange clip such as this. I'd hope to have trbarry's opinion.

Example: TFF clip

A -> A*
B A
A+
B

where A+ = A interpolated with static areas of B.

I sure would like to know why the hell does the clip looks good :sly:

EDIT: PM'ed trbarry. I sure am curious for the answer...

EDIT2: Doesn't work with field dropped sources, tried it on the same clip and it started to look unprocessed after the field drop.

Bilu

[mf]

Still just as bad. What kind of improvements are you seeing? I just see the output getting very crude (as if it's been downsized and upsized again). And I don't modificate your stuff in any way, it's just been pasted in my AVS. Are your eyes bad in not noticing this stuff or am I doing anything wrong?

[bilu]

@mf

Try this one please. You need Decomb 5.1.0 to use IsCombed().

function TMC(clip input) {
input.TomsMoComp(-1, -1, 0)
SeparateFields()
clip1=last.SelectOdd()
clip2=last.SelectEven()
ConditionalFilter(clip1, clip1, clip2, "IsCombed()", "=", "false")
}

Usage: TMC()
No need to define field order, and can handle field drops quite well.

I can provide you my sample clip to show you what improvement I'm seeing. But try this new script on your clip and tell me the results.


Bilu

[mf]

Looks to me like a swapped field order.

Original (http://mf.creations.nl/avs/t62-orig.png)
SSTMC (http://mf.creations.nl/avs/t62-sstmc.png)
BiluTMC (from previous post) (http://mf.creations.nl/avs/t62-bilutmc.png)
BiluTMC2 (from last post) (http://mf.creations.nl/avs/t62-bilutmc2.png)

[bilu]

Try this one now, with field-order guidance but still able to handle dropped fields:

function TMC(clip input,int par) {
input.TomsMoComp(par, -1, 0)
SeparateFields()
clip1=last.SelectOdd()
clip2=last.SelectEven()
(par==1)? ConditionalFilter(clip1, clip1, clip2, "IsCombed()", "=", "false") : ConditionalFilter(clip2, clip2, clip1, "IsCombed()", "=", "false")
}


Bilu

[mf]

Sadly I'll have to report that it still looks like turd.

http://mf.creations.nl/avs/t62-bilutmc3.png

Could I please know what you're trying to achieve before I do more testing?

[bilu]

function BiluTMC3(clip input,int par) {
input.TomsMoComp(par, -1, 0).SeparateFields()
clip1=SelectOdd()
clip2=SelectEven()
ConditionalFilter(clip1, clip1, clip2, "IsCombed()", "=", "false")
}

Could I please know what you're trying to achieve before I do more testing?

A deinterlacer that works over difficult anime using TomsMoComp(x,-1,0) processing. Got inspired by your ideas of supersampling (first time using them myself),remembering the neuron2 forum thread I mentioned and this:

For Avisynth only, a value of -1 is supported. In this case the TomsMoComp filter will not deinterlace but instead assume you already have progressive frames but want to double the vertical size. I found by accident that this could give slightly better apparent detail than regular scaling algorithms and is useful for low bit rate captures that are hard to IVTC/deinterlace or where you have just kept the even fields for some other reason.

Would like you to try it on a difficult anime yourself, one of those that deserve manual matching for being full of garbage.

If this process grabs moving info from one field and static info from both it can do wonders. I'm looking for better movement, not necessary better detail. Still my tests have shown me better results than FieldDeinterlace.

Bilu

#END OF THE OLD THREAD#

[bilu]

function FastTMC(clip input,int par) {
input.TomsMoComp(par, -1, 0).SeparateFields()
clip1=SelectOdd()
clip2=SelectEven()
ConditionalFilter(clip1, clip1, clip2, "IsCombed()", "=", "false")
}

Usage: FastTMC(0) for BFF sources, FastTMC(1) for TFF sources

Requirements: TomsMoComp, Decomb 5.1.0

TomsMoComp(x,-1,0) works like this:
For Avisynth only, a value of -1 is supported. In this case the TomsMoComp filter will not deinterlace but instead assume you already have progressive frames but want to double the vertical size. I found by accident that this could give slightly better apparent detail than regular scaling algorithms and is useful for low bit rate captures that are hard to IVTC/deinterlace or where you have just kept the even fields for some other reason.

TMC uses motion-compensation. The way this script works seems like using moving information from one field only and static information from both fields. It's the only explanation I have for working so good in my Neo Genesis Evangelion sample, which has a weird pattern like this:

A B C
A AC C

B= orphan field
AC= blend between A and C

The conditional filtering part is to handle dropped fields.Since TMC(x,-1,0) seems to create a new field when doubling vertical resolution using interpolation and motion-compensated techniques , and we only want this field, not the original one (which seems to be the original frame untouched).

Seems to work much better than using FieldDeinterlace or discarding a field, keeping much better resolution on static areas, while being cleaner and faster than KernelDeint.

Still the dropped-field handling could need improvement, it looks like bobbing on static parts.


I'd like to see some feedback of using this on real hard anime, the kind that you need YATTA over it :devil:


Best regards,
Bilu

[mf]

Buy Eva Revival :D.

[bilu]

Originally posted by mf
Buy Eva Revival :D. Cheater :D

[mf]

Or actually, since you bought the rights to Eva by getting the old DVDs, you should be able to download revival and get away with it since it's just a different version of something you already own.

[bilu]

I just own a 53 MB VOB sample I've been using for tests.

I borrowed it from Hiro2k when he posted the sample here:
http://forum.doom9.org/showthread.php?s=&threadid=56395

And have been testing whenever a new approach comes up, just for the challenge. :cool:

But what I'd really like is feedback about using this script over nasty anime ;)

Bilu

[SoonUDie]

Bilu, do you know any other place to get that clip? If not, do you think you could host it for a while so I could grab it? I'm going to be working on a new IVTC routine and want some good "challenging" sources. I already have at least one anime which likes to change patterns, and I'm sure I have some telecined movies around here somewhere... but it sounds like EVA is really a bastard, and I'd love to get my hands on a clip :devil:

Oh, and good luck getting your uber-deinterlacing working :D

[bilu]

function TMC(clip input,int par) {
clip1=input.TomsMoComp(par, -1, 0).SeparateFields().SelectOdd()
clip2=input.KernelDeint(par,sharp=false)
ConditionalFilter(clip1, clip1, clip2, "IsCombed()", "=", "false")
}

This is it. It benefits from TomsMoComp(x,-1,0) extra lines and works faster and cleaner than KernelDeint(sharp=false). Still I'm using KD(sharp=false) as backup whenever there are dropped fields - it's still the best preserving static areas. My former script used SelectEven() to grab the extra lines whenever SelectOdd() got combed but it ended up bobbing, with KD the transition is not noticeable.

Static areas are just a tiny bit worse than KD, but motion looks better.

Main advantage is cleaner and faster processing than KD.


Bilu

[acrespo]

If I use TMC with Decimate(5) after TMC(), Avisynth 2.53 crash.

[bilu]

Originally posted by acrespo
If I use TMC with Decimate(5) after TMC(), Avisynth 2.53 crash.

1) Works OK for me, same Avisynth version, build Nov 11 :confused:
Are you using Decomb 5.1.0? You need it for the IsCombed() function.

2) You shouldn't use TMC with Decimate. Use Uncomb() or Telecide() instead. Field matching is different from deinterlacing.
Field matching (Telecined TFF source, D2V flags 2301):

A B B C D -> A B C D D
A B C D D A B C D D

then Decimate(5) will remove the 5th frame because it is a duplicate.

Deinterlacing(same source):

A B B C D -> A B B* C* D
A B C D D A B C* D* D

where B*,C* and D* are post-processed fields. Post-processing is used to hide combing, using information from both fields of the current frame. No field matching is ever made, so no duplicate frame is ever generated. Some may look like dups, but that's pure coincidence.


Best regards,
Bilu

[bilu]

Example:

1st pass using TMC -> 69032 KB
1st pass using KD(sharp=false) -> 80218 KB

Faster, cleaner, gains compressibility, doesn't look blurry. :cool:

If trbarry would made a deinterlacer based on this it would really rock IMHO.


Bilu

[acrespo]

Originally posted by bilu
1) Works OK for me, same Avisynth version, build Nov 11 :confused:
Are you using Decomb 5.1.0? You need it for the IsCombed() function.

2) You shouldn't use TMC with Decimate. Use Uncomb() or Telecide() instead. Field matching is different from deinterlacing.
Field matching (Telecined TFF source, D2V flags 2301):

A B B C D -> A B C D D
A B C D D A B C D D

then Decimate(5) will remove the 5th frame because it is a duplicate.

Deinterlacing(same source):

A B B C D -> A B B* C* D
A B C D D A B C* D* D

where B*,C* and D* are post-processed fields. Post-processing is used to hide combing, using information from both fields of the current frame. No field matching is ever made, so no duplicate frame is ever generated. Some may look like dups, but that's pure coincidence.


Best regards,
Bilu

1-) Yes, I have Decomb 5.1.0 and all function correctly without Decimate(5). If I put Crop filter between TMC(1) and Decimate(5) all function properly again. My source is Xvid decodec with Divx 5.1.1 decoder.

2-) Hummm I understood the problem, and I have other question: It's possible to use TMC with PAL (25fps) source AVI (Xvid) file? If it's possible I will change my VCR to PAL-M mode instead NTSC to capture.

[bilu]

1) TomsMoComp's readme:

Known issues and limitations (for both DScaler and Avisynth):

1) Assumes YUV (YUY2 or YV12) Frame Based input. Use an Avisynth function to convert first if
needed. YV12 is supported by the 2.5 Avisynth Alpha version only.

2) Currently still requires the pixel width to be a multiple of 4. This probably
shouldn't be required but I pilfered the code from some of my other filters.
Sorry, I'll fix it later.

3) So far it has only been tested on SSEMMX machines. (now others)

Decomb's reference manual:

Telecide requires that the input width be a multiple of 2. Deviation from this will cause Telecide to throw an exception.

Maybe Decomb's requirement also applies to Decimate, I don't know.

2)You can use TMC() in both PAL and NTSC interlaced streams, as with KernelDeint, FieldDeinterlacer, TomsMoComp and others. Just don't use it with Telecined streams (also known as FILM).

Best regards,
Bilu

[acrespo]

Strange. My video has the requirements. Resolution of 576X480, Xvid, 29.97 fps. I didn't know that this function not function with telecined material. I did all tests in telecined video :P
But why avisynth crashes with decimate in this case is strange:

OpenDMLSource("d:\capture.avi")
TMC(1)
Decimate(5)
Crop(-2,16,-2,16)

And not crash in this case:

OpenDMLSource("d:\capture.avi")
TMC(1)
Crop(-2,16,-2,16)
Decimate(5)


I will test on PAL video tonight and I will post the results here.

[bilu]

function TMC(clip input) {
clip1=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectOdd()
clip2=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectEven()
clip3=Subtract(input,clip1).Greyscale()
ConditionalFilter(clip3, clip2, clip1, "AverageLuma()", "=", "126.000000")
}

Usage: TMC()

Grabs only the extra fields generated by TomsMoComp(x,-1,0).
Detects field order. Cleaner than KernelDeint, sharper than FieldDeinterlace, and very fast.

My favorite deinterlacer from now on :D


Bilu

[bel]

Hi Bilu!

Could you please give me (a dummy in avs scrip) a full example for using your TMC function so I just replace my d2v file name and do the encode?

Best Regard.

[bilu]

New version:

function TMC(clip input) {
input=input.AssumeFieldBased()
input=(input.IsYV12)? input : input.ConvertToYV12(interlaced=true)
clip1=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectOdd()
clip2=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectEven()
clip3=Subtract(input,clip1).Greyscale()
ConditionalFilter(clip3, clip2, clip1, "AverageLuma()", "=", "126.000000")
AssumeFrameBased()
}

Works better and does the YV12 conversion if needed.


@bel

Loadplugin("D:\avs25\MPEG2Dec3.dll")
Loadplugin("D:\avs25\TomsMoComp.dll")

function TMC(clip input) {
input=input.AssumeFieldBased()
input=(input.IsYV12)? input : input.ConvertToYV12(interlaced=true)
clip1=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectOdd()
clip2=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectEven()
clip3=Subtract(input,clip1).Greyscale()
ConditionalFilter(clip3, clip2, clip1, "AverageLuma()", "=", "126.000000")
AssumeFrameBased()
}

mpeg2source("D:\weird_clips\eva.d2v",idct=2,cpu=0)
TMC()


Best regards,
Bilu

[bel]

Thanks a lot Bilu!
I will try it tonight. And let you know the result.

[bilu]

I've been making this tests on a PIII-500 laptop.
It achieved 17fps previewing the script on a media player.
This new version achieves 21 fps, because it avoids the Subtract function. On some tests using this method I've found interlacing artifacts remaining, but they are very rare. And the result is worth it. Also looks a bit cleaned, like if I used Undot() - not blurred.

function TMC(clip input) {
input=input.AssumeFieldBased()
global input=(input.IsYV12)? input : input.ConvertToYV12(interlaced=true)
global clip1=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectOdd()
clip2=input.TomsMoComp(-1, -1, 0).SeparateFields().SelectEven()
ConditionalFilter(input, clip2, clip1, "LumaDifference(input, clip1) ", "=", "0.000000")
AssumeFrameBased()
}


Bilu

[Le Furet]

Wouldn't it be faster to put the TomsMoComp and the SeparateFields in the line "input = ..." ?

[bilu]

A faster version: 22.5 fps previewing on my PIII-500 laptop.
And more accurate too, input==clip1 may compare more than just luma.

function TMC(clip input) {
(input.IsYV12)? input : input.ConvertToYV12(interlaced=true)
AssumeFieldBased().TomsMoComp(-1, -1, 0).SeparateFields()
clip1=SelectOdd()
clip2=SelectEven()
(input==clip1)? clip2 : clip1
AssumeFrameBased()
}

Usage: TMC()


KernelDeint(sharp=false,twoway=false) achieves 18.5 fps in the same machine.


Bilu

[bilu]

Originally posted by Le Furet
Wouldn't it be faster to put the TomsMoComp and the SeparateFields in the line "input = ..." ?

Would work only if I removed "input=" from that line. Later in the script I need to compare the original clip (input) with a processed one (clip1) to check if I really got the extra fields with SelectOdd() or not. If clip looks like the original then it was not touched and we'll choose the other one.


Bilu

[bel]

Hi Bilu,
The script you wrote for me did not work for my clip.
Progressive frames are unchanged, interlaced frames get worse.
But I have learnt the concept of writing a function and how to use it in an avs script.
Thanks a lot Bilu.

[Wilbert]

@Bilu,

I don't understand anything of your script. For a start, what is the line

(input==clip1)? clip2 : clip1

doing? What happens if the "input" is YUY2?

[scharfis_brain]

@bel: try to use assumetff or assumebff befor calling this function.

[bilu]

Originally posted by Wilbert
@Bilu,

I don't understand anything of your script. For a start, what is the line

(input==clip1)? clip2 : clip1

doing? What happens if the "input" is YUY2? You're right. Should be:

function TMC(clip input) {
input=(input.IsYV12)? input : input.ConvertToYV12(interlaced=true)
input.AssumeFieldBased().TomsMoComp(-1, -1, 0).SeparateFields()
clip1=SelectOdd()
clip2=SelectEven()
(input==clip1)? clip2 : clip1
AssumeFrameBased()
}

Usage: TMC()


Thanks for noticing.

Bilu

[bilu]

Originally posted by scharfis_brain
@bel: try to use assumetff or assumebff befor calling this function.

It shouldn't be needed, but may help though.
TomsMoComp(-1,x,x) autodetects field order and the comparison function ensures that you get the processed field, not the original combed one. :confused:

EDIT: Tried it with AssumeTFF() and AssumeBFF() over an AVI source and an MPEG-2 source. Both ran OK.


Bilu

[bilu]

The little feedback I've been having tells me that it:

- is something between KernelDeint and FieldDeinterlace;
- is better on anime than real life moves;
- is cleaner than KD but may add ghosting over high-action scenes on real life movies;
- may leave some rare but noticeable combing artifacts behind on some real-life scenes;

Probably enough to fill the gap here:
http://forum.doom9.org/showthread.php?s=&postid=397283

as a better alternative to FieldDeinterlace as deinterlacer for anime.


I would appreciate more feedback on this :)


Best regards,
Bilu

[bilu]

Much better one - replace the script with this:

TomsMoComp(-1,-1,0).VerticalReduceBy2()

What's different:
- Slower: 22.5 fps -> 18.5 fps in AVS preview on my laptop;
- IMHO this version is as sharp as KernelDeint;
- Even better compressibility:

80218 KB KernelDeint(sharp=false)
69032 KB TMC()
60923 KB this version


Bilu

[sh0dan]

Originally posted by bilu
(input==clip1)? clip2 : clip1


I see no way at all that this can work.

1) It is not a conditional filter. It will be evaluated at script initialization, and remain the same for the rest of the time.

2) clip1==clip2 does NOT compare frames, but compares if both variables refer to the same clip. Eg.

v1 = colorbars(256,256)
v2 = v1
variable = (v1 == v2)

will evaluate to true.

v1 = colorbars(256,256)
v2 = colorbars(256,256)
variable = (v1 == v2)

will evaluate to false.

[bilu]

Well, but it did work, and was the only way to get the extra fields when the stream had dropped fields. :confused:

Still, TomsMoComp(-1,-1,0).VerticalReduceBy2() is much better, so I've quit thatr method.

Bilu