New filter: SSTomsMoComp

[mf]

Oops I scripted again :D.

I don't know if anyone ever did this before, but as I was playing around with some deinterlacing stuff I just found that this looked pretty nice for a clueless (meaning no combing check) deinterlacer. So I made a function out of it. You can also use it as an alternative to TomsBob. It uses TMC's internal interpolation to give another instance of TMC more stuff to work on (and more room for error - causing less stray pixels). You can also let it keep the interpolated image, if you want to do other things within some kind of supersampling.

http://mf.creations.nl/avs/functions/SSTomsMoComp-v0.1.avs

Usage is described in the AVS itself. If anyone likes this, I'll make an YV12 version, it currently only works in YUY2.
Have fun! :)

[mf]

Ok, so that's the last time I'm making a sharefunction that I have no interest in myself :rolleyes:. So much for the YV12 version or the interlacing mask version ;). Nobody seems interested.

[FredThompson]

17 hours and you're complaining about lack of response?!?! C'mon, I frequently see things that are new/updated which don't make the "headline" news on the doom9 homepage for days.

[mf]

Originally posted by FredThompson
17 hours and you're complaining about lack of response?!?! C'mon, I frequently see things that are new/updated which don't make the "headline" news on the doom9 homepage for days.
I've released things more than once. On the forums things come and go alot quicker. The thread just gets moved down and down until nobody even gets to read it anymore. And my stuff doesn't need to make headline news, I don't think any AVISynth plugins ever did. Some interest over the 127 pageviews can't hurt. I won't say "what, is nobody replying?", but I just say "well, I know this is something I won't have to do again". I certainly don't release something for myself. It's for people that are interested. If they're not, their loss.

[Didée]

Well, I for one came here approx. 2 hours after you had posted the script, downloaded, looked at it, ...

... and believe it or not: I did not have the time to try it out yet.
But even if it'll take me some more days to start playing with it (and this is *very* likely to happen), that does not mean it's forgotten ...
originally posted by Shayne

Peace.
- Didée

[mf]

Originally posted by Didée
But even if it'll take me some more days to start playing with it (and this is *very* likely to happen), that does not mean it's forgotten ...

- Didée
And that's what I make it for :).

[bilu]

Originally posted by mf
Ok, so that's the last time I'm making a sharefunction that I have no interest in myself :rolleyes:. So much for the YV12 version or the interlacing mask version ;). Nobody seems interested. Don't you ever say that again. :angry:
Tried it over two pure FILM (hoping to get some blended artifacts as supposed to) and it didn't. It looked P-E-R-F-E-C-T, with some stuttering because it was FILM (perfectly natural but a good example of difficult deinterlacing).

Also tried on my Neo Genesis Evangelion clip. One word: WOW! :cool:
I hope you do an YV12 version and *if possible* reduce action on static areas like KernelDeint does - that's a tough one, I know. :rolleyes:

Muchas gracias for your functions and keep the good work :)


Bilu

[mf]

Originally posted by bilu
Don't you ever say that again. :angry:
Tried it over two pure FILM (hoping to get some blended artifacts as supposed to) and it didn't. It looked P-E-R-F-E-C-T, with some stuttering because it was FILM (perfectly natural but a good example of difficult deinterlacing).

Also tried on my Neo Genesis Evangelion clip. One word: WOW! :cool:
I hope you do an YV12 version and *if possible* reduce action on static areas like KernelDeint does - that's a tough one, I know. :rolleyes:

Muchas gracias for your functions and keep the good work :)


Bilu
Thanks, and will do :D. I got a masked version but it's T-U-R-D atm :p.

[Chainmax]

This sounds pretty interesting, do you think you could whip up an alternative based off KernelDeint?

[bilu]

@mf

Could you compare your method against this?

TomsMoComp(-1,-1,0).VerticalReduceBy2()


Bilu

[mf]

Originally posted by bilu
@mf

Could you compare your method against this?

TomsMoComp(-1,-1,0).VerticalReduceBy2()


Bilu
Nothing pdersonal, but it still isn't any better :confused:

[bilu]

Can you post screenshots?

I had much better results in my Neo Genesis clip with it than with SSTMC :confused:

Argh.


Bilu

[mf]

Originally posted by bilu
Can you post screenshots?

I had much better results in my Neo Genesis clip with it than with SSTMC :confused:

Argh.


Bilu
That's cause your NGE clip is screwed up to the point that SeparateFields().SelectOdd().PointResize(input.width, input.height) would still look good!
http://mf.creations.nl/avs/t62-bilutmc4.png

[bilu]

Originally posted by mf
That's cause your NGE clip is screwed up to the point that SeparateFields().SelectOdd().PointResize(input.width, input.height) would still look good! Although I don't quite agree (and i'm writing from another PC, so I can't prove it now), I do agree that I need to test other sources better.

This image has a lot of diagonal edges and is good to show the limitations of line interpolation. I'll try to get an example like that on my Read Or Die clip (not crappy like NGE).


Bilu

[FredThompson]

I've found the best filter is to view on a 5 inch B&W TV...

[bilu]

I was wrong about the line interpolation issue. My Read or Die clip looks clean with Force Film, gets clean after TomsMoComp(-1,-1,0).VerticalReduceBy2(). As good as the original.

But I was able to recreate the problem by dropping a field on the green Warner logo in the Matrix intro. My approached sucked and yours worked very good. Exactly the opposite as on my NGE clip.

I was able to get a result as good as yours on the Matrix logo with the following YV12-compatible script:

SeparateFields()
TomsMoComp(0,-1,0)
Weave()
TomsMoComp(1,5,0)
VerticalReduceBy2()

Replace VerticalReduceby2() with LanczosResize() if you like it better.
I'm still trying to find a method that works excellent in both cases.


Bilu

[mf]

Originally posted by bilu
I was able to get a result as good as yours on the Matrix logo with the following YV12-compatible script:

SeparateFields()
TomsMoComp(0,-1,0)
Weave()
TomsMoComp(1,5,0)
VerticalReduceBy2()
That is about the same as SSTMC does *yawn*.

[bilu]

Originally posted by mf
That is about the same as SSTMC does *yawn*.

Yes, I did it on purpose, just to make an YV12 version of it ;)
IMHO the cropping process was useless.


Bilu

[mf]

Originally posted by bilu
Yes, I did it on purpose, just to make an YV12 version of it ;)
IMHO the cropping process was useless.
Bilu
That was to counteract the bobbing in bob mode but I think I screwed up somewhere along the way.

[bilu]

I've found more than one clip that works better with one method while others work better with the other.

My Read or Die clip with Force Film is completely clean and uncombed, and it looks better with my method in some parts.

On the "Matrix WB green logo with a dropped field" sample your method works way better. On my NGE clip my method works way better. On my ROD sample both look good but mine is better in some parts.

I have tried to make some scripts that get best of both worlds, but without success until now. :sly:


Bilu

[scharfis_brain]

After some testing, I found out, that you have to shift the odd frame up by 1/2 pixel to get a clean output:

function TMCkernelbob(clip input, int th, bool TFF)
{
function kernelbob(clip a, int th, int ord)
{
f=a.kerneldeint(order=ord, sharp=true, twoway=false, threshold=th)
e=a.separatefields.trim(1,0).weave.kerneldeint(order=1-ord, sharp=true, twoway=false, threshold=th)
interleave(f,e)
}

function shiftoddhalf (clip clip)
{
i=clip.converttorgb32
j=interleave(i.selecteven,i.selectodd.generalconvolution(0," 0 1 0 0 1 0 0 0 0").sharpen(0,0.5))
j.converttoyuy2
}

a = (TFF == true) ? input.assumetff.addborders(0,4,0,4) : input.assumebff.addborders(0,4,0,4)
a.converttoyuy2(interlaced=true)
separatefields.tomsmocomp(0,-1,0).shiftoddhalf.weave
kernelbob(th,0)
crop(0,0,0,-1).addborders(0,1,0,0).lanczosresize(a.width,a.height).crop(0,4,0,-4)
}

Nice for Natural Video (DV-Interlaced), because most of the deinterlacing-stairstepping is gone and remaining combing is impossible with this method.

The downside are Computer-Implanted horizontal Lines, which get a little bit blurry and crappy.
Usage:

TMCkernelbob(15,TFF=true)

[scharfis_brain]

or how about this one?

function TMCkernelbob(clip input, int "th", bool "TFF")

{
function kernelbob(clip a, int th, int ord)
{
f=a.kerneldeint(order=ord, sharp=false, twoway=false, threshold=th)
e=a.separatefields.trim(1,0).weave.kerneldeint(order=1-ord, sharp=false, twoway=false, threshold=th)
interleave(f,e)
}


th = default(th,7)
input = (TFF == true) ? input.assumetff.addborders(0,4,0,4) : input.assumebff.addborders(0,4,0,4)
input = input.converttoyuy2(interlaced=true)

x=input.separatefields.selecteven.tomsmocomp(0,-1,0).separatefields.selectodd
y=input.separatefields.selectodd.tomsmocomp(0,-1,0).separatefields.selecteven
a=interleave(x,y).weave.swapfields.kernelbob(th,0).crop(0,0,0,-1).addborders(0,1,0,0)
#select TMC's extra-lines and deinterlace them

b=input.kernelbob(th,0)
#take the input clip and deinterlace it

layer(a,b,"fast").crop(0,4,0,-4)
#mix both to get a higher resolution
}
It is a lot faster, than my version above...

[mf]

Actually I thought bob only needed to compensate 1 pixel? Seperated: half a pixel; upsized to normal vertical res: 1 pixel?
Otherwise it would imply that the difference on a seperated clip is a quarter of a pixel :confused:

[Mug Funky]

i think the stutter is a problem of tomsmocomp

just regular bobbing shimmer, really.

i'm working on an interlace masking script that can be used with tomsmocomp. the new Overlay() internal filter is a lifesaver, but there's still problems with beating a decent mask out of avisynth. not enough math functions for overlaying video.

[scharfis_brain]

I had to introduce the shift by a half pixel, because the bobbed output was slightly jumping up and down... (inspected with subtract...)

after shifting the odd frame by 1/2 Pixel, the image was steady...
Don't know why....

but with the 2nd one, I don't need a 1/2-Pixel shift....

could you try both functions and then tell my then, what you do think about it?

[mf]

Originally posted by Mug Funky
i'm working on an interlace masking script that can be used with tomsmocomp. the new Overlay() internal filter is a lifesaver, but there's still problems with beating a decent mask out of avisynth. not enough math functions for overlaying video.
Actually, I already made that but never published it, using MaskTools142.

inputyv12 = last
inputyv12.ConvertToYUY2(interlaced=true)
input = last

input.KernelDeint(order=1, map=true)

#Telecide(order=1)

Subtract(input, last)
ConvertToYV12()
subtracted = last
#inputyv12.Binarize(threshold=0, Y=128, U=128, V=128)
BlankClip(subtracted, color=$808080)
grey = last
OverlayMask(subtracted, grey).GreyScale()
#ConvertToRGB32()
#ResetMask()
#ColorKeyMask($808080, 0)
#return subtracted
#Levels(0, 1.0, 255, 255, 255)
Invert()
lumamask = last

lumamask.PointResize(lumamask.width/2, lumamask.height/2)
chromamask = last

#YToUV(chromamask, chromamask, lumamask)
#mask = last

#maskedmap = last

#white = BlankClip(maskedmap, color=$FFFFFF)
#black = BlankClip(maskedmap, color=$000000)

#Layer(black, maskedmap)
#ConvertToYV12()
#Invert()
#mask = last

input.SSTomsMoComp("bff")
ConvertToYV12()
sstmc = last

MaskedMerge(inputyv12, sstmc, lumamask, U=1, V=1)
y = last

MaskedMerge(inputyv12.UToY(), sstmc.UToY(), chromamask, U=1, V=1)
u = last
MaskedMerge(inputyv12.VToY(), sstmc.VToY(), chromamask, U=1, V=1)
v = last

YToUV(u, v, y)

But because of my failed bob compensation (in hindsight it was a pretty lame attempt), it looked crap, so I never finished/published it. That's also why this is all messy with commented lines and stuff :D.

[Mug Funky]

i've been using this one...
it leaves behind a fair amount of combing though. and some fast moving shapes can throw it out of whack. i had some success with it in R4 FLCL ep3

function FunkyBob(clip c) {

c3=c.horizontalreduceby2().converttoyuy2(interlaced=true)

c2=subtract(c3.bob().selecteven(), c3.bob().selectodd()).greyscale()
c2=subtract(c2,c2.deleteframe(0))

c2=layer(c2,c2.levels(0,1,255,255,0),op="lighten",level=255)
c2=c2.undot().levels(129,1,143,0,255).pointresize(c.width,c.height)

odd = overlay(c, c.tomsmocomp(0,5,0), mask=c2)
even = overlay(c, c.tomsmocomp(1,5,0), mask=c2)

interleave(odd,even)

converttoyv12()
}



edit:

i used to clean up the leftover combing with generalconvolution, but i hate converting to RGB for it.

do a 3x3 convo of:

0 1 0
0 2 0
0 1 0

and then another of:

0 -2 0
0 7 0
0 -2 0

[scharfis_brain]

is

blur(0,1).sharpen(0,1)

able to replace your rgb-convo?

[mf]

Oh, you make your own comb mask. I just leech off of neuron's metrics, those are probably better than my scripting anyway :p.

[Mug Funky]

@scharfis_brain:
i hadn't thought of that... i didn't want to touch the horizontal resolution, but it's possible this will give me nice results and let me stay in YV12. ta.

@mf:
i should probably do that too, but i like to control every step of the process. i haven't actually got masktools to be honest.

note to self: should probably download it.

[edit]

the blur(1).blur(-1) method works brilliantly!

but it's exposed the weakness of my masking technique (mainly with pure interlaced DV and fast moving objects on the "boundary" between 2 frames.) :O

[mf]

Originally posted by scharfis_brain
is

blur(0,1).sharpen(0,1)

able to replace your rgb-convo?
How about Unfilter(0, -1).Unfilter(0, 1) ?

Edit: now this is weird. Unfilter doesn't seem to touch interlacing at all, even with Assumeframebased() :eek:.
Edit2: Okay, unfilter's type of blurring is just weird. Disregard above :rolleyes:.

[Mug Funky]

yeah. i'd have used unfilter but i found the same problem. i was kinda expecting it though.

unfilter is designed to correctly deconvolve the picture, and it does a damn good job of it. especially if the original convolution fits it exactly (as is the case for unfilter(-100).unfilter(100))

blur seems to work pretty well. i haven't bothered with a PSNR test to see what it actually destroys, but the visual result is pleasing to me.

[scharfis_brain]

MugFunky:

blur(1).blur(-1) touches hor. and vert. directions

while blur(0,1).blur(0,-1) only touches the vertical direction.

btw. did anyone tested my function?

[Mug Funky]

oh. i wasn't aware of this function of blur()...

thanks heaps. if only i'd known that earlier

*shaking fist at myself*

[scharfis_brain]

hehe, next try for my perfect deinterlacer!

function TMCBob(clip c, bool TFF, int th) {

c = (TFF == true) ? c.assumetff : c.assumebff
c1=c.converttoyuy2(interlaced=true)

# use DGbob for getting the interlacing-mask
c2a=c1.dgbob(order=1,thresh=th)
c2b=c2a.trim(1,0)
c3a=subtract(c2a.selecteven(), c2a.selectodd())
c3b=subtract(c2b.selecteven(), c2b.selectodd())
c3=interleave(c3a,c3b).greyscale()
c3=layer(c3,c3.levels(0,1,255,255,0),op="lighten",level=255).levels(129,1,143,0,255)

# use TMC(0,-1,0) to get the extra lines, shift the odd frame one line up
c4=c1.separatefields.tomsmocomp(0,-1,0)
c4a=c4.selecteven
c4b=c4.selectodd
c4=interleave(c4a,stackvertical(c4b.crop(0,0,0,-c4.height+1),c4b.crop(0,0,0,-1)))

# fill the interlaced areas with TMC's extra lines
overlay(c1.doubleweave, c4, mask=c3)

}

I've stolen some lines of Code from Mug Funky (building the mask), but found, that abusing dgbob (or any other deinterlacer) for getting the metrics easier and better.

[Mug Funky]

oh, hey. hadn't tried that... i thought of trying it but wanted to keep it down to avisynth internal filters (excluding tomsmocomp, and i think i used undot for DV material with a high noise floor).

i'm flattered you're using my code... i just wish i'd cleaned it up some... those variable names are the very definition of "ad-hoc".

heh. i guess i also avoided dgbob because it seemed strange to use a smart bob to achieve smart bob :O

my computer's busy atm, but i'll give this a try ASAP. see if it gets rid of those awful "outlines" that i get with funkybob in high-contrast movement (doesn't occur often in anime, but it's everywhere in DV footage which i wrote this for).

[scharfis_brain]

Hehe, I sadly cannot use undot yet, because I am currently sitting in front of a K6-2 450Mhz....
but undot needs a iSSE - Capable CPU :(

my much more faster Athlon is currently NA due to a crashed windoze. And I am too lazy to fix that next time....


I saw, that your interlaced map included movements between 3 fields, but optimal would have been 2 fields.

But whatever I tried... I wasn't able to get a dgbob-like interlacing map with avisynth-internal filters...
so I decided to abuse dgbob itself.

btw. I am only testing on real-live video (but this looks really really great with TMCbob!). I do not have any Anime-clips...

[Mug Funky]

oh, okay. i figured bob was so much faster, and the smartness of the bob was not overly important seeing as all i wanted was the difference between 2 fields

but i see that dgbob helps much lower the noise floor of the interlace mask, so i get more info to feed to levels().

there's still the problem of subtract() screwing up on motion-blurred objects - subtract gives "false positives" on such objects, leaving combs behind.

another thing - the reason i used a 2-field mask and a 2-frame mask was to eliminate artifacts that look like interlacing but persist over 1 frame (like graphics with horizontal lines, or some forms of crosstalk)

i'm just about to try your script out... i have high hopes :)

[edit]

hmm... didn't work for me at all. the picture came out either combed slightly (like a progressive frame with 1 field shifted slightly, like halfway between stairstepping and combing) or b0rked (interlaced between fields from different frames).

i'm working with BFF material (PAL DV), but i did a crop.addborders and got a worse result... strange.

perhaps i wasn't holding my mouth right :)

[scharfis_brain]

did you tried
doubleweave.selectodd

between AVISOURCE and TMCbob?

[Mug Funky]

hmm... i wasn't holding my mouth right. not sure what happened, but it's working nice now. very nice. there's none of those horrid "field-border" problems i had with funkybob. little bit of residual combing, but i'll fix that by running blur(0,1).blur(0,-1) afterwards.

[edit]

i think i'll use TMCbob for live action only, as it seems to give jumpy line shimmer in anime (in static scenes). this could probably be fixed by playing with thresholds however.

[soujir0u]

I just tried out this script, and it definitely looks better than standard fielddeinterlace() which I was using on this video clip... (less jaggy and blending). Any chance for a YV12 version?

[mf]

Yeah.
SeparateFields()
TomsMoComp(0,-1,0)
Weave()
TomsMoComp(1,5,0)
VerticalReduceBy2()

But it can't bob.

[soujir0u]

Cool, thanks. Bob doubles the framerate doesn't it? I don't usually use that...

Anyway, I was comparing this script with just tomsmocomp(1,0,0) and the result was really similar...