Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st March 2007, 16:59   #961  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
First of all, generally: I don't claim that things should be done in exactly the way as they're done in MCBob; heaven forbid. In MCBob it's a rough compromise for speed's sake, because some things are just too inefficient when done by limited means of scripting.

What I want is to raise the general idea, because it seems quite feasable. If we have an idea of the local detail level - and that "idea" is rather easy to get by spatial evaluation -, then the allowed range for pixel variance can be derived from it. After thinking a little over it, making the thresholds dependend on the present local variance seems much more logical to me, compared to simply allow "this_fixed_value" for just everything ...


Specifically:

Quote:
The neighborhood for determining the min/max for the motion threshold for 'c' would consist of which pixels?
...
I was thinking it would be the 3 x's above and below 'c', but want to make sure.
Hereby ensured.


Quote:
I've been looking at this a little more and I don't quite understand why the threshold is based on the range (max-min) of the neighborhood.
Using the maximum difference to the center pixel (like you suggested) theoretically is more precise. I used local min/max for two-andahalf reasons:

a) it was much faster. min/max I get with mt_lutxy(mt_inpand,mt_expand). for max-diff I'd need mt_lutxy(mt_lut_xy(clip,clip.mt_inpand),mt_lutxy(clip,clip.mt_expand)).

b) the center "doesn't exist" yet. The center is the pixel we want to construct.
Therefore: calculating the maximum difference of the neighborhood to the interpolated-from-the-neighborhood center value, in the end is the same as taking min/max of the neighborhood, and divide by 2.

c) the "double sized" value of min/max is levelled out trivially: if we want to allow 1/4 pel motion as being static, this means 1/4 pel between neighbored top/bottom fields. But the field motion check is done between same-parity fields, i.e. the doubled distance. So, to allow 1/4 pel between two directly neighbored fields, the motion check has to allow 1/2 pel between same-parity fields ... that's the point where the "bigger" result of min/max gets levelled out. (That's not fully exact science, sure ... but after trying, it seemed to suffice.)


Somehow, talking about it seems to make it more complicated than it really is ...
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 2nd April 2007, 09:42   #962  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
I made an initial implementation of an adaptive-threshold motion-mask that can be used with tdeint (mmask.dll). Currently, it is just a bunch of separate filters: mthresh, mmask, mcombine, and buildMM, that can work with tdeint through the emask option. Be aware that this is straight c and completely unoptimized so it is rather slow. I've implemented it as a set of separate filters to make experimenting easier.

mthresh(clip child, int type) -

This filter takes in a field based clip and will output a threshold map, where the threshold for each pixel is based on it's 3x3 neighborhood. The output frames will be twice the size vertically of the input. The top half of the output is the 1/4 pel threshold and the bottom is the 1/2 pel threshold. 'type' can be 0, 1, 2, or 3 (default is 1). For type = 2, the thresholds are simply:

max(abs(cp-nmin),abs(cp-nmax))*0.25
max(abs(cp-nmin),abs(cp-nmax))*0.50

where cp is the current pixel value and nmin/nmax are the min/max of the current pixel's top/left/right/bottom neighbors. type=3 is the same as 2, but considers all 8 neighbors. type = 0 is the same as 2 but compensates for the 2x difference in distance between lines (due to it being only a field), as well as for the distance differences due to downsampling in chroma. type=1 is the same as 3, but with the compensation as in 0.


mmask(clip child, clip mask, int type, int nt, int minthresh, int maxthresh) -

This filter takes in the mask output by mthresh (the 'mask' clip should be set to this) and the field based video that was given as input to mthresh (each frame needs to be stacked vertically with itself) (the 'child' clip should be set to this). It will then output a mask indicating which pixels are moving (0) or static (255). A pixel is static if the difference between it and the pixel in the other frame is less than min(cthresh,othresh) where cthresh is the threshold determined by mthresh for the current pixel and othresh is the threshold determined by mthresh for the other pixel. type can be set to 0, 1, or 2 (default is 2). type=0 will do difference to previous frame, type=1 will do difference to previous/next, and type=2 will do difference to next. nt is the noise threshold (default 2)... nt will be added to min(cthresh,othresh). minthresh is the minimum threshold value (if min(cthresh,othresh) is less than minthresh then it will be increased to minthresh) (default is 0). maxthresh works the same way as minthresh but sets the maximum (default is 75).


mcombine(clip child, int cstr) -

This filter takes the output from mmask, which is still double size vertically and combines the 1/4 pel and 1/2 masks into one mask. Basically, if a pixel is marked as moving in the 1/4 pel mask but is marked as static in the 1/2 pel mask, then mcombine checks to see how many of that pixels neighbors in its 3x3 neighborhood are marked as static in the 1/4 pel mask. If at least 'cstr' of its neighbors are, than that pixel is considered static in the final mask (default for cstr is 4). All pixels marked as static in the 1/4 pel mask are also considered static in the final mask.


buildMM(clip tfields, clip bfields, int mode, int order, int field, int length, int type) -

This filter takes in two clips corresponding to top fields and bottom fields. These clips should be the output of running mthresh, mmask, and mcombine separately on the top and bottom fields of the video. buildMM assumes that type=2 was used when running mmask(). mode/order/field are exactly the same as the same parameters in tdeint and should match the settings you intend to use in tdeint (defaults are 0/-1/-1, -1 means the same as it does in tdeint). length specifies the static field period for weaving a pixel (to get a 5-field check as in tdeint use length=5, to get an 8 field check as in securedeint use length=8, etc...) (length must be >= 4 and <= 60, default is 8). 'type' specifies whether both neighboring vertical lines of the line attempting to be weaved have to agree on the direction (back, across, or forward) of the static period. If type=0 then they don't, if type = 1 then they do (default is 0). The output of buildMM is a mask usable by tdeint through its 'emask' parameter.


Here's an example of doing bobbing on a bff clip:

mpeg2source("C:\t4.d2v")
bot = separatefields().selecteven()
top = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=0,mode=1,emask=buildMM(topf,botf,mode=1,order=0))

here's what it would be for tff:

mpeg2source("C:\t4.d2v")
top = separatefields().selecteven()
bot = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=1,mode=1,emask=buildMM(topf,botf,mode=1,order=1))

and here's same rate deinterlacing on a tff clip (keeping top fields):

mpeg2source("C:\t4.d2v")
top = separatefields().selecteven()
bot = separatefields().selectodd()
topf = mmask(stackvertical(top,top),maskc=mthresh(top)).mcombine()
botf = mmask(stackvertical(bot,bot),maskc=mthresh(bot)).mcombine()
tdeint(order=1,field=1,mode=0,emask=buildMM(topf,botf,mode=0,order=1,field=1))


Any suggestions are welcome. I've been testing this on some difficult clips and it seems to work considerably better than tdeint by itself. Probably the part most up for change is mcombine(), as the method it uses to combine the 1/4 pel and 1/2 pel masks isn't exactly ideal.

mmask.dll also includes the following two filters:

dilate(clip child, int dstr)
erode(clip child, int estr)

dilate will mark any pixel that is 0 as 255 if >= 'dstr' of its 8 neighbors are not 0.

erode will mark any pixel that is not 0 as 0 if <= 'estr' of its 8 neighbors are not 0.

Those are there for possible processing of the intermediate masks.

EDIT: mthresh, mmask, mcombine, buildMM, dilate, and erode will all work with YV12 or YUY2.

Last edited by tritical; 2nd April 2007 at 10:07.
tritical is offline   Reply With Quote
Old 3rd April 2007, 03:00   #963  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Not sure if anyone else tested, but I found that there were still some artifacts on a couple sequences. Having mmask check differences both 1 frame forward and 2 frames forward and then anding those two masks fixed it though. Gonna post an updated version of mmask.dll later. Hopefully after a little more testing I'll start merging it all into one filter.


New version of mmask.dll. Changes from before:

mmask() -

The 'type' parameter can now be set to 0, 1, 2, 3, 4, or 5 (default is 3).

0 = diff to n-3
1 = diff to n-2
2 = diff to n-1
3 = diff to n+1
4 = diff to n+2
5 = diff to n+3

static vs moving is determined as explained previously. 'minthresh' now defaults to 5. This may result in a tiny amount of residual combing, but was needed to stop the background from wavering in the "scrolling text" test clip from http://media.xiph.org/vqeg/TestSeqences/Reference/.


andMasks(clip clip1, clip clip2, int dis) -

This filter will output an and'd mask of clip1 and clip2. clip1 should be generated with mmask(type=3) (i.e. single frame forward diff), clip2 should be generated with either mmask(type=4) or mmask(type=5) (two frame or three frame forward diff). dis should be set to 2 if you use type=4 or 3 if you use type=5 (default is 2). For the dis=2 case, the filter will grab frame 'n' from clip2 and and it with frames n and n+1 from clip1. If dis=3 then it will and frame 'n' of clip2 with frames n/n+1/n+2 of clip1.


buildMM() -

There is a new parameter called 'dis' (an integer) (default is 1), which if you use andMasks() to create the motion masks should be set to the same value as the 'dis' parameter of andMasks(). Basically, 'dis' tells buildMM how many frames forward a pixel is stationary if it is marked as 255 in the mask. Also, 'type' can now be set to 0, 1, or 2 (default is 1). 1 and 2 are the same as 0 and 1 were before. 0 is now a less restrictive version of 1... theoretically, 0 should weave the most but be the most artifact prone, 2 will weave the least but be the least artifact prone.

bobbing, tff, 1+2 frame diff:

top = separatefields().selecteven()
bot = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=1,mode=1,emask=buildMM(topf,botf,mode=1,order=1,dis=2))

bobbing, bff, 1+2 frame diff:

bot = separatefields().selecteven()
top = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=0,mode=1,emask=buildMM(topf,botf,mode=1,order=0,dis=2))

same rate, tff, keep top field, 1+2 frame diff:

top = separatefields().selecteven()
bot = separatefields().selectodd()
topm = mthresh(top)
botm = mthresh(bot)
tops = stackvertical(top,top)
bots = stackvertical(bot,bot)
topf1 = mmask(tops,topm,type=3)
botf1 = mmask(bots,botm,type=3)
topf2 = mmask(tops,topm,type=4)
botf2 = mmask(bots,botm,type=4)
topf = andmasks(topf1,topf2,dis=2).mcombine()
botf = andmasks(botf1,botf2,dis=2).mcombine()
tdeint(order=1,field=1,emask=buildMM(topf,botf,order=1,field=1,dis=2))

So far this has managed to get through every clip that tdeint, using its internal motion masking, produces motion related artifacts on.

Seems I spoke too soon. Defaulting to minthresh=5 in mmask() to fix the wavering on the scrolling text clip results in artifacts in another video. Those artifacts can be fixed by setting minthresh=2, but that causes problems for the scrolling text clip, or by setting length=10 in buildMM().

Last edited by tritical; 3rd April 2007 at 09:34.
tritical is offline   Reply With Quote
Old 3rd April 2007, 15:23   #964  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
You and Didée makes this all seem like jibber-jabber.
I'll give the new mmask a try. Thanks for the .dll and thorough explaination.
Terranigma is offline   Reply With Quote
Old 3rd April 2007, 23:12   #965  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
I noticed two bugs in buildMM() that would cause it not to weave pixels when it should have. I updated mmask.dll (same link as above). I also changed the default for 'length' in buildMM() to 10. mmask() was also changed so that instead of requiring diff < min(cthresh,othresh) it's now diff <= min(cthresh,othresh) and the default for 'minthresh' was changed to 4 (was 5 previously).
tritical is offline   Reply With Quote
Old 4th April 2007, 23:50   #966  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Cool, if I set length=12 with this mmask thing it fixes a short sequence that causes huge artifacting normally. I guess that's not really a test of the new thresholding though? I'm guessing if TDeint by itself could be set to 12-field check it would also be fine on these frames (but then I don't really understand most of this discussion).
ChiDragon is offline   Reply With Quote
Old 5th April 2007, 00:54   #967  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
I added a new filter to mmask.dll called createMM(), which combines mthresh/mmask/andMasks/mcombine. Syntax:

createMM(clip, int type, int nt, int minthresh, int maxthresh, int cstr)

type is from mthresh
nt/minthresh/maxthresh are from mmask
cstr is from mcombine

dis is fixed at 2. The defaults are the same as for the separate filters.

Now things are a little simpler:

bobbing, tff:

topf = separatefields().selecteven().createMM()
botf = separatefields().selectodd().createMM()
tdeint(mode=1,order=1,emask=buildMM(topf,botf,mode=1,order=1,dis=2))

bobbing, bff:

botf = separatefields().selecteven().createMM()
topf = separatefields().selectodd().createMM()
tdeint(mode=1,order=0,emask=buildMM(topf,botf,mode=1,order=0,dis=2))

same rate, keep top field, tff:

topf = separatefields().selecteven().createMM()
botf = separatefields().selectodd().createMM()
tdeint(mode=0,order=1,field=1,emask=buildMM(topf,botf,mode=0,order=1,field=1,dis=2))


@ChiDragon
You're are probably correct. The longer the static period required the more resistant it will be to artifacts from repeated motion. I think a good example of the adaptive threshold in action is src20_ref__525.yuv from http://media.xiph.org/vqeg/TestSeqences/Reference/. TDeint requires mthreshL=13 to get the same level of stability as createMM()+buildMM(), and actually using mthreshL=13 on any material with motion will lead to lots of residual combing and artifacts which you wont get with createMM()+buildMM().

Would you be willing to upload the short sequence you mentioned earlier to my ftp?
12.216.251.99:17262
upload/upload

Last edited by tritical; 5th April 2007 at 01:01.
tritical is offline   Reply With Quote
Old 5th April 2007, 05:24   #968  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
I uploaded the clip.
ChiDragon is offline   Reply With Quote
Old 5th April 2007, 07:24   #969  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@ChiDragon
Thanks .

@All
I updated mmask.dll again, only changes were a few optimizations. Atm, using tdeint w/ createMM()+buildMM() is about 3x slower for me than using tdeint w/ its internal motion-masking. It can probably be sped up some more, but considering it's doing a 10 field check (by default) vs tdeint's 5 field check and is using per-pixel adaptive thresholding, 3x slower seems pretty reasonable.
tritical is offline   Reply With Quote
Old 5th April 2007, 14:02   #970  |  Link
Adub
Fighting spam with a fish
 
Adub's Avatar
 
Join Date: Sep 2005
Posts: 2,698
Well, it looks like I might have to start checking this out.
Thanks alot for the new filter Tritical!
__________________
FAQs:Bond's AVC/H.264 FAQ
Site:Adubvideo
Adub is offline   Reply With Quote
Old 6th April 2007, 19:30   #971  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Updated mmask.dll again. Changes were all to mthresh(). 'type' now has options 4 and 5, which use 4 and 8 neighbors respectively, but use the range (max-min) of the neighborhood instead of diff to the center pixel. I also added four new parameters: mtqL, mthL, mtqC, mthC which can be used to set hard thresholds instead of using adaptive thresholds. mtqL sets the quarter pel threshold for luma, mthL sets the half pel threshold for luma, mtqC/mthC are the same but for chroma. If these parameters are set to -1 (the default) then an adaptive threshold is used, if they are between 0 and 255 inclusive then the value of the parameter is used as the threshold for every pixel.

The above changes to mthresh() were also included in createMM(), whose type parameter can be set to 4 or 5, and which now has mtqL, mthL, mtqC, and mthC parameters.

I'm planning to create one filter called tMM() that does everything internally so that deinterlacing would simply be:

tdeint(order=xx,mode=xx,field=xx,emask=tMM(order=xx,mode=xx,field=xx))
tritical is offline   Reply With Quote
Old 6th April 2007, 23:56   #972  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Didn't take long... TMM.zip. It includes a readme and source code. Basic usage examples:

bobbing, tff:

tdeint(mode=1,order=1,emask=TMM(mode=1,order=1))

bobbing, bff:

tdeint(mode=1,order=0,emask=TMM(mode=1,order=0))

same rate, keep top field, tff:

tdeint(order=1,field=1,emask=TMM(order=1,field=1))

same rate, keep bot field, bff:

tdeint(order=0,field=0,emask=TMM(order=0,field=0))
tritical is offline   Reply With Quote
Old 6th April 2007, 23:58   #973  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
I'll try this out, thanks. Hey, any news on what we've discussed via pm?


Update:
Very nice work tritical. Would it be possible to use tmm with, lets say, bob() to make it adaptive? If so, how would I go about doing that?

By the way, you forgot to add another bracket at the end of the second example =P
tdeint(order=0,field=1,emask=TMM(order=0,field=1)

Last edited by Terranigma; 7th April 2007 at 00:15. Reason: To Give Credit :)
Terranigma is offline   Reply With Quote
Old 7th April 2007, 00:28   #974  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
I've told about 5 people that I'd look into something for them and haven't gotten around to any of it due to working on this motion-mask thing. Anyways, I did download tprivtc earlier in the week to try and fix the problems you mentioned, but it turns out that it uses fstream.h functions for doing the file/text stuff so I couldn't get it to compile straight away with vs.net 2003 (which doesn't have fstream.h and switching to iostream+fstream didn't fix everything) so I put it away for later. I'll probably mess with it tommorrow.

Quote:
Very nice work tritical. Would it be possible to use tmm with, lets say, bob() to make it adaptive? If so, how would I go about doing that?
Not sure in what fashion you mean, but the main problem with using it in conjunction with bob() via some type of masking would be that bob shifts both fields. If you just want cubic interpolation with motion-adaptation then TMM w/ tdeint(mode=1,type=0) would work.

thanks for the heads up on the typo in the read me.

Last edited by tritical; 7th April 2007 at 00:31.
tritical is offline   Reply With Quote
Old 7th April 2007, 00:34   #975  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Quote:
Originally Posted by tritical View Post
Not sure in what fashion you mean, but the main problem with using it in conjunction with bob() via some type of masking would be that bob shifts both fields. If you just want cubic interpolation with motion-adaptation then TMM w/ tdeint(mode=1,type=0) would work.
That's what I meant. I'll try your suggestion there instead.
Terranigma is offline   Reply With Quote
Old 7th April 2007, 01:18   #976  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Quote:
Originally Posted by tritical View Post
@ChiDragon
Thanks .

@All
I updated mmask.dll again, only changes were a few optimizations. Atm, using tdeint w/ createMM()+buildMM() is about 3x slower for me than using tdeint w/ its internal motion-masking. It can probably be sped up some more, but considering it's doing a 10 field check (by default) vs tdeint's 5 field check and is using per-pixel adaptive thresholding, 3x slower seems pretty reasonable.
Only three times slower? That's still ~20% faster than SecureBob/Old TDeint+EEDI2, ~220% faster than MVBob and ~310% faster than MCBob! tritical, you're a genius .
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.
Chainmax is offline   Reply With Quote
Old 7th April 2007, 03:18   #977  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
You might still want to use eedi2 with tdeint's edeint parameter though, which will slow it down a bit more. Using emask+edeint, there's hardly any reason to keep tdeint around though.
foxyshadis is offline   Reply With Quote
Old 7th April 2007, 16:11   #978  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Well, like I said, this is still ~20% faster than Old TDeint+EEDI2. Besides, someone here said that using EEDI2 with the edeint parameter yielded a softer picture than plain TDeint. Not to mention that EEDI2 has artifacts of its own.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.
Chainmax is offline   Reply With Quote
Old 7th April 2007, 17:39   #979  |  Link
canuckerfan
Registered User
 
Join Date: Jul 2005
Posts: 317
so are we better off leaving out EEDI2 or is it best left to our discretion?
canuckerfan is offline   Reply With Quote
Old 7th April 2007, 20:08   #980  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I never noticed the differences myself, but then again I never conducted a test either. Theoretically speaking, using EEDI2 might have some benefits in specific situations so I'd advise you to conduct a few tests with a variety of sources (at least 1 real life clip and one animation clip) and see if you can spot a difference.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.
Chainmax is offline   Reply With Quote
Reply

Tags
tdeint, tivtc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:09.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.