TDeint and TIVTC

[ChiDragon]

@tritical:
My mistake with the l/c/p thing, the "l +" matches show up as "c (OVR)" and "COMBED FRAME". The ones that show up as "p (OVR)" and "COMBED FRAME" are listed in the input file as "p +". The "p +" ones ARE shown as "l" in TDecimate's display listing though, except when PP=1 (then they're "p").

I just checked and the problem with TDecimate and hints only happens when batch=true in TDec (batch=true in TFM doesn't matter). For what it's worth, this occurs even if the ovr file is blank.

[tritical]

Reproduced it now. I originally added the batch parameter to tdecimate as a way to be able to load a script with tdecimate(mode=5,...) without having complete input files... that way you could set up the two pass mode=4/5 process in vdub's job control all at once. For that too work, it had to set some internal arrays to fake, but usable, values. The problem is that if mode isn't set to 5 then tdecimate thinks the values are good and uses them. Anyways, I forgot to have it check that mode is indeed set to 5 before setting the arrays to fake values. It'll be fixed in the next version. Thanks for the reports.

The "p +" matches showing as "l" in tdecimate is how it is suppose to work. Specifically, any combed frame, regardless of the match, should show up as either "l" or "h" in tdecimate if pp>1 in tfm. If pp=1, then only combed c matches will be translated into "l" or "h" and the others will stay as they are. It's not a pretty system, but it gets the job done .

[Chainmax]

Originally Posted by tritical

@Chainmax
...Then just change the tdecimate line back to what it was originally, add input="stats.txt" to it, and uncomment out all the other lines. That way it wont have to process 5000 frames everytime you seek to frame in a new cycle.

I don't understand that. The stats file was created and the Mode2 and Mode0 scripts are:

MPEG2Source("X:\wherever\myvideo.d2v")

Trim(0,1982)+Trim(2355,0)

SeparateFields().DeBlock_QED().ColorYUV(off_y=-3,gain_y=20,cont_u=0,cont_v =51.2,opt="coring").Weave()

Interp = SeparateFields().EEDI2(field=3,maxd=29,nt=80,pp=3)
TDeint(mode=1,order=1,type=3,mtnmode=3,tryweave=true,edeint=Interp,slow=2)

Crop(22,4,688,554,align=true)
Lanczos4Resize(656,448)

TDecimate(mode=2,rate=29.97,input="stats.txt")

MPEG2Source("X:\wherever\myvideo.d2v")

Trim(0,1982)+Trim(2355,0)

SeparateFields().DeBlock_QED().ColorYUV(off_y=-3,gain_y=20,cont_u=0,cont_v =51.2,opt="coring").Weave()

Interp = SeparateFields().EEDI2(field=3,maxd=29,nt=80,pp=3)
TDeint(mode=1,order=1,type=3,mtnmode=3,tryweave=true,edeint=Interp,slow=2)

Crop(22,4,688,554,align=true)
Lanczos4Resize(656,448)

TDecimate(cycleR=2003,cycle=5000,input="stats.txt")

The Mode0 version, just like before, can't even load the video. The Mode2 version nets an access violation error pointing to the TDecimate line .

[tritical]

There seem to be a few logic errors in the mode 0/1 code making it still go through and request all the frames in the cycles even though it has all the stats info. I'll fix that.

On the mode 2 access violation, does it occur on load or during processing? and is it only if you use an input file?

[Chainmax]

The access violation occurs when trying to load the script. Upon editing the script and commenting out the stats loading portion the script loads fine.

[tritical]

Could you add "debug=true" into the tdecimate line, have DebugView open, and then try to load the mode 2 script that crashes? Post the output you get back here... should give some idea of where the problem is since I can't seem to reproduce it.

[Chainmax]

I bet it's just one of those random Avisynth issues, but I'll try your suggestion as soon as I can and report back.

[ChiDragon]

@tritical:
Would it be possible to add a "flags" option for D2V match use that only checks scene changes for being combed? Like a limited flags=4. Checking with flags=1, scene changes seem to be the only places where there are combed D2V matches in my files.

By the way, I've noticed some frames reported as d2vdups by TDecimate that aren't dupes at all. I haven't seen it actually decimate these frames, so I guess it doesn't matter, but I thought it was weird.

[tritical]

A flags option that checks the d2v matches for combing only around scenechanges would be possible to add.

To diagnose the d2v dup problem I'd need the d2v file and an output file from tfm generated with a run using:

mpeg2source("d2v.d2v")
tfm(d2v="d2v.d2v",output="output.txt",+any other options)

Also, a list of some of the frames being incorrectly reported.

The requirement to be a d2v dup is that the combination of the match used in tfm and the rff flags in the d2v file indicate that one field is present in both the current and previous frame.

[zoinbergs]

Hey guys! I got a quick question for ya.. hopefully it means a quick answer is on the other end...

I'm currently trying to field match some 25p footage that has been 3:2 telecined from 20fps, and it's fields are (unfortunately) blended.

Here's a quick example:

http://www.sendspace.com/file/zp3t0z


Could anybody possibly define some basic terms for TFM (I believe I would need to use) to give me a push in the right direction?

Somebody told me that TFM can match field blended frames, so I presume I'm in the right place, am I not?

I've been trying for days to get "restore24" to work, as that is exactly what I need to do (if one were to consider my footage 3:2 pulldowned from 23.976 to 29.97), but I just cannot seem to get my script to work with it! You can have a look at my thread here on it if you'd like:

http://forum.doom9.org/showthread.php?t=117006

I've almost given up on restore24, and am looking for any possible alternatives. TFM and TDecimate look promising I must say! Wow can you guys code!

.....................................................................

So I believe I can start with TFM(mode=7) right? I read that in the handy tutorial, but that's all I've figured out so far.

Any and all help is most definitely appreciated, and is considered to be priceless for me! I would be forever in debt to somebody who can help me figure this one out.

Thank you in advance, and keep up the AWESOME work.



EDIT: My prayers have been answered with the wonderful tool FixBlendIVTC!!!! Have a nice day guys, and thank you anyway!

[ChiDragon]

Well, I started opening a bunch of instances of VDub for the false d2vdups as I found them, until I noticed that they disappear when the script is reopened! This is true for the 7 or so that I found up to that point. What the...

[tritical]

Declaring a d2v dup requires that tfm knows the match that was used for the previous frame. If you reopen a script or seek to a frame without processing the frame before it (and you aren't using an input file), then it doesn't have that information... in which case it wont declare the frame a d2v dup. What is the script you're using?

[ChiDragon]

Actually, I was using an input file. I did try seeking once, forward and back, to see if the d2vdup would reappear and it didn't.

V = MPEG2Source("J:\PVR Extractions\211.d2v")
A = NicMPASource("J:\PVR Extractions\211 T01 DELAY 0ms.mpa")
AudioDub(V,A)
AmplifydB(15)

Trim(1589,4289)+Trim(8092,27742)+Trim(56212,75941)+Trim(81492,101986)+Trim(107616,109331)

TFM(d2v="J:\PVR Extractions\211.d2v",trimIn="cap3_TI.txt",mode=0,slow=2,micmatching=0,PP=5,blockx=32,blocky=32,MI=160,input="cap3_TFM.txt",ovr="I:\cap3_TFM_ovr.txt")
TDecimate(mode=1,dupThresh=0.4,hybrid=1,vidThresh=3,vidDetect=1,input="cap3_TDec.txt",ovr="I:\cap3_TDec_ovr.txt")

Crop(4,4,-18,-4)
LanczosResize(512,384)

I'm actually using the previous version of TIVTC BTW, to avoid that hints issue.

[tritical]

Could you send me the following:

211.d2v
cap3_TI.txt
cap3_TFM.txt
cap3_TFM_ovr.txt
cap3_TDec.txt
cap3_TDec_ovr.txt

Even if it goes away after moving around a few frames, if just getting it the first time (opening the script and requesting a set of specific frames in a specific order) is reproducible then that should be enough.

You can zip the files and email them to me as an attachment, or upload them to:

66.189.231.68:17252
user/pass: upload/upload

or if you have a place to host then that's fine too. I'd like to get this issue figured out before releasing another version as the d2v dup detection is a major part of the decimation routines. There are some safety checks in place that will keep it from doing anything really stupid (a d2v dup also has to have a difference metric <= to the lowest in the cycle for most stuff), but having it correct would be best.

[ChiDragon]

Sorry, I wasn't clear about the seeking thing. When I said "I did try seeking once, forward and back, to see if the d2vdup would reappear and it didn't" I meant that in reference to reopening the script in VDub via F2. Seeking back and forth when a false d2vdup does appear doesn't cause it to go away, only reopening the script does.

I thought I mentioned this before, but I guess not: the false dupes only seem to occur in cycles where there already is a real d2vdup.

I actually deleted those files once I was done with that encode, sorry... But I've found a place in another ep that I can reproduce by seeking a specific way.

Open the script in VDub, "Go To Frame" 4133 (all frame numbers are post-decimation), seek forward to 4275, then backward to 4147 and the first false d2vdup occurs, at the end of the cycle. Seek back further, to the next cycle, and there's another at the end of the cycle. The start of this cycle is at 4140 (2 cycles after the initial frame sought).

If you seek further in the initial forward-seeking, to say 4300, there are more false d2vdups, now appearing in the backward-seeking as early as 4195 and continuing each cyle backwards until 4140. More false dupes can be added by going further forwards and then backwards again, even without first reopening the script.

Those results are when an input file is used for TDec (input/no input for TFM gave the same results either way). If no input file is used, the false d2vdups appear "sooner" in the backwards-seeking (4133->4300=4243 rather than 4195).

Using this:

TFM(d2v="J:\PVR Extractions\212.d2v")
Trim(1709,4412)+Trim(8906,30480)+Trim(34974,50748)+Trim(55692,73536)+Trim(78932,101454)+Trim(107747,109462)
TDecimate(display=true)

Rather than this:

Trim(1709,4412)+Trim(8906,30480)+Trim(34974,50748)+Trim(55692,73536)+Trim(78932,101454)+Trim(107747,109462)
TFM(d2v="J:\PVR Extractions\212.d2v",trimIn="cap7_TI.txt")
TDecimate(display=true)

causes them to appear one cycle "sooner" still, at 4247.

Using that final script with no TrimIn file, I decided to try using Go for both 4133 and 4300 rather than seeking to 4300, and the result was d2vdups as early as 4295 (2 cycles before 4300) when I did backwards-seeking from there. Seeking backwards seems to be the key, at least in this instance; just jumping forward and backward with Go didn't cause it.

Weirdness all around... Hope my explanation of this isn't too confusing.

[tritical]

Could you test this build: [link removed] and see if it fixes all the false d2v dup detections? The false detections were caused by a combination of random access, hints, and a bug in the d2v dup decision code. It also has the fixes for all the other issues.

[ChiDragon]

It all appears to be fixed, thanks!

[tritical]

Only 18 months since v1.0 beta 1 ... here is [link removed]. Changes:

+ added blend deinterlacing option (type = 4)
- changed denoise default to false
- pixels detected as moving, but with absolute difference < 4 to both vertical 
     neighbors are no longer automatically weaved (should fix problems with slow fades)

[anonymez]

finally

thanks tritical

[Mug Funky]

thanks tons!

will try it out immediately.

[edit]

awesome performance on fades! picture doesn't bob around, but smoothly comes up the way it should. excellent. i can significantly raise my thresholds now.

the uppermost line appears to behave strangely on odd fields (when deinting top-first stuff). it appears to be getting weaved.