@len0x: auto-IVTC methods for AutoGK

[bilu]

Hi len0x,

I haven't tried AutoGK yet but the automatic IVTC detection process you use got me curious.

I had a batch tool to use with Robot4Rip called R4R_ENC. It was a primitive tool based on batch files and Avisynth scripts and was the inspiration for TaZit. It had some pretty steps for auto-resizing based on compressibility as well, faster than TaZit (less steps). You may find some ideas useful, I hope.

http://forum.doom9.org/showthread.php?s=&threadid=51228

I stopped development because I couldn't get a decent AutoIVTC, mainly because you can't trust only in the DVD2AVI output. Example: hardware telecined movies with no MPEG-2 Telecined flags.

http://forum.doom9.org/showthread.php?s=&threadid=53393

Please take a look on my approach of using Telecide's debug information.I do a XVID Null encode with Telecide(FOrder(),guide=1,post=0,debug=true) and capture the output to a file.

http://forum.doom9.org/showthread.php?s=&threadid=64649

FOrder() is a function to get the field order from MPEG-2 sources and use them in Telecide, DGBob or SmartDecimate.

#FOrder works only with MPEG-2 sources
function FOrder(clip c) {
fo= GetParity(c)? 1:0
return fo
}

As for hybrid encodes, process would be easier on Matroska's variable framerate than on AVI's constant framerate. Because with constant framerate you also may need to calculate Decimate hybrid thresholds on mode=1 and mode=3, and collect data for doing so.

This was an experience of mine some time ago, as an example:
http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?t=126

On Matroska process a filter could just decide which parts are Telecined and which are not. And make a process like this even simpler:

http://forum.doom9.org/showthread.php?s=&postid=395247#post395247


Would like your feedback on these. I think this subject deserves a thread for itself.


Best regards,
Bilu

[len0x]

heh, I finally got you interested :)

All the links you've posted I studied before I started working on auto-IVTC. Some ideas are nice, some can be done simpler.

Have a go with AutoGK first and you'll see how it works (and you'll see familiar getOrder function :) ). I don't use d2v statistics at all (apart from detecting pure FILM). I work on direct analysis of motion frames, grab combed statistics and examine the source in portions of 5 frames in a row.

Anyway, to detect the source is easy for me. What I would ideally would like to see is automatic threshold parameter adjustment for Decimate(). Coz say you've detected that source is hybrid and know which mode you gonna run decimate with, but it would be nice if you can automatically adjust default threshold=1.0 used for hybrid sources... I don't know if statistics from telecide can be used for that since I haven't explored that path at all...

*Edit*
I didn't see that link from DG's board about threshold caclulation. Can you sum up what's the algorythm without implementation details?

[bilu]

Originally posted by len0x
heh, I finally got you interested :)I'm glad I can say the same about you :)
Your work can bring much value and attention to the auto-IVTC issues.

Originally posted by len0x
I didn't see that link from DG's board about threshold caclulation. Can you sum up what's the algorythm without implementation details?[/B] Output from Decimate(mode=0,debug=true):

!Decimate: 0: 0.00
!Decimate: 1: 2.87
!Decimate: 2: 0.48
!Decimate: 3: 2.35
!Decimate: 4: 2.49

These values are differences between frames, the same you use on the the hybrid threshold.

On my algo I would account how much frame metrics fit between 0.0 and 0.1, 0.1 and 0.2, and so on. But I would read this values only on frames detected as duplicates. I was accounting only between 0.0 and 2.0, seemed more than enough for me.

Example from a Neo Genesis Evangelion clip (false detected as IVTC,but enough as an example):

Each value is a threshold range count, first is 0.0<range<=0.1 and so on.

14 60 94 53 23 22 20 17 13 6 9 6 4 4 4 4 6 6 6 3

This means that
14 frames accounted as 0.0 - 0.1 range
60 " " " 0.1 - 0.2 range
and so on.

Then it would sum this values from beginning until 90% of the duplicates or the last accounted value represent less than 3% of the duplicates. Choice would be 0.9 in this case. Of course percentages could be redefined.

About false detected IVTC clip:
http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?t=174

Another very nice approach for smoother 30fps->24fps video parts on hybrid encoding, wish someone would do a filter based on this:

http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?t=216


Bilu

[len0x]

Originally posted by bilu

These values are differences between frames, the same you use on the the hybrid threshold.


Do we know exact algorytm for caclulation of those differences or we need to ask DG ?

Originally posted by bilu

On my algo I would account how much frame metrics fit between 0.0 and 0.1, 0.1 and 0.2, and so on. But I would read this values only on frames detected as duplicates. I was accounting only between 0.0 and 2.0, seemed more than enough for me.


I don't get the part about duplicates. First why value them, second how do you detect them? (duplicate is either true duplicate, or static scene, or very dark image - I see no way to distinguish between them)

*Edit* or by duplicates you mean those with values < 2.0, meaning static frames? That makes sense actually. I just wonder - I do motion detection myself, so in general I could do the same without telecide help if I can map my motion values to telecide's one

[bilu]

What I'd like to see one of these days:

A Decimate() based filter that would have two modes: VFR and CFR.

VFR mode would simply chop duplicates and output timestamps to a log file as 24 fps for the 4 former frames prior to the duplicate. This would be for the Matroska format.

CFR mode would chop duplicates and do one of scharfis_brain 30fps->24fps approaches, mode=3 is my favorite but haven't tried the others myself.
Scharfis method is smoother than Decomb's or SmartDecimate's hybrid approaches on the video parts. And with no thresholds :)


Bilu

[len0x]

Originally posted by bilu

VFR mode would simply chop duplicates and output timestamps to a log file as 24 fps for the 4 former frames prior to the duplicate. This would be for the Matroska format.


I doubt it'll become really popular (VFR I mean) as it will require quite a support from hardware...

[bilu]

Originally posted by len0x
Do we know exact algorytm for caclulation of those differences or we need to ask DG ? Better ask :)

Originally posted by len0x
I don't get the part about duplicates. First why value them, second how do you detect them? (duplicate is either true duplicate, or static scene, or very dark image - I see no way to distinguish between them) [/B]

I explained badly. Duplicates are not necessarily true duplicates, as in analog hardware-telecined. Decomb's reference:


threshold (decimal, default 0.0)

When mode=1, frames determined to be the most similar to their preceding frame can be treated in two possible ways: 1) they can be blended as described above, or 2) they can be passed through if the threshold parameter is non-zero and the difference metric exceeds the threshold. By setting an appropriate threshold, you can have duplicates get blended while passing through frames that have new content, i.e., which differ significantly from the previous frame. This allows hybrid film/nonfilm clips to be dealt with intelligently: the film portions will have blends and the nonfilm portions will not. The threshold parameter has no effect when mode=0. Use show=true (or debug=true in conjunction with the DebugView utility) to view the difference metrics and thereby determine an appropriate threshold for your clip.

the threshold parameter also affects mode=3 in a similar way.


Bilu

[bilu]

Originally posted by len0x
I doubt it'll become really popular (VFR I mean) as it will require quite a support from hardware...

You're right, but it would be an option for VGA or TV-out users.


Bilu

[bilu]

If mode=3, instead of discarding the most similar frame, Decimate() will apply different decimation strategies for 3:2 pulldown material (film) and for pure video (nonfilm) cycles. Film cycles are decimated in the normal way. Nonfilm cycles are decimated by applying a frame blending decimation of the cycle from 5 frames to 4 frames. The resulting frame rate is the film rate (decimated by 1/5).

I guess threshold in mode=3 would mean blending 5 frames into 4 if threshold is exceeded by the frame which was supposed to be a duplicate.


Bilu

[len0x]

Originally posted by bilu
I explained badly. Duplicates are not necessarily true duplicates, as in analog hardware-telecined. Decomb's reference:


I'm slowly getting there... we want only most similar frames to be blended, right? But in any case we gonna blend static VIDEO parts of the source as well ?

[len0x]

Originally posted by bilu

I guess threshold in mode=3 would mean blending 5 frames into 4 if threshold is exceeded by the frame which was supposed to be a duplicate.


that makes sense as well.
May be I outta do some experiments on how my motion detection relates to telecide's one... If I can duplicate it(roughly) in AVS, then it would be nice...

[bilu]

Originally posted by len0x
*Edit* or by duplicates you mean those with values < 2.0, meaning static frames? That makes sense actually. I just wonder - I do motion detection myself, so in general I could do the same without telecide help if I can map my motion values to telecide's one

I really have to try AutoGK to see how your motion detection process works, but won't have the time today.

But what would you think about checking which parts are Video or Film and using pure Film IVTC on the Film part and scharfis approach on the Video part instead of using Decimate hybrid modes?

You really have to look at it. Very good and without thresholds.

EDIT - Scharfis_brain script page with videos:

http://home.arcor.de/scharfis_brain/60ito24p.html

[len0x]

it works real simple:

global diff = 0.50*YDifferenceFromPrevious(clip) + 0.25*UDifferenceFromPrevious(clip) + 0.25*VDifferenceFromPrevious(clip)

About scharfis's method - I really have to put myself together and read that thread when my brain is still working OK (i.e. in the morning) :) I can't think now at all :)

[bilu]

Check the videos to see the potential.
I won't have time for more until tomorrow too.

Bye.

[manono]

Hi-

While scharfis_brain's approach is certainly valid, I think it comes down to personal preference as to whether or not you use his way, or Decimate(Mode=3), or SmartDecimate to convert the video portions to 23.976fps. I generally try to avoid the blurring/blending effect as much as possible, and prefer a clearer/cleaner result, even if it leaves the video slightly jerky.

But I'm glad that you're finally involved in this bilu. If you run some of your video clips through AutoGK, you may be able to help refine len0x's methods some more. But I think you'll agree that a lot has been accomplished since you first mentioned the idea of Auto IVTC detection, something I didn't even think possible at the time.

[bilu]

Originally posted by manono
While scharfis_brain's approach is certainly valid, I think it comes down to personal preference as to whether or not you use his way, or Decimate(Mode=3), or SmartDecimate to convert the video portions to 23.976fps. I generally try to avoid the blurring/blending effect as much as possible, and prefer a clearer/cleaner result, even if it leaves the video slightly jerky. Having options was allways a tradition in Gordian Knot. I don't want to break that tradition :)

Have you checked mode=1 and mode=2 on schafis script? No blending used, it stutters just a bit, so you could find it better than Decomb.Mode=3 uses blending and is the smoothest, too smooth for some :) Mode=1 seems the most Film-like IMHO.

Still you would need to separate Film from Video in a script to be able to use something else than Decimate() on the video parts. And decision of which frames should be treated as Film or as Video can also be hard.

Check out this example:
http://forum.doom9.org/showthread.php?s=&threadid=64649

I've been exchanging some mails with lighty about the possibilities for this sort of decision. But it's been hard to establish contact with him in the last days. He was developing a program that would parse Telecide output like in that anime thread I mentioned to do the variable framerate timecode log automatically for Matroska muxing later.

Specs on the last mail:


Parameters
=======
Minimum cycle for IN-PATTERN (default 1)
Minimum cycle for OUT-PATTERN (default 1)

Pattern Detection
===========
Use cycles in parameters instead of frames to garantee that the pattern matching changes when finding a complete pattern or lack of pattern. Internal processing should be still in frames, of course.

Example: 11111 11111 11000 11111 11111 would keep the 3 out-of-pattern frames as in-pattern, since there is not a complete lack of pattern. Should work better with false detections this way.

The framerange for a detected pattern doesn't have to be a multiple of 5, just the minimum threshold.
Hence the use of cycles instead of frames on the parameters, to enforce this point.

Example: 11111 11111 11000 00000 11100 would detect the first 12 frames as in-pattern, because it respects the minimum 5 frame (1 cycle) threshold. Then it would detect 8 frames as non-pattern. The last 5 frames would still be detected as non-pattern because 111 was not enough for the minimum 1 cycle threshold.

Frame Reduction
==========
X = nr of frames before IVTC
Y = nr of frames after IVTC

Y = INT (( X - (X MOD 5)) * 0.8) + X MOD 5

INT( ( 841 - 1) * 0.8 ) + 841 MOD 5 = 672 + 1 = 673
INT( ( 842 - 2) * 0.8 ) + 842 MOD 5 = 672 + 2 = 674
INT( ( 843 - 3) * 0.8 ) + 843 MOD 5 = 672 + 3 = 675
INT( ( 844 - 4) * 0.8 ) + 844 MOD 5 = 672 + 4 = 676
INT( ( 845 - 0) * 0.8 ) + 845 MOD 5 = 676 + 0 = 676
INT( ( 846 - 1) * 0.8 ) + 846 MOD 5 = 676 + 1 = 677

Why do 844 and 845 frames before IVTC end up both with 676 frames? It's simple.

5 5 5 4 becomes 4 4 4 4
5 5 5 5 becomes 4 4 4 4

Where 5 means the number of frames in each pattern, 4 on the upper left example is the rest (not enough to fill a pattern) and 4 4 4 means patterns after decimating.
This means that everytime you get an IVTC range you need to reduce the framenumber in the timecodes by (X-Y).
Most info here is useful IMHO. Also I think that a block of 2 consecutive IVTC cycles instead of 1 would be useful to avoid false detections that would end up stuttering by chopping a real frame instead of a duplicate.


Bilu

[bilu]

Originally posted by manono
But I'm glad that you're finally involved in this bilu. If you run some of your video clips through AutoGK, you may be able to help refine len0x's methods some more. But I think you'll agree that a lot has been accomplished since you first mentioned the idea of Auto IVTC detection, something I didn't even think possible at the time. I'm very glad to see key developers and moderators such as you and len0x having a look on this challenging problem. I'm sure this will bring more attention from the encoding community to this issue, more than ever.

And as more attention this gets, more problematic clips get checked, and more solutions arise. It's a pity for us that neuron2 is taking a sabbatical this year, but we shouldn't be selfish :) Kevin Atkinson (SmartDecimate) is a bit offline now too.

Maybe we could convince trbarry to work on this subject one of these days ... ;)


Bilu

[manono]

Hi-

Still you would need to separate Film from Video in a script to be able to use something else than Decimate() on the video parts.

Yes, and although I can't pretend to speak for len0x, I don't think, because of time considerations, that he's prepared to run a null encode on the whole thing to determine exactly which portions are Film and which are Video. He samples the source to determine whether it's Film, Video, or Hybrid, and if Hybrid, if it's mostly Video, or mostly Film. So I don't know if sharfis_brain's ideas can be used in AutoGK. Gordian Knot, maybe, but not AutoGK, where the whole process has to be automated as quickly and as accurately as possible. But he does get reports every so often where the user says AutoGK said it was one thing, when it was something different. Maybe those things have been fixed now. I don't really know, because I see so little Hybrid material these days.

But it would be pretty cool to be able to run a null encode on the DVD, and then IVTC the Film portions normally, and put on 60i->24p for the video parts using sharfis_brain's methods, particularly if it results in less blurring.

I'm very glad to see key developers and moderators such as you and len0x having a look on this challenging problem.

Nothing key about me. :) I'm just a user always looking for a better way.

It's a pity for us that neuron2 is taking a sabbatical this year, but we shouldn't be selfish Kevin Atkinson (SmartDecimate) is a bit offline now too.

Yep, I agree 100%. I noticed before that neuron2 was going to tackle the blended fields problem, to try and weed them out, and give us back clean frames. Something like that would help me personally the most. RePAL works for some things, but I've never had much luck with Unblend. I saw a sharfis_brain script for that, but it was kind of over my head. Just have to be patient, I guess.

[bilu]

Originally posted by manono
Yes, and although I can't pretend to speak for len0x, I don't think, because of time considerations, that he's prepared to run a null encode on the whole thing to determine exactly which portions are Film and which are Video. He will still have to do it to find Decimate hybrid thresholds IMHO.
And that means a script using Telecide(debug=true) and Decimate(debug=true). But if we could use a 30fps -> 24 fps method that wouldn't require thresholds we would avoid Decimate(debug=true) in the Null encode. But I do have to check how AutoGK is working now, I don't know yet if len0x is using Telecide or Uncomb before his motion detection method. Uncomb would be faster, but has no 3-way match. Also SmartDecimate is faster, but leaves interlaced artifacts sometimes.Originally posted by manono
I don't really know, because I see so little Hybrid material these days. Mostly Sci-fi series like Babylon 5 or Star Trek TNG and some anime.
I wouldn't know about DVD extras, never encoded it.

About blended fields - they sure are a problem. The Neo Genesis sample I have has lots of this:
A AC C
A B C
where B is an orphan field, the other half was replaced by a blend from former and next frames. And the clip is full of this. It's enough to confuse any pattern matching as you can see in the anime thread I mentioned before.


Bilu

[bilu]

Originally posted by len0x
Have a go with AutoGK first and you'll see how it works (and you'll see familiar getOrder function :) ). I don't use d2v statistics at all (apart from detecting pure FILM). I work on direct analysis of motion frames, grab combed statistics and examine the source in portions of 5 frames in a row. Do I have a way to use AutoGK with just VOB samples? :)

Originally posted by len0x
Can you sum up what's the algorythm without implementation details? My turn to ask :D

I understand how your motion detection process works, it's based on the HomieFR script. But how are you doing field match before motion detection, is it Telecide? And are you using Telecide(guide=1) for NTSC?

I'm asking this because PAL to NTSC conversions (guide=3) are rare IMHO, and the gthresh feature from pattern guidance is very good for anime, where there are lots of static parts. I have a sample of "Read Or Die" anime that gives completely different results when you use (guide=0) or (guide=1).

How are you collecting the combing statistics?
What use do you have for it in your process?


Bilu

[len0x]

Originally posted by bilu
Do I have a way to use AutoGK with just VOB samples? :)


yes, they should be in the form VTS_??_?.vob and contain VTS_?? - Stream Information.txt file produced by DVDDecrypter (you can fake that one, but make sure PAL or NTSC is correctly specified there) or ifo file.

Originally posted by bilu
I understand how your motion detection process works, it's based on the HomieFR script. But how are you doing field match before motion detection, is it Telecide? And are you using Telecide(guide=1) for NTSC?


I have two statistics for each frame:
- is it motion frame (based on diff)
- is it combed frame (based on isCombed() from decomb 5.1)

So I don't use telecide at all.
After that I parse output file (very tricky bit) in portions of 5 frames to determine if _those_5_frames_ are:

- progressive pattern
- interlaced pattern
- telecined pattern

I only work on motion frames and discard static ones since as I said I have no way to distinguish between static FILM/interlaced scenes and static progressive scenes.
This seems to be very robust method and it has only two variables to tune: threshold parameter for isCombed and threshold for motion frames.

*Edit* This is very diffrent approach from what ppl tried before - but this is one-to-one mapping from opening the source manually and use your eyes to determine what kind of source you're dealing with.

[len0x]

Originally posted by bilu
Having options was allways a tradition in Gordian Knot. I don't want to break that tradition :)


Not in the AutoGK - we have to use the best solution without an option :)

[len0x]

Originally posted by manono
Yes, and although I can't pretend to speak for len0x, I don't think, because of time considerations, that he's prepared to run a null encode on the whole thing to determine exactly which portions are Film and which are Video.


You're right - I'm not prepared to do that. :)
When I have nothing more to do I'll start integrating such methods like YATTA and VFR :D But before that - it's not worth it...

Originally posted by manono

But he does get reports every so often where the user says AutoGK said it was one thing, when it was something different. Maybe those things have been fixed now. I don't really know, because I see so little Hybrid material these days.


I see it failing only for one reason: what can we call hybrid? which percentage of FILM and NTSC ? Now I have thresholds 6-94% for hybrid. The rest is either pure FILM or pure NTSC. I had one user who said that with 94.5%(was it 93.5%) he still wants to be able to just deinterlace no matter what. I can raise this to 10-90% to be absolutely sure. But then some might complain that 10% of the movie is not in a good shape.

Another problem is threshold parameter for decimate which has to be adjusted, but as Bilu said that we can deal with as well. So I don't see any problems.

[len0x]

Originally posted by bilu
He will still have to do it to find Decimate hybrid thresholds IMHO. And that means a script using Telecide(debug=true) and Decimate(debug=true).

Not really - even if I use telecide for that I will use only 10% of the source. I need an estimation anyway...

[bilu]

Originally posted by len0x
I have two statistics for each frame:
- is it motion frame (based on diff)
- is it combed frame (based on isCombed() from decomb 5.1)

So I don't use telecide at all.
After that I parse output file (very tricky bit) in portions of 5 frames to determine if _those_5_frames_ are:

- progressive pattern
- interlaced pattern
- telecined pattern

I only work on motion frames and discard static ones since as I said I have no way to distinguish between static FILM/interlaced scenes and static progressive scenes.
This seems to be very robust method and it has only two variables to tune: threshold parameter for isCombed and threshold for motion frames.

*Edit* This is very diffrent approach from what ppl tried before - but this is one-to-one mapping from opening the source manually and use your eyes to determine what kind of source you're dealing with. I have to try it out with my Neo Genesis Evangelion clip.
Haven't tried it yet but I already think that this is probably the most robust method available :)

I feel that this approach is the right one for the blended fields.
Telecide's pattern guidance could be fooled more easily IMHO.

But what do you do with your decision? Would you do a Telecide(guide=1).Decimate(debug=true) to collect Decimate hybrid threshold?

I have an idea: your method using IsCombed on moving areas can help predicting which frames would become duplicates after Telecide.

Then you could do a smaller analisys using Telecide(guide=1).Select( frame before dup, dup frame).Decimate(debug=true). Since the debug would output the difference from the last frame, and you already know the dup framenumbers, it would be much faster to collect this data.

What do you think?

EDIT: How do you guess which frame blocks are telecined? 2 combed frames in every 5? :)

Bilu

[len0x]

Originally posted by bilu
But what do you do with your decision? Would you do a Telecide(guide=1).Decimate(debug=true) to collect Decimate hybrid threshold?


I don't do that atm at all. Just select appropriate mode for decimate
(0 for normal ivtc, mode 1 or 3 for hybrid with threshold=1.0) for actual encoding. That's why I was looking for a method to adjust that threshold.

Originally posted by bilu

I have an idea: your method using IsCombed on moving areas can help predicting which frames would become duplicates after Telecide.

Then you could do a smaller analisys using Select( frame before dup, dup frame).Telecide(guide=0).Decimate(debug=true). Since the debug would output the difference from the last frame, and you already know the dup framenumbers, it would be much faster to collect this data.


Interesting. So you're saying we just need stats from dupes, right?
I really wonder if I can produce similar statistics as telecide uses via avisynth scripts. Then we can avoid using telecide at all...

[len0x]

Originally posted by bilu
EDIT: How do you guess which frame blocks are telecined? 2 combed frames in every 5? :)


That's correct. but only if out of 5 frames more than 3 are motion frames. Otherwise I discard the whole block. The beauty of this method is that you can recognize progressive patterns in hybrid sources as well very easy. (you can have FILM/progressive hybrid sources)

[bilu]

Originally posted by len0x
I really wonder if I can produce similar statistics as telecide uses via avisynth scripts. Then we can avoid using telecide at all... You'd have to collect frame difference after Telecide (or Uncomb, much faster) between (frame before dup, dup frame) and "synchronize" your values with Decimate. I guess this can be faster than any Decimate(debug=true) process...

Bilu

[len0x]

Originally posted by bilu
You'd have to collect frame difference after Telecide (or Uncomb, much faster) between (frame before dup, dup frame) and "synchronize" your values with Decimate. I guess this can be faster than any Decimate(debug=true) process...


If I'm able use my motion value somehow, then there is no need for second pass of processing since I already have the motion value when deciding if the frames is moving or not, so instead of just result(true/false) I can output motion values and process them in one go within my program.

[bilu]

Originally posted by len0x
If I'm able use my motion value somehow, then there is no need for second pass of processing since I already have the motion value when deciding if the frames is moving or not, so instead of just result(true/false) I can output motion values and process them in one go within my program. But isn't your motion value applied without any field matching first?
You would need to match fields with Uncomb or Telecide and then collect frame differences!


Bilu

[len0x]

Originally posted by bilu
But isn't your motion value applied without any field matching first?
You would need to match fields with Uncomb or Telecide and then collect frame differences!


What do mean by field matching? (I might be new to this...)

[len0x]

Can I get hold of that one somehow ?

[bilu]

Originally posted by len0x
What do mean by field matching? (I might be new to this...) You aren't, believe me :)

Telecide(post=0) or Uncomb do field matching. FieldDeinterlace or KernelDeint are deinterlacers.

The difference is that deinterlacers apply post-processing over the fields of the current frame to get rid of interlacing interfacts, blending or interpolating.

ABCDE -> A*B*C*D*E*
ABCDE A*B*C*D*E*

Relation between A and A* is that A* fields are the result of blending or interpolating A fields.

Field matchers check fields from the next or previous frames if a frame looks combed, then replace a field with the most similar one, and then post-process if you choose to.

Example 1: Telecide(back=0) - match fields in current or next frame

Order=1 (Top Field First)
AABCD -> ABCCD
ABCCD ABCCD

Example 2: Uncomb - matches fields in current or previous frame

No need to define order (can be problematic with missing fields)
AABCD -> AABCD
ABCCD AABCD

That's where the definition of Telecine duplicate frame comes from :)
And that's why you need it before measuring the duplicate threshold.

Bilu

[bilu]

Originally posted by len0x
Can I get hold of that one somehow ?

I can upload to you that one and other problematic clips.
PM me an FTP server if you can. If you can't I'll try to set up something and tell you when ready.

I also got some problematic AVI clips that we could try to convert into VOBs if needed. I think they could be helpful for research too.

EDIT: about measuring.

1) Motion Detection
2) IsCombed()
3) Telecide(post=2,back=0)
4) Dup threshold measure

Optimizations:

AABCD
ABCCD

frames 2 and 3 are combed. After field matching will become

ABCCD
ABCCD

frame 4 will be the duplicate.So we would select frames 3 and 4 for dup threshold measurement.

Conclusion: Whenever the 5 frame pattern is FILM, if

frame A is combed
frame B is not combed

then B will be the dup and (A,B) will be the framerange for dup threshold.

EDIT2: As you can see in the Uncomb example it works the opposite way: if frame A is not combed and B is combed then B will be the duplicate.

EDIT3:
1) Motion detection
2) IsCombed()
3) If pattern=FILM then dup=(not combed frame after combed frame)
4) Select(frame before dup, dup frame)
It will look like this:
ABCDEF
BBDDFF

5) Telecide(guide=0,post=2,back=0)
It will look like this:
BBDDFF
BBDDFF

6) Calculate dup threshold (even frames may have post-processing and be slightly different from odd frames, enough to generate a dup threshold. Most times between 0.0 (pure telecine) and 1.0 (somewhat dirty) but it can go to 2.0 or more)


Bilu

[bilu]

I don't know much about the missing field problem. But I do understand why it would mess with Uncomb:

AABCD -> AABCD or AABCD
AB*DD AABBD AABDD

It would choose to match current (C top and D bottom) or previous (C top with B top, since there was no C bottom in frame 3).

Telecide supports 3 way matching as an option, but it can cause stuttering (mostly in real life video, non-anime).

back (0-2, default 0) selects the matching mode. When back=0, the backward match is never tested. This means that at bad edit cuts, a progressive frame may not be found and the resulting frame must be deinterlaced by postprocessing. When back=1, if a frame is still combed after field matching (according to a test against 'bthresh' -- see below), then the backward match is tried. This requires post > 0 and may allow a good progressive frame to be found at bad edit cuts. When back=2, the backward match is always considered. Some clips, especially those with a lot of blended fields, may be handled better with back=2, which tests for matching with the previous, current, and next frames. But use it with caution, because it can make some clips jerky.

By better handling with back=2 in blended fields neuron2 means a part of the blended frame may match better with previous than next or current frames. Not all blends are 50-50.

Example on 30fps pure video:

AB0DE
ABCDE

if 0 is a blend more close to B than to D then frame 3 will be

B
C

with post-processing on top, and the result can look closer to the previous frame than to current or next frames. Hence the stuttering.

bthresh (0.0-255.0, default 50) sets the combing detection threshold for conditional backward matching (back=1). The backward match will be considered if the candidate match has combing greater than 'bthresh'. Typically you set a high 'vthresh' to catch stray combed frames, together with a lower 'bthresh' to catch bad edits.

If you raise vthresh and lower bthresh it will make it easier for Telecide to choose 3-way. And the stuttering problem gets bigger too.
This should be used only in anime IMHO. On real live movies post-processing (deinterlacing) would be a better choice.

vthresh (0.0-255.0, default 50.0) sets the combed frame detection threshold for the postprocessing. You may want to increase this value if too many good frames are being deinterlaced, or reduce it if some combed frames are not getting caught. The default is a reasonable general purpose value. Note that this threshold determines whether a frame is considered combed and needs to be deinterlaced; it is not the threshold you might be familiar with in Smart Deinterlacer. That threshold is determined by dthresh (below); it is the threshold for deinterlacing the frames detected as combed.

As you've seen before, if you want 3-way matching you may end up dealing with vthresh values too.

dthresh (0.0-255.0, default 7.0) sets the threshold for deinterlacing frames detected as combed. Note that this threshold is the threshold you might be familiar with in Smart Deinterlacer.



I'd rather use telecide(back=0) than Uncomb or any other option.
It's safer on motion and on missing fields too.

Bilu

[len0x]

Originally posted by bilu
I can upload to you that one and other problematic clips.
PM me an FTP server if you can. If you can't I'll try to set up something and tell you when ready.


I'll try to organize something about it.

Originally posted by bilu

EDIT3:
1) Motion detection
2) IsCombed()
3) If pattern=FILM then dup=(not combed frame after combed frame)
4) Select(frame before dup, dup frame)
It will look like this:
ABCDEF
BBDDFF
5) Telecide(guide=0,post=2,back=0)
It will look like this:
BBDDFF
BBDDFF
6) Calculate dup threshold (even frames may have post-processing and be slightly different from odd frames, enough to generate a dup threshold. Most times between 0.0 (pure telecine) and 1.0 (somewhat dirty) but it can go to 2.0 or more)


OK, looks pretty nice.
By select you mean that I have to construct this:

clip=Select(3,4)+Select(8,9)+...

right?

Over the weekend I'll try to compare telecide's statictics with mine and see if mine can be used for this purpose (I wish there was a way to dump stats from telecide to an external file without special utilities).

P.S. Now I see the big picture and it doesn't sound that scary anymore as it looks :)

[bilu]

Originally posted by len0x
By select you mean that I have to construct this:

clip=Select(3,4)+Select(8,9)+...

right?yep, that's right :)Originally posted by len0x
Over the weekend I'll try to compare telecide's statictics with mine and see if mine can be used for this purpose (I wish there was a way to dump stats from telecide to an external file without special utilities). Are you a mind reader? :D
Before reading your post I was writing this one:

http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?p=2685#2685

EDIT: you need Decimate-like statistics, not Telecide.
And Decimate(debug=true) output looks like this:

!Decimate: 0: 0.00
!Decimate: 1: 5.50
!Decimate: 2: 6.11
!Decimate: 3: 6.55
!Decimate: 4: 0.00
!Decimate: in frm 0, use frm 0
!Decimate: dropping frm 4
!Decimate: in frm 1, use frm 1
!Decimate: dropping frm 4
!Decimate: in frm 2, use frm 2
!Decimate: dropping frm 4
!Decimate: in frm 3, use frm 3
!Decimate: dropping frm 4
!Decimate: in frm 4, use frm 5
!Decimate: dropping frm 9
!Decimate: 5: 6.87
!Decimate: 6: 7.00
!Decimate: 7: 6.69
!Decimate: 8: 6.73
!Decimate: 9: 0.00

You would want metrics from frame 4 and 9 here.
Threshold would be 0.0 (Clean Telecine)

Eventually you could replace Decimate's output with conditional filtering like you're doing now.

But then again, dup frame decision already says what's the dup frame in "dropping frm X" and we could parse that metric.

It's up to you:

A) use Telecide + calculate and collect frame differencing on 40% of the sample (2 frames on every 5) after predicting which would be dup frame;

B) use Telecide + Decimate(debug=true) and parse an output file. Would run Telecide/Decimate on 100% of the sample but would avoid predicting the dup frame (just parse "dropping frm") and calculating the frame difference;

Which would you think that would be faster? :)


Bilu

[bilu]

Originally posted by len0x
I wish there was a way to dump stats from telecide to an external file without special utilities My method:

NULL.BAT
========
reg import null.reg
dbgproxy avs2avi MOVIE.AVS MOVIE.AVI -p 0 -c xvid 2> debug.txt

( 2> is to output STDERR )

NULL.REG
========
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\GNU\XviD]
"mode"=dword:00000006

DBGProxy: http://ivan.vecerina.com/code/dbgproxy/
Capture program logs generated using the DebugString call, and output them to STDERR (standard error output) prefixed with a '!'

Bilu

[len0x]

btw, there is no need for null encode. I'm doing it with vdubmod - direct stream copy mode on avs which crops image to 4x4 and therefor avi output is really small and data is not propagated to the codec (which seems to be 1-2fps faster then null encode)

[len0x]

Originally posted by bilu
Which would you think that would be faster? :)


For sure the first one. And considering that motion statistics is gathered on 10% of the source, then 40% of that would be 4% of total size which is less then comp test - should be real fast...

[bilu]

http://neuron2.net/ipw-web/bulletin/bb/viewtopic.php?p=2688#2688

Let's see if we can convince sh0dan to do it... ;)


Bilu

[bilu]

Originally posted by len0x
For sure the first one. And considering that motion statistics is gathered on 10% of the source, then 40% of that would be 4% of total size which is less then comp test - should be real fast... The other advantage of this method is that you don't need any debug=true output :)

[len0x]

Originally posted by bilu
The other advantage of this method is that you don't need any debug=true output :)

exactly, but more work to be done for mapping values to threshold parameters though...

[bilu]

Yes, but it's your work. Wouldn't depend on others (sh0dan modifying decomb, etc.) or parsing tools. So you won't have to wait for others to continue your work on AutoIVTC :)

[bilu]

Originally posted by len0x
exactly, but more work to be done for mapping values to threshold parameters though...

Would looking at Decomb.dll source code help?

[len0x]

Originally posted by bilu
Would looking at Decomb.dll source code help?

it would, but it may take really a while for a person who didn't write any avisynth plugins to figure out what's going on there :)

[bilu]

Originally posted by bilu
A) use Telecide + calculate and collect frame differencing on 40% of the sample (2 frames on every 5) after predicting which would be dup frame;

B) use Telecide + Decimate(debug=true) and parse an output file. Would run Telecide/Decimate on 100% of the sample but would avoid predicting the dup frame (just parse "dropping frm") and calculating the frame difference; C) use Telecide(guide=1).Select(frame before dup, frame dup).Decimate(debug=true) and parse the frame difference over 40% of the sample. Would avoid mapping values and still use 2 in every 5 frames(just in Decimate, not in Telecide).

D) use Select(frame before dup, frame dup).Telecide(guide=0).Decimate(debug=true) and parse the output file.
Would be even faster since it would throw 2 frames in every 5 both in Telecide and Decimate, not like in C).

The D) process could be less accurate because guide=0 doesn't use gthresh as (guide=1) does, but pattern guidance can also get fooled by blended fields. Since this is a statistical approach it wouldn't be important to use pattern guidance, since you work only on moving areas.

D) would be faster than C), use 40% of the sample and would avoid mapping values, hence becoming faster to implement. And almost as fast as A) I think. My new favorite :)

EDIT: You would still have to predict the dup frame.

Bilu

[bilu]

Dup Prediction
==============
if IsCombed(frm-1)=true and IsCombed(frm)=false then dup=frm

Hybrid threshold measure Part I
===============================
Select(Dup-1,Dup)
Telecide(guide=0,back=0,post=0)

Hybrid threshold measure Part II (not decided)
================================
A)Decimate(cycle=2,debug=true)
Collect and parse output data

or

B)Scripted frame differencing


Comparison
==========

A) Output values will be correct. Will have to collect and parse using third-party apps.

B) Output values need to be scaled. No need for other apps.

Both) Use 40% of the sample.


Your opinion?

[len0x]

Looks good. Dup prediction will be done inside my program anyway (which will parse isCombed() output) and then avs script will be generated for threshold measure.

And it might not be 40% actually since not all patterns are telecined in the source...

[bilu]

Originally posted by len0x
Looks good. Dup prediction will be done inside my program anyway (which will parse isCombed() output) and then avs script will be generated for threshold measure. But between A) and B) from my last post, which do you prefer? Collect Decimate's output or scale frame differencing in Avisynth?

Originally posted by len0x
And it might not be 40% actually since not all patterns are telecined in the source... [/B] By 40% I mean 40% of the pattern cycle (2 in every 5, 2 is 40% of 5).If a pattern is telecine, you'll use 40% of it :)