PDA

View Full Version : telecide question - possible enhancement idea


wotef
12th December 2002, 14:08
hi, have come across a problematic segment of a movie that was low-motion and very, very dark

the movie needs NTSC 3:2 pulldown; telecide and decimate do very well on normally lit and/or fast motion scenes, but trips up on the dark, slow scenes - matching sub-optimally between p/c/n

the problem gets compounded if it throws off pattern guidance later on, right?

- specifying manual telecide overrides solves the issue, but going through frame by frame by hand is unmanageable for long segments

- something i tried was to turn telecide's debug on, then to destaturate and tweak the contrast up real high with tweak(sat=0,cont=11), prior to telecide(guide=1,debug=true),decimate(cycle=5), then i logged the debug output with debugview and constructed an entire override file with the new telecide field-matching

p.s. tweaking brightness made no difference; contrast was the key

--> removed the tweak command and told telecide to use the new override file for the entire segment, and bingo, the results were MUCH better visually

following the same procedure on some bright scenes predictably seems to push the image into clipping, so the downside is that telecide will then fail again, this time cause it can't see lacing on whited-out frames

so i wonder what your thoughts would be on a possible enhancement for telecide --> in order to increase accuracy of field-matching on both (1) very dark, slow motion scenes & (2) very bright scenes that are already near clipping --> give telecide a way to measure average contrast on a frame by frame basis such that when below a certain floor, it will create a clip segment (where contrast is tweaked up and the field pattern is applied accordingly), and if the contrast is above a certain cap, it will create a clip segment (where contrast is tweaked down and the field pattern is applied accordingly)...all without touching the colour balance of the output clip, else...behave normally and integrate the disparate outputs into one progressive stream

not sure of the best way this would be implemented, perhaps switches/flags, threshold parameters...

but first of all, what do you think of the principle?

Guest
12th December 2002, 14:36
It shouldn't be necessary. Telecide compares the combing factor for the three candidate matches, and even if they are all low, it shouldn't matter because the relative relationship should still be there. What might be happening is that you are falling into the range of my noise rejection. Can you provide me a test input clip that shows the problem?

Richard Berg
12th December 2002, 14:37
Looks like Wotef & I cross-posted. There are sample clips in my thread (http://forum.doom9.org/showthread.php?s=&threadid=40294).

In fact, if you have the ability as a moderator to move that post here, that would be fine. The Tweak() + DebugView phenomenon was Wotef's discovery, anyway.

Guest
12th December 2002, 14:40
Thank you, Richard, I am getting your clips.

Guest
13th December 2002, 00:52
@Richard

I got your clip and tried it with and without the Tweak. I did not see any improvement with Tweak. Please provide your exact scripts and tell me what frame numbers you want me to look at. Thank you.

In parallel, I will look for ways to improve things for such clips. Basically, this is like the "mouth problem"; the area of combing is very small relative to noise.

wotef
13th December 2002, 05:58
hi neuron2, i'll answer this one -

let's look at the default script first and make things easier to see:

avisource("testhuff02.avi")
telecide(guide=1,chroma=true,blend=false,threshold=57,dthreshold=10,debug=true)
decimate(cycle=5,quality=3)
crop(8,100,-0,-104)
lanczosresize(720,552)

i look at (what becomes after decimation) frame 157 (i believe this originally frame number 195 or so)

if i look at the planet, i get this:
http://www.zenadsl5618.zen.co.uk/w00tage/157_default.jpg

i pop the debug output / field choices into here:
http://www.zenadsl5618.zen.co.uk/w00tage/normal.csv

next, i run this script
avisource("testhuff02.avi")
tweak(sat=0,cont=11)
telecide(guide=1,chroma=true,blend=false,threshold=57,dthreshold=10,debug=true)
decimate(cycle=5,quality=3)
crop(8,100,-0,-104)
lanczosresize(720,552)

and i pop the tweaked debug field choices into here:
http://www.zenadsl5618.zen.co.uk/w00tage/tweaked.csv

if i compare a random row, there ARE differences, perhaps due to the noise threshold as you said, but i would interpret it as follows - in (low contrast/slow moving/albeit noisy frames) default telecide lace detection will consider certain frames as equivalent in terms of combing and be prone to choosing field patterns sub-optimally when compared to tweaking the same frame (desaturating, with high contrast); in the case of a low contrast tweaked frame, it would appear that the tweak can tip the balance and reveal resultant combing better such that telecide can make a more precise choice later about field patterns (edit - although i'm a bit confused now cause my example debug output shows telecide doesn't choose the lowest combed frame!! but it does change from a varying pattern to a cccnn cadence)

e.g.
normal:
Telecide: frame 194: p=5 c=4 n=10 using c
tweaked:
Telecide: frame 194: p=2502 c=2527 n=2601 using n

so, proof in the pudding, i now use the debug output from the tweaked job as an override file (http://www.zenadsl5618.zen.co.uk/w00tage/tweaked.ovr) for the final run here:
avisource("testhuff02.avi")
telecide(guide=1,chroma=true,blend=false,threshold=57,dthreshold=10,ovr="tweaked.ovr")
decimate(cycle=5,quality=3)
crop(8,100,-0,-104)
lanczosresize(720,552)

and with frame 157 i'll now get:
http://www.zenadsl5618.zen.co.uk/w00tage/157_tweaked_override.jpg

should this not work? the frame choices are clearly influenced by the saturation and contrast tweak and the resultant file is visually much better

for now, my previous ideas still seem to produce beneficial results, even if my explanation as to why it works is not correct (didn't know about a noise detection threshold in telecide)...also never seen the mouth clip

...if there is benefit, then i still think the (contrast cap and floor) enhancement approach would work, too

eagerly awaiting your thoughts

ArdenDag
13th December 2002, 06:09
hmm, sounds very intriguing, would love to hear what donald has to say about this :)

Guest
14th December 2002, 02:23
Hmmmmmmm....

Thank you for your extensive data and analysis, wotef.

Yes, it did turn out to be that my theory was correct, i.e., by increasing contrast you were pushing pixels up out of my noise threshold rejection range. Interestingly, it is not just the good image pixels that can contribute. Any noise larger than a pixel and correlated between fields can contribute to improving the field matching! I had added the threshold originally to handle some very noisy video captures that had uncorrelated noise. Looks like it's time for more parameters. '100fps.com' can diss me some more. :(

So, attached is a beta version: Decomb 4.02.1. It adds two parameters:

agg: set agg=true to enable aggressive pattern guidance as in DecombYV12 (default is false).

nt: set nt=100 for the previous Decomb behavior (it's a square of pixel value)(default is 100).

So, Richard's clip can now be correctly rendered with:

Telecide(guide=1,agg=true,nt=0,post=false)

As an aside, I am exploring other ways to improve field matching. This was just a quick hack to explain the apparent improvement when increasing contrast.

I want to finish off DecombYV12 and then bring the versions into line before I make any formal releases.

I also have an idea of really expanding the manual override capabilities. You'll be able to specify patterns to apply to given frame ranges, specify different thresholds for different ranges, etc.

wotef
14th December 2002, 11:41
hey man, you wrote tweak and decomb, so i don't really think anybody can diss you!

christmas has come early if the new beta helps us breeze through the IVTC for the rest of this movie now, so cheers, and we'll report results through on how it goes :D

wotef
14th December 2002, 11:56
p.s. if you get a chance, can you explain a little more on how the noise detection and threshold works - the >1 pixel (between fields) precision sounds very interesting and could have potential benefit for a denoising engine filter, no?

Guest
14th December 2002, 15:03
Originally posted by wotef
i don't really think anybody can diss you! I got dissed at swimming practice this morning. :(

Really, they dissed me for having too many parameters to Telecide() making it unusable by average human beings, in their view. The fact that the default options handle the majority of cases apparently escaped their attention.

christmas has come early if the new beta helps us breeze through the IVTC for the rest of this movie now, so cheers, and we'll report results through on how it goes :D Let me know if you have any further problems with it.

if you get a chance, can you explain a little more on how the noise detection and threshold works - the >1 pixel (between fields) precision sounds very interesting and could have potential benefit for a denoising engine filter, no? The combing metric (used for field matching) goes like this (prev is previous line pixel, curr is current line pixel, next is next line pixel):


metric = 0
for all pixels
comb = (prev - curr) * (next - curr)
if comb > nt then metric++;The parameter nt is now configurable.

What I have found is that a progressive frame can have 'noise' that is taller than a pixel. So for field matching purposes, you can consider it to be image content! Think of it as like a tiny smudge on a film frame. It's 'noise' but it can aid field matching. This is to be distinguished from noise that changes with every field, such as that resulting from a noisy off-air analog capture. In the latter case, you want to reject the noise; in the former, you want to include it.

Richard's clip seemed to have a lot of low-level field correlated noise.

I don't know if this observation has any significance for denoising.