TDeint and TIVTC

[flossy_cake]

Originally Posted by StainlessS

DGIndex on a DVD VOB d2v would do it better, gives TFM more to work with.

Sounds complicated and difficult

Isn't it interesting though how human vision can tell immediately if there are combed pixels in the frame, but coming up with an algorithm for it is very difficult. Just thinking about it gives me a headache, I mean how are we going to tell if the pixels are real image data or combing artefacts? The difference in colour between neighbouring pixels could be virtually anything and that could still be a real texture in the image.

It seems Tritical and D.G have created their own algorithms for dealing with it (metric=0/1) but both of them still seem to need per-scene or per-video thresholding.

The only suggestion I could come up with was to perhaps apply the combing detection on a per-block basis, but I'm guessing Tritical and D.G had thought of that already.

NVidia's DXVA deinterlacing has combing detection (controlled by the "use inverse telecine" checkbox in NVCP) and it seems to avoid the issue by playing it safe with a very low threshold so it needs only a small amount of combing before it reverts to interpolation. Similar thing with my PVR's deinterlacer (chipset HiSilicon 3790mv200).

Still, TFM seems to do a pretty good job with PP=0 -- I'd rather have the rare 1-2 frame combing artefact on scene splices than tens of frames dropping to 240p.

[StainlessS]

Sounds complicated and difficult

Really.

d2v="...Whatever.d2v"
Mpeg2Source(d2v) # OOoops, was DGIndex(d2v). I use templates, so never have to type it in myself.
TFM(d2v=d2v, ... additional args you use)

Allows TFM to see the source d2v and detect illegal field transitions etc, instead of hoping that the source filter (eg LWLibavVideoSource or whatever) did everything correctly.
(allows it to see how the source was actually encoded, and DGIndex settings eg "Honour Pulldown Flags" etc)

TFM() on an decoded clip, cannot see any illegal field transitions or exactly how it is encoded, eg soft telecine, only has access to the decoded video provided by the source filter.
DGSource file type dgi, cannot be used by TFM, never implemented.

EDIT:

DGIndex settings eg "Honour Pulldown Flags" etc

Well, this lot from d2v

Stream_Type=1
MPEG_Type=2
iDCT_Algorithm=6
YUVRGB_Scale=1
Luminance_Filter=0,0
Clipping=0,0,0,0
Aspect_Ratio=4:3
Picture_Size=720x576
Field_Operation=0    Honour/Ignore Pulldown Flags/Forced Film 
Frame_Rate=25000 (25/1)
Location=0,0,3,4757c

900 5 0 2048 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 319488 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 710656 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1075200 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1474560 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1847296 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 2228224 0 1 1 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2

# ...

900 5 3 598341632 0 1 8 92 b2 b2 a2 a2
900 5 3 598458368 0 2 1 d2 ff

FINISHED  100.00% VIDEO

If TFM(d2v=d2v) detects illegal field transition, it will create a OriginalName_fixed.d2v (or something like that) file, and throw error telling you to change to the fixed d2v.
TFM() without d2v cannot tell illegal field transitions, nor create fixed d2v.

d2v for TFM() just makes for less guesswork by TFM.

EDIT:
I'm not sure if DGIndex MPeg2Source(d2v).Tfm(d2v=d2v)
has any advantage over eg SomeOtherVideoSource(d2v).Tfm(d2v=d2v) [But if SomeOtherVideoSource() is not frame accurate, then DGIndex Mpeg2Source probably will be better]

But both gotta be better than DGSource(dgi).Tfm()
or SomeOtherVideoSource(dgi OR d2v).Tfm()

EDIT:
I believe that DGIndex Mpeg2Source() is 100% frame accurate, SomeOtherVideoSource() may tend to vary.

[hello_hello]

Originally Posted by flossy_cake

Indeed there shouldn't be, but TFM incorrectly detects there are because of the stippled patterns on various objects in the scene. Disabling TFM and stepping through each frame at a time shows no combing artefacts on that motion sequence. But what I learned recently was that this is only happening because the source filter is disobeying the repeat field flags -- if they are obeyed then I see the expect pattern of 2 combed frames followed by 3 progressive frames.

You'll probably also be decoding at the wrong frame rate. If you're using ffms2 or lasmash and they're not obeying the repeat flags, they output the average frame rate. I thought one of them was changed to make obeying repeat flags the default recently. I could be wrong. I'm using the old lsmash that plays with XP and possibly not the latest ffms2.

Decoding with lsmash or ffms2 it looks progressive, except for around the scene changes, and Info() says:

LWLibavVideoSource("D:\soft telecine.lwi", repeat=true)
29.97fps, 317 frames, 10.577 sec

LWLibavVideoSource("D:\soft telecine.lwi")
29.97fps, 256 frames, 8.541 sec

FFVideoSource("E:\soft telecine.mkv", cachefile="D:\soft telecine.ffindex", threads=1, RFFMode=1)
29.97fps, 318 frames, 10.610 sec

FFVideoSource("E:\soft telecine.mkv", cachefile="D:\soft telecine.ffindex", threads=1)
24.3789fps, 256 frames, 10.5 sec

The frame rate for the second ffms2 example is usually the indicator there's repeat flags being ignored, because there's hard telecined or video sections too, making the average frame rate non standard. It'll also mess with the audio sync. I don't know why lsmash didn't output the same frame rate.

I don't know why the repeat flags are resulting in such horrible telecined fields, or why
Field Repeats: 0
from the log file I posted earlier appears to be wrong.

FYI, you can split MKVs into sections with MKVToolNixGUI.
Even if I rip a DVD with MakeMKV (which I rarely do) I always remux the MKV as a TS file with TSMuxer so I can index it with the old DGIndex as it obeys the repeat flags by default. I never use "force film" mode.

Edit: I tried showing the d2v (index) file to TFM after indexing with DGIndex (decoding with DGDecode) but it didn't seem to help.

DGDecode_mpeg2source("D:\soft telecine.d2v")
TFM(d2v="D:\soft telecine.d2v")

[flossy_cake]

Originally Posted by hello_hello

I don't know why the repeat flags are resulting in such horrible telecined fields, or why

I suspect it's where the video editing equipment back in the 90's/00's didn't make sure to maintain field order when the next scene was spliced in. It just spliced it on whatever field happened to be the last at that point when it should have gone 1 more to make sure the next scene started at top field or whatever the order was. It's so annoying cause it damages the video in a way that makes it really difficult to undo. If previewed on interlaced CRT the viewer probably wouldn't be able to tell anything was wrong as it just scans field after field anyway kind of like how if simple bob deinterlacing was used that would also avoid it.

The back of the DVD case says "Every possible effort has been made to produce the highest quality DVD release. Due to the quality of some of the original elements, some visual imperfections may be experienced". Maybe this is what they are referring to but then again maybe they're just covering themselves against refunds.

edit: actually now that I re-read your comment, I think you are referring to why the repeat flags don't match whatever muckup the 90's scene splicer did. In other words, if splicer cut the end of the scene 1 field too early then the repeat flag should reflect that. The answer is of course: I don't know

[Emulgator]

I wanted to say MPEG2Source of course, just correcting that.
Yes, framecuts go right through the wrong place, leaving an orphaned field.
I remember CCE accepting a .i32 pulldown file to avoid that.
Means extra work that might have been neglected.

[flossy_cake]

Is it possible to have TFM or TDeint set frame property _Combed like eg.

hints = TFM(hint=true, PP=1)
ConditionalFilter(last, deint, last, """propGetAny(hints, "_Combed")""", "=", "true", show=true)

Trying to make it output clip "deint" when _Combed=true

edit: I should probably use

IsCombedTIVTC(clip, int "cthresh", int "MI", bool "chroma", int "blockx", int "blocky")

but it would be nice to know how the hints work as well.

[flossy_cake]

Kind of stuck with the conditional filters. Trying to evaluate luma diff since prev frame in a 25fps clip and then based on that value conditionally select one of two 50fps output clips.

Problem is this doesn't seem possible due to the way Avisynth evaluates things in real time -- the evaluation of the 25fps clip slows down the 50fps clip to half speed as the evaluation is always performed in the same "code block".

I can kind of get around it by doubling the fps of the 25fps stream but then this breaks things like luma comparisons with the previous frame since it's comparing it to a repeat of itself so it often evaluates to 0 change in luma when it shouldn't. Then tried to work around this with some bodge code but can't get it to do what I want. It seems the only way to get what I want is to perform the evaluations on the original 25fps clip.

LWLibavVideoSource("576i25 mixed cadences 1-1 and 2-2.mkv")
orig25 = last
orig50 = ChangeFPS(orig25, 50)
bwdif50 = ChangeFPS(BWDIF(last, field=-2, thr=3), 50)

ConditionalFilter(orig50, bwdif50, orig50, 
\"IsCombedTIVTC(cthresh=9, MI=64) || (YDifferenceFromPrevious() < 2.0)", 
\ "=", "true", show=true)

I need to swap the blue orig50 for orig25 to get the correct evaluations, but that causes the playback of the conditionally selected clip (bwdif50/orig50) to become slow. Seems the Avisynth thread that performs the conditional evaluation is somehow tied to the timing of the input clip which it's evaluating.

Tried using FrameEvaluate on orig25 prior to the conditional like this:

FrameEvaluate(orig25, "global isCombed = IsCombedTIVTC(cthresh=9, MI=64)")
FrameEvaluate(orig25, "global yDiff = YDifferenceFromPrevious())

but couldn't access the vars isCombed and yDiff inside the conditional.

edit: in case it's not obvious what I'm trying to do here. Basically I want to use Tritical's IsCombedTIVTC() with one modification: if IsCombedTIVTC says the frame is NOT combed, then I will only obey that if the luma diff since the last frame was suitably small. This is my attempt at trying to mitigate combing artefacts from scenes where there is just a tiny bit of motion and IsCombedTIVTC is erroneously reporting "not combed".

[hello_hello]

Edit: I haven't read your latest post yet. Got to visit the real world for a bit. Hopefully someone else will come along in the mean time.

TFM's hints have been around for a long time (I can't remember how they work exactly). They're not the same thing as frame properties which are relatively new. Not many plugins support them yet (nor do Avisynth's internal filters).
I can't imagine it could work the way you're doing it.

TIVTC has ShowCombedTIVTC & IsCombedTIVTC functions you can use to check for combed frames. The help file says they use TFM's comb detection.

Here's some functions you may want to play with. They might give you some ideas. I put them together for a discussion in a thread a long time ago, but I'm hopeless with conditional filters too. I remember a lot of frustrated messing around to get them to work and I've never used them myself so I only vaguely remember them. The script has to exist. That is, they probably won't work in AvsPmod until the script is saved.

FindCombing1() writes a log file (run an analysis pass) with the frame number of any combed frame and a running count of combed frames.

12  -  Combing      1
13  -  Combing      2
14  -  Combing      3
15  -  Combing      4
16  -  Combing      5
17  -  Combing      6
18  -  Combing      7
19  -  Combing      8

FindCombing2() as three MI thresholds and writes a log file of the threshold levels with a running count for each.

91  -  Thresh Hi      75
92  -  Thresh Hi      76
93  -  Thresh Low      1
94  -  Thresh Low      2
95  -  Thresh Hi      77
96  -  Thresh Hi      78
97  -  Thresh Hi      79
98  -  Thresh Hi      80
99  -  Thresh Hi      81
100  -  Thresh Med      1
101  -  Thresh Low      3
102  -  Thresh Med      2

FindCombingRed() - I'm not 100% sure. It was a while ago.
It looks like it creates a B/W version of the clip, runs TFM without de-interlacing followed by TDecimate, makes the combed pixels red, runs IsCombedTIVTC on the B/W clip and also checks the second clip for the amount of red, or something....
It displays the clip with the combed pixels red and the log file looks like this:

76  -  Combing 62  -  12.74
77  -  Combing 63  -  112.00
78  -  Combing 64  -  112.00
79  -  Combing 65  -  112.00
80  -  Combing 66  -  14.88
82  -  Combing 67  -  13.51
83  -  Combing 68  -  112.00

I think the idea of that one was to field match and decimate without repairing combing and then checking for it. Or something....

# -------------------------------------------------------------------------------
#            FindCombing1
# -------------------------------------------------------------------------------

function FindCombing1(clip Vid, bool "Append", string "FileName", bool "All")  {

Append = default(Append, false)
FileName = default(FileName, "")
All = default(All, false)

global Count = 0
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
Dir = (StrLen(ScriptDir()) < 2)  ? "D:" : ""
File = ((FileName != "") ? FileName : \
Dir + ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName)) + "_Combing.txt"

Vid = FrameEvaluate(Vid, """

A = IsCombedTIVTC()
Count = A ? Count + 1 : Count
TheCombing = A ? string(Count, "  -  Combing % 6.0f") : ""

""")

Vid = !All ? WriteFileIf(Vid, File, "A", "current_frame", "TheCombing", Append=Append) : \
WriteFile(Vid, File, "current_frame", "TheCombing", Append=Append)

return Vid  }

# -------------------------------------------------------------------------------
#            FindCombing2
# -------------------------------------------------------------------------------

function FindCombing2(clip Vid, bool "Append", string "FileName", bool "All")  {

Append = default(Append, false)
FileName = default(FileName, "")
All = default(All, false)

ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
Dir = (StrLen(ScriptDir()) < 2)  ? "D:" : ""
File = ((FileName != "") ? FileName : \
Dir + ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName)) + "_Combing.txt"

global FCA = 0
global FCB = 0
global FCC = 0
Vid=Vid

Vid = FrameEvaluate(Vid, """

A = IsCombedTIVTC(MI=30)
B = IsCombedTIVTC(MI=80)
C = IsCombedTIVTC(MI=130)

FCA = A && !B && !C ? FCA + 1 : FCA
FCB = B && !C ? FCB + 1 : FCB
FCC = C ? FCC + 1 : FCC

TheCombing = "  -  " + \
(A && !B && !C ? string(FCA, "Thresh Low % 6.0f") : \
B && !C ? string(FCB, "Thresh Med % 6.0f") : \
C ? string(FCC, "Thresh Hi  % 6.0f") : "")

""")

Vid = !All ? WriteFileIf(Vid, File, "A || B || C", "current_frame", "TheCombing", Append=Append) : \
WriteFile(Vid, File, "current_frame", "TheCombing", Append=Append)

return Vid  }

# -------------------------------------------------------------------------------
#            FindCombingRed
# -------------------------------------------------------------------------------

function FindCombingRed(clip Vid, bool "Append", string "FileName", bool "All", bool "Write")  {

Append = default(Append, false)
FileName = default(FileName, "")
All = default(All, false)
Write = default(Write, true)

ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
Dir = (StrLen(ScriptDir()) < 2)  ? "D:" : ""
File = ((FileName != "") ? FileName : \
Dir + ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName)) + "_Combing.txt"

global Count = 0
Vid = Vid.GreyScale()
global Vid1 = Vid.TFM(pp=1).TDecimate()
CombTest = BlankClip(Vid, Color=color_red)
Vid2 = Vid.TFM(Clip2=CombTest).TDecimate()
global Everything = All

Out = FrameEvaluate(Vid2, """

IsComb = IsCombedTIVTC(Vid1)
IsRed = (AverageChromaV() > 128)
IsEnd = (FrameCount() == (current_frame + 1))
Count = IsComb ? Count + 1 : Count

WriteThis = \
(!IsComb && !IsRed && !Everything ? "" : string(current_frame)) + \
(!IsComb && !IsRed ? "" : \
IsComb && IsRed ? (string(Count, "  -  Combing %.0f") + string(AverageChromaV()-128, "  -  %.2f")) : \
IsComb ? string(Count, "  -  Combing %.0f") : \
("  -  No Combing" + string(AverageChromaV()-128, "  -  %.2f"))) + \
(!IsEnd ? "" : \
(!IsComb && !IsRed && !Everything ? "" : chr(10)) + string(FrameCount(), "%.0f  -  Frame Count"))

""")

Out = !Write ? Vid2 : \
!All ? WriteFileIf(Out, File, "IsComb || IsRed || IsEnd", "WriteThis", Append=Append) : \
WriteFile(Out, File, "WriteThis", Append=Append)

return Out  }

[flossy_cake]

Thanks hello_hello, I'll get stuck in again tomorrow as I already spent the day working on this without luck.

The Avisynth wiki gives this code example which seems to indicate that I should be able to use FrameEvaluate to perform the IsCombedTIVTC test on the 25fps clip and then reference the results of it from elsewhere in the script:

function g(clip c)
{
  global w = c
  c2 = ScriptClip(c, "subtitle(t)")
  c3 = FrameEvaluate(c2, "me()")
  c4 = FrameEvaluate(c3, "global text = YDifferenceFromPrevious(w)")
  return c4
}

function me()
{
  global t = String(text)
}

But I had no such luck, every time I try to reference it I get the error "I don't know what <varname> means".

On another note...

Originally Posted by poisondeathray

You can disable post processing globally with TFM(pp=0)

Somehow I missed this and indeed TFM can weave cheese slices silly weird cadence just fine with pp=0. In fact pp=0 seems to do well on actual content as well, even dodgy stuff like my Caroline in the City NTSC DVDs which has all these weird 1-field edits on scene splices, while MadVR's [deint=film] chokes badly with lots of stutter (tfm's mode=0 helps with that too, and then tdecimate's hybrid=1 to blend over single-field splices:

TFM(mode=0, PP=1)
TDecimate(mode=1, hybrid=1, hint=true, cycle=5, cycleR=1,
		  \ viddetect=2, vidthresh=2.2, display=false)

So in fact I am very grateful and pleased that we have TFM as it's doing a significantly better job than Nvidia's DXVA and MadVR's film mode!

So a big to Tritical

[StainlessS]

That Advanced Scripting on Wiki is way headbanging stuff, and I personally think the necessity is questionable.
Everything that can be done in the weird back-to-front and upside down order can more easily be done with scriptclip.
(and in sane order)

but anyways, for anybody wants to play with this [equivalent of flossy prev given script from Wiki]

#v = AviSource("E:\Temp\Test3\atomic_kitten.avi").ConvertToYV12
KCLIP=BlankClip(Pixel_Type="YV12",color=$000000)
WCLIP=KCLIP.BlankClip(color=$FFFFFF)
MCLIP=KCLIP.BlankClip(color=$808080)

V = Interleave(KCLIP,WCLIP,MCLIP)

function g(clip c)
{
  global w = c
  c2 = ScriptClip(c, """RT_DebugF("%d] Subtitle(t)='%s'",current_frame,t) subtitle(t)""")
  c3 = FrameEvaluate(c2, """RT_DebugF("%d] Calling me()",current_frame) me()""")
  c4 = FrameEvaluate(c3, """RT_debugF("\n%d] Setting Global text with YDifferenceFromPrevious(w)",current_frame) global text = YDifferenceFromPrevious(w)""")
  return c4
}

function me()
{
# RT_DebugF("%d] me(): Setting Global t to %s",current_frame,string(text))  # DONT WORK. current_frame not available here
  RT_DebugF("me(): Setting Global t to %s",string(text))
  global t = String(text)
}

g(v)

shows this

00004140    15:29:36.029    RT_DebugF:
00004141    15:29:36.029    RT_DebugF: 0] Setting Global text with YDifferenceFromPrevious(w)
00004142    15:29:36.029    RT_DebugF: 0] Calling me()
00004143    15:29:36.029    RT_DebugF: me(): Setting Global t to 0.000000
00004144    15:29:36.029    RT_DebugF: 0] Subtitle(t)='0.000000'
00004145    15:29:37.761    RT_DebugF:
00004146    15:29:37.761    RT_DebugF: 1] Setting Global text with YDifferenceFromPrevious(w)
00004147    15:29:37.761    RT_DebugF: 1] Calling me()
00004148    15:29:37.777    RT_DebugF: me(): Setting Global t to 219.000000
00004149    15:29:37.777    RT_DebugF: 1] Subtitle(t)='219.000000'
00004150    15:29:38.662    RT_DebugF:
00004151    15:29:38.662    RT_DebugF: 2] Setting Global text with YDifferenceFromPrevious(w)
00004152    15:29:38.662    RT_DebugF: 2] Calling me()
00004153    15:29:38.662    RT_DebugF: me(): Setting Global t to 109.000000
00004154    15:29:38.662    RT_DebugF: 2] Subtitle(t)='109.000000'
00004155    15:29:39.548    RT_DebugF:
00004156    15:29:39.548    RT_DebugF: 3] Setting Global text with YDifferenceFromPrevious(w)
00004157    15:29:39.548    RT_DebugF: 3] Calling me()
00004158    15:29:39.548    RT_DebugF: me(): Setting Global t to 110.000000
00004159    15:29:39.548    RT_DebugF: 3] Subtitle(t)='110.000000'
00004160    15:29:41.511    RT_DebugF:
00004161    15:29:41.511    RT_DebugF: 4] Setting Global text with YDifferenceFromPrevious(w)
00004162    15:29:41.511    RT_DebugF: 4] Calling me()
00004163    15:29:41.511    RT_DebugF: me(): Setting Global t to 219.000000
00004164    15:29:41.511    RT_DebugF: 4] Subtitle(t)='219.000000'

Current_Frame is not available in me() func, but can use this and supply current_frame as arg to new me2() func

#v = AviSource("E:\Temp\Test3\atomic_kitten.avi").ConvertToYV12
KCLIP=BlankClip(Pixel_Type="YV12",color=$000000)
WCLIP=KCLIP.BlankClip(color=$FFFFFF)
MCLIP=KCLIP.BlankClip(color=$808080)

V = Interleave(KCLIP,WCLIP,MCLIP)

function g(clip c)
{
  global w = c
  c2 = ScriptClip(c, """RT_DebugF("%d] Subtitle(t)='%s'",current_frame,t) subtitle(t)""")
  c3 = FrameEvaluate(c2, """RT_DebugF("%d] Calling me2()",current_frame) me2(current_frame)""")
  c4 = FrameEvaluate(c3, """RT_debugF("\n%d] Setting Global text with YDifferenceFromPrevious(w)",current_frame) global text = YDifferenceFromPrevious(w)""")
  return c4
}

function me2(int n)
{
# current_frame = n # Cheat if current_frame required for something
 RT_DebugF("%d] me2(%d): Setting Global t to %s",n,n,string(text))  # current_frame provided as arg n
  global t = String(text)
}

g(v)

result

00000246    15:43:25.380    RT_DebugF:
00000247    15:43:25.380    RT_DebugF: 0] Setting Global text with YDifferenceFromPrevious(w)
00000248    15:43:25.380    RT_DebugF: 0] Calling me2()
00000249    15:43:25.380    RT_DebugF: 0] me2(0): Setting Global t to 0.000000
00000250    15:43:25.380    RT_DebugF: 0] Subtitle(t)='0.000000'
00000251    15:43:27.315    RT_DebugF:
00000252    15:43:27.315    RT_DebugF: 1] Setting Global text with YDifferenceFromPrevious(w)
00000253    15:43:27.315    RT_DebugF: 1] Calling me2()
00000254    15:43:27.315    RT_DebugF: 1] me2(1): Setting Global t to 219.000000
00000255    15:43:27.315    RT_DebugF: 1] Subtitle(t)='219.000000'
00000256    15:43:28.523    RT_DebugF:
00000257    15:43:28.523    RT_DebugF: 2] Setting Global text with YDifferenceFromPrevious(w)
00000258    15:43:28.523    RT_DebugF: 2] Calling me2()
00000259    15:43:28.523    RT_DebugF: 2] me2(2): Setting Global t to 109.000000
00000260    15:43:28.523    RT_DebugF: 2] Subtitle(t)='109.000000'
00000261    15:43:29.360    RT_DebugF:
00000262    15:43:29.360    RT_DebugF: 3] Setting Global text with YDifferenceFromPrevious(w)
00000263    15:43:29.360    RT_DebugF: 3] Calling me2()
00000264    15:43:29.360    RT_DebugF: 3] me2(3): Setting Global t to 110.000000
00000265    15:43:29.360    RT_DebugF: 3] Subtitle(t)='110.000000'
00000266    15:43:30.346    RT_DebugF:
00000267    15:43:30.346    RT_DebugF: 4] Setting Global text with YDifferenceFromPrevious(w)
00000268    15:43:30.346    RT_DebugF: 4] Calling me2()
00000269    15:43:30.346    RT_DebugF: 4] me2(4): Setting Global t to 219.000000
00000270    15:43:30.346    RT_DebugF: 4] Subtitle(t)='219.000000'
00000271    15:43:31.295    RT_DebugF:
00000272    15:43:31.295    RT_DebugF: 5] Setting Global text with YDifferenceFromPrevious(w)
00000273    15:43:31.295    RT_DebugF: 5] Calling me2()
00000274    15:43:31.295    RT_DebugF: 5] me2(5): Setting Global t to 109.000000
00000275    15:43:31.295    RT_DebugF: 5] Subtitle(t)='109.000000'

EDIT: I guess that you could Ultra Cheat and use

Function me3(int current_frame) { ... }

But I had no such luck, every time I try to reference it I get the error "I don't know what <varname> means".

You did not post what it was that you were trying to do, but whatever it was, it could only be used within runtime environment eg ScriptClip.
(and maybe only with After_Frame=true)

ScriptClip(clip clip, string filter [, bool show, bool after_frame, AVS+ bool local]
AVS+ ScriptClip(clip clip, function func [, bool show, bool after_frame, bool local] )

If after_frame=true, the script should be evaluated after the frame has been fetched from the filters above. Default is false. 

ScriptClip on Wiki:- http://avisynth.nl/index.php/Conditi...ter#ScriptClip

Note,

SSS """
    # ...
    Y = AverageLuma() # After any kind of fetching/sampling of current_frame frame in ScriptClip, 
    #    code following will act as if After_frame=True
    #    ie, all upstream filters would have been called to provide the frame for AverageLuma()
    # ...
"""
ScriptClip(SSS,After_Frame=FALSE)  # FALSE is default

If using eg Scriptclip(...,After_frame=False), and some FrameEvaluate() style function is supposed to set a global var,
on FIRST FRAME, that global var may not have been set EVER unless upstream filters have been processed ie FrameEvaluate() not yet called,
and on subsequent frames, the global var set by FrameEvaluate would be as when set on the FrameEvaluate of previous frame.
So may need the After_frame=true thingy.
(if that makes any sense)

[flossy_cake]

Thanks, got it working with ScriptClip and after_frame=true.

I still need to keep the input and output clips at the same frame rate (in this case 50fps since bwdif will be outputting 50fps) otherwise I encountered other sync issues, but managed to work around it by making the luma check ignored if the diff is < 0.1, i.e if it was a duplicate frame, i.e 25fps. But it still seems to need denoising otherwise a static shot can have luma diff of 0.3 ish.

edit: neither of those ended up being reliable solutions. Should be able to workaround it tho with YDifferenceToNext(-2) which should always look at the real previous frame instead of the duplicate. Actually I'm impressed with what is possible with these realtime functions. It's giving me an explosion of ideas and possibilities like detecting 4 combed frames in a row could be used to infer 1:1 cadence and then if that was untrue it may even be possible to dynamically switch to TFM(). The problem is with handling of edge cases. So many edge cases it seems

[flossy_cake]

Originally Posted by StainlessS

Presume this is some kind of diff overflow bug in FieldDiff(),

#WhateverSource(...)
OPT=4 # (default) Auto
#OPT=0 # Use C routines
FieldDiff(debug=true,display=true,opt=OPT)
return last

Similar to both display and DebugView, thoughout video clip.


OPT=4 (Default)

00007315    3.68521953  FieldDiff:  Frame = 14233  Diff = -207589436161 (sad)

Were you able to resolve this?

I'm asking as I need to use more metrics as I'm finding that IsCombedTIVTC still results in a lot of false negatives compared to the one used by TDeint, and I can't compensate with YUV diffing as there are still edge cases, so I'm looking at using CFieldDiff as well to somehow mitigate that but...yeah

Maybe it's possible to workaround by subtracting/multiplying/bitshifting it by something and still get a meaningful value out of it? What is the range of the value supposed to be?

[StainlessS]

Were you able to resolve this?

Nope.

There is this lot for 8 bit only.

****************************************************
********** RUNTIME CLIP COMPARISON FUNCTIONS ********
*****************************************************

 The compiletime/runtime clip functions share some common characteristics.
 The 'n' and (where used) 'n2' args are the optional frame numbers and default to 'current_frame' if not specified.
 The x,y,w,h, coords specify the source rectangle under scrutiny and are specified as for Crop(), the default 0,0,0,0 is full frame.
 If 'interlaced' is true (sometimes use Altscan arg name instead), then every other line is ommited from the scan, so if eg x=0,y=1, then 
 scanlines 1,3,5,7 etc are scanned, if eg x=0,y=4 then scanlines 4,6,8,10 etc are scanned. The 'h' coord specifies the full height scan ie 
 same whether interlaced is true or false, although it will not matter if the 'h' coord is specified as eg odd or even, internally the 
 height 'h' is reduced by 1 when interlaced=true and 'h' is even.
 Some of the functions have 'x2' and 'y2' args for a second frame (which could be the same frame).
 Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
    v2.0, Where two clip comparison, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to
 0 which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions
 as c.


 RT_YDifference(clip c,int "n"=current_frame,int "delta"=1,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
      bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between frame n area x,y,w,h, and frame (n+delta) area x2,y2,w,h.
  Note, by default it will be equivalent to YDifferenceToNext as delta defaults to 1 and x,y,w,h defaults full frame.
  BEWARE, x2 and y2 default to x and y.

  Eg, RT_YDifference(clip,delta=0,y2=1,interlaced=true)
    Would difference the even and odd lines of the same frame,

***
***
***

RT_LumaDifference(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
    int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between clip c frame (n+delta) area x,y,w,h, and clip c2 frame (n2+delta2) area x2,y2,w,h.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

***
***
***

RT_LumaCorrelation(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "n2"=current_frame,
    int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600||clip2.Width>1100||clip2.Height>600?3:2))
  Returns FLOAT value luma correlation (-1.0 -> 1.0) between clip c frame (n+delta) area x,y,w,h, and clip c2 frame (n2+delta2) area x2,y2,w,h.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.
  
  Pearson's Sample Correlation Coefficient.
  http://en.wikipedia.org/wiki/Correlation_and_dependence
  http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
  Uses equivalent routine to the JMac698's JCorr plugin here:
  http://forum.doom9.org/showthread.php?t=165386
  and here:
  http://forum.doom9.org/showthread.php?p=1495098#post1495098


***
***
***

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
        int "n2"=current_frame,int "delta2"=0,int "x2"=x,int"y2"=y,bool "interlaced"=false,            \
        int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

    Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns amount of pixels
    whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
    Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
    If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
    Thresh Default 0, returns number of pixels that are not exactly the same.
    Return value is in range 0.0 (meaning none) to 255.0 meaning 100% of pixels.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

***
***
***

RT_LumaPixelsDifferentCount(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
        int "n2"=current_frame,int "delta2"=0,int "x2"=x,int"y2"=y,bool "interlaced"=false,            \
        int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

    Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
    whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
    Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
    If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
    Thresh Default 0, returns number of pixels that are not exactly the same.
    Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

***
***
***

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0,
        int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,bool "Altscan"=false,bool "ChromaI"=false, 
        Int "Matrix"=(Width>1100||Height>600)?3:2)
    Returns difference between clip c frame n area x,y,w,h and clip c2 frame n2 area x2,y2,w,h. (range 0.0 -> 255.0).    

    Args:-
    c, c2, clips to difference.
    n,  default = current_Frame. Frame from clip c.
    n2, default = current_Frame. Frame from clip c2.
    ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
        Y8, returns same as LumaDifference. ChromaWeight ignored.
        YUV,
           Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
        RGB,
           If ChromaWeight >  0.0, then returns same as RT_RGBDifference()  [ie average RGB pixel channel difference].
           If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.                    
    x,y,w,h. All Default 0 (full frame), for clip c.
    x2,y2. x2 defaults x, y2 defaults y, for clip c2.
    Altscan, default false. If true then scan only every other scanline starting at clip c y coord, and clip c2 y2 coord.
    ChromaI, default false. If YV12 and ChromaI, then do YV12 interlaced chroma scanning. 
        Ignored if not YV12. Both YV12 clips must be same, cannot difference one that is progressive and one that has interlaced chroma.     
    Matrix, default (c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600)?3:2
        Conversion matrix for conversion of RGB to YUV-Y Luma.  0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709.
        Default = (c.Width > 1100 OR c.Height>600 OR c2.Width > 1100 OR c2.Height>600) then 3(PC709) else 2(PC601). YUV not used
        Matrix only Used if RGB and ChromaWeight == 0.0, where returns same as RT_LumaDifference().
    Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

Using Delta, Interlaced (later renamed to AltScan for some functions), and Y args to compare pretty much any fields.

EDIT: From the RT_YDifference example given,

diff = RT_YDifference(clip,delta=0,y2=1,interlaced=true)
 #   Would difference the even and odd lines of the same frame,

and

diff = RT_YDifference(clip,delta=1,y2=1,interlaced=true)
 #   Would difference the current_frame even and Next_frame odd lines, etc

[flossy_cake]

I guess there is also something like

LumaDifference(SeparateFields().SelectEven(), SeparateFields().SelectOdd())

But I don't know how meaningful it is.

edit: yep, not very meaningful it seems. I'm getting more meaningful numbers out of CFieldDiff / 1000000 + 207590 -- a value of about 0.25 on noncombed pans and 7 on combed pans. If it's less than about 0.125 then it seems safe to believe the result of IsCombedIVTC.

I'll have to try your RT_ funcs to see how they compare -- thanks.

[StainlessS]

LumaDifference takes two clips (where could be same clip).

similar is

RT_LumaDifference(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
    int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between clip c frame (n+delta) area x,y,w,h, and clip c2 frame (n2+delta2) area x2,y2,w,h.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

[StainlessS]

Maybe of interest,
mod of snippit posted by TheCoreyBurton, [cant find his post, TheCoreyBurton account deleted]
https://forum.doom9.org/showthread.p...99#post1785099

Function DoubleRate(clip c,Bool "Blend") {
    c
    Blend=Default(Blend,False)                                      # Avoid blends at scene change (copy previous frame)
    prefilt   = DeGrainMedian()                                     # some smoothing
    superfilt = MSuper(prefilt, hpad=16, vpad=16, rfilter = 4)      # all levels for MAnalyse
    super     = MSuper(pel=2, hpad=16, vpad=16,rfilter=4,levels=1)  # one level is enough for MRecalculate
    bw_1      = MAnalyse(superfilt, chroma = false, isb = true,  blksize = 16, OverLap=2, searchparam = 3, plevel = 0, search = 3, badrange = (-24))
    fw_1      = MAnalyse(superfilt, chroma = false, isb = false, blksize = 16, OverLap=2, searchparam = 3, plevel = 0, search = 3, badrange = (-24))
    bw_2      = MRecalculate(super, chroma = false, bw_1, blksize = 8, searchparam = 1, search = 3)    # No Overlap (currently broken for final vectors)
    fw_2      = MRecalculate(super, chroma = false, fw_1, blksize = 8, searchparam = 1, search = 3)    # No Overlap
    MBlockFps(super, bw_2, fw_2, num=FramerateNumerator(c) * 2, den=FramerateDenominator(c), mode=0, Blend=Blend)
    Return Last
}

Function HybridDoubleRate_QTGMC(clip c,String "Preset",Bool "Blend",Bool "Show") {
    c
    Preset=Default(Preset,"Very Slow")
    Blend=Default(Blend,False)      # for DoubleRate. False dont blend scene change, copy previous frame, true blend.
    Show=Default(Show,False)
    DeInt=QTGMC(preset=Preset)  Deint=(Show) ? Deint.AddBorders(0,0,0,20,$FF0000).Subtitle("QTGMC "+Preset+" Deinterlaced",Align=1) : Deint
    Dbl=DoubleRate(c,Blend)     Dbl  =(Show) ? Dbl.AddBorders(0,0,0,20,$000000).Subtitle("DoubleRate",Align=1) : Dbl
    A=ConditionalFilter(last,DeInt.SelectEven,Dbl.SelectEven,"IsCombedTIVTC","=","true")
    B=ConditionalFilter(last,DeInt.SelectOdd, Dbl.SelectOdd, "IsCombedTIVTC","=","true")
    Interleave(A,B)
    Return Last
}

Function HybridDoubleRate_NedYad(clip c,Bool "Blend",Bool "Show") {
    c
    Blend=Default(Blend,False)      # for DoubleRate. False dont blend scene change, copy previous frame, true blend.
    Show=Default(Show,False)
    NED  = NNedi3(Field=-2, nns=2)                        # Field -2 = double rate, internal parity value to start (set by AssumeT/BFF)
    DeInt= YadifMod(Order=-1,Mode=1,edeint=NED)           # Order -1 = Internal Parity.  Mode 1=double rate do spactial
    Deint=(Show) ? Deint.AddBorders(0,0,0,20,$FF0000).Subtitle("Ned/Yad Deinterlaced",Align=1) : Deint
    Dbl=DoubleRate(c,Blend)     Dbl  =(Show) ? Dbl.AddBorders(0,0,0,20,$000000).Subtitle("DoubleRate",Align=1) : Dbl
    A=ConditionalFilter(last,DeInt.SelectEven,Dbl.SelectEven,"IsCombedTIVTC","=","true")
    B=ConditionalFilter(last,DeInt.SelectOdd, Dbl.SelectOdd, "IsCombedTIVTC","=","true")
    Interleave(A,B)
    Return Last
}


VideoFileName   ="D:\BAB-59\VIDEO_TS\bab59.d2v"
MPEG2Source(VideoFileName)
AssumeTFF

#HybridDoubleRate_QTGMC(Preset="Slow",Show=True)
HybridDoubleRate_NedYad(Show=True)

Is really pretty good.
Deinterlace interlaced sections, and MC doubled framerate for progressive sections.
Perhaps change YdifMod to YadifMod2


EDIT: something I was playing with (Work in Progress - not sure if in working condition or not)

vExt = GetFilenameExtension(VideoFileName)

If(vExt == ".dgi")  { DGSource(VideoFileName)   }
Else                { Mpeg2Source(VideoFileName)}


AudioExt=RT_GetFileExtension(AudioFileName)

Audio=      (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
        \ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
        \ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
        \ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
        \ : (AudioExt==".w64") ? RaWavSource(AudioFileName,samplerate=6)
        \ : 0

Assert(!isInt(Audio),"NO AUDIO")


(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP    # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP

####### CONFIG ##########

/*
    PalHybridToProgressive().

    Optional:
        Calls TFM() to match fields for Phase Shifted. [When showing Stacked result, shows original pre-TFM clip as Src]
        Calls Spotless() on Separated Fields and re-weaves. [more temporally stable than separate calls for interlaced/progressive]

    IsCombedTIVTC() Interlaced de-interlaced via YadifMod2, Progressive double rated via JohnFPS() [even fields Lossless].

    Optional:
        McDegrainSharp() on now fully progressive result. [also helps smooth and migrate detail into even result frames when returning singlerate (well thats the idea anyway)].

*/



If(vExt == ".d2v") {
    TFM(d2v=VideoFileName) # field match, fix Perverse Telecine + other crap
} Else {
    TFM()
}


return last

DBLRATE     = True            # Double Rate result.
DOTFM       = false           # Basic fix phase shifted
TFM_Mode    = 0
DS_RADT     = 2               # Spotless Radius
DS_THSAD    = 128*(8/2*8/2)   # noise + small spots on interlaced [255*(8/2*8/2) / 2 : /2 for interlaced field despot]
DS_PEL      = 1               #
CMB_CTHRESH = 9               # 9)
CMB_MI      = 80              # 80)
CMB_Chroma  = False           # False)
CMB_BlockX  = 16              # 16)
CMB_BlockY  = 16              # 16)
CMB_Metric  = 1               # 0)
MCD_FRAMES  = 1               #
MCD_CSHARP  = 0.3             #
Show        = True            # True to switch on subs etc
Stack       = True            # True to Show Source as well as result (If Show)

# Requires setting Field Order before calling.
PalHybridToProgressive(DBLRATE, DoTfm=DOTFM,TFM_mode=TFM_Mode, DS_RadT=1, DS_ThSAD=DS_THSAD,DS_Pel=DS_PEL,CMB_CThresh=CMB_CTHRESH,CMB_Mi=CMB_MI,CMB_Chroma=CMB_CHROMA,CMB_Metric=CMB_Metric,Mcd_Frames=MCD_FRAMES,Mcd_Csharp=MCD_CSHARP,Show=SHOW,Stack=STACK)

Return Last

Prefetch(4)
Return Last

Function PalHybridToProgressive(clip c, Bool "DblRate",bool "Chroma",
    \   Bool "DoTfm", Int "TFM_Mode",
    \   Int "DS_RadT", Int "DS_ThSAD", Int "DS_ThSAD2", Int "DS_Pel", Int "DS_BlkSz", Int "DS_TM", String "DS_Dc",        [* Spotless Args  *]
    \   Int "CMB_CThresh",Int "CMB_MI",Bool "CMB_Chroma", Int "CMB_BlockX",Int "CMB_BlockY",int "CMB_Metric",             [* IsCombedTIVTC  *]
    \   Int "MCD_Frames", Float "MCD_CSharp",                                                                             [* McDegrainSharp *]
    \   Bool "Show", Bool "Stack"                                                                                         [* Debug *]
    \ )
    {
    DblRate     = Default(DblRate, True)
    Chroma      = Default(Chroma,  True)                # Make False for Greyscale.
    #
    DoTfm       = Default(DoTfm,False)                  # False) True=Field matching. Basic fix for Phase Shifted (AKA Perverse) Telecine
    TFM_Mode    = Default(TFM_Mode,1)                   # 0) If DoTfm then TFM(Mode=TFM_Mode). If more exotic TFM() required, then set DoTfm=False & call Your Tfm() before this function.
    # SpotLess, Set frame radius DS_RADT 1 or more to switch on.
    DS_RadT     = Default(DS_RadT, 0)                   # 0) SpotLess temporal radius (spots on RadT consecutive frames)
    DS_ThSAD    = Default(DS_ThSAD,  255 * (8/2 * 8/2)) # 255 * (8/2 * 8/2) = 4080 = 1/4 of 8x8 block @ 255 or Maybe eg 24 * (8*8) = 1536 for general noise. ::: Spotless/MvTools  default = 10,000, about 156*(8*8)
    DS_ThSAD2   = Default(DS_ThSAD2, DS_ThSAD - ((DS_RadT-1)*(8*8)))  #     A bit less.
    DS_Pel      = Default(DS_Pel, 2)                    # 2), or max = 4
    DS_BlkSz    = Default(DS_BlkSz, 8)                  # 8) Maybe dont change
    DS_Tm       = Default(DS_Tm, False)                 # False). TrueMotion, some like true, some like False.
    DS_Dc       = Default(DS_Dc,"RemoveGrain(22).Blur(0.3)")

    #             CombedTIVTC args, Defaults, 1st after comment hash.
    CMB_CThresh = Default(CMB_CThresh, 8)               # 8), Area combing threshold [ ~ 8 -> 12].
                                                        # Valid settings are from -1 (every pixel will be detected as combed) to 255 (no pixel will be detected as combed).
                                                        # This is basically a pixel difference value. A good range is between 8 to 12.
    CMB_Mi      = Default(CMB_Mi, 80)                   # 80), Number of combed pixels inside any of the BlockX by BlockY size blocks on the frame, for the frame to be detected as combed.
    CMB_Chroma  = Default(CMB_Chroma,False)             #
    CMB_BlockX  = Default(CMB_BlockX, 16)               # 16), X-axis size of the window used during combed frame detection.
    CMB_BlockY  = Default(CMB_BlockY, 16)               # 16), Y-axis size of the window used during combed frame detection.
    CMB_Metric  = Default(CMB_Metric,1)                 #
                                                        #      So in any single block, if 85[MI] out of 256[BlockX x BlockY] pixels pass CThresh as combed, then is Interlaced.
    #
    MCD_Frames  = Default(MCD_Frames,   0)              # 0) Acts on both Deinterlaced and JohnFPS doubled Progressive frames. McDegrainSharp(frames=MCD_Frames), (0=Dont McDegrainSharp)
    MCD_CSharp  = Default(MCD_CSharp, 0.3)              # 0.3) McDegrainSharp() , defaults to 0.6 in McdegrainSharp.

    #
    Show        = Default(Show,False)                   # Show Titlebar and Deinterlaced Sub.
    Stack       = Default(Stack,True)                   # Shows Src as well as result [Ignored if Show=False]
    ###
    c
    Stack       = (Show&&Stack)
    Src         = Last
    myName      = "PalHybridToProgressive: "
    Assert(!c.Is420 || Height % 4 == 0,myName+"Interlaced Height MUST be mod 4")

    COMBDET = "IsCombedTIVTC(cthresh="+string(CMB_cthresh)+",MI="+String(CMB_MI)+",chroma="+String(CMB_Chroma)+",blockx="+String(CMB_blockx)+",blocky="+String(CMB_blocky)+",metric="+String(CMB_metric)+")"

    MatchedSrc = (DoTfm) ? Src.TFM(Mode=TFM_Mode,PP=0,Slow=1,mchroma=false,micMatching=0) : Src

    # Field based despotting, for both interlace and progressive [avoid inconstancy between interlaced and progressive]
    if(DS_RadT>0) {
        DeSpot_E = MatchedSrc.SeparateFields.SelectEven.SpotLess(Radt=DS_RadT,thsad=DS_ThSAD,thsad2=DS_ThSAD2,Pel=DS_Pel,chroma=Chroma,blksz=DS_BlkSz,tm=DS_Tm,glob=True,dc=MatchedSrc.SeparateFields.SelectEven.Eval(DS_Dc))
        DeSpot_O = MatchedSrc.SeparateFields.SelectOdd.SpotLess( Radt=DS_RadT,thsad=DS_ThSAD,thsad2=DS_ThSAD2,Pel=DS_Pel,chroma=Chroma,blksz=DS_BlkSz,tm=DS_Tm,glob=True,dc=MatchedSrc.SeparateFields.SelectOdd.Eval (DS_Dc))
        DeSpot = Interleave(DeSpot_E,DeSpot_O).Weave
    } Else { DeSpot = MatchedSrc }

#    Deinterlaced = DeSpot.YadifMod2(Order=-1,Mode=1,edeint=DeSpot.NNedi3(Field=-2, nns=2))    # Order -1 = Internal Parity.  Mode 1=double rate do spatial : Field-2=double rate internal parity,
    Deinterlaced = DeSpot.BWDif(Field=-2,edeint=DeSpot.NNedi3(Field=-2, nns=2))               # field -2=Double rate internal parity.
#    Deinterlaced = DeSpot.QTGMC(Preset="fast",NoisePreset="Fast",SourceMatch=2,Lossless=2,EZKeepGrain=0.4,Sharpness=0.1, tr2=0)

    Progressive  = Despot.JohnFPS()

    DoubleRateE = GConditionalFilter(MatchedSrc, Deinterlaced.SelectEven, Progressive.SelectEven, COMBDET,"=","true")
    DoubleRateO = GConditionalFilter(MatchedSrc, Deinterlaced.SelectOdd,  Progressive.SelectOdd,  COMBDET,"=","true")
    DoubleRate  = Interleave(DoubleRateE,DoubleRateO)
    Last = (MCD_Frames>0) ? DoubleRate.McDeGrainSharp(frames=MCD_Frames,csharp=MCD_CSharp,chroma=Chroma)  : DoubleRate        # McDegrainSharp on resultant doublerate Progressive

    if(SHOW) {
        Tit         = (!DblRate) ? "SingleRate" : "DoubleRate"
        SrcD        = Interleave(Src,Src)
        MatchedSrcD = Interleave(MatchedSrc,MatchedSrc)
        ResultBar_K = Last.BlankClip(height=20,Color=$000000).ScriptClip("""Subtitle(""""+"(P)"+Tit+"""")""")
        ResultBar_R = Last.BlankClip(height=20,Color=$FF0000).ScriptClip("""Subtitle(""""+"(I)"+Tit+"""")""")
        ResultBar   = GConditionalFilter(MatchedSrcD, ResultBar_R, ResultBar_K, COMBDET,"=","true")
        Last        = StackVertical(ResultBar,Last)
        if(Stack) {
            Tit = (!DblRate) ? "Src" : "Src (DblRate:Dupes)"
            if(DoTfm) {
                bpc    = Src.BitsPerComponent()
                Src8D  = (bpc==8)   ? SrcD        : SrcD.ConvertBits(8)
                MSrc8D = (bpc==8)   ? MatchedSrcD : MatchedSrcD.ConvertBits(8)
                LftWin_A  = StackVertical(Last.BlankClip(height=20).ScriptClip("""Subtitle(""""+Tit+"""")"""),Src8D)
                WW_L = (LftWin_A.Width / 4) * 2
                WW_R = LftWin_A.Width - WW_L
                LftWin_BL = LftWin_A.Crop(0,0,WW_L,0)
                LftWin_BR = StackVertical(MSrc8D.BlankClip(Width=WW_R,height=20,Color=$0000FF).Subtitle("TFM"),MSrc8D.Crop(WW_R,0,0,0))
                LftWin_B  = StackHorizontal(LftWin_BL,LftWin_BR)
                LftWin    = LftWin_A.GConditionalFilter(LftWin_A,LftWin_B,"LumaDifference(Src8D,MSrc8D)","=","0.0",Args="LftWin_B,Src8D,MSrc8D",Local=True)
                #SSS="return (LumaDifference(Src8D,MSrc8D) == 0.0) ? Last : LftWin_B"
                #LftWin = LftWin_A.GScriptClip(SSS ,Args="LftWin_B,Src8D,MSrc8D",Local=True)
            } Else {
                LftBar = Last.BlankClip(height=20,Color=$000000).ScriptClip("""Subtitle(""""+Tit+"""")""")
                LftWin=StackVertical(LftBar,SrcD)
            }
            StackHorizontal(LftWin,Last)
        }
    }
    Return ((DblRate) ? Last : Last.SelectEven).AudioDubEx(Src)
}






Function GetFilenameExtension(String fn) {
    fn=fn.RevStr    i1=fn.FindStr("\")  i2=fn.FindStr("/")
    i=(i1==0) ? i2 : (i2==0) ? i1 : min(i1,i2)
    fn=(i>0) ? Fn.LeftStr(i-1) : fn
    i=fn.FindStr(".")
    Return (i>0)?fn.LeftStr(i).RevStr:""
}

# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
    c.BlankClip(height=20,Color=Default(Col,0))
    (Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Trim(0,-1).Subtitle(Tit)
    Return StackVertical(c).AudioDubEx(c)
}

EDIT: Above EDITED

EDIT: qCombed() v1.03, mod of IsCombed (c)Donald Graft.:- https://forum.doom9.org/showthread.p...95#post1776495

[flossy_cake]

I couldn't get DG's IsCombed() working on my system, it always returns false regardless of threshold param. But your qCombed mod works -- cheers. I'll have to play around with it for a few hours to see what it thinks of various content. Preliminary results are that it is similar to IsCombedTIVTC but with a lower threshold.

Honestly though I'm a bit frustrated as I can't seem to get a grip on how NTSC soft telecine actually works. For instance, do the repeat flags mean repeat FIELD or repeat FRAME? Is the stream really 24p with repeat flags, or is it the same 30i stream but with duplicate fields simply containing no picture data and just a "repeat me" flag. One source filter behaves different to another even with the same "repeat" param, so I can't even do experiments to get enough data to make a reliable inference.

Anyway, I decided to dig out my old Sony DVD player to see how it handles it. The test video was S03E01 of NTSC Caroline in the City (from which that laundry scene was taken). The rest of that season is hard telecined, so I can easily compare how the DVD player handles both types.

The hard telecined episodes are motion compensated deinterlaced (same as bwdif with threshold set to a value of around 2-3). So static stuff gets weave, anything that moves drops to 240p. Therefore on hard telecine content, TFM is producing superior results compared to my DVD player, and this is highly satisfying. The only issue with TFM is that because some scenes begin on a combed frame, the first field of the scene is only a half image without any other field to match it to, so it gets dropped, resulting in a repeated frame at the beginning of some scene changes. This is actually noticeable and kind of annoying, but I think still preferable to 240p on motion. I've tried to get TDecimate to produce a blend on such frames but couldn't get it to work reliably. Played around with the scene threshold params too but couldn't find the magic setting.

As for the soft telecined episodes, those seem to be handled better by the DVD player: progressive frames are presented unaltered, while any combing on scene changes is interpolated. And it seems to get this perfect -- never interpolates when it shouldnt, never weaves when it shouldn't. The final output format of the DVD player is p60 though, so 3:2 judder. But all frames are progressive and contain maximum resolution available at all times. I'm not able to achieve this with TFM or TDecimate unfortunately, so far.

Seems to be an issue with the source filter. I think the source filter needs to signal repeat FIELDS correctly so that downstream knows which exact fields need to be interpolated. I can't seem to get this happening, so for now I'm just forcing the soft telecined episodes to hard telecined by setting "obey repeat flag" to true in the source filter, and letting TFM and TDecimate convert it back to 24p again.

I'm not keen on adding a lot of extra post processing like additional decombing or deint passes and such, as those can degrade the image and I'm yet to find a combing detection filter that is reliable enough to avoid false positives and therefore cause unnecessary drops to 240p (or conversely, failing to detect combing).

So I keep coming back to this idea that the source filter needs to flag FIELDS as needing interpolation, and the downstream processor/renderer needs to use that to decide when to interpolate FIELDS. And, I need this to be done in realtime, just like with a DVD player. This seems possible in theory, but in practice?

[StainlessS]

do the repeat flags mean repeat FIELD or repeat FRAME?

I believe only fields are repeated (but I live in pal land and rarely have NTSC 'stuff'.)

So I keep coming back to this idea that the source filter needs to flag FIELDS as needing interpolation, and the downstream processor/renderer needs to use that to decide when to interpolate FIELDS.

I think that is why its best to use

d2v = "..."
Mpeg2Source(d2v)
TFM(d2v=d2v,...) # maybe PP = 0 for no post processing

[tormento]

As I am getting mad at Cowboy Bebop, I need your help as expert "defielders".

There have been different attempts in the past but nowadays we have a lot more of powerful plugins, so I decided to give it a personal run.

The video is an "intentional" mix of 23.976p (animations, the 80% of the video), 29.97p (pannings) and 60i (CGI), all soft tagged as 29.97p. The author tells us that that was the best way to preserve the "feeling" and "crispness" of the anime.

I'd like to have a uniform fps, possibly a standard one, such as 23.976p being the most used there, without sacrificing quality too much (CGI has different images at every 60i frame).

Can you, please, help me with a proper script to deal with it?

Here is a sample.

[PoeBear]

LWLibavVideoSource("Bebop.m2ts")
TFM().TDecimate()

Unless you're going to VFR or comb through them by hand and do something special with the 29.97/59.94 scenes, that's all you really can do. Your output would match how the US Blu-ray was done