Inverse Telecine NTSC Simpsons DVDs

[edumj]

Hi,

I’m trying to deinterlace (IVTC) these DVDs, but the 2:3 pattern (2 interlaced and 3 progressive frames) it’s only at the begining of the video, later it doesn’t stop changing from to 3 or 4 interlaced frames, followed by 4 or 5 progresive frames (or maybe static interlaced frames), and sometimes 5 or 6 different progressive frames. Maybe some speed changing telecine??

Here’s the beginning of the episode I’m working with (vob demuxed and indexed):
https://mega.nz/#!KUtgDZpB!d9bKTHFil...FpxbD9XDY6Bwi4

I’ve tried TFM+TDecimate and Telecide+Decimate, but they (the decimates) always delete 1 of 5 frames no matter if they are repeated or not, and I can’t find a way to delete 1 of 5 frames but only if they are repeated, and keeping in sync by deleting the next repeated frames it finds.

How can I do this automatically with deleteframe()?

I’m also trying to inverse telecine and delete repeated frames with manually counting how many fields correspond to every original film frame. I do it with with SeparateFields() and writing every changing frame, so I get the pulldown pattern: 2:2:2:3 2:2:2:3 2 and 4:5:4:5:4:5 5:4:5:4:5:4 etc...
I made a script to delete:
· 1 field when the number of fields is 3 (to keep 1 full frame and eliminate the repeated field which causes interlacing),
· 1 field (or 3 to keep in sync) when the number is 5 (2 and a half frames)
· 0 fields (or 2 to keep in sync) when the number is 4 (2 repeated frames)
· 0 fields when then number is 2 (because no repeated fields). And it works!

How can I do this automatically?

I’m trying to write one txt file with these pulldown cadence changes but I don’t know enough about avisynth conditional functions to make it work correcty. I can post my (messy) script.

I do SeparateFields(), and select even fields and odd fields separately, and with YDifferenceToNext it detects (more or less) when a frame changes (when the next top or bottom field will change, so there are no more repeated fields) so I must write the number of the counter variable (the number of fields that 1 frame represents) in a txt, and reinitialize. I’m trying to make the 2 YDifferenceToNext at a time to make only one txt file, but it should “advance” 2 frames when it finds a repeated field to work the right way... Does anyone understands it??



PD: Why it's so hard to correctly answer "Random Questions"??

[real.finder]

it's has even 60i, use some bob and see

[edumj]

Originally Posted by real.finder

it's has even 60i, use some bob and see

Well, it's only when the Simpsons logo fades with the garage, and it's quite well deinterlaced just with TFM (better than Telecide).

The rest seems like speeding up (too much different consecutive frames) and down (too much repeated frames).

[edumj]

This is my script, I 've translated the most part to english:

v=MPEG2Source("VTS_03_1_HonorFlags+Demux.d2v").trim(190,-10)

global Counter=0
global Counter2=0

global RepeatedField=false
global WriteTXT=false

Media="50%+25%+25%="
abre_parentesis=" ("
cierra_parentesis=")"
YY="Y="
UU="U="
VV="V="


function SeRepiteTop()
{
global RepeatedField = (diff_top < 1.0) ? true : false

	# If repeated TOP Field, increase counter
	global Counter=   RepeatedField==true ? Counter+1 : Counter

	# If TOP Field moving, WriteTXT=true
	global WriteTXT=   RepeatedField==false ? true : false
	
}

function SeRepiteBottom()
{
global RepeatedField = (diff_bottom < 1.0) ? true : false

	# If repeated BOTTOM Field, increase counter
	global Counter = RepeatedField==true ? Counter+1 : Counter
	
	# If TOP Field moving, WriteTXT=true, else keep looking for more duplicated TOP (and BOTTOM) fields, until the image moves
	global WriteTXT=   RepeatedField==false ? true : WriteTXT
	
	# If WriteTXT=true add 2 to counter, and save to a different variable to Initialize Counter
	global Counter2= WriteTXT==true ? Counter+2 : Counter2
	# Initialize Counter to zero
	global Counter= WriteTXT==true ? 0 : Counter

	
}

function CombingInfo(clip c, clip d)
{
#filec = "campos_repes_top_y_bottom.log"
filed = "campos_repes_top_y_bottom.log"
global top = c
global bottom = d

c= ScriptClip(c,"Poner_Nombre(Media+string(diff_top)+abre_parentesis+string(WriteTXT)+cierra_parentesis +string(Counter2) ,5)")
c= ScriptClip(c,"Poner_Nombre(YY+string(Ydiff_top) ,1)")
c= ScriptClip(c,"Poner_Nombre(UU+string(Udiff_top) ,2)")
c= ScriptClip(c,"Poner_Nombre(VV+string(Vdiff_top) ,3)")
d= ScriptClip(d,"Poner_Nombre(Media+string(diff_bottom)+abre_parentesis+string(WriteTXT)+cierra_parentesis +string(Counter2) ,5)")

#c = WriteFileIf(c, filec, "WriteTXT", """ "frame " """, "current_frame", """ "=" """, "Counter2", """ " fields " """)
c = FrameEvaluate(c, "SeRepiteTop")
c = FrameEvaluate(c,"global diff_top = 0.50*YDifferenceToNext(top) + 0.25*UDifferenceToNext(top) + 0.25*VDifferenceToNext(top)")
c = FrameEvaluate(c,"global Ydiff_top = YDifferenceToNext(top)")
c = FrameEvaluate(c,"global Udiff_top = UDifferenceToNext(top)")
c = FrameEvaluate(c,"global Vdiff_top = VDifferenceToNext(top)")


d = WriteFileIf(d, filed, "WriteTXT", """ "frame " """, "current_frame", """ "=" """, "Counter2", """ " fields" """)
d = FrameEvaluate(d, "SeRepiteBottom")
d = FrameEvaluate(d,"global diff_bottom = 0.50*YDifferenceToNext(bottom) + 0.25*UDifferenceToNext(bottom) + 0.25*VDifferenceToNext(bottom)")

# Initialize??
#global WriteTXT=false
	
return stackvertical (c,d)
}



	# Función para poner texto, y posición?? o número de línea (multiplicar por 15 o algo, para simular ir bajando líneas)
function Poner_Nombre(clip "c", "nombre", int "linea") {
	linea = Default (linea, 1)*18
	c = Default (c, last)
	c=c.Subtitle(nombre, x=8, y=linea, font="Arial", size=18.0, text_color=$11FFFF00, halo_color=$000000, align=4, spc=0, font_width=0, font_angle=0.0, interlaced=false)
	return c
}


CombingInfo(v.SeparateFields().SelectOdd(), v.SeparateFields().SelectEven())

The resulting .txt is:

frame 0=2 fields
frame 1=3 fields
frame 2=2 fields
frame 3=3 fields
frame 4=2 fields
frame 5=2 fields
frame 6=3 fields
frame 7=2 fields
frame 8=3 fields

What it is 2:3:2:3 2 2:3:2:3 (22 fields)

But It Should be:

frame 0=2 fields
frame 1=3 fields
frame 2=2 fields
frame 3=3 fields
frame 5=2 fields
frame 6=3 fields
frame 7=2 fields
frame 8=3 fields

So, it's 2:3:2:3 2:3:2:3 (20 fields)

[manono]

Originally Posted by edumj

How can I do this automatically with deleteframe()?

You can't. Multidecimate and DeDup can remove dupe frames.

The intros to animations are often different from the main body of the episodes. They're often edited as video with no concern for us trying to make sense of it all later on. For DVD and television broadcast it only has to output interlaced 29.97fps (or progressive 59.94fps). If the intent is to make it all progressive 23.976fps and without losing audio synch, you might have to remove unique frames. Both Decomb and TIVTC have blend modes (hybrid mode 1 in TIVTC) for turning the 29.97p parts to progressive 23.976fps, if you want to try.

[edumj]

Originally Posted by manono

You can't. Multidecimate and DeDup can remove dupe frames.

Well, I meant to automatically remove 1 of every 5 frames, but only if it is duplicated. And, if it has not deleted 1/5 of current frames, then start removing the next duplicated frames it founds, until the number of removed frames is 1/5 of current frame.

I think Multidecimate still deletes 1 of 5 even if it's not duplicated, and DeDup creates VFR.

I can do this (but with my other function, I didn't post, to delete duplicated fields) counting every field I delete, etc... and should be something like this with frames:

global frames_should_be_deleted = (current_frame)/5

global frames_still_to_delete = frames_should_be_deleted - deleted_frames

And if it's "frames_still_to_delete > 0" I delete 3 fields instead of only 1, etc...
But I need to know the changing patterns and the script I posted doesn't work well...

Originally Posted by manono

The intros to animations are often different from the main body of the episodes. They're often edited as video with no concern for us trying to make sense of it all later on. For DVD and television broadcast it only has to output interlaced 29.97fps (or progressive 59.94fps). If the intent is to make it all progressive 23.976fps and without losing audio synch, you might have to remove unique frames. Both Decomb and TIVTC have blend modes (hybrid mode 1 in TIVTC) for turning the 29.97p parts to progressive 23.976fps, if you want to try.

Well, if you see frame by frame the zooming Simpsons logo you'll see only a few "doubled combed"?? frames... but I could try that blending modes even it's seems ok just with TFM (but needs decimation).

[manono]

Originally Posted by edumj

I think Multidecimate still deletes 1 of 5 even if it's not duplicated...

It can be set to decimate in a couple of ways From the included doc:

It can remove all the duplicate frames from a clip, or it can remove N out of every M frames, where N and M can be selected by the user, removing the frames most similar to their predecessors. Special modes are available to protect static scenes from decimation.

For example, if 1 in 5 is no good (and it isn't a lot of the time), then set it to remove 5 in 25 or 10 in 50, whatever you like. I don't work on this show, but isn't the main body of the episodes more 'regular' or easier to work with? Or use that blend mode in TDecimate to blend frames when a sequence is greater than 23.976fps.

[edumj]

With Multidecimate it's better, but still leaves some duplicated frames and deletes some unique frames, even setting 10 of every 50 frames.

I've changed a bit the script I posted, and now it seems to work. One error was the field order:

CombingInfo(v.SeparateFields().SelectOdd(), v.SeparateFields().SelectEven())

It should be:

CombingInfo(v.SeparateFields().SelectEven(), v.SeparateFields().SelectOdd())

Because the top field is the even filed, and 0 is the first field, and must be even...

And it seems to only work well with DifferenceToNext and not with DifferenceFromPrevious, I still don´t know why.

But, I don't know what filters to use to better detect duplicated fields. Some spatial or temporal filter to clean bad compressed flat areas?

I'm testing dfttest, TTempsmoothF, TemporalSoften and MipSmooth, but still some fields have much difference when they are the same, because some flickering or shacking video? After filtering I have to lower the threshold to detect duplicated, but then some little movements are not detected as unique fields.

Is there some way to "enhance" real movements, and stabilyze flickering (background lines or far and very detailed objects moves a little in each field) to better detect duplicated fields? Maybe Sharp edges or drawing lines?

EDIT: I also tried blurring or downscaling to avoid flicker, but then some little movements (like the clouds at the beginning) are detected as duplicated.

[StainlessS]

detect duplicated fields

RT_Stats:- http://forum.doom9.org/showthread.php?t=165479

RT_LumaDifference(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
    int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

Delta2=1 for dif to next, -1 for dif to prev.
y=0, and y2=1 to difference fields of the same frame (or whatever frames specified by delta and delta2).
EDIT: Interlaced=true arg compares fields rather than frames, I later used AltScan arg name in other plugs
to mean the same, ie Alternate ScanLines.

EDIT: some additional text describing family of functios

*****************************************************
********** RUNTIME CLIP COMPARISON FUNCTIONS ********
*****************************************************

 The compiletime/runtime clip functions share some common characteristics.
 The 'n' and (where used) 'n2' args are the optional frame numbers and default to 'current_frame' if not specified.
 The x,y,w,h, coords specify the source rectangle under scrutiny and are specified as for Crop(), the default 0,0,0,0 is full frame.
 If 'interlaced' is true, then every other line is ommited from the scan, so if eg x=0,y=1, then scanlines 1,3,5,7 etc are scanned,
 if eg x=0,y=4 then scanlines 4,6,8,10 etc are scanned. The 'h' coord specifies the full height scan ie same whether interlaced is true
 or false, although it will not matter if the 'h' coord is specified as eg odd or even, internally the height 'h' is reduced by 1 when
 interlaced=true and 'h' is even.
 Some of the functions have 'x2' and 'y2' args for a second frame (which could be the same frame).
 Note, 'x2' and 'y2' default to 'x' and 'y' respectively.


 RT_YDifference(clip,int "n"=current_frame,int "delta"=1,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
      bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between frame n area x,y,w,h, and frame (n+delta) area x2,y2,w,h.
  Note, by default it will be equivalent to YDifferenceToNext as delta defaults to 1 and x,y,w,h defaults full frame.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

  Eg, RT_YDifference(clip,delta=0,y2=1,interlaced=true)
    Would difference the even and odd lines of the same frame,

Also, dont know if this of any use (not for interlaced, need Bob):- http://forum.doom9.org/showthread.php?t=171339

ApparentFPS: by StainlessS.

    Plugin dll's for Avisynth versions v2.58 and v2.6.

    Shows underlying framerate where a clip has had many duplicates inserted, easier than counting unique frames.

    ApparentFPS(clip c,Float "DupeThresh"=0.5,Float "FrameRate"=clp.FrameRate,Int "Samples"=Int(Ceil(FrameRate)),
        \ Float "ChromaWeight"=1.0/3.0,String "Prefix"="", Bool "Show"=True,Bool "Verbose"=True, Bool "Debug"=False,
        \ Int "Mode"=0,
        \ Int "Matrix"=(c.Width>1100||c.Height>600)?3:2,    # 2=Pc.601 : 3=PC.709
        \ Int "BlkW"=64, Int "BlkH"=BlkW, Int "oLapX"=BlkW/2, Int "oLapY"=BlkH/2
        \)

    DupeThresh,   Float, Default 0.5 (Greater than 0.0), suggest 0.25 -> 1.0. FrameDifference below or equal to this is a dupe.
    FrameRate,    Float, Default clp.FrameRate.               # FrameRate of clp. Probably of no great use, just use default.
    Samples,      Int,   Default Int(Ceil(FrameRate))         # (1 or more). Min Frames to sample for underlying framerate detection. (>= FrameRate).
    ChromaWeight, Float, Default 1.0/3.0, (0.0 -> 1.0).       # Suggest 1.0/3.0 -> 1.0/2.0. (Use 0.0 for GreyScale YUV).
    Prefix,       String, Default "".                         # Prefix for returned Local vars, Default "" = None returned.
    Show,         Bool, Default True.                         # Show Info
    Verbose,      Bool, Default True.                         # Additional Info, Sequence as binary digits.
    Debug,        Bool, Default False.                        # Not of any great use.
    ### Added v1.5
    Mode          Int, default 0. (0 -> 1).
                    0) Standard RT_FrameDifference mode.
                    1) FrameMovement mode, difference is greatest difference for any BlkWxBlkH block.
    Matrix,       Default (c.Width>1100||c.Height>600)?3:2.     2)=PC.601 : 3=Pc.709. Used in Conversion of RGB to Luma-Y.
    BlkW          Int, Default 64.   (8 < BlkW, Even Only).     Block Width for Mode 1 (not used for Mode=0).
    BlkH          Int, Default BlkW. (8 < BlkH, Even Only).     Block Height for Mode 1 (not used for Mode=0).
    oLapX         Int, Default BlkW/2. (0 <= oLapX).            Horizontal block Overlap for Mode = 1.
    oLapY         Int, Default BlkH/2. (0 <= oLapY).            Vertical block Overlap for Mode = 1.
    ###

    Function samples over Samples frames, and shows instantaneous ApparentFPS and MaxApparentFPS, the real and perhaps most useful
    result is MaxApparentFPS.

    If Prefix = "" (Default) then does not return any info Local vars. When set to eg "P_" will set Local P_ApparentFPS
       and P_MaxApparentFPS as Float.
    Also returns Locals P_MinAboveDupeDif, P_MaxBelowDupeDif and P_CurrentDif which may assist in choosing DupeThresh, they are min and
    max values so far, and difference between current and previous frame. P_MinAboveDupeDif is the difference to previous frame of the
    frame which has lowest difference that what considered NOT a dupe (above DupeThresh), MaxBelowDupeDif max difference of frame that WAS
    considered a dupe (below DupeThresh). Ideally in a clip containing duplicates, there would be a distinct gap between these two
    above/below dup dif values with DupeThresh somewhere in between.

    !!! NOTE !!!, It may be better to set DupeThresh a little too high rather than too low, (where some motion frames will be mistaken as
    dupes), as it is likely that there will be at least one sequence in clip where there is sufficient motion to accurately set
    MaxApparentFPS.

    Can switch Off metrics (Show=False), and set eg Prefix = "P_" to return Locals P_ApparentFPS and P_MaxApparentFPS with rough
    instantaneous estimate of current underlying framerate and maximum underlying framerate detected so far.
    NOTE, ApparentFPS can range from 0.0 in static scene (can be measure of how 'active' a scene is).
    'Verbose' (Default True) shows additional info, a string of 1's and 0's showing duplicate and unique frames. Also shown in info is
    the Unique frame count for the current Sample spread (which will be reduced at either end of clip), current ApparentFPS is calculated as
    (FrameRate * UniqueFrameCount / CurrentSampleSpread), so where UniqueFrameCount == CurrentSampleSpread, the result is FrameRate, However,
    if CurrentSampleSpread is reduced due to being near clip ends, then current ApparentFPS will NOT be assigned to P_MaxApparentFPS even
    where greater than current P_MaxApparentFPS.

    The rest should be pretty straight forward.
    
    v1.01, Extended maximum samples from 256 to 1024 (silently limited to 1024).
        Maximum number of binary digit flags limited to 64 when Verbose=True.
        Max number of Debug Hex Values limited to 8 (ie 8 * 32 bits = 256 bit flags).
        
    v1.03, Added 'UniqMax' Verbose metrics indicator, maximum number of unique frames in Sample range, so far.

    NOTE, a clip that has been edited after dupe insertion may seem to jump up in MaxApparentFPS at edits, due to dupes being cut out.
       You can reduce this effect by setting eg Samples=Int(Ceil(FrameRate))*10, but it will not show any MaxApparentFPS until you have
       scanned forward Samples/2 frames (Samples are centered around current frame). It will also take a little time to start up as it
       has to sample a lot of frames at startup. NOTE, Samples is silently (no error) limited to 1024.
        
    v1.05.
       Added Some args, Matrix and Mode/blk args.
       Mode=1, ie RT_FrameMovement mode, measures frame differences using a Block mode, chops the frame into BlkWxBlkH blocks, and difference
       is the maximum difference for any one block. Blocks can also be OverLapping, by default block sizes / 2, so by default each frame
       is multiply sampled 4 times (ie is slower than Mode=0, but can set oLapX and oLapY to 0, for no overlaps).
       Mode=1, might be of use where there is very little movement in a sequence, but for most uses, probably best left at default Mode=0.
       Also Note, in v1.05, ChromaWeight also alters differencing method used for RGB. Where ChromaWeight=0.0, will use LumaDifferencing,
       and where ChromaWeight > 0.0, then will use (AveRedDif+AveGrnDif+AveBluDif)/3.0, for both Mode=0 and Mode=1.

EDIT: What it looks like (center digit 0 is a little brighter than others representing current_frame):-

[edumj]

Thanks, but I have no idea how to make this work...





It shows differences between frames like Y(U/V)DifferenceToNext?

I use DifferenceToNext with even and odd fields, but without cleaning and maybe stabilizing before, it's not very precise detecting duplicated frames.
And I don't know if I should make it Grayscale to ignore bad compressed flat areas?

[StainlessS]

It shows differences between frames like Y(U/V)DifferenceToNext?

FrameDifference does cope with Y/U/V, just use same clip for c and c2, and eg n2=current_frame+1 or n2=current_frame-1 for next or prev frame.
Where ChromaWeight > 0.0, returns Y/U/V difference combined, not separate results.

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0)
    Returns difference between clip c frame n and clip c2 frame n2 (0.0 -> 255.0).
    ChromaWeight range 0.0 -> 1.0.
    For RGB, returns same as RGBDifference.
    For Y8 returns same as LumaDifference.
    For non Y8 Planar returns Lumadif + Chromadif with chroma weighted by ChromaWeight arg.
      Returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).

LumaDifference can use same clip for c and c2, (or different clips).
You can use n, n2, Delta, Delta2, y, y2 and interlaced to choose fields or frames, with any offset (not just next or prev), and
without having to do the SeparateFields/SelectEven/SelectOdd stuff.

I use DifferenceToNext with even and odd fields

Assuming not field separated

  CurEvenToCurOdd  = RT_LumaDifference(c,c,y2=1,Interlaced=True)
  CurEvenToNxtEven = RT_LumaDifference(c,c,Delta2=1,Interlaced=True)                            # Same as below line
  CurEvenToNxtEven = RT_LumaDifference(c,c,n2=current_frame+1,Interlaced=True)                  # Same as above line
  CurEvenToNxtEven = RT_LumaDifference(c,c,n=current_frame,n2=current_frame+1,Interlaced=True)  # Same as above line
  CurEvenToNxtOdd  = RT_LumaDifference(c,c,Delta2=1,y2=1,Interlaced=True)
  CurOddToCurEven  = RT_LumaDifference(c,c,y=1,y2=0,Interlaced=True)        	# Same as CurEvenToCurOdd
  CurOddToNxtEven  = RT_LumaDifference(c,c,y=1,Delta2=1,y2=0,Interlaced=True)
  CurOddToNxtOdd   = RT_LumaDifference(c,c,y=1,Delta2=1,Interlaced=True)

  You can mix and match depending upon what is easiest (Delta can also be -1 for prev frame or +2 for frame after next etc).

  Delta is relative to n (n by default is current_frame)
  Delta2 is relative to n2 (n2 by default is current_frame)

  If you dont find easier for some job then dont use.

[Katie Boundary]

I think edumj is trying to create variable-framerate output, decimating the natively film content back to 24 hz while leaving the natively NTSC stuff at 30 or 60 hz. AVIsynth can't do that. You're best off bobbing everything to 60 Hz or learning how to make MP4 files from multiple AVIsynth scripts with different framerates.

[hello_hello]

Originally Posted by Katie Boundary

I think edumj is trying to create variable-framerate output, decimating the natively film content back to 24 hz while leaving the natively NTSC stuff at 30 or 60 hz. AVIsynth can't do that. You're best off bobbing everything to 60 Hz or learning how to make MP4 files from multiple AVIsynth scripts with different framerates.

Avisynth itself is constant frame rate but that doesn't prevent TIVTC from having a variable frame rate mode, and with a timecodes file x264 is capable of variable frame rate encoding and the Avisynth output framerate becomes irrelevant.

TIVTC effectively only understands how to output combinations of film and NTSC in variable frame rate mode, but I'm pretty sure Dedup or ExactDedup can delete duplicate frames in no particular pattern and create a timecodes file for encoding.

edumj,
I haven't read through the whole thread but have you tried TIVTC in 2 pass mode, even if you're after a constant frame rate output? It's usually pretty good in 2 pass mode.

Something like this creates the metrics files:

TFM(Output="D:\TFM.txt")
TDecimate(mode=4, Output="D:\TDecimate.txt")

and this does the encoding, although hybrid mode isn't supported in mode 2 so you can't use it for combinations of film and 29.970. You can specify any frame rate though.

TFM(Input="D:\TFM.txt")
TDecimate(mode=2, Input="D:\TDecimate.txt", TFMIn="D:\TFM.txt", rate=23.976)

[edumj]

Originally Posted by hello_hello

edumj,
I haven't read through the whole thread but have you tried TIVTC in 2 pass mode, even if you're after a constant frame rate output? It's usually pretty good in 2 pass mode.

Hi, it also drops some frames when more than 5 unique frames in a row.

Originally Posted by Katie Boundary

I think edumj is trying to create variable-framerate output, decimating the natively film content back to 24 hz while leaving the natively NTSC stuff at 30 or 60 hz.

Hi Katie, what I'm trying to do is to restore the suposed original 23.976 fps before it was telecined. I've seen the HD versions of The Simpsons, and they are 23.976 fps, but I've seen also dropped frames in this same episode (I think they were the same dropped frames than if I decimate 1 or 5!).

I can't find an Inverse Telecine filter which decimates at the same time as matching fields, even then it must leave the video as VFR.

I made a script some years ago, to delete frames while detelecining (for some Linkin Park interlaced videos), using SelectEvery. It was something like this:

deinterlaced= frame==1 ? deinterlaced.SelectEvery(10, 0,1, 2,3, 5,6, 7,8).Weave() :\
		frame==2 ? deinterlaced.SelectEvery(10, 0,1, 3,4, 5,6, 8,9).Weave() :\
		frame==3 ? deinterlaced.SelectEvery(10, 1,2, 3,4, 6,7, 8,9).Weave() :\
		frame==4 ? deinterlaced.SelectEvery(10, 1,2, 4,5, 6,7, 9,10).Weave() :\
		frame==5 ? deinterlaced.SelectEvery(10, 0,1, 2,3, 4,5, 7,8).Weave() :last

And I had to look frame by frame, looking for the 2:3 patterns, (which changes every scene change) and writing down what was the first type of frame (1 to 5) at the begining of the scene change, and making lots of trims!

Now, I've made a script with 6 videos (original video, and the 5 possibilities), to easly detect every scene first frame type, with IsCombedTIVTC and pointrezising to better see the combed lines).

But all this only works with regular 2:3 pattern (very few scenes of the Simpsons are "regular" ).

I didn't try TFM then (Linkin Park videos), but for this strange pulldown patterns I found in The Simpsons, my idea is that the only way to restore the original 23.976 fps without deleting unique frames (appart from the 60 fps "simPsons" fading with the garage, and the inital credits which fades in every field) is to delete duplicated fields when they are after more tan 4 unique frames, trying to keep in sync.

I supose the Simpsons were made at 12 fps (like most cartoons), but camera movements should be at a maximun of 24 fps (I don't think they do the camera pans in video!?) so the original fps must be 24 (also like most cartoons). While the 60i parts comes from edit in video, like fades to black, fading one scene inside another (P->garage or Bart throw the school window and writing in the board).
So the only explanation I see is that telecining was made at a variable framerate, so "accelerating" slow parts (deleting duplicates when needed) should be ok.

But, to know which fields I must delete, I must first know where are the duplicated fields (or frames if I use TFM), and I'm trying to make a script to write in a txt file how many fields corresponds to every unique frame, like the pulldown patterns, but instead of 2:3:2:3 it's something like:

2
3
2
3
5
4
5
4
5
4
...

But it must detect real duplicated fields (even and odd separately) and I still haven't found a perfect filters combination to clean fields coming from combed frames (to be detected as duplicate) and to avoid "line flickering"? which are not detected as duplicate.

I will try what StainlessS says, if I can make it work whith my the FrameEvaluate's in my script, but right now I'm with:

Grayscale().ColorYUV(cont_y=40, off_u=0, off_v=0).MosquitoNR.Convolution3D (0,255, 255, 16, 64, 10, 0)	.converttoyuy2().SpatialSoften(7,20,0).ConvertToYV12()	.TemporalSoften(3, 24, 0, scenechange=15, mode=2)

Any better filter method??

[Katie Boundary]

Originally Posted by edumj

Hi Katie, what I'm trying to do is to restore the suposed original 23.976 fps before it was telecined.

While the main body of a Simpsons episode might be animated at 24 FPS, the opening credits are not. Parts of the opening credits are native NTSC, and you'll never get a clean IVTC out of that.

Originally Posted by edumj

I've seen the HD versions of The Simpsons, and they are 23.976 fps, but I've seen also dropped frames in this same episode

The Simpsons was originally made in SD. Any HD copy of it that you find will either (a) feature completely redone animation, or (b) be missing frames. The Blu-Ray copies of Star Trek: the Next Generation, for example, had to have the special effects completely re-done because the original special effects were done in native NTSC.

Originally Posted by edumj

I can't find an Inverse Telecine filter which decimates at the same time as matching fields

Those are two things that you shouldn't want to do at the same time. You should want to match fields first, then decimate.

Originally Posted by edumj

I supose the Simpsons were made at 12 fps (like most cartoons), but camera movements should be at a maximun of 24 fps (I don't think they do the camera pans in video!?) so the original fps must be 24 (also like most cartoons). While the 60i parts comes from edit in video, like fades to black, fading one scene inside another (P->garage or Bart throw the school window and writing in the board).

It sounds like the show was animated at 24 frames per second, then telecined and transferred to tape, then edited on tape. This was pretty common for anything made between 1980 and 2000. It also leaves a lot of orphaned fields and disruptions in the pulldown pattern. Best advice: don't worry about the pulldown pattern and don't worry about deleting duplicate frames. Humpty-dumpty cannot be put back together again. Settle for 30 frames per second with occasional duplicates.

[hello_hello]

Originally Posted by edumj

Hi, it also drops some frames when more than 5 unique frames in a row.

Did you try increasing the maxndl option? I've found it works quite well, at least for video. I rarely encode animation..

A little snippet from the help file.

While maxndl stands for "max non-duplicate length", it is actually more like a trade off between maintaining video sync and producing a smooth result. The larger maxndl is the more the decimation can be non-uniformly spread throughout the video, which helps the smoothness of the result in cases where duplicates are not evenly spread.

[edumj]

Originally Posted by Katie Boundary

While the main body of a Simpsons episode might be animated at 24 FPS, the opening credits are not. Parts of the opening credits are native NTSC, and you'll never get a clean IVTC out of that.

I know, but there will be only a few missing or deinterlaced frames.

Originally Posted by Katie Boundary

The Simpsons was originally made in SD. Any HD copy of it that you find will either (a) feature completely redone animation, or (b) be missing frames. The Blu-Ray copies of Star Trek: the Next Generation, for example, had to have the special effects completely re-done because the original special effects were done in native NTSC.

Yes? Then they could just encode it interlaced, like with Columbo BluRays (which has no fx!). I've also seen some tv series with FX that looks made in video too.

Originally Posted by Katie Boundary

Those are two things that you shouldn't want to do at the same time. You should want to match fields first, then decimate.

Why? For me it seems more natural to just discard the 2 repeated fields coming from combed frames, which are dirty (for chroma subsampling or whatever) rather than matching fields and then expect that decimate deletes the right (less ugly) frame. When I was testing TFM I think it sometines left the dirtiest frame.

With this regular 2:3:2:3 pattern:

At Bt Bt Ct Dt
Ab Bb Cb Db Db

It's ok to do:

SelectEvery(10, 0,1, 2,3, 5,6, 8,9).Weave()

So I delete the duplicated B Top field and D Bottom field.

But whith other patterns it's hard to know which is the cleaner field...

Originally Posted by Katie Boundary

It sounds like the show was animated at 24 frames per second, then telecined and transferred to tape, then edited on tape. This was pretty common for anything made between 1980 and 2000. It also leaves a lot of orphaned fields and disruptions in the pulldown pattern. Best advice: don't worry about the pulldown pattern and don't worry about deleting duplicate frames. Humpty-dumpty cannot be put back together again. Settle for 30 frames per second with occasional duplicates.

Well, I live in a PAL country and 30 fps looks horrible, I have to try to recover the 23.976 fps and then make it PAL.

I don't know if I said that before, but PAL Simpsons DVDs are worst than NTSC. They have even more combed frames and also blended ones!

[edumj]

Originally Posted by hello_hello

Did you try increasing the maxndl option? I've found it works quite well, at least for video. I rarely encode animation..

A little snippet from the help file.

While maxndl stands for "max non-duplicate length", it is actually more like a trade off between maintaining video sync and producing a smooth result. The larger maxndl is the more the decimation can be non-uniformly spread throughout the video, which helps the smoothness of the result in cases where duplicates are not evenly spread.

I think I read that, but don't remeber if I tried...

EDIT: Yes, I tried with maxndl=30 but also drops frames, and leaves some animations (like when they are swimming to the sofa) with one frame repeated 3 times, followed by just 1 frame (instead of 2 and 2 duplicated frames).

[edumj]

Originally Posted by StainlessS

FrameDifference does cope with Y/U/V, just use same clip for c and c2, and eg n2=current_frame+1 or n2=current_frame-1 for next or prev frame.
Where ChromaWeight > 0.0, returns Y/U/V difference combined, not separate results.

I've tried RT_YDifference and RT_FrameDifference, but it seems to do the same as DifferentFromNext?? Because when I use DifferentFromNext I also mix Y/U/V information using a code a found somewhere:

c = FrameEvaluate(c,"global diff_top = 0.50*YDifferenceToNext(top) + 0.25*UDifferenceToNext(top) + 0.25*VDifferenceToNext(top)")

Although now I'm testing with grayscale, so only Y seems needed.

So, I also need to clean bad compressed frames before.

Any good filter to clean flat color areas?

Or any way to perhaps mask lines to only take into account color changing areas?? To avoid bakcground or Apu's "candy" line flickering.

Thanks.

[StainlessS]

Originally Posted by edumj

but it seems to do the same as ...

c = FrameEvaluate(c,"global diff_top = 0.50*YDifferenceToNext(top) + 0.25*UDifferenceToNext(top) + 0.25*VDifferenceToNext(top)")

Framedifference is the same, but uses 0.33 combined chroma weight: to .66 luma, by default ie (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0) versus above 0.5.