Duplicity2/DropDeadGorgeous v2.13 - Dupe Tool - 17 Nov 2018

[StainlessS]

Post #1 of 2.

/*
Duplicity2(), v2.13, by StainlessS @ Doom9:- https://forum.doom9.org/showthread.php?t=175357

A two faced scheming and double dealing [on your behalf] dupe tool.

Three modes of operation,
0) Write duplicate Frames file only (as single frames or ranges).
1) Replace duplicates with exact duplicates. (Can also write Frames file).
2) Replace duplicates with Interpolated or Blended frames (control via MaxDuplen and MaxInterp). (Can also write Frames file).

Req:-
RT_Stats v2.00_Beta_11 + CallCmd (c) StainlessS,
GSCript, Grunt, (c) Gavino,
MaskTools2, MvTools2, (c) Manoa/Fizick/Pinterf & Others (Pinterf Versions).
RemoveGrain, (c) Kassandro.

GScript not needed if Avs+ [(c) Ultim, Pinterf and others.].
If CallCmd available then will AutoDelete DBase on clip closure, otherwise requires manual deletion.

ColorSpace:- All v2.6 standard PLANAR colorspaces except YV411, no support for YUY2, RGB24 or RGB32.

Function Duplicity2(clip c,"Mode"=0,Float "ThG"=1.0,Float "ThB"=ThG*2.0, Int "ThP"=64,Float "IgP"=0.0,Int "ThK"=7,
\ Int "MaxDupLen"=9,Int "LagMax"=MaxDupLen,Int "MaxInterp"=Min(9,MaxDupLen),
\ int "BlkW"=64,int "BlkH"=BlkW,int "OLapX"=BlkW/2,Int "OLapY=BlkH/2",
\ Int "X"=0,Int "Y"=0,Int"W"=0,Int "H"=0,Float "CW"=0.0,Bool "ChromaI"=False,Bool "AltScan"=False,
\ Int "SPad"=16, Int "SSharp"=1, Int "SRFilter"=4, [* MSuper *]
\ Int "ABlkSize"=16, Int "AOverlap"=4, Int "ASearch"=3,Int "ADct"=0, [* MAnalyse *]
\ Int "RBlkSize"=8, Int "ROverlap"=2, Int "RthSAD"=100, [* MRecalculate *]
\ Float "Iml"=200.0, Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
\ Int "OthSCD1"=400,Int "OthSCD2"=130, [* Dupe Detect Override @ Scene Change *]
\ String "OverRide"="",String "Frames"="Frames.txt",Bool "Ranges"=True,Bool "FrameAsRange"=False,
\ Bool "Show"=True,Bool "Verb"=True,Bool "ShowBlk"=True,Bool "Dim"=True,Bool "ShowDot"=False,
\ Bool "InterpFast"=True,String "DBase"=""
\ )

Duplicity2() uses 3 RT_Stats measurement functions, RT_FrameDifference(), RT_FrameMovement(), and RT_LumaPixelsDifferent().
the first two may difference using either luma only, or with a combined weighted chroma difference. Also see CW,
and ChromaI args. RT_LumaPixelsDifferent() differences luma only.

DeDuppers use some method to determine if a frame is a duplicate of its predecessor, a typical measuring method might be
RT_FrameDifference (average absolute difference per pixel), unfortunately the result of such measure may be due to a small
frame global change (lots and lots of pixels change by a small amount), or a smaller number of pixels change by a
larger amount, both these cases could produce the exact same result. Another case may be that of a very small number of
pixels change by a very large amount (eg a person blinks an eye with remainder of frame static) and such measure might seem
to be a duplicate as it could produce much smaller numbers than a frame global change of a small amount.

Duplicity2() uses a triple measurement system (maybe it should be called Triplicity), Primary measurement (RT_FrameDifference)
to detect frames 'Unique' to their immediate predecessor (large or 'frame Global' changes), and second and third methods to
determine if 'Non-Unique' frames are either 'Low Motion' or 'Duplicate' frames. The second method is a block differencing
method (RT_FrameMovement) where it splits the frame up into BlkW by BlkH blocks and finds the maximum average pixel difference
of any one block. The third measurement method is RT_LumaPixelsDifferent (result 0.0->255.0 where 255.0 is equivalent to 100.0%,
user args for this method actually require Duplicity2 arg 'IgP' in range 0.0->100.0%, and also show results in range 0->100.0%).
RT_LumaPixelsDifferent, compares frames and shows percent of pixels whose corresponding absolute luma pixel difference is
greater than Int arg 'ThP', if this percent difference is greater than 'IgP', then it is deemed to be a 'Low motion' frame and if
less or equal to 'IgP' then is a duplicate ('IgP' can be 0.0, meaning any pixels at all with difference greater than 'ThP' is a
'Low Motion' Frame). Where both 'ThP' and 'IgP' are 0 and 0.0 respectively, will show percentage of pixels that are not Exactly the same.
'IgP' could be set to 0.0 (Default) in which case the function is governed solely by any pixel differences exceeding 'ThP', or can set
'IgP'= eg 0.001% meaning the it will disregard as noise 1 pixel difference exceeding 'ThP' in every 100,000 pixels
[eg 1 pixel in every 316x316 region]. It is probably easiest to always use IgP of 0.0 and only adjust ThP.

The function examines the DBase to determine if status for current frame is already known, if not already known then,
the function determines if the current frame is a 'Unique' frame or not (via 'ThG' RT_FrameDifference to predecessor), if Unique,
then the job is done and is considered a Non-Duplicate, these Unique frames are marked in a DBase as Unique. If a frame is
'Non-Unique', then the function scans both backwards and forwards looking for 'Unique Start' and 'Unique End' frames (and if
necessary, marking them in DBase). This Backwards/Forwards scanning permits jumping around in the clip for perusal and getting
exact same results as if travelling forwards only (even backwards traversal only, should work just fine). When the
'Unique Start' and 'Unique End' frames are established, we then know the extent of the unknown (unvisited) frames, and can scan
and measure those with the second and third measurement methods (RT_FrameMovement and RT_LumaPixelsDifferent) to determine
Low Motion/Dupe status. When determining Low Motion/Dupe status, we compare with an 'Anchor Frame', which at the beginning of a
'Non-Unique' sequence is the 'Unique Start' frame. When scanning a non-unique sequence the 'Anchor Frame' is the predecessor frame
to the very first frame in the current sub sequence of the 'Non-Unique' sequence, so potential duplicates are not necessarily
compared with their immediate predecessor. When a frame is determined to be 'Low Motion' (not a dupe), any detected duplicates are
written to the DBase and also to the output Frames file, and it switches to detecting sequences of 'Low Motion' frames within the
'Non-Unique' sequence. On each detected 'Low Motion' frame, the Anchor frame for the next frame (whether it be another
'Low Motion' frame or a new 'Dupe' frame) will be switched to that newly detected 'Low Motion' frame. With this method,
all frames BETWEEN (but excluding) the 'Unique Start' and 'Unique End' in the 'Non-Unique' sequence are detected as perhaps
multiple 'Low Motion' and 'Duplicate' sub sequences.
Setting LagMax arg to some low number (min 1), will limit distance of the Anchor frame prior to the current frame, eg setting
LagMax=1, would always do LoMo/Dupe compare with the current frame and its predecessor. A LagMax greater than 1 allows better detection
of Low-Motion frames where there is only very slow creeping movement in the sequence.

The above method allows for reasonably fast processing, where a 'unique' frame is encountered [where it is clearly different from
its predecessor] but where not so clear, will examine the LoMo/dupe suspect frames more closely. ThG sets the boundary between
'Unique' frames and non unique, and so has some effect on speed, setting too high for your source will slow processing speed as many
non dupes will be examined more closely, and setting ThG too low for your source, may increase speed but may miss duplicates in noisy
sources (probably better to set too high than too low).

Although you might be able to scan backwards and get same detections, results are written to the Frames file in visited sequence and
you should NOT jump about whilst writing the final frames file. Also, this is a Multi-Instance script, can use multiple instances
at the same time (view side by side metrics) but you should really choose a unique 'Frames' filename for each instance.

Args:-
Mode, Default 0, (Range 0->2)
0) Write Frames file only (where Frames file="" then not written so only Show=true does anything).
1) Make duplicate sequences Exact Dupes.
2) Interpolate/Blend Duplicates (depends upon MaxDupLen and MaxInterp).

ThG, Default 1.0 (0.0 < ThG <= 255.0) Suggest 1.0, never below 0.5.
Primary Global Unique detect threshold. [current_frame-1 <-> current_frame].
If using 255.0 will likely scan all of the way to both Start and End frames, and then
detect LoMo/Dupe frames of entire clip using the 2nd & 3rd detection methods, will seem to pause for a long time,
dont do it (no error if using daft numbers).

ThB, Default Float ThG * 2.0, (Range ThG <= ThB < 255.0, Suggested about 1.5->3.5).
Secondary Local LoMo/Dupe Block detect threshold, [Anchor<->current_frame]
Tune for source and Block Size. (For smaller block size, just by random chance there may be several noisy pixels
in duplicate frame block, that produce considerably higher metric than most other blocks).

ThP, Default Int 64. (Advised Range ThB < ThP <= 64, or 255=OFF)
Ternary Local [single pixel] LoMo/Dupe detect threshold, [Anchor<->current_frame].
On a test with 256 artificial generated 640x480 identical frames of random coloured pixels, encoded at
MeGUI AVC CRF 23.0(default), RT_LumaPixelsDifferent(Thresh=1) found no frames where any single pixel was
different by more than 1 luma level when compared with the first encoded frame from that sequence. Mpeg2
and flv might not fare as well. This test was on a totally clean artificial source without artefacts
and such, you will likely need considerably higher values of ThP than 1.
I have seen multiple clips exhibiting some kind of minute (perhaps sub pixel) frame shift (both horizontal and
vertical) where frame should have been an exact duplicate, no idea to the cause.
This setting is an additional knob to twiddle, and you could turn it off and rely on ThG and ThB
completely [ThP=255, turns off]. The default of 64 and the below setting IgP=0.0, will consider any single pixel
with a luma difference of 64 or more, to be a non dupe, unfortunately any clip having above mentioned sub pixel
shifting at dupe frames, may need a high setting to not trigger a lomo detection, I have not I think
seen so far any clip that needed ThP more than about 48. 64 might seem ridiculously high but if frames are shifted,
then a little preposterous-ness is likely necessary, in a true non-dupe, there will most usually be at least
one pixel that differs in luma by more than 64.
We have thusly decided to make the default an emergency fallback detect value of 64, the user may decide to lower
this value or indeed turn it off completely[ThP=255].

IgP, Default 0.0, (Range 0.0 <= IgP < 100.0). Percent noise threshold for Ternary Local [pixel] LoMo/Dupe Detect,
Note, default 0.0% means that any frame with any single pixel with abs luma difference greater than ThP will
be detected as LoMotion.

ThK, Default = 7, (0 <= ThK <= 255). 0 = K Overrides off.
If RT_YPlaneMinMaxDifference(Threshold=0.2) of current_frame is less ThK then will Override any detection,
ie black (or narrow luma range) frames are ignored as not duplicates.
[Threshold=0.2 ignores as noise, 1 in 500 extreme pixels when establishing loose minimum and loose maximum & luma range].

MaxDupLen, Default 9. Maximum permitted detection length of duplicate run, If detected dupe run exceeds this setting,
then it is totally ignored and presumed to be a static scene.

LagMax, Default MaxDupLen. Setting to a low number (min 1) will limit the distance of the Anchor frame prior to
the current frame, eg setting LagMax=1, would always do LoMo/Dupe compare with the current frame and its
predecessor. Maybe best left alone.

MaxInterp, Default Min(MaxDupLen,9), (Range 0->9). For Mode=2(Interpolate/Blend mode) only. Max length of interpolated frames.
Sequences longer than this but less or equal to MaxDupLen will be blended instead.

BlkW, Default 64, 8 <= BlkW, Mod 2, Best Mod 4. Width of Blocks for Secondary Metric.
BlkH, Default BlkW, 8 <= BlkH, Mod 2, Best Mod 4. Height of Blocks for Secondary Metric.
OLapX, Default BlkW/2, 0 <= OLapX <= BlkW/2, Horizontal block overlap for Secondary Block metric.
OLapY, Default BlkH/2, 0 <= OLapY <= BlkH/2, Vertical block overlap for Secondary Block metric.
Where CW==0.0 and AltScan==True, might try eg OLapY=(BlkH/2)-1, where might be faster due to AltScan and,
also in overlapped 'Y_Block' scan will then scan alternate scanlines that were not scanned in previous non overlapped
scan (if that makes any sense, ie even scanlines sampled in first block, odd lines sampled in overlapped block scan).

X,Y,W,H, All Default 0, Coordinates of test area, as in crop (all 0 = full frame).
The x,y coords will be rounded up mod4, and w,h coords rounded down mod 4.
Ignore eg animated logo.

CW [ChromaWeight] Default 0.0. (0.0 <= CW <=1.0).
For Primary Frame Global and Secondary Block Mode difference detection, weighting of combined chroma diff to luma diff.
If required, recommend about 0.1, where difference would be weighted as
TotDif = ((1.0 - CW) * YDif) + (CW * ((UDif+VDif)/2.0))
Suggest always CW=0.0 where Night Capture.


See Post #22 for current script

Zip Includes DropDeadGorgeous()

For Zip, See Mediafire in sig below this post, or SendSpace.

[StainlessS]

Post #2 of 2.


ChromaI Default False, Primary + Secondary Block Mode setting. YV12 Only(otherwise ignored), Set True if Interlaced Chroma.
Also ignored if CW=0.0. DO NOT use Interlaced material with Mode=2(Interpolate/Blend).

AltScan Default False. Secondary (only) Block Mode setting. Scans only every other scanline of blocks.

MvTools2 Args, for Interpolation.
MSuper args:
SPad, Default 16, See MvTools MSuper(hpad=8,vpad=8) [We use same for both].
SSharp, Default 1, See MvTools MSuper(sharp=2)
SRFilter, Default 4, See MvTools MSuper(rfilter=2)

MAnalyse args:
ABlkSize, Default 16, See MvTools MAnalyse(BlkSize=8,BlkSizeV=8) [We use same for both].
AOverLap, Default 4, See MvTools MAnalyse(OverLap=0)
ASearch, Default 3, See MvTools MAnalyse(Search=4)
ADct, Default 0, See MvTools MAnalyse(Dct=0). Using other than default value can be VERY slow.

MRecalculate args:
RBlkSize, Default 8, See MvTools MRecalculate(BlkSize)
NOTE, RBlkSize = 0 Will Switch OFF MRecalculate, probably a bit faster at possible expense of quality.
ROverlap, Default 2, See MvTools MRecalculate(Overlap)
RthSAD, Default 100, See MvTools MRecalculate(thSAD=200)

MFlowInter args:
Iml, Default 200.0, See MvTools MFlowInter(ml=100.0)
IBlend, Default True, See MvTools MFlowInter(Blend=True). Blend or copy frame at scene change.
IthSCD1, Default 400, See MvTools MFlowInter(thSCD1=400) : (0 < IthSCD1 <= (8*8)*255)
IthSCD2, Default 130, See MvTools MFlowInter(thSCD1=130) : (0 < IthSCD1 <= 255)
Threshold which sets how many blocks have to change for the frame to be considered as a
scene change. It is ranged from 0 to 255, 0 meaning 0 %, 255 meaning 100 %.
Default is 130 (which means 51.0%).
Used by MvTools2 MFlowInter during interpolation.

Duplicate detection Override due to scene change following dupes, args: (Modes 0 & 2 only)
OthSCD1, Default 400. as MVTools MSCDetection arg. (0 < OthSCD1 <= (8*8)*255). See MvTools2.
OthSCD2, Default 130. as MVTools MSCDetection arg. (0 < OthSCD2 <= 255)
These two thresholds govern whether frames either side of duplicate sequence are determined to be
from the same scene, if not from same scene then duplicate detection is overridden and 'E' flag is set in metrics,
on what would have been the duplicate sequence.
NOTE, If OthSCD1 >= 8*8*255 or OthSCD2 >= 255 then switches OFF scene change testing, will NOT override
any duplicate detections (will likely result in blended scene transitions where frame after dupes is from
a different scene).
NOTE, For Mode=1(Replace dupes with exact dupes), scene change detection is omitted as no blending nor
interpolation is done and so no blending across scene change is possible and so not necessary to detect.

OverRide, Default "" (ie not used).
Provide list of frames/ranges to OverRide as NOT DUPLICATES, can give filename or string of Overrides, eg
".\Overrides.txt", text file "Overrides.txt" in current directory containing something like
the contents between to ############ lines.
############
100,199 # Frames 100 to 199 overridden (as NOT dupes)
300,-100 # Frames 300 to 399 overridden (100 frames)
500 # Frame 500 Overridden.
############
Or Preceded in string by "*" Star or Asterisk character, then remainder of string as contents of above file example.

Frames, Default "Frames.txt", Frames filename for output, ("" will not write Frames file).
Ranges, Default True (True=Write comma separated frame ranges to Frames file, else single frames only).
FrameAsRange, Default False. If Ranges=True and FrameAsRange=default false, and single frame, then writes single frame number and
not range. If Ranges=True and FrameAsRange=True then writes single frame as range eg '23,23' for frame 23.

Show, Default True, Show metrics.
Verb, Default True, Additional Verbose Metrics when Show=True (Verbose Metrics not shown below).
ShowBlk, Default True. If true (and Show=True), then mark block on frame where secondary Block Detections occur.
Motion detection shown in Orange/Yellow, and greatest non 0.0 non detection block shown in Green.
Perhaps of use in tuning for ThB.
Dim, Default True. If True (AND Show=True AND Mode==0), then dims any outer area excluded by X,Y,W,H args.
When Mode!=0 then Dim is Ignored [ie only valid when Mode=0(Write Frames file mode)], ignored when modifying
source frames ie Mode=1(Make dupes exact dupes) or Mode=2(Interpolate dupes).
Perhaps switch to Mode=0, to examine X,Y,W,H, coords correct.
ShowDot, Default False. If Show AND ShowDot both True, then switches OFF Show and instead shows
an Asterisk/Star in top left corner of duplicates detections (after frame mods eg Exact copy or Interpolation etc).

InterpFast, Default True. If True, then Interpolation clips are pre-calculated, if false then calculations done in real time.
Pre-Calc clips a bit (maybe 10%) faster, but perhaps not quite as good quality.
NOTE, If experiencing Out Of Memory errors, then try switch to InterpFast=False.

DBase, Default "", If named DBase suppled, then is not deleted on clip closure.
[ If name supplied, result DB may be re-used by other scripts, eg DropDeadGorgeous() function ].
NOTE, If using multiple instances and named DBase, then give each DBase as different name.

Metrics show eg
"nnnn] OKSE:GBP:ULD {1/32 : L1/5 : uS=12 : uE=45}"
"pG= p.gggggg : pB= p.bbbbbb : pP%= p.pppppp"
"aG= a.gggggg : aB= a.bbbbbb : aP%= a.pppppp
"{K=200 : a=567 : Lag=1}"

Where,
'nnnn' = current frame
'OKSE:GBP:ULD' = Flags highlighted where relevant.
OKSE, OverRide flags (overridden to not duplicates),
O=User OverRide, (via OverRide arg)
User_Overrides are set in DBase to 'Not A Duplicate', prior to scanning and cannot be changed via the
detection scan.
K=Low Luma Range Override (black or low luma range frames not detected as dupes), via ThK arg.
K_Override via ThK can override all of the other conditions (except User Override as above).
S=Static Scene OverRide, via MaxDupLen arg.
Static_OverRide flagged if detected as Duplicate run but exceeds MaxDupLen.
E=End+1 of current dupe sequence is change of scene, so if Mode=2 then Interpolation
or Blending is skipped for that dupe sequence.
NOTE, Mode=0(Write Frames), also processes scene change detection, unless disabled via eg OthSCD2=256.
Mode 1(replace with exact dupes), skips scene change detection as no purpose to it.

GBP, Motion detection reason flags,
G=Frame Global detection via Primary detector. [Orange, Unique to current - 1]
B=Block detection via Secondary Detector. [Orange, motion between current and anchor in LoMo seq]
P=Pixel detection via Ternary Detector. [Orange, motion between current and anchor in LoMo seq]

ULD, Current frame detection Status.
U=Unique Frame, detect via ThG
L=Low_Motion Frame, detect via ThB, ThP and IgP.
D=Dupe Frame, not Unique or LoMotion.

'1/32' = The 1st of 32 frames in a 'Non Unique', perhaps mixed LoMo/Dupe sequence.
'L1/5' = The 1st of 5 frames of Low Motion sub sequence of the 'Non Unique' sequence
D1/5=1st of 5 dupes.
I1/5=1st of 5 dupes, Where Mode=2, and Interpolated.
B1/5=1st of 5 dupes, Where Mode=2, and Blended.
E4/7=4th of 7 dupes that were not processed because frame after dupes is from a different scene.
Will not occur when Mode=1 (replace with exact dupes).
K3/4 3rd of 4 K_Override frames.
S2/99 2nd of 99 Static_OverRide frames.

'uS=12 : uE=45' Unique Start and End frames, 'Non Unique' LoMo/Dupe sequence is between these two frames (exclusive).

'pG= p.gggggg.' = Primary 'Unique Frame' Frame difference metric between current and Predecessor frame.
'pB= p.bbbbbb.' = Secondary 'Low Motion/Dupe' Block difference metric between current and Predecessor frame. (info only)
'pP= p.pppppp.' = Ternary 'Low Motion/Dupe' Pixel difference metric between current and Predecessor frame. (Info Only)
'aG= a.gggggg.' = Primary Unique Frame difference metric between current and Anchor frame in Non Unique sequence.
'aB= a.bbbbbb.' = Secondary 'Low Motion/Dupe' Block difference metric between current and Anchor frame in Non Unique sequence.
'aP= a.pppppp.' = Ternary 'Low Motion/Dupe' Pixel difference metric between current and Anchor frame in Non Unique sequence.
'K=200' = Current_frame RT_YPLaneMinMaxDifference(Threshold=0.2), Unique or Non Unique sequences.
'a=567' = Comparison Anchor frame number in Non Unique sequence.
'Lag=1' = Frame Count between current frame and Anchor Frame (Lag=1, Anchor is Predecessor) in Non Unique sequence.

For 'Unique' frames, Only shows "nnnn] OKSE:GBP:ULD" on top line and also the 2nd line & 'K=2', 'Low Motion/Dupe' show the remainder.
*/


Greatest low motion block detected shown by Yellow rectangular marker. Dimm'ed area shows X,Y,W,H excluded area.
Just inside that block marker @ top LHS corner, there is an almost invisible fox (the light in doorway is due to motion detector).
Difficult scene, shimmering road due to raining.
https://i.postimg.cc/XYk0NgBb/Duplicity_2.00_Beta_2.jpg (https://postimages.org/)

And again in frame where fox (below doorway) after activating motion sensor (sniffing at something).
https://i.postimg.cc/ZKgLBk3P/Duplicity_2.00_Beta_2_b.jpg (https://postimages.org/)


See Post #22 for script

EDIT: Posted elsewhere about DropDeadGorgeous2():

Duplicity2(), is a dupe Detector, Dupe frame list creator, Dupes replaced with exact dupes, Dupes replaced with interpolated frames + DBase of dupes creator.
DropDeadGorgeous2() uses Duplicity2 DBase to do the almost impossible, smoothing out jerking clips (where source has jerks and dupes, much better than simple interpolation of dupes).
DropDeadGorgeous2 can also [EDIT: help] up framerate (eg 25.0 -> 29.97) keeping all source frames verbatim, and creating smooth result
[squeezes out dupes, where are replaced by interpolated frames between frames of most movement - non dupe frames are relocated between jerk interpolations and dupes to squeeze out dupes (hard to explain it)].

EDIT: Perhaps this explains better

abcXdefgDDhi Src, Two dupe [D] frames preceded (within a set range) by a jerk frame. Jerk frame X, has greater motion to its predecessor than do d, e, f, or g.
abc12Xdefghi DropDeadGorgeous dupe removal and Jerk interpolation

Lower case non dupe non Jerk
X = Frame of greatest motion from predecessor (Jerk)
D = Dupe Frames
Digit = Interpolated frames between X and its predecessor c (Same Count as D dupes)


# below assumes that 2 frames both dropped and duped, similar where 1 frame dropped. [2 frames dropped is extreme and more difficult case, we describe the more difficult]

abc12Xdefghi # How it should have been in capture, 1,2 will both be dropped, X is jerk frame following frames that will be dropped.
abcXdefgDDhi # when 1,2 dropped by CAP device, and DD dupes inserted.
abc12Xdefghi # DropDeadGorgeous() fixed. (1,2 are both interpolated frames {with c and X as interp source frames}, original 1,2 GONE $4E4 [forever])

[StainlessS]

Script Removed. See post #22 for script.


@ Motenai Yoda,
I see you on-line there, give a whirl please, took some time to do, sorry. Somewhat better (I hope) than previous posted in FrameSel thread. Still need use Frames rather than ranges with the other script, I'll get down to doing it as a FrameSel plug mod soon.
@ Orum, hope you can do better than this with your new plug in devs forum, good luck.
@ JohnMeyer, I know that you aint ever completely happy with anything video related, but maybe this tickles your fancy, just a tad :)

[StainlessS]

SPARE for Growth

[Motenai Yoda]

Sorry, I wasn't avoiding to try previous one, just there is something dependency related I have to fix.
btw CallCmd work on x64 avs+?

[StainlessS]

CallCmd work on x64 ?

Not yet, I'll make it work.
Can just comment out that line, delete file by hand.

EDIT: 2nd last line of script function, ie

CallCmd(close=RT_String("""CMD /C chcp 1252 && del "%s" """,DB), hide=true, Synchronous=7) # Auto delete DBase file on clip closure.

[Motenai Yoda]

you should delete those last 2 lines.
also it throw a "Script error: There is no function named 'GScript'."

[StainlessS]

There is no function named 'GScript'."
I guess that is something that was forgotten during the emulation implementation.

Req RT_Stats v1.43 + CallCmd (c) StainlessS, + GSCript + Grunt (c) Gavino.

Even if Avs+ has GScript functionality, you are likely to have problems having to comment out lines in GScript using scripts that can run on either avs standard and avs+.
I always have Gscript and Grunt [required for GScriptClip(arg=Args,Local=True)] in my plugins, whether avs or avs+ is in use.

you should delete those last 2 lines.
Which other line are you talking about, the GScript(InstS) thing actually calls the function, and so of course cannot be commented out and still function as expected (or at all).

[StainlessS]

try

# GScript(InstS)
Eval(InstS)


Works for me in avs+ 32bit.

EDIT: I'll fix it so that it works on avs 2.6 standard or avs+(without GSCript or script editing).

[Motenai Yoda]

https://thumbs2.imgbox.com/2a/37/J2rfcfDs_t.png (http://imgbox.com/J2rfcfDs)
I have those 2 lines at the end, from .zip

[StainlessS]

Link and posts updated to v1.01.

EDIT: Actually, I mistakenly used RT_VarExist("GScript") when I should have used RT_FunctionExist("Gscript"),
but it works OK anyway, arh well, I'll leave as it is for just now, I'll fix it for next time.

EDIT: I'll fix in the posted script, I'll just leave the zip as it is for now.

[johnmeyer]

I'll be out for several days but would like to try this. I do a LOT of work that involves detecting and then doing something with duplicates.

However ...

The most typical scenario that I come across that creates duplicates is from either a capture card or streaming video drops a frame and then, to keep the audio in sync, adds a duplicate frame. In those scenarios, the duplicate is a true duplicate, and no fancy logic is needed to find it. The much, much bigger issue is find the place where the dropped frame happened because it is at THAT point you have to do something, typically synthesizing a new frame using motion estimation.

As for how to detect differences and thereby determine which frames are dups, I haven't used RTStats for a while so I don't remember all the functions, but when using native AVISynth functions, it is the YDifference functions that are the most useful because, if I remember correctly, they compute their metric by the absolute differences between each pixel position in one frame with the same pixel position in the adjacent frame and then summing all of those differences. It will be interesting to see how your new function compares with that.

[StainlessS]

RT_LumaDifference can use two separate clips if required + any two frame numbers (one for each clip), [YDifference is single clip], but same measurement.

Yoda wanted to list dupe frames and temporarily remove them for MC processing, and then replace with identical dupes from the
MC processed clip.

ie here:- https://forum.doom9.org/showthread.php?p=1832814#post1832814

I think he may want this for vfr stuff, not sure.

[Motenai Yoda]

Yep something about full of dups stuff
but as lossy codecs will introduce artifacts between dups, loss of detail et simila iirc TDecimate uses a threshold to filter out small differences (default 0) and a block based metric to find slight differences in small area only.

[StainlessS]

I use a block based metric in SecurityCamExtractMotion.avs as shown in action here:- https://forum.doom9.org/showthread.php?p=1775305#post1775305

Is too big to post, but included in RT_Stats v2.0 beta.

I did at one time use it in the Duplicity script, maybe I'll add as an option or something.

[StainlessS]

If anybody has wanted to try SecurityCamExtractMotion.avs as mentioned in previous post but has had trouble down to Gscript and Avs+,
find fixed version works on AVS/AVS+ :- SecurityCamExtractMotion.AVS v2.02 :- https://www.mediafire.com/file/c6kqosweqmk82my/SecurityCamExtractMotion.AVS/file

Or on SendSpace below in my sig.

Looks like this
https://s20.postimg.cc/5ix37af7h/SCMD2_zpsiuzq505m.png (https://postimg.cc/image/uc6n7xy7t/)

[johnmeyer]

I used that security cam motion detect logic several years ago and it worked incredibly well. This looks like a much later version because I don't remember being able to draw a box on the screen to define the "live" area for the detection logic to use (or ignore).

[StainlessS]

The dimmed area stuff (edge of frame) is highlited in Blue, the marker stuff hilited in purple, was updated at least 12 months ago.


# SecurityCamExtractMotion.AVS, v2.01, By StainlessS.
#
# Requires RT_Stats v2.0, FrameSel v1.03, GScript, Grunt, mt_tools_2. [GScript not needed if Avs+]
#
# Clip, Planar, YUY2 or RGB clip, YV411 not supported due to non support in Overlay.
# (RGB, will likely require different thresholds)
#
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN MotDetect.
#
# MotDetect() creates four .txt command files for KEEP frames (movement), DROP frames (not movement), DUPE frames and HOLD frames.
# HOLD frames are combination of all KEEP frames and, DROP frames where block with greatest difference is in close proximity to previous
# KEEP frame block with greatest difference (ie there seems to be possible movement very near previously identified movement).
# Using SelectFrames() you can select frames types of each of the 4 command files, or using the Reject mode of FrameSel, you can
# instead return the frames NOT in the command file, so using the 4 created command files, you can extract 8 variations of frame types
# after a single detection scan.
#
############################################
Th = 6.0 # Greater or equal KEEP (movement), below is DROP(or DUPE, or possibly HOLD).
# Need check against source, block size and almost all other settings.
#
dTH = 4.0 # Below or equal is DUPE, 0.0 <= dTh < Th (above dTh and below Th is DROP [or possibly HOLD]).
#
BLKW = 64 # Suggest about 16 -> 64 (even only, mod 4 else probably slower). Metrics will change with block size.
BLKH = BLKW # Localized movement, adjust for smallest size of moving objects to detect, makes only little difference to speed,
# bigger, may be a bit faster.
# For human target, suggest box encompassing head + shoulders of most distant human.
# RT_FrameMovement returns greatest average pixel difference for any one of the blocks.
# Block searches may be overlapped by up to half a block both horizontally and vertically and so total x,y,w,h area
# of each pair of frames is compared up to about 4 times due to the half block overlaps.

#
X = 16 # Area Of Interest, avoid eg passing traffic or timecode (or could use eg Overlay to blot out stuff for dclip).
Y = 16 # Suggest cut out eg sky unless looking for flying saucers (less to search, faster).
W = -16 # As for Crop. Best MOD 4 only
H = -16
#
CW = 0.0 # ChromaWeight, 0.0->1.0. 0.0 = Luma ONLY, about 0.1 suggested for color (luma ONLY faster for YUV).
# For RGB, Matrix used where CW==0.0 to convert to Luma-Y. If CW != 0.0, then returns average pixel differences,
# (apdR+apdG+apdB)/3.0.
# Perhaps always CW=0.0 [luma only] for YUV night captures.
# Can convert RGB to YUV for Luma-Y detect and SelectFrames() using original RGB, if required.
#
AltScan=false # If True, Scan only every other raster line starting from Y (maybe faster, especially YUV Luma scan only).
#
ChromaI=False # If True, YV12 chroma is interlaced (ignored if not YV12 or if Luma ONLY scan [CW==0.0]).
#
Matrix=(Width>1100||Height>600)?3:2 # Matrix used for RGB conversion to Luma-Y.
# Default, If (Width>1100 || Height>600) then 3(PC.709), else 2(PC.601). (0=Rec.601, 1=Rec.709)
# NOTE, default PClevels conversions will likely require different thresholds to YUV because of the
# greater differences in Y (after RGB-Luma-Y conversion), probably about (255.0/219.0) times greater.
#
OLapX= BlkW/2 # X Block Overlap. 0->BLKW/2 (bigger more accurate, slower)
OLapY= BlkH/2 # Y Block Overlap. 0->BLKH/2
# Where YUV, CW==0.0 and AltScan==True, might try eg OLapY=(BlkH/2)-1, where would be faster due to AltScan and,
# also in overlapped 'Y_Block' scan will then scan alternate scanlines that were not scanned in previous non overlapped
# scan (if that makes any sense, ie even scanlines sampled in first block, odd lines sampled in overlapped block scan).

LAG = 1 # Lag (default 1, ie no lag. 1 or more). Compares current frame n with frame n-Lag. Detection is always
# initially compared(n,n-1), if neither DUPE nor KEEP detected and Lag>1, then will redetect using comparison(n,n-Lag)
# where n-lag frame is limited to no earlier than the most recent KEEP frame.
# A moderate Lag (up to about 50 or so), may be more sensitive to very small amount of movement,
# it introduces a time delay between compared frames. There is no upper limit to Lag arg.
# NOTE, Lag > 1 will likely produce more KEEP and HOLD frames (ie identify more movement), and also a bit slower
# due to 2nd detection scan where initial scan identified DROP frame (ie DROP may be converted to KEEP or HOLD).
# Lag will not make any huge difference to results, small amount of movement at beginning of sequence of movements may be
# detected, use lag > 1 only where every possible movement frame is required.

SHOWDCLIP=false # If True, then shows DClip in MotDetect rather than source clip (Only useful if dclip pre-processed in some fashion)
#
dclip=Last # Detection clip.
# Or some kind of denoise or whatever, used for detection (visible in ShowMetrics if SHOWDCLIp==True).
# Use SPATIAL denoise ONLY.
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).VagueDenoiser(threshold=0.8,method=1,nsteps=6,chromaT=0,interlaced=false) # SLOWISH
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).hqdn3d(2.0, 0.0,0.0,0.0,0) # A bit FASTER

###
### Show Metrics ONLY
DIM=False # Dim/darken area outside of the x,y,w,h search area. A little faster with Dim==False when showing metrics only.
SHOWMISS =true # Show block 'Marker' for greatest moving DROP block (miss), (markers never shown for dupes).
MISSCOLOR =$AAFF00 # Color, block 'Marker' for greatest DROP block (when SHOWMISS==True).
HITCOLOR =$FFFF44 # Color, block 'Marker' for greatest KEEP block (hit).
HOLDCOLOR =$FF8800 # Color, block 'Marker' for HOLD block (Miss converted to HIT due to Miss proximity to previous Hit).
VERBOSE=True # Show additional metrics, Th, dTh, BLK Args


worked incredibly well
With RT_Stats v2.0, have implemented RT_FrameMovement (I think may have been implemented as RT_LumaMovement, ie luma only) in version that your tried,
yep, it works damned lovely :)

The Block marker in yellow show area of max change/movement.

EDIT:
HOLD frames are combination of all KEEP frames and, DROP frames where block with greatest difference is in close proximity to previous
KEEP frame block with greatest difference (ie there seems to be possible movement very near previously identified movement).


I remember about time I did the above frame cap, that a bloke walking down the street went behind a car, and you could only just see the top of his hat above the car/van bobbing up and down, and the HOLD frame thing above managed to track only a few pixels of the oscillating hat until he emerged a few yards on, was quite fascinating to see it actually working profoundly well, in the words of Harry Potter, "I Love Magic", :)

[`Orum]

Yoda wanted to list dupe frames and temporarily remove them for MC processing, and then replace with identical dupes from the MC processed clip.
Interesting. Why does he want the duplicates put back in? Does some playback device not support VFR?

Edit: Thinking about this, it shouldn't be too hard to do; just parse the timestamp file and insert duplicate frames when necessary.
@ Orum, hope you can do better than this with your new plug in devs forum, good luck.
Heh, thanks for the vote of confidence! Anyway if you give it a shot let me know what you think. It really needs block-level metrics but that should be coming very soon (probably in the next release).

[StainlessS]

Interesting. Why does he want the duplicates put back in? Does some playback device not support VFR?

I have never played with vfr, but I'm guessin' that he wants to put back [EDIT: Identical] dupes so that some other vfr stuff can then work with the
inserted identical duplicates to produce the vfr 'thingy'.

Here is a working demo for removing the dupes for some MC processing of only non-dupes, and then to replace the removed dups with
identical dups from the newly processed MC result, I'm gonna add the "MakeReDupCmd()" functionality to FrameSel plug.
https://forum.doom9.org/showthread.php?p=1832814#post1832814

Looking forward to seeing what your plug can do, Peace bro :)

[Motenai Yoda]

Nope just leave it cfr as is, since I don't like how most tdecimate vfr looks, and "true" vfr as dedup ones doesn't even play on a lot of devices/players
I tried it and work well, I'm looking forward to Orum's plug too

[StainlessS]

Post #1 of Many.

Duplicity2() v2.13 (see post #1 for docs)

Function Duplicity2(clip c,"Mode",Float "ThG",Float "ThB", Int "ThP",Float "IgP",Int "ThK",
\ Int "MaxDupLen",Int "LagMax",Int "MaxInterp",
\ int "BlkW",int "BlkH",int "OLapX",Int "OLapY",
\ Int "X",Int "Y",Int"W",Int "H",Float "CW",Bool "ChromaI",Bool "AltScan",
\ Int "SPad", Int "SSharp", Int "SRFilter", [* MSuper *]
\ Int "ABlkSize", Int "AOverlap", Int "ASearch",Int "ADct", [* MAnalyse *]
\ Int "RBlkSize", Int "ROverlap", Int "RthSAD", [* MRecalculate *]
\ Float "Iml", Bool "IBlend", Int "IthSCD1", Int "IthSCD2", [* MFlowInter *]
\ Int "OthSCD1",Int "OthSCD2", [* Duplicates Detection Override *]
\ String "OverRide",String "Frames",Bool "Ranges",Bool "FrameAsRange",
\ Bool "Show",Bool "Verb",Bool "ShowBlk",Bool "Dim",Bool "ShowDot",
\ Bool "InterpFast",String "DBase"
\ ){

Function D2_Hit_Marker(int W, Int H, Bool "Hit", Int "BW", Int "BH",Int "PW",Int "PH",Bool "YV12",Bool "Mod2") {
Hit=Default(hit,False) BW=Max(Default(BW,W/8),1) BH=Max(Default(BH,H/8),1)
BW2=(W>4)?BW*2:BW BH2=(H>4)?BH*2:BH
PW=Max(Default(PW,BW/8),1) PH=Max(Default(PH,BH/8),1) is26 = VersionNumber>=2.6
YV12=(!is26) ? True : Default(YV12,False) Mod2=Default(Mod2,True) Mod = (YV12||Mod2) ? 2 : 1
CanvasW=(W+Mod-1)/Mod*Mod CanvasH=(H+Mod-1)/Mod*Mod
Rpn= RT_String("x %d < x %d >= | y %d < | y %d >= | x %d < x %d >= | & y %d < y %d >= | & ",BW,W-BW,BH,H-BH,BW2,W-BW2,BH2,H-BH2)
Rpn= (Hit) ? Rpn + RT_String("x %d < x %d >= | y %d < | y %d >= | | ",PW,W-PW,PH,H-PH) : Rpn
Rpn= Rpn + RT_String("x %d < y %d < & & 255 0 ?",W,H)
Blankclip(width=CanvasW,height=CanvasH,Length=1,pixel_type=YV12?"YV12":"Y8").Killaudio
return mt_lutspa(relative = false,yExpr=Rpn, chroma = "-128")
}
c myName="Duplicity2: " VER=2.13
Assert(RT_Version() >= 1.99911,RT_String("%sNeed RT_Stats v2.00Beta11(%s)",myName,RT_VersionString))
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
HasGRunt =RT_FunctionExist("GScriptClip") HasMvTools=RT_FunctionExist("MSuper")
HasMaskTools=RT_FunctionExist("MT_Lutxy") HasRemoveGrain=RT_FunctionExist("RemoveGrain")
HasCallCmd = RT_FunctionExist("CallCmd") IsAvs26=VersionNumber>=2.6
###
Mode = Default(Mode,0)
ThG=Default(ThG,1.0) ThB=Default(ThB,ThG*2.0) ThP=Default(ThP,64) IgP=Default(IgP,0.0)
ThK=Default(ThK,7) MaxDupLen=default(MaxDupLen,9) LagMax=default(LagMax,MaxDupLen) MaxInterp = Default(MaxInterp,9)
BlkW=Default(BlkW,64) BlkH=Default(BlkH,BlkW) OLapX=Default(OLapX,BlkW/2) OLapY=Default(OLapY,BlkH/2)
X=default(X,0) Y=default(Y,0) W=default(W,0) H=default(H,0)
CW=Default(CW,0.0) ChromaI=Default(ChromaI,False) AltScan=Default(AltScan,False)
SPad = Default(SPad,16) SSharp = Default(SSharp,1) SRFilter= Default(SRFilter,4)
ABlkSize = Default(ABlkSize,16) AOverlap= Default(AOverlap,4) ASearch = Default(ASearch,3) ADct = Default(ADct,0)
RBlkSize = Default(RBlkSize,8) ROverlap= Default(ROverlap,2) RthSAD = Default(RthSAD,100)
Iml = Default(Iml,200.0) IBlend = Default(IBlend,True) IthSCD1 = Default(IthSCD1,400) IthSCD2= Default(IthSCD2,130)
OthSCD1 = Default(OthSCD1,400) OthSCD2 = Default(OthSCD2,130)
Override=Default(OverRide,"") Frames=Default(Frames,"Frames.txt") Ranges=Default(Ranges,True) FrameAsRange=Default(FrameAsRange,False)
Show=Default(Show,True) Verb=Default(Verb,True)
Showblk=Default(ShowBlk,true) Dim=Default(Dim,true) ShowDot=Show?Default(ShowDot,False):False
InterpFast=Default(InterpFast,True) DBase=Default(DBase,"")
####
Assert(0 <= Mode <= 2,RT_String("%s0 <= Mode(%d) <= 2",myName,Mode))
Assert(1 <= MaxDupLen,RT_String("%s1 <= MaxDupLen(%d)",myName,MaxDupLen))
Assert(1 <= LagMax,RT_String("%s1 <= LagMax(%d)",myName,LagMax))
Assert(0 <= MaxInterp <= 9,RT_String("%s0 <= MaxInterp(%d) <= 9",myName,MaxInterp))
Assert(IsAvsPlus || HasGScript,RT_String("%sNeed either GScript or AVS+",myName))
Assert(HasGRunt,RT_String("%sNeed GRunt:-https://forum.doom9.org/showthread.php?t=139337 ",myName))
Assert(HasMVTools,RT_String("%sNeed MvTools2:-http://forum.doom9.org/showthread.php?t=131033",myName))
Assert(HasMaskTools,RT_String("%sNeed MaskTools",myName))
Assert(Mode!=2 || MaxInterp==0 || HasRemoveGrain,RT_String("%sNeed RemoveGrain",myName))
Assert(IsYV12||(IsAvs26&&(IsY8||IsYV16||IsYV24)),RT_String("%sOnly Standard v2.6 Planar Supported (excepting YV411)",myname))
MaxInterp=Min(MaxInterp,MaxDupLen)
W = (W > 0) ? (W + X) - Width : W H = (H > 0) ? (H + Y) - Height : H # Make -ve relative width,height
X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP (inwards) Mod 4
W = (W-3) / 4 * 4 H = (H-3) / 4 * 4 # Round DOWN(inwards) Mod 4
W = Width - X + W H = Height - Y + H # +ve width,height
DC=Crop(X,Y,W,H) BlkW=Min(DC.Width,BlkW) BlkH=Min(DC.Height,BlkH)
(Show && !ShowDot && DIM && Mode==0 && (W!=Width || H!=Height)) ? levels(16,1.0,235,16+48,235-48,Coring=False).Overlay(DC,x=X,y=Y) : Last
DB=(DBase=="") ? RT_GetFullPathName("~@@@_"+RT_LocalTimeString+".DB") : RT_GetFullPathName(DBase)
FuncS="""
Function EndOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at EOS
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
bv=sup.MAnalyse(isb=true, delta=1,blksize=16)
Return c.MSCDetection(bv,thSCD1=thSCD1,thSCD2=thSCD2)
}
###
Function ChrIsNul(String S) {return RT_Ord(S)== 0}
Function ChrIsHash(String S) {return RT_Ord(S)==35}
Function ChrIsStar(String S) {return RT_Ord(S)==42}
Function ChrIsComma(String S) {return RT_Ord(S)==44}
Function ChrIsHyphen(String S) {return RT_Ord(S)==45}
Function ChrIsDigit(String S) {C=RT_Ord(S) return C>=48&&C<=57}
Function ChrEatWhite(String S) {i=1 C=RT_Ord(S,i) While(C==32||C>=8&&C<=13) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatDigits(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
###
Function Fn@@@(clip c,clip dc,String DB,Int Mode,Float ThG,Float ThB,Int ThP,Float IgP,Int ThK,
\ Int MaxDupLen, Int LagMax,Int MaxInterp,
\ Int BlkW,Int BlkH,Int OLapX,Int OLapY,Int X,Int Y,Float CW,Bool ChromaI,Bool Altscan,
\ Int OthSCD1,Int OthSCD2,String Frames,Bool Ranges,Bool FrameAsRange,
\ Bool Show,Bool Verb,Bool ShowBlk,Bool ShowDot,Bool InterpFast
\) {
c n=current_frame FC=c.FrameCount
Stat=RT_DBaseGetField(DB,n,FLD_STAT@@@)
if(Stat == 0) { # In here if unvisited
if(RT_FrameDifference(dc,dc,n=n,n2=n-1,ChromaWeight=CW,chromai=ChromaI) > ThG) { # dif to prev
# Uniq_KOverride OR Unique
Stat=(ThK>0 && RT_YPlaneMinMaxDifference(dc,n=n,Threshold=0.2)<ThK)?FLG_UK@@@:FLG_UU@@@
RT_DBaseSetField(DB,n,FLD_STAT@@@,Stat)
Global nDone@@@=nDone@@@+1
if(nDone@@@==FC) { RT_DBaseSetId(DB,1,FC) }
} else {
# MUST scan backwards to find frame that is Unique to its predecessor(we're not necessarily a dupe of that frame)
StartU = -1
for(i=n-1,0,-1) { # Exit with i==-1 (should NEVER HAPPEN, frame 0 is Unique)
if(RT_DBaseGetField(DB,i,FLD_STAT@@@)==0) { # Stat Unknown
if(RT_FrameDifference(dc,dc,n=i,n2=i-1,ChromaWeight=CW,chromai=ChromaI) > ThG) {
StartU=i
# Set Unique/Uniq_KOverride Stat
RT_DBaseSetField(DB,StartU,FLD_STAT@@@,(ThK>0&&RT_YPlaneMinMaxDifference(dc,n=i,Threshold=0.2)<ThK)?FLG_UK@@@:FLG_UU@@@)
Global nDone@@@=nDone@@@+1
if(nDone@@@==FC) { RT_DBaseSetId(DB,1,FC) }
i = -1 # Break, Exit with i==-2
}
} Else { StartU=i i = -2 } # Break, Exit with i==-3
}
# Assert(i!=-1,myName@@@+"Uniq Not Found backwards")
EndU = FC
for(i=n+1,FC-1) { # Exit with i==FC (should NEVER HAPPEN, last frame is Unique)
if(RT_DBaseGetField(DB,i,FLD_STAT@@@)==0) { # Stat Unknown
if(RT_FrameDifference(dc,dc,n=i,n2=i-1,ChromaWeight=CW,chromai=ChromaI) > ThG) {
EndU=i
# Set Unique/Uniq_KOverride Stat
RT_DBaseSetField(DB,EndU,FLD_STAT@@@,(ThK>0 && RT_YPlaneMinMaxDifference(dc,n=i,Threshold=0.2)<ThK)?FLG_UK@@@:FLG_UU@@@)
Global nDone@@@=nDone@@@+1
if(nDone@@@==FC) { RT_DBaseSetId(DB,1,FC) }
i = FC # Break, Exit with i==FC=+1
}
} Else { EndU=i i = FC+1 } # Break, Exit with i==FC+2
}
# Assert(i > FC,myName@@@+"Uniq Not Found forwards")
Anchor=StartU
for(i=StartU+1,EndU-1) {
Count=1 Stat=0
if(ThK>0 && RT_YPlaneMinMaxDifference(dc,n=i,Threshold=0.2)<ThK) {
Anchor=Max(Anchor,i-LagMax)
RT_DBaseSetField(DB,i,0,FLG_NUK@@@,StartU,EndU,Anchor,0) # Count Field NOT SET
for(j=i+1,EndU-1) {
Anchor=j-1
if(RT_YPlaneMinMaxDifference(dc,n=j,Threshold=0.2)<ThK) {
Count=Count+1
RT_DBaseSetField(DB,j,0,FLG_NUK@@@,StartU,EndU,Anchor,j-i) # Count Field NOT SET
} else { j=EndU }
}
} Else {
Anchor=Max(Anchor,i-LagMax)
if((RT_FrameMovement(dc,dc,n=i,n2=Anchor,ChromaWeight=CW,AltScan=AltSCan,
\ ChromaI=ChromaI,BlkW=BlkW,BlkH=BlkH,OLapX=OLapX,OLapY=OLApY,BlkTh=ThB)>ThB) ||
\(RT_LumaPixelsDifferent(dc,dc,n=i,n2=Anchor,Thresh=ThP)>IgP) ||
\(RT_FrameDifference(dc,dc,n=i,n2=Anchor,ChromaWeight=CW,chromai=ChromaI)> ThG)
\) { # Start of LoMotion SubSequence
RT_DBaseSetField(DB,i,0,FLG_LM@@@,StartU,EndU,Anchor,0) # Count Field NOT SET
for(j=i+1,EndU-1) {
Anchor=j-1
if(
\ (
\ (RT_FrameMovement(dc,dc,n=j,n2=Anchor,ChromaWeight=CW,AltScan=AltSCan,
\ ChromaI=ChromaI,BlkW=BlkW,BlkH=BlkH,OLapX=OLapX,OLapY=OLApY,BlkTh=ThB)>ThB) ||
\ (RT_LumaPixelsDifferent(dc,dc,n=j,n2=Anchor,Thresh=ThP)>IgP) ||
\ (RT_FrameDifference(dc,dc,n=j,n2=Anchor,ChromaWeight=CW,chromai=ChromaI)>ThG)
\ ) && (ThK<=0 || RT_YPlaneMinMaxDifference(dc,n=j,Threshold=0.2)>=ThK)
\ ) {
Count=Count+1
RT_DBaseSetField(DB,j,0,FLG_LM@@@,StartU,EndU,Anchor,j-i) # Count Field NOT SET
} Else { j=EndU }
}
} Else { # Start of Dupe SubSequence
RemAnchor=Anchor # Remember Anchor at start of dupe run
for(j=i+1,EndU-1) {
Anchor=Max(Anchor,j-LagMax)
if(
\ (RT_FrameMovement(dc,dc,n=j,n2=Anchor,ChromaWeight=CW,AltScan=AltSCan,
\ ChromaI=ChromaI,BlkW=BlkW,BlkH=BlkH,OLapX=OLapX,OLapY=OLApY,BlkTh=ThB)<=ThB) &&
\ (RT_LumaPixelsDifferent(dc,dc,n=j,n2=Anchor,Thresh=ThP)<=IgP) &&
\ (RT_FrameDifference(dc,dc,n=j,n2=Anchor,ChromaWeight=CW,chromai=ChromaI) <= ThG) &&
\ (ThK<=0 || RT_YPlaneMinMaxDifference(dc,n=j,Threshold=0.2)>=ThK)
\ ) {
Count=Count+1
} Else { j=EndU }
}
Anchor=RemAnchor # restore duprun anchor
if(Count>MaxDupLen) {Stat=FLG_DS@@@} # MaxDupLen OverRide
Else {
if(Mode!=1 && i+Count==EndU && OthSCD2 < 255 && OthSCD1 < 8*8*255) {
Tmp_c = c.Trim(i-1,-1) + c.Trim(i+Count,-1)
Stat=(EndOfSceneClip(Tmp_c,OthSCD1,OthSCD2).RT_AverageLuma(n=0,w=1,h=1).Int != 0)?FLG_DE@@@:FLG_DUP@@@
} Else {Stat=FLG_DUP@@@}

[StainlessS]

Post #2 of Many.


if(Stat==FLG_DUP@@@) {
if(Frames!="") {
if(!Ranges) { for(j=0,Count-1) { RT_WriteFile(Frames,"%d",i+j,Append=True) }}
Else if(Count>1) { RT_WriteFile(Frames,"%d,%d",i,i+Count-1,Append=True)}
Else if(!FrameAsRange) { RT_WriteFile(Frames,"%d",i,Append=True)}
Else if(i==0 && Count==1) { RT_WriteFile(Frames,"%d,-1",i,Append=True)}
Else { RT_WriteFile(Frames,"%d,%d",i,i,Append=True)}
}
if(Show) {
Global nDupeSeq@@@=nDupeSeq@@@+1
if(Count > DupeLenMax@@@) {Global DupeLenMax@@@=Count Global DupePosMax@@@=i}
Global nDupes@@@=nDupes@@@+Count
}
}
}
for(j=i,i+Count-1) {
Anchor=Max(Anchor,j-LagMax)
RT_DBaseSetField(DB,j,0,Stat,StartU,EndU,Anchor,j-i) # Count Field NOT SET
}
}
RT_DBaseSetField(DB,i,FLD_COUNT@@@,Count) # set SubSequence Start count field
}
i=i+Count-1 # Last K frame (next time for/next incr to i+1)
} # end for i
seqLen=(EndU-StartU-1)
Global nDone@@@=nDone@@@+seqLen
if(nDone@@@==FC) { RT_DBaseSetId(DB,1,FC) }
if(Show) {
Global nSeq@@@=nSeq@@@+1
if(seqLen>MaxSeqLen@@@) { Global MaxSeqLen@@@=seqLen Global MaxSeqPos@@@=StartU+1 }
}
Stat=RT_DBaseGetField(DB,n,FLD_STAT@@@) # Get Status of 1st SubSequence
} # End, If(pG > ThG)
} # End Stat==0
###
if(Show || (Mode>0 && Stat==FLG_DUP@@@)) {
Index=RT_DBaseGetField(DB,n,FLD_INDEX@@@)
SStart=n-Index # Start of Current SubSequence
Count=RT_DBaseGetField(DB,SStart,FLD_COUNT@@@) # Count stored only in first Record of Sub Sequence
if(Mode == 1 && Stat==FLG_DUP@@@) { # Mode==1, Exact Dupe
c.Trim(SStart-1,-1)
ShowDot?RT_Subtitle(DotE@@@):NOP
} Else if(Mode==2 && Stat==FLG_DUP@@@) { # Mode==2, Interpolate/Blend
if(Count > MaxInterp) { # Blend Morph
Weight = (Index + 1).Float / (Count+1)
Merge(c.Trim(SStart-1,-1),c.Trim(SStart+Count,-1), Weight)
ShowDot?RT_Subtitle(DotB@@@):NOP
} else if(InterpFast) { # Interpolate
if (Count==1) { I10@@@ }
Else if (Count==2) { Index==0?I20@@@:I21@@@ }
Else if (Count==3) { Index==0?I30@@@:(Index==1)?I31@@@:I32@@@ }
Else if (Count==4) { Index==0?I40@@@:(Index==1)?I41@@@:(Index==2)?I42@@@:I43@@@ }
Else if (Count==5) { Index==0?I50@@@:(Index==1)?I51@@@:(Index==2)?I52@@@:(Index==3)?I53@@@:I54@@@ }
Else if (Count==6) { Index==0?I60@@@:(Index==1)?I61@@@:(Index==2)?I62@@@:(Index==3)?I63@@@:(Index==4)?I64@@@:I65@@@ }
Else if (Count==7) { Index==0?I70@@@:(Index==1)?I71@@@:(Index==2)?I72@@@:(Index==3)?I73@@@:(Index==4)?I74@@@:
\(Index==5)?I75@@@:I76@@@ }
Else if (Count==8) { Index==0?I80@@@:(Index==1)?I81@@@:(Index==2)?I82@@@:(Index==3)?I83@@@:(Index==4)?I84@@@:
\ (Index==5)?I85@@@:(Index==6)?I86@@@:I87@@@ }
Else if (Count==9) { Index==0?I90@@@:(Index==1)?I91@@@:(Index==2)?I92@@@:(Index==3)?I93@@@:(Index==4)?I94@@@:
\ (Index==5)?I95@@@:(Index==6)?I96@@@:(Index==7)?I97@@@:I98@@@ }
ShowDot?RT_Subtitle(DotI@@@):NOP
} Else {
fillMax=3
LS=Max(SStart-fillMax,0) RS=SStart+Count # Range limit for trim (cannot eg trim prior frame 0)
LCnt=SStart-LS RCnt=Min(FC-RS,fillMax)
Tmp_c = c.Trim(LS,-LCnt) + c.Trim(RS,-RCnt) # Dupes trimmed out, with ideally two frames before & two after.
Prefilt=Tmp_c.RemoveGrain(22)
Super=Tmp_c.MSuper(hpad=SPad@@@,vpad=SPad@@@,levels=1,sharp=SSharp@@@,rfilter=SRFilter@@@)
Superfilt=Prefilt.MSuper(hpad=SPad@@@,vpad=SPad@@@,sharp=SSharp@@@,rfilter=SRFilter@@@)
bv=Superfilt.MAnalyse(isb=true ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
fv=Superfilt.MAnalyse(isb=False ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
if(RBlkSize@@@>0) {
bv=Super.MRecalculate(bv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
fv=Super.MRecalculate(fv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
}
Tmp_I=Tmp_c.MFlowInter(Super,bv,fv,time=100.0*(Index+1)/(Count+1),ml=Iml@@@,blend=IBlend@@@,thSCD1=IthSCD1@@@,thSCD2=IthSCD2@@@)
Tmp_I.Trim(LCnt-1,-1) # NO SHIFT, fixed frame @ LCnt-1 ie same as Left PreDupe frame
ShowDot?RT_Subtitle(DotI@@@):NOP
} # End, Count > MaxInterp
} Else If(Mode==0 && Stat==FLG_DUP@@@ && ShowDot) {
ShowDot?RT_Subtitle(DotD@@@):NOP
}
if(Show&&!ShowDot) {
pG = RT_FrameDifference(dc,dc,n=n,n2=n-1,ChromaWeight=CW,chromai=ChromaI) # dif to prev
pB = RT_FrameMovement(dc,dc,n=n,n2=n-1,ChromaWeight=CW,AltScan=AltSCan,ChromaI=ChromaI,
\ BlkW=BlkW,BlkH=BlkH,OLapX=OLapX,OLapY=OLApY,BlkTh=ThB,Prefix="p@@@")
pP = RT_LumaPixelsDifferent(dc,dc,n=n,n2=n-1,Thresh=ThP)
kG = RT_YPlaneMinMaxDifference(dc,n=n,Threshold=0.2)
if(Stat<=FLG_UU@@@&&Stat>=FLG_OO@@@) { # Unique-ish (Anchor not tested)
if (ShowBlk && pB>0.0) {
if(pB>ThB) { Overlay(HitMrk@@@, x=X+p@@@XOFF,y=Y+p@@@YOFF,Mask=HitMsk@@@, Mode="Blend",output=OutCS@@@) }
Else { Overlay(MissMrk@@@,x=X+p@@@XOFF,y=Y+p@@@YOFF,Mask=MissMsk@@@,Mode="Blend",output=OutCS@@@) }
}
if(Stat==FLG_OO@@@) { RT_Subtitle(FmtOO@@@,n,pG,pB,pP*100.0/255.0,kG) } # 1 User Override FLG_OO@@@
Else if(Stat==FLG_UK@@@) { RT_Subtitle(FmtUK@@@,n,pG,pB,pP*100.0/255.0,kG) } # 2 K Override
Else if(Stat==FLG_UU@@@) { RT_Subtitle(FmtUU@@@,n,(pB>ThB)?45:76,(pP>igP)?45:76,pG,pB,pP*100.0/255.0,kG) } # 3 Unique
} Else {
StartU=RT_DBaseGetField(DB,n,FLD_START@@@)
EndU=RT_DBaseGetField(DB,n,FLD_END@@@)
Anchor=RT_DBaseGetField(DB,n,FLD_ANCHOR@@@)
UIndex=n-StartU # LoMo Index relative StartU (1 relative)
UCount=EndU-StartU-1 # Total LoMo sequence count
aG=RT_FrameDifference(dc,dc,n=n,n2=Anchor,ChromaWeight=CW,chromai=ChromaI)
aB=RT_FrameMovement(dc,dc,n=n,n2=Anchor,ChromaWeight=CW,AltScan=AltSCan,ChromaI=ChromaI,
\ BlkW=BlkW,BlkH=BlkH,OLapX=OLapX,OLapY=OLApY,BlkTh=ThB,Prefix="a@@@")
aP=RT_LumaPixelsDifferent(dc,dc,n=n,n2=Anchor,Thresh=ThP)
if (ShowBlk && aB>0.0) {
if(aB>ThB) { Overlay(HitMrk@@@, x=X+a@@@XOFF,y=Y+a@@@YOFF,Mask=HitMsk@@@, Mode="Blend",output=OutCS@@@) }
Else { Overlay(MissMrk@@@,x=X+a@@@XOFF,y=Y+a@@@YOFF,Mask=MissMsk@@@,Mode="Blend",output=OutCS@@@) }
}
if(Stat==FLG_LM@@@) { # LoMo
GCol=(aG>ThG)?33:76 BCol=(aB>ThB)?33:76 PCol=(aP>IgP)?33:76
RT_Subtitle(FmtLM@@@,n,GCol,BCol,PCol,UIndex,UCount,Index+1,Count,StartU,EndU,
\ pG,pB,pP*100.0/255.0,GCol==33?33:45,aG,BCol==33?33:45,aB,PCol==33?33:45,aP*100.0/255.0,kG,Anchor,n-Anchor)
} Else if(Stat==FLG_DUP@@@) { # Dupe
ch=Mode==2?(Count>MaxInterp?66:73):68
RT_Subtitle(FmtDUP@@@,n,UIndex,UCount,ch,Index+1,Count,StartU,EndU,pG,pB,pP*100.0/255.0,aG,aB,aP*100.0/255.0,kG,Anchor,n-Anchor)
} Else if(Stat==FLG_DE@@@) { # Dupe_EndOfScene_Override FLG_DE@@@
RT_Subtitle(FmtDE@@@,n,UIndex,UCount,Index+1,Count,StartU,EndU,pG,pB,pP*100.0/255.0,aG,aB,aP*100.0/255.0,kG,Anchor,n-Anchor)
} Else if(Stat==FLG_NUK@@@) { # Dupe_KOverride FLG_NUK@@@
RT_Subtitle(FmtNUK@@@,n,UIndex,UCount,Index+1,Count,StartU,EndU,pG,pB,pP*100.0/255.0,aG,aB,aP*100.0/255.0,kG,Anchor,n-Anchor)
} Else if(Stat==FLG_DS@@@) { # Dupe_MaxDupLen_Override FLG_DS@@@
RT_Subtitle(FmtDS@@@,n,UIndex,UCount,Index+1,Count,StartU,EndU,pG,pB,pP*100.0/255.0,aG,aB,aP*100.0/255.0,kG,Anchor,n-Anchor)
} Else {
Assert(False,RT_String("%s[%d] Error Unknwn Stat(%d)",myName@@@,n,Stat))
}
}
if(Verb) {
EstFps=(nDone@@@-nDupes@@@)*FrameRate/nDone@@@
RT_SubTitle(vFmt@@@,
\ Mode,ThG,ThB,ThP,IgP*100.0/255.0,ThK,MaxDupLen,MaxInterp,LagMax,nSeq@@@,MaxSeqLen@@@,MaxSeqPos@@@,
\ nDupes@@@,DupeLenMax@@@,DupePosMax@@@,nDupeSeq@@@,nDone@@@*100.0/FC,EstFPS,Align=1)
}
}
} # End, if(Show || (Mode>0 && Stat==FLG_DUP@@@))
Return Last
}

[StainlessS]

Post #3 of many.



if(Frames!="") {
Frames=RT_GetFullPathName(Frames) RT_FileDelete(Frames)
RT_WriteFile(Frames,"###\n# Duplicate frames file. [ Generated by Duplicity2() ]\n###\n\n")
}
RT_DBaseAlloc(DB,FrameCount,"iiiiii")
### DBase Fields:-
Global FLD_STAT@@@ = 0 # Holds Status FLGS
Global FLD_START@@@ = 1 # Unique Start Frame
Global FLD_END@@@ = 2 # Unique End Frame
Global FLD_ANCHOR@@@ = 3 # Anchor Frame
Global FLD_INDEX@@@ = 4 # Index into Count (0 Relative, 0->Count-1)
Global FLD_COUNT@@@ = 5 # Sequence Frame Count
### Status Flags in Field FLD_STAT@@@
Global FLG_OO@@@ = 1 # User Override
Global FLG_UK@@@ = 2 # K Override
Global FLG_UU@@@ = 3 # Unique Frame
Global FLG_NUK@@@ = 4 # Non Unique Sequence K Override
Global FLG_LM@@@ = 5 # LoMotion Frame
Global FLG_DS@@@ = 6 # Static Sequence [detected as duplicates but Longer than MaxDupLen]
Global FLG_DE@@@ = 7 # Duplicate Sequence is followed by Scene Change (will NOT occar when Mode==1)
Global FLG_DUP@@@ = 8 # Duplicate Sequence
###
RT_DBaseSetField(DB,0,FLD_STAT@@@,FLG_OO@@@) RT_DBaseSetField(DB,FrameCount-1,FLD_STAT@@@,FLG_OO@@@) # Set first and Last Frames to Override Frame
RT_DBaseSetId(DB,0,VER)
Global nDone@@@=2 # Start and End frames already done
Global nDupes@@@=0 Global nDupeSeq@@@=0 Global DupeLenMax@@@=0 Global DupePosMax@@@=0
Global nSeq@@@=0 Global MaxSeqLen@@@=0 Global MaxSeqPos@@@=0
if(OverRide!="") {
if(ChrIsStar(Override)) {
OverRide=MidStr(OverRide,2)
} else {
Assert(Exist(OverRide),RT_String("%sOverRide File does not exist",myName))
OverRide=RT_ReadTxtFromFile(Override)
}
LINES=RT_TxtQueryLines(OverRide)
for(i=0,LINES-1) {
SS=ChrEatWhite(RT_TxtGetLine(Override,i))
S=SS
if(!ChrIsNul(S) && !ChrIsHash(S)) {
if(ChrIsDigit(S)) {
StartF = S.RT_NumberValue
EndF=StartF
S=S.ChrEatDigits.ChrEatWhite
if(ChrIsComma(S)){S=S.MidStr(2).ChrEatWhite}
IsNeg = ChrIsHyphen(S)
if(IsNeg){S=S.MidStr(2).ChrEatWhite}
if(ChrIsDigit(S)) {
EndF=S.RT_NumberValue
S=S.ChrEatDigits
if(EndF==0) {EndF=FrameCount-1 }
Else if(IsNeg) {EndF=StartF+(EndF)-1}
Assert(StartF<=EndF,RT_String("%sOverRide: Error@Line_%d, StartFrame(%d) > EndFrame(%d)\n%s",myName,i+1,StartF,EndF,SS))
} Else if(IsNeg) {Assert(False,RT_String("%sOverRide: Error@Line_%d, Expecting End Frame\n%s",myName,i+1,SS))}
Assert(EndF<FrameCount,RT_String("%sOverRide: Error@Line_%d, EndFrame(%d) Out Of Clip\n%s",myName,i+1,EndF,SS))
RT_DebugF("OverRiding Range %d,%d",StartF,EndF,name=myName)
for(n=StartF,EndF) {
if(n!=0 && n!= FrameCount-1) {
RT_DBaseSetField(DB,n,FLD_STAT@@@,FLG_OO@@@)
Global nDone@@@=nDone@@@+1
}
}
}
S=ChrEatWhite(S)
if(ChrIsHash(S)) {S=""}
Assert(ChrIsNul(S),RT_String("%sOverride: Error@Line_%d, *** NON-PARSE ***\n'%s'",myName,i+1,SS))
}
}
}
####
Global myName@@@ = myName
if(Mode==2 && MaxInterp>0) {
if(InterpFast) {
Prefilt = c.RemoveGrain(22)
Super = c.MSuper(hpad=SPad,vpad=SPad,levels=1,sharp=SSharp,rfilter=SRFilter) # One level is enough for MRecalculate
Superfilt= Prefilt.MSuper(hpad=SPad,vpad=SPad,sharp=SSharp,rfilter=SRFilter) # All levels for MAnalyse
For(i=1, MaxInterp) {
Eval(RT_String("bv=Superfilt.MAnalyse(isb=true ,blksize=ABlkSize,overlap=AOverLap,search=ASearch,delta=%d,dct=ADct)",i+1))
Eval(RT_String("fv=Superfilt.MAnalyse(isb=false,blksize=ABlkSize,overlap=AOverLap,search=ASearch,delta=%d,dct=ADct)",i+1))
if(RBlkSize>0) {
Eval(RT_String("bv=Super.MRecalculate(bv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)"))
Eval(RT_String("fv=Super.MRecalculate(fv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)"))
}
for(j=1,i) {
Eval(RT_String("Global I%d%d@@@=c.MFlowInter(Super,bv,fv,time=100.0*%d/%d," +
\ "ml=Iml,blend=IBlend,thSCD1=IthSCD1,thSCD2=IthSCD2).SelectEvery(1,%d).Trim(0,-c.FrameCount)", i,j-1, j,i+1,-j))
}
}
} Else {
Global SPad@@@=SPad Global SSharp@@@=SSharp Global SRFilter@@@=SRFilter
Global ABlkSize@@@=ABlkSize Global AOverLap@@@=AOverLap Global ASearch@@@=ASearch Global ADct@@@=ADct
Global RBlkSize@@@=RBlkSize Global ROverLap@@@=ROverLap Global RthSAD@@@=RthSAD
Global Iml@@@=Iml Global IBlend@@@=IBlend Global IthSCD1@@@=IthSCD1 Global IthSCD2@@@=IthSCD2
}
}
###
Global HitMsk@@@ = D2_Hit_Marker(BlkW,BlkH,true)
Global HitMrk@@@ = HitMsk@@@.BlankClip(Color=$FFCC00,pixel_type="YV12")
Global MissMsk@@@ = D2_Hit_Marker(BlkW,BlkH,false)
Global MissMrk@@@ = MissMsk@@@.BlankClip(Color=$00FF00,pixel_type="YV12")
Global OutCS@@@=IsYV12?"YV12":IsY8?"Y8":IsYV16?"YV16":"YV24"
####
Global FmtOO@@@="%d] \a!O\aLKSE\a-:\aLGBP:ULD\a- \npG=%9.6f : pB=%9.6f : pP%%=%9.6f\n{K=%d}"
Global FmtUK@@@="%d] \aLO\a!K\aLS\a-:\aLGBP\a-:\aLULD\a-\n\a!pG=%9.6f\a- : pB=%9.6f : pP%%=%9.6f\n{K=%d}"
Global FmtUU@@@="%d] \aLOKSE\a-:\a!G\a%cB\a%cP\a-:U\aLLD\a- \n\a!pG=%9.6f\a- : pB=%9.6f : pP%%=%9.6f\n{K=%d}"
Global FmtLM@@@="%d] \aLOKSE\a-:\a%cG\a%cB\a%cP\a-:\aLU\a-L\aLD\a- {%d/%d:L%d/%d:uS=%d:uE=%d}\npG=%9.6f : pB=%9.6f : pP%%=%9.6f\n"
\ + "\a%caG=%9.6f : \a%caB=%9.6f\a- : \a%caP%%=%9.6f\a-\n{K=%d : a=%d : Lag=%d}"
Global FmtDUP@@@="%d] \aLOKSE\a-:\aLGBP\a-:\aLUL\a-D\aL\a- {%d/%d:%c%d/%d:uS=%d:uE=%d}\npG=%9.6f :" +
\ " pB=%9.6f : pP%%=%9.6f\naG=%9.6f : aB=%9.6f : aP%%=%9.6f\n{K=%d : a=%d : Lag=%d}"
Global FmtDE@@@="%d] \aLOKS\a!E\a-:\aLGBP\a-:\aLULD\a- {%d/%d:E%d/%d:uS=%d:uE=%d}\npG=%9.6f : pB=%9.6f : pP%%=%9.6f\naG=%9.6f :" +
\ " aB=%9.6f : aP%%=%9.6f\n{K=%d : a=%d : Lag=%d}"
Global FmtNUK@@@="%d] \aLO\a!K\aLSE\a-:\aLGBP\a-:\aLULD\a- {%d/%d:K%d/%d:uS=%d:uE=%d}\npG=%9.6f : pB=%9.6f : pP%%=%9.6f\naG=%9.6f :" +
\ " aB=%9.6f : aP%%=%9.6f\n{\a!K=%d\a- : a=%d : Lag=%d}"
Global FmtDS@@@="%d] \aLOK\a!S\aLE\a-:GBP\a-:\aLULD\a- {%d/%d:S%d/%d:uS=%d:uE=%d}\npG=%9.6f : pB=%9.6f : pP%%=%9.6f\naG=%9.6f :" +
\ " aB=%9.6f : aP%%=%9.6f\n{K=%d : a=%d : Lag=%d}"
Global VFmt@@@="Md=%d:ThG=%.3f:ThB=%.3f:ThP=%d:IgP%%=%.3f\nThK=%d:MaxDupLen=%d:MaxInterp=%d:LagMax=%d\nNuSeq: Cnt=%d:LenMax=%d@%d\nDupe:" +
\ " Cnt=%d:MaxLen=%d@%d:nSeq=%d\nComplete%%=%.3f:EstFPS=%.3f"
Global DotD@@@="\a2*\a!D\a9*\n\a- D\n\a4*\a9D\a!*"
Global DotE@@@="\a2*\a!E\a9*\n\a- E\n\a4*\a9E\a!*"
Global DotI@@@="\a2*\a!I\a9*\n\a- I\n\a4*\a9I\a!*"
Global DotB@@@="\a2*\a!B\a9*\n\a- B\n\a4*\a9B\a!*"
###
IgP=IgP*255.0/100.0 # To range 0.0 -> 255.0
ARGS = "dc,DB,Mode,ThG,ThB,ThP,IgP,ThK,MaxDupLen,LagMax,MaxInterp,BlkW,BlkH,OLapX,OLapY,X,Y,CW,ChromaI,Altscan," +
\ "OthSCD1,OthSCD2,Frames,Ranges,FrameAsRange,Show,Verb,ShowBlk,ShowDot,InterpFast"
ScriptLine = "Fn@@@("+ARGS+")"
Last.GScriptClip(ScriptLine, local=true, args=ARGS)
"""
GIFunc="DUPLICITY2" # Function Name, Supply unique name for your multi-instance function.
GIName=GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# Write current instance Debug GScript File to current directory (line numbers will correspond to error messages)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
HasGScript ? GScript(InstS) : Eval(InstS,"GS_EVAL") # Use GSCript if installed (loaded plugs override builtin)
# if UnNamed DBase && CallCmd available, Auto delete DBase file on clip closure.
(DBase=="" && HasCallCmd) ? CallCmd(close=RT_String("""CMD /C chcp 1252 && del "%s" """,DB), hide=true, Synchronous=7) : NOP
Return Last
}

[StainlessS]

Post #4 of Many.

Client

Import(".\Duplicity2.Avsi")

######
FN = "D:\G.AVI" DBase = "D:\G_Det.DB" ThB=2.0
#FN = "D:\GTEST.AVI" DBase = "D:\GTEST_Det.DB" ThB=2.0
#FN = "D:\HAR.AVI" DBase = "D:\HAR_Det.DB" ThB=2.5
#FN = "D:\DeDupTest.AVI" DBase = "D:\DeDupTest_Det.DB" ThB=2.0
######
Avisource(FN)
SRC=Last
###
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
ThG=1.0 # Primary Frame Global Detect
ThP=64 # Pixel Count Detect
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
InterpFast=true
Show=True
ShowDot=False
ShowBlk=true
VERB=False
###
TITLE=False
STACK=True
STACK4=False
###

Result=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=False,InterpFast=InterpFast,DBase=(STACK4||!SHOW)?DBase:"")

Metrics=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=True,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,InterpFast=InterpFast,DBase=(STACK4||!SHOW)?"":DBase)

Dif=Subtract(Result,SRC)

SRC = (TITLE) ? SRC.TSub("Src") : SRC
Result = (TITLE) ? Result.TSub("Result") : Result
Metrics = (TITLE) ? Metrics.TSub("Metrics") : Metrics
Dif = (TITLE) ? Dif.TSub("Dif(Result,Src)") : Dif

LFT=StackVertical((STACK4||!SHOW)?Result:Metrics,SRC)
RGT=StackVertical(Metrics,Dif)
STK4=StackHorizontal(LFT,RGT)

(STACK4) ? STK4 : (STACK) ? LFT : (Show)?Metrics:Result

Return Last


# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}

[StainlessS]

Post #5 of 6.

Post removed. NOW SPARE

[StainlessS]

Post #6 of 6.

Post removed. NOW SPARE

[StainlessS]

Update, Thread Renamed, to Duplicity2, see first post.

Posts #22 to 27 updated to current script.

[StainlessS]

Anybody wanna test this out, my eyes dont see any difference (crap eyes).

Duplicity2_v2.02_Beta LINK REMOVED

Have implemented FillDrops modes.


Function Duplicity2(clip c,"Mode"=0,Float "ThG"=1.0,Float "ThB"=ThG, Int "ThP"=64,Float "IgP"=0.0,Int "ThK"=7,
\ Int "MaxDupLen"=9,Int "LagMax"=MaxDupLen,Int "MaxInterp"=Min(9,MaxDupLen),
\ int "BlkW"=64,int "BlkH"=BlkW,int "OLapX"=BlkW/2,Int "OLapY=BlkH/2",
\ Int "X"=0,Int "Y"=0,Int"W"=0,Int "H"=0,Float "CW"=0.0,Bool "ChromaI"=False,Bool "AltScan"=False,
\ Int "SPad"=16, Int "SSharp"=1, Int "SRFilter"=4, [* MSuper *]
\ Int "ABlkSize"=16, Int "AOverlap"=4, Int "ASearch"=3,Int "ADct"=0, [* MAnalyse *]
\ Int "RBlkSize"=8, Int "ROverlap"=2, Int "RthSAD"=100, [* MRecalculate *]
\ Float "Iml"=200.0, Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
\ Int "OthSCD1"=400,Int "OthSCD2"=130, [* Dupe Detect Override @ Scene Change *]
\ String "OverRide"="",String "Frames"="Frames.txt",Bool "Ranges"=True,Bool "FrameAsRange"=False,
\ Bool "Show"=True,Bool "Verb"=True,Bool "ShowBlk"=True,Bool "Dim"=True,Bool "ShowDot"=False,
\ Int "FillDrops"=0
\ )

...


FillDrops, Default 0 (OFF). [Valid only for Mode=2, Interpolate]
0) Standard precalculated Interpolation clips mode. (Fastest)
1) Standard Interpolated clip mode, except for N_Dupes==1, where Precalculated FillDrops clip method.
2) FillDrops Interpolated mode, except for N_Dupes==1, where Precalculated FillDrops clip.
3) Pure FillDrops mode where all Interpolation clips created during frameserving. (Slowest).


Gavino explanation of FillDrops mode with Delta=1.
https://forum.doom9.org/showthread.php?p=1409378#post1409378
You're right, the function certainly does work, and now I see why.
What happens is that if a duplicate is found at frame n, it interpolates between n and n+1, since delta=1.
At first sight, this is wrong, but since frame n is a duplicate of frame n-1, it is equivalent to the desired interpolation between n-1 and n+1, so the final result comes out OK. (Whether by accident or by design, I'm not sure. )

And Also here:- https://forum.doom9.org/showthread.php?p=1409954#post1409954
Didée's explanation is the same as mine (but clearer!), and would suggest that the delta=2 approach cannot be improved upon. Your results are therefore puzzling and invite further explanation, which I believe I have found.

By looking at the source code, I have discovered that for occlusion areas, MVFlowInter also uses information from the two further surrounding frames (current-delta and current+2*delta).
So the diagram becomes, when interpolating between C and C+d,
----> <----
C-d C C+d C+2d
----> <----
It is this additional information that will be more accurate when delta=1, explaining your findings and making your two-stage approach a better one. Well done.

Just to clarify further my remark about ConditionalFilter, what I am saying is this.
In ConditionalFilter(c1, c2, c3, "...", "...", "..."), 'last' is always set to the value of 'c1' before the expression strings are evaluated (on each frame). Therefore your code
goodframes = ConditionalFilter(clip, trim(clip,1,0), clip, \
"(YDifferenceFromPrevious(clip)/YDifferenceFromPrevious(loop(clip,2,0,0))) * \
(YDifferenceToNext(clip)/YDifferenceToNext(trim(clip,1,0)))", "greaterthan", "bad_thresh")
can be replaced by
goodframes = ConditionalFilter(c, trim(c,1,0), c, \
"(YDifferenceFromPrevious()/YDifferenceFromPrevious(loop(2,0,0))) * \
(YDifferenceToNext()/YDifferenceToNext(trim(1,0)))", "greaterthan", "bad_thresh")
and similarly for 'fixed', with the global variable being removed.
Using a global variable for this purpose in a function is in general a bad idea as the function will not work if called more than once.

Above in BLUE what I mean by FillDrops mode, allows extra data from additional frames where occlusions occur, and so better quality interpolation. The FillDrops=3 mode, we also trim out all dupes and so even better quality than original FillDrops method should be expected.
For the stuff about extra data from additional frames where occlusions occur, because in the filldrop original method the n-1 frame is the frame that n is a duplicate of, and so part of the extra data (where occlusions) comes from n-1 (dupe) and so is no better than using n,
the only extra info comes from n+2. For our Filldrops mode=3, all dupes are trimmed out (no matter what length, 0-9) and interpolation using frames either side of dupe sequence, so we expect even better than original FillDrops method.

We can only pre-create Interpolated FillDrops clips (created before framesering starts) where number of consecutive dupes==1.
N_dupes==1 pre-calc clip used for FillDrops modes 1 and 2 (exactly same as in FillDrops script), but for FillDrops mode==3,
then duplicates are trimmed out (for identical to FillDrops mode with delta=1, the frame of which the single dupe is a copy, is not trimmed out
and so takes some small part in result, whereas for FillDrops mode=3, all dupes trimmed out and so play NO PART in result).


# FillDrops Mode==2 and longer than single dupe (For single dupe, fix with precalc clip I10@@@ above)
# OR, FillDrops==3 and all length dupes interpolated Slow FillDrops method
# NOTE, FillDrops mode uses DELTA=1, and NO SHIFT,
fillMax=3
LS=Max(SStart-fillMax,0) RS=SStart+Count # Range limit for trim (cannot eg trim prior frame 0)
LCnt=SStart-LS RCnt=Min(c.FrameCount-RS,fillMax)
Tmp_c = c.Trim(LS,-LCnt) + c.Trim(RS,-RCnt) # Dupes trimmed out, with ideally two frames before & two after.
Prefilt=Tmp_c.RemoveGrain(22)
Super=Tmp_c.MSuper(hpad=SPad@@@,vpad=SPad@@@,levels=1,sharp=SSharp@@@,rfilter=SRFilter@@@)
Superfilt=Prefilt.MSuper(hpad=SPad@@@,vpad=SPad@@@,sharp=SSharp@@@,rfilter=SRFilter@@@)
bv=Superfilt.MAnalyse(isb=true ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
fv=Superfilt.MAnalyse(isb=False ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
if(RBlkSize@@@>0) {
bv=Super.MRecalculate(bv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
fv=Super.MRecalculate(fv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
}
Tmp_I=Tmp_c.MFlowInter(Super,bv,fv,time=100.0*(Index+1)/(Count+1),ml=Iml@@@,blend=IBlend@@@,thSCD1=IthSCD1@@@,thSCD2=IthSCD2@@@)
Tmp_I.Trim(LCnt-1,-1) # NO SHIFT, fixed frame @ LCnt-1 ie same as Left PreDupe frame


@Zorr, in part, is what you asked in this thread:- https://forum.doom9.org/showthread.php?t=175373

@John Meyer, The N_Dupes==1, for both FillDrops modes=1 and 2, then should be (I think) identical result to your filldrops method.
For N_Dupes > 1 and FillDrops mode=2, or any N_dupes>0 and FillDrops mode==3, then all dupes trimmed out and play no part in
interpolation, so perhaps even better than original FillDrops where result affected by single dupe.


Also John, any chance you could post all yiou know about the FillDrops weirdness, I have a clip where there seems to be 2 dropped frames(big jump)
and then some time later (some times immediately after) there are two dupes,
ALSO, sometimes single frame dropped, and single dupe some time later.
Is there a maximum distance between drops, max number of dropped, max number of dupes etc, any info/rules that you may have observed, thanx.

EDIT: Included in zip, shows difference (where Mode=2) between two Filldrops modes.

#Import("Duplicity2.Avs")

FN="d:\HAR.AVI"
AviSource(FN)
#########
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
ThG=1.0 # Primary Frame Global Detect
ThB=1.5 # Block Detect
ThP=64 # Pixel Count Detect
Show=True
ShowDot=false
ShowBlk=True
VERB=True
FillDropsA=1 # For A clip only
FillDropsB=3 # For B clip only (return default if STACK4=False)
###
STACK4=True
ORG=Last

A=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsA
\ )

B=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsB
\ )

D1=Subtract(A,B).Levels(125,1,127,0,255,coring=false)
LFT=StackVertical(A,B)
RGT=StackVertical(ORG,D1)
Return STACK4?StackHorizontal(LFT,RGT):B

[bblogoss]

I am testing a clip full of exact duplicates (metric=0.00000), so detection is not a problem, but I still get some frames that are not interpolated (generally when there are 2 or more consecutives dups) even with MaxDupLen and MaxInterp = 9
Is it normal ?

[StainlessS]

No idea, what do the metrics show at the dupes.
What is the length of the skipped seqs, ? (greater than MaxDupLen ?)
Are they flagged as Low motion ?

Post image of non detect dupe on PostImage.org, with show=true, Verb=True.

Are you using most recent script ?

[bblogoss]

I think I get it, the frames are flagged as E=End+1

like it says in the script E=End+1 of current dupe sequence is change of scene, so if Mode=2 then Interpolation or Blending is skipped for that dupe sequence

It's a big jump detected as a scene change. But now even when trying to switch OFF scene change detection with OthSCD1 and OthSCD2 the are stilled flagged as E

What should we exactly do to avoid scene change detection ?

OthSCD2=256 or OthSCD1=8*8*255 and no change at all.

[StainlessS]

OK, I think I see what problem is, perhaps I should have labored it in docs a little more.

OthSCD2 >=255 will switch off End + 1 scene change detect (which would avoid any attempt to interpolate or count sequence as duplicates,
might want to avoid any processing at all where possibility of of creating blends across scene change [also omit from any Frames file]).

In addition to End + 1 scene change detect, the Interplator (MFlowInter) also has scene change detection args, which may need switching off also.
Take special note of ALL below args

Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
Int "OthSCD1"=400,Int "OthSCD2"=130, [* Dupe Detect Override @ Scene Change *]


EDIT:
are stilled flagged as E
Sorry, missed that, which script are you using ?

EDIT: Ignore most of above, me a bit confused.
Post a sequence (PM if like), with about 5 frames prior to first in any enclosing low motion sequence, + 5 frames after end frame,
also say which script you are using (v2.01 or v2.02) and provide client script.
If you provide OthSCD2>=255 then it should not set E flag ie

if(Count>MaxDupLen) {Stat=FLG_DS@@@} # MaxDupLen OverRide ### STATIC SCENE
Else {
if(Mode!=1 && i+Count==EndU && OthSCD2 < 255 && OthSCD1 < 8*8*255) {
Tmp_c = c.Trim(i-1,-1) + c.Trim(i+Count,-1) ### IN HERE ONLY IF OthSCD2 < 255
### BELOW ONLY PLACE WHERE stat set to FLG_DE@@@
Stat=(EndOfSceneClip(Tmp_c,OthSCD1,OthSCD2).RT_AverageLuma(n=0,w=1,h=1).Int != 0)?FLG_DE@@@:FLG_DUP@@@
} Else {Stat=FLG_DUP@@@}
# ...
}

[bblogoss]

Ok, problem is solved. I was using V2.01 and still having the E flag no matter if OthSCD2 or IthSCD2 >=255

in V2.02 after changing mode!=1 to mode!=2 in the part of the script you provide it seems Ok now

if(Count>MaxDupLen) {Stat=FLG_DS@@@} # MaxDupLen OverRide ### STATIC SCENE
Else {
if(Mode!=2 && i+Count==EndU && OthSCD2 < 255 && OthSCD1 < 8*8*255) {
Tmp_c = c.Trim(i-1,-1) + c.Trim(i+Count,-1) ### IN HERE ONLY IF OthSCD2 < 255
### BELOW ONLY PLACE WHERE stat set to FLG_DE@@@
Stat=(EndOfSceneClip(Tmp_c,OthSCD1,OthSCD2).RT_AverageLuma(n=0,w=1,h=1).Int != 0)?FLG_DE@@@:FLG_DUP@@@
} Else {Stat=FLG_DUP@@@}
# ...
}

Thanks for the help :)

[StainlessS]

That will not be best fix, should defo not enter that code block if OthSCD2 >= 255,
That fix just kludge fix and will move problem to mode==1 (will not skip END+1 switch off if mode==1 [Replace With Exact Dupes mode]).

Try replace with this, what does debugview show


if(Count>MaxDupLen) {Stat=FLG_DS@@@} # MaxDupLen OverRide ### STATIC SCENE
Else {
if(Mode!=1 && i+Count==EndU && OthSCD2 < 255 && OthSCD1 < 8*8*255) {
RT_DebugF("%d] OthSCD1=%d OthSCD2=%d",i,OthSCD1,OthSCD2)
Tmp_c = c.Trim(i-1,-1) + c.Trim(i+Count,-1) ### IN HERE ONLY IF OthSCD2 < 255
### BELOW ONLY PLACE WHERE stat set to FLG_DE@@@
Stat=(EndOfSceneClip(Tmp_c,OthSCD1,OthSCD2).RT_AverageLuma(n=0,w=1,h=1).Int != 0)?FLG_DE@@@:FLG_DUP@@@
RT_DebugF("%d] Stat=%d",i,Stat)
} Else {Stat=FLG_DUP@@@}
# ...
}

Above i, is first duplicate in dupe run.

[bblogoss]

I'm not so used with debug mode. What should I exactly do after adding the two RT_DebugF lines ?

[StainlessS]

DebugView, by SysInternals (now owned by M$):- https://docs.microsoft.com/en-us/sysinternals/downloads/debugview

NOTE, some other software may also output to debugview.

There is a massive number of free utilities from same site, and can download the whole lot as a single zip.
(SysInternals Suite, a few lines below in the LHS pane)

EDIT: I also like (amongst others from that site) AutoLogon, does what it says on the tin.

[bblogoss]

Well here is what Debugview shows

00000001 0.00000000 [15600] FFMS2 avs plugin: Initializing...
00000002 0.00003070 [15600] FFMS2 - avs 2.6 mode
00000003 22.96657753 [15600] RT_DebugF: 522] OthSCD1=400 OthSCD2=130
00000004 22.97030830 [15600] RT_DebugF: 522] Stat=8
00000005 23.06452370 [15600] RT_DebugF: 525] OthSCD1=400 OthSCD2=130
00000006 23.06820679 [15600] RT_DebugF: 525] Stat=8
00000007 23.30171204 [15600] RT_DebugF: 531] OthSCD1=400 OthSCD2=130
00000008 23.30541229 [15600] RT_DebugF: 531] Stat=8
00000009 23.53751755 [15600] RT_DebugF: 538] OthSCD1=400 OthSCD2=130
00000010 23.54120636 [15600] RT_DebugF: 538] Stat=8
00000011 23.73814964 [15600] RT_DebugF: 544] OthSCD1=400 OthSCD2=130
00000012 23.74173164 [15600] RT_DebugF: 544] Stat=8
00000013 23.93748283 [15600] RT_DebugF: 550] OthSCD1=400 OthSCD2=130
00000014 23.94121933 [15600] RT_DebugF: 550] Stat=8
00000015 24.13818169 [15600] RT_DebugF: 556] OthSCD1=400 OthSCD2=130
00000016 24.14183044 [15600] RT_DebugF: 556] Stat=8
00000017 24.33157349 [15600] RT_DebugF: 562] OthSCD1=400 OthSCD2=130
00000018 24.33519936 [15600] RT_DebugF: 562] Stat=8
00000019 24.53227234 [15600] RT_DebugF: 568] OthSCD1=400 OthSCD2=130
00000020 24.53590775 [15600] RT_DebugF: 568] Stat=8
00000021 24.73389816 [15600] RT_DebugF: 574] OthSCD1=400 OthSCD2=130
.......
........

That's the default settings. Am I missing something or it seems like the OthSCD2 setting in the client part of the script is not loaded ?
After manually changing the default setting of OthSCD2 in the script there is no E flag and DebugView says logically nothing

so here is the client part of the script

Import("Duplicity2_v2.02_Beta.Avs")

FN="server.vdr"
AviSource(FN).ConvertToYV24(matrix="Rec601")
#########
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
ThG=1.0 # Primary Frame Global Detect
ThB=1.5 # Block Detect
ThP=64 # Pixel Count Detect
OthSCD2=255
Show=true
ShowDot=false
ShowBlk=false
VERB=false
FillDropsA=1 # For A clip only
FillDropsB=3 # For B clip only (return default if STACK4=False)
###
STACK4=false
ORG=Last

A=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsA
\ )

B=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsB
\ )

D1=Subtract(A,B).Levels(125,1,127,0,255,coring=false)
LFT=StackVertical(A,B)
RGT=StackVertical(ORG,D1)
Return STACK4?StackHorizontal(LFT,RGT):B

[StainlessS]

OOoops, how embarrassing, I did not spot the fact that we did not call the functions wth OthSCD2, ie


Import("Duplicity2_v2.02_Beta.Avs")

FN="server.vdr"
AviSource(FN).ConvertToYV24(matrix="Rec601")
#########
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
ThG=1.0 # Primary Frame Global Detect
ThB=1.5 # Block Detect
ThP=64 # Pixel Count Detect
OthSCD2=255
IthSCD2=255
Show=true
ShowDot=false
ShowBlk=false
VERB=false
FillDropsA=1 # For A clip only
FillDropsB=3 # For B clip only (return default if STACK4=False)
###
STACK4=false
ORG=Last

A=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsA,OthSCD2=OthSCD2,IthSCD2=IthSCD2
\ )

B=ORG.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,
\ MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ CW=0.0,ChromaI=False,AltScan=False,
\ Ranges=True,FrameAsRange=False,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,FillDrops=FillDropsB,OthSCD2=OthSCD2,IthSCD2=IthSCD2
\ )

D1=Subtract(A,B).Levels(125,1,127,0,255,coring=false)
LFT=StackVertical(A,B)
RGT=StackVertical(ORG,D1)
Return STACK4?StackHorizontal(LFT,RGT):B


Just too obvious I guess :(

[bblogoss]

Everything works like a charm now :thanks:

Also, does it change something if we save the main script as an AVSI file in the plugin directory and then make a final script in a classic way ?

AviSource("videofile.avi")
Duplicity2(mode=2, show=false, ShowBlk=false, OthSCD2=255, IthSCD2=255, FillDrops=3, IBlend=false)

[StainlessS]

Should work fine as avsi in plugs directory.

[StainlessS]

Duplicity2, v2.10, new version, see 1st post.

Main diffs.

Function Duplicity2(clip c,"Mode"=0,Float "ThG"=1.0,Float "ThB"=ThG*2.0, Int "ThP"=64,Float "IgP"=0.0,Int "ThK"=7,
\ Int "MaxDupLen"=9,Int "LagMax"=MaxDupLen,Int "MaxInterp"=Min(9,MaxDupLen),
\ int "BlkW"=64,int "BlkH"=BlkW,int "OLapX"=BlkW/2,Int "OLapY=BlkH/2",
\ Int "X"=0,Int "Y"=0,Int"W"=0,Int "H"=0,Float "CW"=0.0,Bool "ChromaI"=False,Bool "AltScan"=False,
\ Int "SPad"=16, Int "SSharp"=1, Int "SRFilter"=4, [* MSuper *]
\ Int "ABlkSize"=16, Int "AOverlap"=4, Int "ASearch"=3,Int "ADct"=0, [* MAnalyse *]
\ Int "RBlkSize"=8, Int "ROverlap"=2, Int "RthSAD"=100, [* MRecalculate *]
\ Float "Iml"=200.0, Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
\ Int "OthSCD1"=400,Int "OthSCD2"=130, [* Dupe Detect Override @ Scene Change *]
\ String "OverRide"="",String "Frames"="Frames.txt",Bool "Ranges"=True,Bool "FrameAsRange"=False,
\ Bool "Show"=True,Bool "Verb"=True,Bool "ShowBlk"=True,Bool "Dim"=True,Bool "ShowDot"=False,
\ Bool "InterpFast"=True,String "DBase"=""
\ )



InterpFast, Default True. If True, then Interpolation clips are pre-calculated, if false then calculations done in real time.
Pre-Calc clips a bit (maybe 10%) faster, but perhaps not quite as good quality.

DBase, Default "", If named DBase suppled, then is not deleted on Function exit.
[ If name supplied, result DB may be re-used by other scripts, eg DropDeadGorgeous() function ].
NOTE, If using multiple instances and named DBase, then give each DBase as different name.

[StainlessS]

Also see previous post (needed by this script).

Here its play time,


/*

DropDeadGorgeous() v -1.0, Beta -1, by the Gorgeous & Truly Scrumptious StainlessS @ Doom9

Using DBase as Created by Duplicity2 v2.10,
Corrects problems where frames were dropped in clip, and then some time later duplicates were inserted to maintain timing.
ScanAhead is number of frames scanned ahead of current frame looking for duplicates, currently no maximum, default 9.
If duplicates found (no current limit on length of duplicate run), then biggest frame jump (movement) prior to dupes is
located, and a number of frames interpolated prior to it (same number as in dupe run).
Fixes jumpy jaunty clips that constantly and repeatedly jump and then freeze.

Simple metrics on Show, "nnn] CIR", where nnn is frame number, and character Flags highlited where,
C=Copy frame (original as source frame)
I=Interpolated frame
R=Relocated frame from earlier in clip (interpolated frames shove following frames to later in clip, removing dupes)

Can use Duplicity Modes 1 or 2 in creating DBase.

*/

Function DropDeadGorgeous(clip c,String DB,Int "ScanAhead",
\ Int "X",Int "Y",Int"W",Int "H",
\ Int "SPad", Int "SSharp", Int "SRFilter", [* MSuper *]
\ Int "ABlkSize", Int "AOverlap", Int "ASearch",Int "ADct", [* MAnalyse *]
\ Int "RBlkSize", Int "ROverlap", Int "RthSAD", [* MRecalculate *]
\ Float "Iml", Bool "IBlend", Int "IthSCD1", Int "IthSCD2", [* MFlowInter *]
\ Bool "Show"
\ ){
c myName="DropDeadGorgeous: "
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
HasGRunt =RT_FunctionExist("GScriptClip") HasMvTools=RT_FunctionExist("MSuper")
HasMaskTools=RT_FunctionExist("MT_Lutxy") HasRemoveGrain=RT_FunctionExist("RemoveGrain")
HasCallCmd=RT_FunctionExist("CallCmd")
ScanAhead = Default(ScanAhead,9)
X=default(X,0) Y=default(Y,0) W=default(W,0) H=default(H,0)
SPad = Default(SPad,16) SSharp = Default(SSharp,1) SRFilter= Default(SRFilter,4)
ABlkSize = Default(ABlkSize,16) AOverlap= Default(AOverlap,4) ASearch = Default(ASearch,3) ADct = Default(ADct,0)
RBlkSize = Default(RBlkSize,8) ROverlap= Default(ROverlap,2) RthSAD = Default(RthSAD,100)
Iml = Default(Iml,200.0) IBlend = Default(IBlend,True) IthSCD1 = Default(IthSCD1,400) IthSCD2= Default(IthSCD2,130)
Show = Default(Show,False)
Assert(0 < ScanAhead,RT_String("%s0 < ScanAhead(%d)",myName,ScanAhead))
####
Assert(DB!="",RT_String("%sDB Cannot be ''",myName))
DB1 = RT_GetFullPathName(DB)
DB2 = RT_GetFullPathName("~@@@_"+RT_LocalTimeString+".DB") RT_DBaseAlloc(DB2,FrameCount,"iii")
W = (W > 0) ? (W + X) - Width : W H = (H > 0) ? (H + Y) - Height : H # Make -ve relative width,height
X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP (inwards) Mod 4
W = (W-3) / 4 * 4 H = (H-3) / 4 * 4 # Round DOWN(inwards) Mod 4
W = Width - X + W H = Height - Y + H # +ve width,height
DC=Crop(X,Y,W,H)
Global G_DIFMOD=1 # Tmp setting
FuncS="""
Function Fn@@@(clip c,clip dc,String DB1,String DB2,int ScanAhead,Bool Show) {
c n=current_frame
Stat2 = RT_DBaseGetField(DB2,n,0)
if(Stat2 == 0) {
Stat1 = RT_DBaseGetField(DB1,n,FLD_STAT@@@)
if(Stat1 < FLG_DE@@@ && SOS@@@.RT_AverageLuma(n=n,w=1,h=1).Int == 0) { # Non dupe and NOT Start Of Scene
de = Min(n+(ScanAhead*2),FrameCount-1)
DupN = -1
for(i=n+1,de) { if(RT_DBaseGetField(DB1,i,FLD_STAT@@@) >= FLG_DE@@@) { DupN = i i=de } }
if(DupN<0) {
ds=Max(de-ScanAhead,n+1)
for(i=n,ds-1) { RT_DBaseSetField(DB2,i,0,1) } # Copy
} Else { # Both n and DupN, NOT SOS
ds = Max(DupN - ScanAhead, n)
for(i=DupN-1,ds, -1) {
if(SOS@@@.RT_AverageLuma(n=i,w=1,h=1).Int != 0) { tmp=i i=ds ds=tmp+1}
}
for(i=n,ds-1) { RT_DBaseSetField(DB2,i,0,1) } # Copy
if(ds<DupN) {
Index=RT_DBaseGetField(DB1,DupN,FLD_INDEX@@@)
SSTart=DupN-Index
Count=RT_DBaseGetField(DB1,SSTart,FLD_COUNT@@@) # Dup Count stored only in first Record of Sub Sequence
mxi = ds
if(G_DIFMOD==0) { mxdif = MFPC@@@.RT_AverageLuma(n=ds) }
Else { mxdif = dc.RT_LumaDifference(dc,n=ds,n2=ds-1) }
for(i=ds+1,DupN-1) {
if(G_DIFMOD==0) { dif = MFPC@@@.RT_AverageLuma(n=i) }
Else { dif = dc.RT_LumaDifference(dc,n=i,n2=i-1) }
if(dif>mxDif) {
mxDif=dif
mxi = i
}
}
for(i=mxi+Count,DupN+Count-1) {
RT_DBaseSetField(DB2,i,0,2) # Copy Frame relative
RT_DBaseSetField(DB2,i,1,COUNT) # Copy Frame
}
for(i=mxi,mxi+Count-1) {
RT_DBaseSetField(DB2,i,0,3) # Interp
RT_DBaseSetField(DB2,i,1,COUNT) # Interp count
RT_DBaseSetField(DB2,i,2,i-mxi) # Index
}
for(i=ds,mxi-1) {
RT_DBaseSetField(DB2,i,0,1)
} # copy
}
}
} Else { RT_DBaseSetField(DB2,n,0,1) } # Dupe or Start Of Scene
Stat2 = RT_DBaseGetField(DB2,n,0)
}
if(Stat2==1) { # Copy (already correct frame)
Stat1 = RT_DBaseGetField(DB1,n,0)
if(Show) { RT_Subtitle("%d] C\aLIR",n) }
} Else if(Stat2==2) { # Copy previous relative
Count = RT_DBaseGetField(DB2,n,1)
Stat1 = RT_DBaseGetField(DB1,n,0)
c.Trim(n-Count,-1)
if(Show) { RT_Subtitle("%d] \aLCI\a-R",n) }
} Else if(Stat2==3) {
Index = RT_DBaseGetField(DB2,n,2)
Count = RT_DBaseGetField(DB2,n,1)
SStart= n-Index
fillMax=3
LS=Max(SStart-fillMax,0) RE=Min(SStart+FillMax-1,c.FrameCount-1)
LCnt=SStart-LS
Tmp_c = c.Trim(LS,-(RE-LS+1))
Prefilt=Tmp_c.RemoveGrain(22)
Super=Tmp_c.MSuper(hpad=SPad@@@,vpad=SPad@@@,levels=1,sharp=SSharp@@@,rfilter=SRFilter@@@)
Superfilt=Prefilt.MSuper(hpad=SPad@@@,vpad=SPad@@@,sharp=SSharp@@@,rfilter=SRFilter@@@)
bv=Superfilt.MAnalyse(isb=true ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
fv=Superfilt.MAnalyse(isb=False ,blksize=ABlkSize@@@,overlap=AOverLap@@@,search=ASearch@@@,delta=1,dct=ADct@@@)
if(RBlkSize@@@>0) {
bv=Super.MRecalculate(bv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
fv=Super.MRecalculate(fv,blksize=RBlkSize@@@,overlap=ROverLap@@@,thSAD=RthSAD@@@)
}
Tmp_I=Tmp_c.MFlowInter(Super,bv,fv,time=100.0*(Index+1)/(Count+1),ml=Iml@@@,blend=IBlend@@@,thSCD1=IthSCD1@@@,thSCD2=IthSCD2@@@)
Tmp_I.Trim(LCnt-1,-1) # NO SHIFT, fixed frame @ LCnt-1 ie same as Left PreDupe frame
Stat1 = RT_DBaseGetField(DB1,n,0)
if(Show) { RT_Subtitle("%d] \aLC\a-I\aLR",n) }
} Else {
RT_Subtitle("%d] %d %d BUG BUG BUG",n,Stat1,Stat2)
}
Return Last
}
Function StartOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at SOS, else 0
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MSCDetection(fv,thSCD1=thSCD1,thSCD2=thSCD2)
}
Function MMaskFromPrevClip(clip c,Int "MaskType",Float "Gamma",Int "thSCD1",Int "thSCD2") {
# MaskType:- 0=Motion, 1=SAD, 2=Occlusion, 3=Horizontal, 4=Vertical, 5=ColorMap.
MaskType=Default(MaskType,0) Gamma=Default(Gamma,1.0) thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MMask(fv,Gamma=1.0,kind=MaskType,thSCD1=thSCD1,thSCD2=thSCD2)
}
Global SOS@@@ = dc.StartOfSceneClip(thSCD1=IthSCD1,thSCD2=IthSCD2)
Global MFPC@@@= dc.MMaskFromPrevClip(Gamma=2.0,MaskType=0,thSCD1=IthSCD1,thSCD2=IthSCD2)
### DB1 Fields:-
Global FLD_STAT@@@ = 0 # Holds Status FLGS
Global FLD_START@@@ = 1 # Unique Start Frame
Global FLD_END@@@ = 2 # Unique End Frame
Global FLD_ANCHOR@@@ = 3 # Anchor Frame
Global FLD_INDEX@@@ = 4 # Index into Count (0 Relative, 0->Count-1)
Global FLD_COUNT@@@ = 5 # Sequence Frame Count
### DB1 Status Flags in Field FLD_STAT@@@
Global FLG_OO@@@ = 1 # User Override
Global FLG_UK@@@ = 2 # K Override
Global FLG_UU@@@ = 3 # Unique Frame
Global FLG_NUK@@@ = 4 # Non Unique Sequence K Override
Global FLG_LM@@@ = 5 # LoMotion Frame
Global FLG_DS@@@ = 6 # Static Sequence [detected as duplicates but Longer than MaxDupLen]
Global FLG_DE@@@ = 7 # Duplicate Sequence is followed by Scene Change (will NOT occar when Mode==1)
Global FLG_DUP@@@ = 8 # Duplicate Sequence
###
Global SPad@@@=SPad Global SSharp@@@=SSharp Global SRFilter@@@=SRFilter
Global ABlkSize@@@=ABlkSize Global AOverLap@@@=AOverLap Global ASearch@@@=ASearch Global ADct@@@=ADct
Global RBlkSize@@@=RBlkSize Global ROverLap@@@=ROverLap Global RthSAD@@@=RthSAD
Global Iml@@@=Iml Global IBlend@@@=IBlend Global IthSCD1@@@=IthSCD1 Global IthSCD2@@@=IthSCD2
###
ARGS = "dc,DB1,DB2,ScanAhead,Show"
ScriptLine = "Fn@@@("+ARGS+")"
Last.GScriptClip(ScriptLine, local=true, args=ARGS)
"""
GIFunc="DROPDEADGORGEOUS" # Function Name, Supply unique name for your multi-instance function.
GIName=GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# Write Debug GScript File to current directory (line numbers will correspond to error messages)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
HasGScript ? GScript(InstS) : Eval(InstS,"GS_EVAL") # Use GSCript if installed (loaded plugs override builtin)
# if CallCmd available, Auto delete DBase file on clip closure.
HasCallCmd?CallCmd(close=RT_String("""CMD /C chcp 1252 && del "%s" """,DB2), hide=true, Synchronous=7):NOP
Return Last
}

FN = "D:\G.AVI" DBase = "D:\G_Det.DB" # DBase Created by Duplicity2
#FN = "D:\HAR.AVI" DBase = "D:\HAR_Det.DB"
Avisource(FN)
ORG=Last
SHOW=True
DropDeadGorgeous(DBase,show=Show)
StackVertical(Last,ORG)

EDIT: Fixed I flag, was not hilted.

Strictly sequential, play forwards ONLY.

[StainlessS]

Duplicity2 v2.11 bugfix update, see first post.

Two bugs, first one probably a copy paste of existing bug, suspect other scripts might suffer from same bug.

Was

X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP Mod 4
W = (W<=0) ? Width() - X + W : W H = (H<=0) ? Height() - Y + H : H
W = W / 4 * 4 H = H / 4 * 4 # Round DOWN Mod 4


Fixed

W = (W > 0) ? (W + X) - Width : W H = (H > 0) ? (H + Y) - Height : H # Make -ve relative width,height
X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP (inwards) Mod 4
W = (W-3) / 4 * 4 H = (H-3) / 4 * 4 # Round DOWN(inwards) Mod 4
W = Width - X + W H = Height - Y + H # +ve width,height

Rounding up X mod 4 should also have affected width, but did not, I always use -ve width, height (as for crop) where problem does not arise.
Same fix applied to previous post above, DropDeadGorgeous() script.

2nd bug, in Client script

Import(".\Duplicity2.Avsi")


######
FN = "D:\G.AVI" DBase = "D:\G_Det.DB" ThB=2.0
#FN = "D:\HAR.AVI" DBase = "D:\HAR_Det.DB" ThB=2.5
######
Avisource(FN)
SRC=Last
###
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
ThG=1.0 # Primary Frame Global Detect
ThP=64 # Pixel Count Detect
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
InterpFast=true
Show=True
ShowDot=False
ShowBlk=True
VERB=True
###
TITLE=true
STACK=False
STACK4=True
###

Result=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=False,InterpFast=InterpFast,DBase=DBase)

Metrics=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=Show,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,InterpFast=InterpFast)

Dif=Subtract(Result,SRC)

SRC = (TITLE) ? SRC.TSub("Src") : SRC
Result = (TITLE) ? Result.TSub("Result") : Result
Metrics = (TITLE) ? Metrics.TSub("Metrics") : Metrics
Dif = (TITLE) ? Dif.TSub("Dif(Result,Src)") : Dif

LFT=StackVertical((STACK4||!SHOW)?Result:Metrics,SRC)
RGT=StackVertical(Metrics,Dif)
STK4=StackHorizontal(LFT,RGT)

(STACK4) ? STK4 : (STACK) ? LFT : (Show)?Metrics:Result

Return Last


# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}

Moved DBase to result clip call instead of metrics clip (would not create named DBase if not showing metrics).

NOTE. I have found a problem in AVS+ (dont happen in Avs 2.6 standard), not sure if avs+ bug or not.
With all metrics showing and also SHOWBLK=True, and INTERPFAST=True, after playing about 1000 or so frames,
go backwards a number of frames and you get out of memory error. Does not happen on AVS standard, or with InterpFast=False.
Possibly a different system resources requirement of Avs+ (dont know). [Tested using VDub2 and VDubMod, Win7 x64 Avs/+ 32bit]

I've also noticed that ClipClop() included DoctorFrames script now fails at MaxInterp of 15 for Avs+ where using same method as used in Duplicity2
when InterpFast=True. Avs 2.6 standard achieves about 20 sets of precalculated clips before failing (with SetMemoryMax could get as hi as 22).

EDIT: Zip on mediafire/SendSpace updated, scripts from posts #22 updated
(also version number update @ top of 1st doc page on post #1).

EDIT: This is the pre-calculated set of interpolation clips where INTERPFAST=True in duplicity2, and similar used in ClipClop DoctorFrames().

if(Mode==2 && MaxInterp>0) {
if(InterpFast) {
Prefilt = c.RemoveGrain(22)
Super = c.MSuper(hpad=SPad,vpad=SPad,levels=1,sharp=SSharp,rfilter=SRFilter) # One level is enough for MRecalculate
Superfilt= Prefilt.MSuper(hpad=SPad,vpad=SPad,sharp=SSharp,rfilter=SRFilter) # All levels for MAnalyse
For(i=1, MaxInterp) {
Eval(RT_String("bv=Superfilt.MAnalyse(isb=true ,blksize=ABlkSize,overlap=AOverLap,search=ASearch,delta=%d,dct=ADct)",i+1))
Eval(RT_String("fv=Superfilt.MAnalyse(isb=false,blksize=ABlkSize,overlap=AOverLap,search=ASearch,delta=%d,dct=ADct)",i+1))
if(RBlkSize>0) {
Eval(RT_String("bv=Super.MRecalculate(bv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)"))
Eval(RT_String("fv=Super.MRecalculate(fv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)"))
}
for(j=1,i) {
Eval(RT_String("Global I%d%d@@@=c.MFlowInter(Super,bv,fv,time=100.0*%d/%d," +
\ "ml=Iml,blend=IBlend,thSCD1=IthSCD1,thSCD2=IthSCD2).SelectEvery(1,%d).Trim(0,-c.FrameCount)", i,j-1, j,i+1,-j))
}
}
} Else {
Global SPad@@@=SPad Global SSharp@@@=SSharp Global SRFilter@@@=SRFilter
Global ABlkSize@@@=ABlkSize Global AOverLap@@@=AOverLap Global ASearch@@@=ASearch Global ADct@@@=ADct
Global RBlkSize@@@=RBlkSize Global ROverLap@@@=ROverLap Global RthSAD@@@=RthSAD
Global Iml@@@=Iml Global IBlend@@@=IBlend Global IthSCD1@@@=IthSCD1 Global IthSCD2@@@=IthSCD2
}
}



EDIT: Could not get Avs+ to fail where STACK and STACK4 both false in client script, ie where only single instance of Duplicity2 called [EDIT: Displayed].

EDIT: Updated Duplicty2 Client script a little bit in post 25 (No zip update).

[StainlessS]

Duplicity2 v2.12, update, see first post.

Zip now included DropDeadGorgeous() script versioned at v0.00.

SCRIPT REMOVED.

[StainlessS]

Duplicity2(), DropDeadGorgeous(), v2.13 updates, see first post.

See post #22 for current Duplicity2 script, or download see 1st post (re-directs to MediaFire or SendSpace below this post).

Current DropDeadGorgeous() reversioned to v2.13, will now always be same version as Duplicity2 (and require similar version Duplicity2).

DropDeadGorgeous.avsi part 1

/*

DropDeadGorgeous() v2.13, by the Gorgeous & Truly Scrumptious StainlessS @ Doom9
Same Requirements as Duplicity2(). https://forum.doom9.org/showthread.php?t=175357

Using DBase as Created by Duplicity2 v2.13.
Corrects problems where frames were dropped in clip, and then some time later duplicates were inserted to maintain timing,
ie 'jumpy' clips that repeatedly jerk and then freeze.
ScanAhead is number of frames scanned ahead of current_frame looking for duplicates, default 9. Any detected dupe run will
start between current_frame and current_frame + ScanAhead.
If duplicates found (limit on length of duplicate run set in Duplicity2 via MaxDupLen), then a JerkFrame is found.
A JerkFrame is searched for backwards, from DupeStartFrame-1 to to current_frame and search stops if a StartOfScene is detected.
The JerkFrame is the frame prior to first dupe, with greatest movement/jerk when compared with its predecessor.
If JerkFrame is found, a number of frames are interpolated starting at the JerkFrame position, (same number as in dupe run,
and source frames to interpolation are JerkFrame and JerkFrame-1), interpolated frames shove following frames to later in clip,
removing dupes.
If no JerkFrame is found, ie where duplicates are of a StartOfScene frame, then so long as DupeEndFrame+1 is not another StartOfScene
frame, then duplicates are interpolated using DupeStartFrame-1 and DupeEndFrame+1 as interpolation source frames.

Function DropDeadGorgeous(clip c,String DB,Int "ScanAhead"=9,
\ Int "X"=0,Int "Y"=0,Int"W"=0,Int "H"=0,
\ Int "SPad"=16, Int "SSharp"=1, Int "SRFilter"=4, [* MSuper *]
\ Int "ABlkSize"=16, Int "AOverlap"=4, Int "ASearch"=3,Int "ADct"=0, [* MAnalyse *]
\ Int "RBlkSize"=8, Int "ROverlap"=2, Int "RthSAD"=100, [* MRecalculate *]
\ Float "Iml"=200.0, Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
\ Bool "Show"=True, Bool "Verb"=True,Int "SOSthSCD2"=130
\ )

DB, DBase as created by Duplicity2, v2.13, error if wrong version DBase.

ScanAhead, Default 9, 1 <= ScanAhead <= 60. Distance in frames to scan ahead of current frame. If first duplicate is always eg
3 frames after a jerk, then set this to 3.
ScanAhead, is deliberatley set quite high by default, then maybe reduce ScanAhead to MostCommonScanDist after full scan.
ALSO SEE Verbose Metrics MostCommonScanDist.

X,Y,W,H, All Default 0, Coordinates of test area, as in crop (all 0 = full frame).
The x,y coords will be rounded up mod4, and w,h coords rounded down mod 4.
Ignore eg animated logo.

MvTools2 Args, for Interpolation.
MSuper args:
SPad, Default 16, See MvTools MSuper(hpad=8,vpad=8) [We use same for both].
SSharp, Default 1, See MvTools MSuper(sharp=2)
SRFilter, Default 4, See MvTools MSuper(rfilter=2)

MAnalyse args:
ABlkSize, Default 16, See MvTools MAnalyse(BlkSize=8,BlkSizeV=8) [We use same for both].
AOverLap, Default 4, See MvTools MAnalyse(OverLap=0)
ASearch, Default 3, See MvTools MAnalyse(Search=4)
ADct, Default 0, See MvTools MAnalyse(Dct=0). Using other than default value can be VERY slow.

MRecalculate args:
RBlkSize, Default 8, See MvTools MRecalculate(BlkSize)
NOTE, RBlkSize = 0 Will Switch OFF MRecalculate, probably a bit faster at possible expense of quality.
ROverlap, Default 2, See MvTools MRecalculate(Overlap)
RthSAD, Default 100, See MvTools MRecalculate(thSAD=200)

MFlowInter args:
Iml, Default 200.0, See MvTools MFlowInter(ml=100.0)
IBlend, Default True, See MvTools MFlowInter(Blend=True). Blend or copy frame at scene change.
IthSCD1, Default 400, See MvTools MFlowInter(thSCD1=400) : (0 < IthSCD1 <= (8*8)*255)
IthSCD2, Default 130, See MvTools MFlowInter(thSCD1=130) : (0 < IthSCD1 <= 255)
Threshold which sets how many blocks have to change for the frame to be considered as a
scene change. It is ranged from 0 to 255, 0 meaning 0 %, 255 meaning 100 %.
Default is 130 (which means 51.0%).
Used by MvTools2 MFlowInter during interpolation.

Show, Default True, Show Metrics.
Verb, Default True, If Verb And Show, then show additional metrics.

SOSthSCD2 Default 130. As for MSCDetection(thSCD2=SOSthSCD2). Halt backwards search for JerkFrame where StartOfScene.

Simple metrics on Show, "nnnn] UD : S : CIR", where nnnn is frame number, and character Flags highlited where,
Duplicity2 Status:
U = Unique frame status from Dupliicty DBase (non dupe, includes low motion static scene etc).
D = Dupe frame status from Dupliicty DBase.

:
S = Current frame is Start Of Scene.

: DropDeadGorgeous Decisions :
C = Copy frame (original as source frame)
I = Interpolated frame
Interpolated frames also show eg {I1/2:ILft=1275,IRgt=1276}
where, first of 2 Interpolated frames, and interpolate source frames are 1275 and 1276(JerkFrame usually, unless DupeS-1=SOS).
R = Relocated frame from earlier in clip (interpolated frames shove following frames to later in clip, removing dupes)
Reloacted frames also show eg {R1/8:Src=1276,n-2}
where, first of 8 relocated frames, source for current relocated frame is frame 1276, which is current frame-2.

If Verb=True, also shows eg below
Motion(n-1,n)=0.367 # Motion metric between current frame and current frame-1 (JerkFrame search metric).
ScanMaxdist=9 (Cnt=7) @ 3458 # Maximum ScanAhead distance found to dupes (count of instances found), and most recent @ frame 3458.
MostCommonScanDist=8 (Cnt=190) # The most common ScanAhead distance found between jerkframe and dupes, and instance count.
# Maybe reduce ScanAhead arg to value as shown for MostCommonScanDist.

Can use any Duplicity Mode in creating DBase, the Duplicity2 End+1 Scene change thing has no effect here (not necessary).

It may be possible to up framerate to a non multiple of source whilst keeping all source frames verbatim, where eg source
is 24 FPS and required = 25 FPS, and using,
Src24.ChangeFPS(25) to produce a new clip at 25 FPS with an exact duplicate every 24 source frames and save new source as eg Src25.avi.
Process new source with eg src25.Duplicity2(ThG=0.1,DBase="D:\Src25_Det.DB"), to produce a DBase (perhaps with MaxDupLen=1, perhaps not).
Then use Src25.DropDeadGorgeous("D:\Src25_Det.DB",ScanAhead=12) to interpolate dupe frames achieving 25.0 and with all original
source frames. With a ScanAhead of eg 12, will locate dupes and then backwards scan up to 12 frames before the dupes looking for
frame of greatest movement, insert an interpolated frame prior to that greatest movement frame and shove all following frames along one
to remove the dupe. Might be worth someone experimenting.
Or, you could just use eg Duplicity2(Mode=2,ThG=0.1) to interpolate the exact dupes directly and without DropDeadGorgeous but that might
produce a more rhythmic stutter every 25 frames.

*/

[StainlessS]

DropDeadGorgeous.avsi part 2


Function DropDeadGorgeous(clip c,String DB,Int "ScanAhead",
\ Int "X",Int "Y",Int"W",Int "H",
\ Int "SPad", Int "SSharp", Int "SRFilter", [* MSuper *]
\ Int "ABlkSize", Int "AOverlap", Int "ASearch",Int "ADct", [* MAnalyse *]
\ Int "RBlkSize", Int "ROverlap", Int "RthSAD", [* MRecalculate *]
\ Float "Iml", Bool "IBlend", Int "IthSCD1", Int "IthSCD2", [* MFlowInter *]
\ Bool "Show", Bool "Verb",Int "SOSthSCD2"
\ ) {
c myName="DropDeadGorgeous: " VER=2.13
# RT_Version(), x.yyyzz where yyy==999, denotes version (x.yyy+0.001) Beta zz ie 1.99911 = v2.00 Beta 11.
Assert(RT_Version() >= 1.99911,RT_String("%sNeed RT_Stats v2.00Beta11(%s)",myName,RT_VersionString))
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
HasGRunt =RT_FunctionExist("GScriptClip") HasMvTools=RT_FunctionExist("MSuper")
HasMaskTools=RT_FunctionExist("MT_Lutxy") HasRemoveGrain=RT_FunctionExist("RemoveGrain")
HasCallCmd=RT_FunctionExist("CallCmd")
ScanAhead = Default(ScanAhead,9)
X=default(X,0) Y=default(Y,0) W=default(W,0) H=default(H,0)
SPad = Default(SPad,16) SSharp = Default(SSharp,1) SRFilter= Default(SRFilter,4)
ABlkSize = Default(ABlkSize,16) AOverlap= Default(AOverlap,4) ASearch = Default(ASearch,3) ADct = Default(ADct,0)
RBlkSize = Default(RBlkSize,8) ROverlap= Default(ROverlap,2) RthSAD = Default(RthSAD,100)
Iml = Default(Iml,200.0) IBlend = Default(IBlend,True) IthSCD1 = Default(IthSCD1,400) IthSCD2= Default(IthSCD2,130)
Show = Default(Show,True) Verb=Show?Default(Verb,True):False SOSthSCD2=Default(SOSthSCD2,130)
Assert(0 < ScanAhead <= 60,RT_String("%s0 < ScanAhead(%d) <= 60",myName,ScanAhead))
####
Assert(DB!="",RT_String("%sDB Cannot be ''",myName))
DB1 = RT_GetFullPathName(DB)
NameOnly=RT_FilenameSplit(DB,12) # Just name and extension parts, no Drive/Path
Assert(RT_DBaseGetID(DB,0) == VER,RT_String("%s'%s' Wrong version DBase(v%.2f), require v%.2f",myName,NameOnly,RT_DBaseGetID(DB,0),VER))
Assert(RT_DBaseGetID(DB,1) == c.Framecount,RT_String("%s'%s' DBase is incomplete",myName,NameOnly))
W = (W > 0) ? (W + X) - Width : W H = (H > 0) ? (H + Y) - Height : H # Make -ve relative width,height
X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP (inwards) Mod 4
W = (W-3) / 4 * 4 H = (H-3) / 4 * 4 # Round DOWN(inwards) Mod 4
W = Width - X + W H = Height - Y + H # +ve width,height
DC=Crop(X,Y,W,H)
FuncS="""
Function Fn@@@(clip c,clip dc,String DB1,String DB2,int ScanAhead,Int SOSthSCD2,Bool Show,Bool Verb) {
c n=current_frame
Stat2 = RT_DBaseGetField(DB2,n,0)
if(Stat2 == 0) {
Stat1 = RT_DBaseGetField(DB1,n,FLD_STAT@@@)
if(Stat1 >= FLG_DE@@@) { # Is a dupe (user musta jumped about), do backwards scan
Index=RT_DBaseGetField(DB1,n,FLD_INDEX@@@)
DupS=n-Index # Start of duplicate run
Count=RT_DBaseGetField(DB1,DupS,FLD_COUNT@@@) # Dup Count stored only in first Record of Sub Sequence
DupE=DupS+Count-1 # End of dupe run
if(DupS < 2) { # Not enough prev frames to interpolate
for(i=0,DupE) { RT_DBaseSet(DB2,i,1,0,0,0) } # Copy from 0, cant do anything else.
} Else {
ScanS = Max(DupS - ScanAhead, 1)
for(i=DupS-1,ScanS,-1) { # Make sure no dupes in backwards scan range
if(RT_DBaseGetField(DB1,i,FLD_STAT@@@) >= FLG_DE@@@) { ScanS=i+1 i=-1 }
}
mxi = -1
mxDif = -1.0
for(i=DupS-1,ScanS,-1) {
if(SOS@@@.RT_AverageLuma(n=i).Int!=0) { i=-1 } # Break at Start Of Scene with current max
Else {
dif = MFPC@@@.RT_AverageLuma(n=i)
if(dif>mxDif) { mxDif=dif mxi = i }
}
}
if(mxi<0) { # No JerkFrame found, ie DupS-1 is StartOfScene
if(DupE+1==c.FrameCount || SOS@@@.RT_AverageLuma(n=DupE+1).Int!=0) {
for(i=ScanS,DupE) { RT_DBaseSet(DB2,i,1,0,0,0) } # Copy all, Cannot Interpolate.
} Else { # DupS-1 is SOS, and DupE+1 is NOT SOS, Interpolate dupes using source DupS-1 and DupE+1
for(i=ScanS,DupS-1) { RT_DBaseSet(DB2,i, 1,0,0,0) } # Copy frames up to SOS frame
for(i=DupS,DupE) { RT_DBaseSet(DB2,i, 2,COUNT,i-DupS,1) } # Interp, Type SOS
}
} Else {
for(i=mxi,mxi+Count-1) { RT_DBaseSet(DB2,i, 2,COUNT,i-mxi,0) } # Interpolate
Cnt2=DupE-(mxi+Count)+1
for(i=mxi+Count,DupE) { RT_DBaseSet(DB2,i, 3,COUNT,i-mxi,Cnt2) } # Relocate Frame relative (frame n = frame n - COUNT)
if(Verb) {
scandist = DupS - mxi
cnt = RT_DBaseGetAttrib(DB2,scandist) + 1
RT_DBaseSetAttrib(DB2,scandist,cnt)
if(cnt >= ScanComCnt@@@) {
if(cnt == ScanComCnt@@@) {
Global ScanComDist@@@=Max(ScanComDist@@@,scandist)
} Else {
Global ScanComCnt@@@ = cnt
Global ScanComDist@@@ = scandist
}
}
if(scandist>=ScanMaxDist@@@) {
if(scandist==ScanMaxDist@@@) {
Global ScanMaxPos@@@ = Max(mxi,ScanMaxPos@@@)
} Else {
Global ScanMaxDist@@@=scandist
Global ScanMaxPos@@@=mxi
}
}
}
}
}
# Frame Following this dupe run is a NON DUPE.
} Else { # Non dupe
ScanE = Min(n+(ScanAhead*2),c.FrameCount-1)
DupS = -1
for(i=n+1,ScanE) { if(RT_DBaseGetField(DB1,i,FLD_STAT@@@) >= FLG_DE@@@) { DupS = i i=ScanE } } # find 1st dupe
if(DupS<0) { # dupe not found
for(i=n,Max(ScanE-ScanAhead,n)) { RT_DBaseSet(DB2,i,1,0,0,0) } # Copy
} Else {
Count=RT_DBaseGetField(DB1,DupS,FLD_COUNT@@@) # Dup Count stored only in first Record of Sub Sequence
DupE=DupS+Count-1
ScanS = Max(DupS - ScanAhead, n)
mxi = -1
mxDif = -1.0
for(i=DupS-1,ScanS,-1) { # Search for JerkFrame
if(SOS@@@.RT_AverageLuma(n=i).Int!=0) { i=-1 } # Break @ Start Of Scene with current max
Else {
dif = MFPC@@@.RT_AverageLuma(n=i)
if(dif>mxDif) { mxDif=dif mxi = i }
}
}
if(mxi<0) { # No JerkFrame found, ie DupS-1 is StartOfScene
if(DupE+1==c.FrameCount || SOS@@@.RT_AverageLuma(n=DupE+1).Int!=0) {
for(i=n,DupE) { RT_DBaseSet(DB2,i,1,0,0,0) } # Copy all, Cannot Interpolate.
} Else { # DupS-1 is SOS, and DupE+1 is NOT SOS, Interpolate dupes using source DupS-1 and DupE+1
for(i=n,DupS-1) { RT_DBaseSet(DB2,i, 1,0,0,0) } # Copy frames up to SOS frame
for(i=DupS,DupE) { RT_DBaseSet(DB2,i, 2,COUNT,i-DupS,1) } # Interp, Type=SOS
}
} Else {
for(i=n,mxi-1) { RT_DBaseSet(DB2,i,1,0,0,0) } # Copy frames prior to frame of greatest movement
for(i=mxi,mxi+Count-1) { RT_DBaseSet(DB2,i, 2,COUNT,i-mxi,0) } # Interp
Cnt2=DupE-(mxi+Count)+1
for(i=mxi+Count,DupE) { RT_DBaseSet(DB2,i, 3,COUNT,i-mxi,Cnt2) } # relocate frames
if(Verb) {
scandist = DupS - mxi
cnt = RT_DBaseGetAttrib(DB2,scandist) + 1
RT_DBaseSetAttrib(DB2,scandist,cnt)
if(cnt >= ScanComCnt@@@) {
if(cnt == ScanComCnt@@@) {
Global ScanComDist@@@=Max(ScanComDist@@@,scandist)
} Else {
Global ScanComCnt@@@ = cnt
Global ScanComDist@@@ = scandist
}
}
if(scandist>=ScanMaxDist@@@) {
if(scandist==ScanMaxDist@@@) {
Global ScanMaxPos@@@ = Max(mxi,ScanMaxPos@@@)
} Else {
Global ScanMaxDist@@@=scandist
Global ScanMaxPos@@@=mxi
}
}
}
}
}
}
Stat2 = RT_DBaseGetField(DB2,n,0)
}
if(Stat2==2) { # Interpolated
Count = RT_DBaseGetField(DB2,n,1)
Index = RT_DBaseGetField(DB2,n,2)
iType = RT_DBaseGetField(DB2,n,3) # 0) Normal Interpolate with shove duplicates, 1) DupS-1=SOS
if(iType==0) {
TmpC=c
FixFrm=n-Index-1
} Else {
DupS=n-Index DupE=DupS+Count-1
IFrms=3
LS=Max(DupS-IFrms,0)
LCnt=DupS-LS RCnt=Min(c.FrameCount-1-DupE,IFrms)
TmpC = c.Trim(LS,-LCnt) ++ c.Trim(DupE+1,-RCnt)
FixFrm=LCnt-1
}
Prefilt=TmpC.RemoveGrain(22)
Pad=SPad@@@ Sharp=SSharp@@@ srfilter=SRFilter@@@ Ablksize=ABlkSize@@@ AOverLap=AOverLap@@@ ASearch=ASearch@@@ ADct=ADct@@@
Super=TmpC.MSuper(hpad=Pad,vpad=Pad,levels=1,sharp=Sharp,rfilter=srfilter)
Superfilt=Prefilt.MSuper(hpad=Pad,vpad=Pad,sharp=Sharp,rfilter=srfilter)
bv=Superfilt.MAnalyse(isb=true ,blksize=Ablksize,overlap=AOverLap,search=ASearch,delta=1,dct=ADct)
fv=Superfilt.MAnalyse(isb=False ,blksize=Ablksize,overlap=AOverLap,search=ASearch,delta=1,dct=ADct)
if(RBlkSize@@@>0) {
RBlkSize=RBlkSize@@@ ROverLap=ROverLap@@@ RthSAD=RthSAD@@@
bv=Super.MRecalculate(bv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)
fv=Super.MRecalculate(fv,blksize=RBlkSize,overlap=ROverLap,thSAD=RthSAD)
}
TmpC=TmpC.MFlowInter(Super,bv,fv,time=100.0*(Index+1)/(Count+1),ml=Iml@@@,blend=IBlend@@@,thSCD1=IthSCD1@@@,thSCD2=IthSCD2@@@)
TmpC.Trim(FixFrm,-1)
if(Show) {
DStat=(RT_DBaseGetField(DB1,n,0)>=FLG_DE@@@)
sos = (SOS@@@.RT_AverageLuma(n=n).Int!=0)
if(iType==0) { iLft=FixFrm iRgt=iLft+1 }
else { iLft=DupS-1 iRgt=DupE+1 }
if(Verb) {
mot = MFPC@@@.RT_AverageLuma(n=n)
ScanMaxDist=ScanMaxDist@@@
ScanMaxDistCnt=RT_DBaseGetAttrib(DB2,ScanMaxDist)
RT_Subtitle(Fmtv_I@@@,n,DStat?76:45,DStat?45:76,sos?45:76,Index+1,Count,ILft,IRgt,
\ mot,ScanMaxDist,ScanMaxDistCnt,ScanMaxPos@@@,ScanComDist@@@,ScanComCnt@@@)
} Else { RT_Subtitle(Fmt_I@@@,n,DStat?76:45,DStat?45:76,sos?45:76,Index+1,Count,ILft,IRgt) }
}
} Else if(Stat2==3) { # Relocated
DStat=(RT_DBaseGetField(DB1,n,0)>=FLG_DE@@@)
RelocOffset = RT_DBaseGetField(DB2,n,1)
c.Trim(n-RelocOffset,-1) # Relocated, n - count
if(Show) {
Count = RT_DBaseGetField(DB2,n,3)
Index = RT_DBaseGetField(DB2,n,2) - RelocOffset
sos = (SOS@@@.RT_AverageLuma(n=n).Int!=0)
if(Verb) {
mot = MFPC@@@.RT_AverageLuma(n=n)
ScanMaxDist=ScanMaxDist@@@
ScanMaxDistCnt=RT_DBaseGetAttrib(DB2,ScanMaxDist)
RT_Subtitle(Fmtv_R@@@,n,DStat?76:45,DStat?45:76,sos?45:76,Index+1,Count,n-RelocOffset,RelocOffset,
\ mot,ScanMaxDist,ScanMaxDistCnt,ScanMaxPos@@@,ScanComDist@@@,ScanComCnt@@@)
} Else { RT_Subtitle(Fmt_R@@@,n,DStat?76:45,DStat?45:76,sos?45:76,Index+1,Count,n-RelocOffset,RelocOffset) } }
} Else if(Show) { # Copy original
DStat=(RT_DBaseGetField(DB1,n,0)>=FLG_DE@@@)
sos = (SOS@@@.RT_AverageLuma(n=n).Int!=0)
if(Verb) {
mot = MFPC@@@.RT_AverageLuma(n=n)
ScanMaxDist=ScanMaxDist@@@
ScanMaxDistCnt=RT_DBaseGetAttrib(DB2,ScanMaxDist)
RT_Subtitle(Fmtv_C@@@,n,DStat?76:45,DStat?45:76,sos?45:76,
\ mot,ScanMaxDist,ScanMaxDistCnt,ScanMaxPos@@@,ScanComDist@@@,ScanComCnt@@@)
} Else { RT_Subtitle(Fmt_C@@@,n,DStat?76:45,DStat?45:76,sos?45:76) }
}
Return Last
}

[StainlessS]

DropDeadGorgeous.avsi part 3


Function StartOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at SOS, else 0
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MSCDetection(fv,thSCD1=thSCD1,thSCD2=thSCD2)
}
Function MMaskFromPrevClip(clip c,Int "MaskType",Float "Gamma",Int "thSCD1",Int "thSCD2") {
# MaskType:- 0=Motion, 1=SAD, 2=Occlusion, 3=Horizontal, 4=Vertical, 5=ColorMap.
MaskType=Default(MaskType,0) Gamma=Default(Gamma,1.0) thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MMask(fv,Gamma=1.0,kind=MaskType,thSCD1=thSCD1,thSCD2=thSCD2)
}
DB2 = RT_GetFullPathName("~@@@_"+RT_LocalTimeString+".DB") RT_DBaseAlloc(DB2,FrameCount,"iiii")
Global ScanComDist@@@=0 Global ScanComCnt@@@=0
Global ScanMaxDist@@@=0 Global ScanMaxPos@@@=0
Global SOS@@@ = dc.StartOfSceneClip(thSCD2=SOSthSCD2)
Global MFPC@@@= dc.MMaskFromPrevClip(Gamma=1.0,MaskType=0,thSCD1=8*8*255,thSCD2=255) # Motion, Switched OFF SC detection, we want all results.
### Some DB1 Fields:-
Global FLD_STAT@@@ = 0 # Holds Status FLGS
Global FLD_INDEX@@@ = 4 # Index into Count (0 Relative, 0->Count-1)
Global FLD_COUNT@@@ = 5 # Sequence Frame Count
### Some DB1 Status Flags in Field FLD_STAT@@@
Global FLG_DE@@@ = 7 # Duplicate Sequence is followed by Scene Change (below is non dupe, greater or equalis dupe)
###
Global SPad@@@=SPad Global SSharp@@@=SSharp Global SRFilter@@@=SRFilter
Global ABlkSize@@@=ABlkSize Global AOverLap@@@=AOverLap Global ASearch@@@=ASearch Global ADct@@@=ADct
Global RBlkSize@@@=RBlkSize Global ROverLap@@@=ROverLap Global RthSAD@@@=RthSAD
Global Iml@@@=Iml Global IBlend@@@=IBlend Global IthSCD1@@@=IthSCD1 Global IthSCD2@@@=IthSCD2
Global Fmt_C@@@ ="%d] \a%cU\a%cD\a- : \a%cS\a- : C\aLIR"
Global Fmt_I@@@ ="%d] \a%cU\a%cD\a- : \a%cS\a- : \aLC\a-I\aLR\a- {I%d/%d:Lft=%d,IRgt=%d}"
Global Fmt_R@@@ ="%d] \a%cU\a%cD\a- : \a%cS\a- : \aLCI\a-R\a- {R%d/%d:Src=%d,n-%d}"
Global Fmtv_C@@@=Fmt_C@@@ + "\a-\nMotion(n-1,n)=%.3f\nScanMaxDist=%d (Cnt=%d) @ %d\nMostCommonScanDist=%d (Cnt=%d)"
Global Fmtv_I@@@=Fmt_I@@@ + "\a-\nMotion(n-1,n)=%.3f\nScanMaxDist=%d (Cnt=%d) @ %d\nMostCommonScanDist=%d (Cnt=%d)"
Global Fmtv_R@@@=Fmt_R@@@ + "\a-\nMotion(n-1,n)=%.3f\nScanMaxDist=%d (Cnt=%d) @ %d\nMostCommonScanDist=%d (Cnt=%d)"
###
ARGS = "dc,DB1,DB2,ScanAhead,SOSthSCD2,Show,Verb"
ScriptLine = "Fn@@@("+ARGS+")"
Last.GScriptClip(ScriptLine, local=true, args=ARGS)
"""
GIFunc="DROPDEADGORGEOUS" # Function Name, Supply unique name for your multi-instance function.
GIName=GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# Write Debug GScript File to current directory (line numbers will correspond to error messages)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
HasGScript ? GScript(InstS) : Eval(InstS,"GS_EVAL") # Use GSCript if installed (loaded plugs override builtin)
# if CallCmd available, Auto delete DBase file on clip closure.
HasCallCmd?CallCmd(close=RT_String("""CMD /C chcp 1252 && del "%s" """,DB2), hide=true, Synchronous=7):NOP
Return Last
}

[StainlessS]

From post #46,
It may be possible to up framerate to a non multiple of source whilst keeping all source frames verbatim, where eg source
is 24 FPS and required = 25 FPS, and using,
Src24.ChangeFPS(25) to produce a new clip at 25 FPS with an exact duplicate every 24 source frames and save new source as eg Src25.avi.
Process new source with eg src25.Duplicity2(ThG=0.1,DBase="D:\Src25_Det.DB"), to produce a DBase (perhaps with MaxDupLen=1, perhaps not).
Then use Src25.DropDeadGorgeous("D:\Src25_Det.DB",ScanAhead=12) to interpolate dupe frames achieving 25.0 and with all original
source frames. With a ScanAhead of eg 12, will locate dupes and then backwards scan up to 12 frames before the dupes looking for
frame of greatest movement, insert an interpolated frame prior to that greatest movement frame and shove all following frames along one
to remove the dupe. Might be worth someone experimenting.


I tried above converting [EDIT: true] 25.0 FPS to 30.0FPS, using ChangeFPS(30.0), and Duplicity2(ThG=0.1,MaxDuplen=1),
Then, used DropDeadGorgeous(ScanAhead=5), result produced is 30FPS with all original source frames, and with good fluid motion throughout.
ChangeFPS(30.0) inserts duplicate after every five source frames, and DropDeadGorgeous(ScanAhead=5) introduces
an interpolated frame within 5 frames prior to the duplicate (between frames of greatest motion) and pushes the following frames one position
along to squeeze out the duplicate.
Works pretty good.
EDIT: If source clip was badly decimated from eg 30FPS to 25 and with jerky motion due to decimation, above should produce pretty much the best possible result and repair/tween the jerk frames produced by the decimation.
EDIT: Should also perform well in cases where cadence changes, eg where source was decimated from 29.97 FPS to 25 FPS,
or where source was edited after decimation, neither one should be any problem.

[Frank62]

A few months ago this could have saved my life...
I have some sources to test - when I will have time (can last longer) - I will tell you the result!

[StainlessS]

Good, thank you, been hoping for some kind of feedback.

[Puffy2010]

Hi, I am unable to download RT_Stats_25&26_x86_x64_dll_v2.00Beta12_20181125.zip from your mediafire. I have tried importing it to my mediafire and it still gets stuck in a loop requesting new download keys. I can download all the other files on your mediafire but this one seems broken.

I would really like to give Duplicity2 a whirl :)

Can you please re-upload it?

[ChaosKing]

Yeah download_repair.php does not sound good. It's an endless loop :eek:

Why don't you switch to dropbox or something similar (you know github :D). The biggest benefit for you would be: The links doesn't change if you update the file in dropbox. I think what was one of your complaints about mediafire..

[StainlessS]

Download failed for me also, was new problem, dont know what happend.
Have re-compressed as 7z instead of zip, and re-upped to both MediaFire and SendSpace, Mediafire downed ok.
Strange, failed down had weird interface that I aint seen before, anyways, working now.

Github, one day.
Am about to down VSCodian [or whatever the pure open source version of Visual Studio Code is called], so github almost at my fingertips. :)

[TheUpsideDown]

Hi StainlessS,

Thanks for all your tools! I've been making my way through some of them and the functionalities they offer are astounding!

We occasionally get get material in that exhibits skipped--and then duplicated frames. We don't have to correct the material--we just have to flag that there are duplicates and inform the distributor.

I've been using Duplicity and the output of frames.txt, pasted into excel with a formula to return non-sequential frame numbers to spot duplicates.

I'm wondering if there's a way to "reverse" the output of frames.txt so that it only writes the frame numbers of duplicate frames. Or, perhaps a way to accomplish this that I'm not gleaning from the notes. Or is there another better suited to this task (I've only dabbled with AVIsynth over the years).

Thanks for any help you can provide! :)

[StainlessS]

Maybe tommrroew,
I'll tyr, but at the momomentt, I a bit less that totally cohernet. Maybe when I'm lees overwhelmed.

[StainlessS]

OK, I'm assuming that you only want single frames written and not ranges (although this threw me a little bit ""reverse" the output of frames.txt").

first few lines of the client script [Added in BLUE]

Import(".\Duplicity2.Avsi")

######
FN = ".\Earth.AVI" DBase = "D:\Earth.DB" ThB=2.0
#FN = "D:\G.AVI" DBase = "D:\G_Det.DB" ThB=2.0
#FN = "D:\GTEST.AVI" DBase = "D:\GTEST_Det.DB" ThB=2.0
#FN = "D:\HAR.AVI" DBase = "D:\HAR_Det.DB" ThB=2.5
#FN = "D:\DeDupTest.AVI" DBase = "D:\DeDupTest_Det.DB" ThB=2.0
######
Avisource(FN)
SRC=Last
###
Mode=0 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
ThG=1.0 # Primary Frame Global Detect
ThP=64 # Pixel Count Detect
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
InterpFast=true
Ranges=False # Added
Show=True
ShowDot=False
ShowBlk=true
VERB=False
###
TITLE=False
STACK=True
STACK4=False
###

Result=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=False,InterpFast=InterpFast,Ranges=RANGES,DBase=(STACK4||!SHOW)?DBase:"")

Metrics=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=True,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,InterpFast=InterpFast,Ranges=RANGES,DBase=(STACK4||!SHOW)?"":DBase)


Frames, Default "Frames.txt", Frames filename for output, ("" will not write Frames file).
Ranges, Default True (True=Write comma separated frame ranges to Frames file, else single frames only).
FrameAsRange, Default False. If Ranges=True and FrameAsRange=default false, and single frame, then writes single frame number and
not range. If Ranges=True and FrameAsRange=True then writes single frame as range eg '23,23' for frame 23.


So this

###
# Duplicate frames file. [ Generated by Duplicity2() ]
###

90,92
320
382
384
386
388,389
439
490
492
527
553
567
583
617
628
635
639
650
660
669
673
682
684
686
688,689

would become this

###
# Duplicate frames file. [ Generated by Duplicity2() ]
###

90
91
92
320
382
384
386
388
389
439
490
492
527
553
567
583
617
628
635
639
650
660
669
673
682
684
686
688
689


Perhaps I have misinterpreted your request, if so then please restate.

[StainlessS]

And this would convert Frames.txt to a list of only non duplicate frames, requires FrameSel():- https://forum.doom9.org/showthread.php?t=167971


AviSource(".\Earth.avi")
trim(0,-10000) # Trim exact same as Duplicity2 source clip

InFRAMES=".\Frames.txt"
OutFRAMES=".\OutFrames.txt"
REJECT=True # output True=NON Dupes : False = Dupes [but as for below RANGES]
RANGES=False # single frames not ranges

FrameSel_CmdReWrite(OutFrames,Cmd=InFRAMES,Reject=REJECT,Ordered=True,Range=RANGES,Prune=False)

Return Last

Note, you do not have to play the clip to generate the outFrames.txt file, is created before first frame is produced.

EDIT: And welcome to the forum :)

EDIT: And to repair dupes, might like to try DropDeadGorgeous2() with SCANAHEAD = (Dupe<===>Dupe Distance - 1) with all original non-dupe frames kept in output [SCANAHEAD max 60].

EDIT: Added config REJECT=True, output non dupes, but can change to false to just change to output single frames or ranges as for RANGES setting,
so can output either dupes or non dupes, and ranges or single frames.

[TheUpsideDown]

Note, you do not have to play the clip to generate the outFrames.txt file, is created before first frame is produced.

EDIT: And welcome to the forum :)

EDIT: And to repair dupes, might like to try DropDeadGorgeous2() with SCANAHEAD = (Dupe<===>Dupe Distance - 1) with all original non-dupe frames kept in output [SCANAHEAD max 60].

EDIT: Added config REJECT=True, output non dupes, but can change to false to just change to output single frames or ranges as for RANGES setting,
so can output either dupes or non dupes, and ranges or single frames.

Thanks for the warm welcome! :thanks:
Apologies for the late reply--and it seems I have a confession to make:

I had originally tried using Duplicity2 to flag repeated frames as it seems that is an easily exposed function (mode=0) but ran into issues with setup I couldn't overcome.*

I was successful in using ExactDeDup--this is what my question was actually referencing--sorry! I can get ExactDeDup to write an output file of all the frames that are unique, but couldn't figure out how to replicate the idea of only writing the duplicated frames.


* my issue(s)
I installed base AVISynth and then UniversalAvisynthInstaller to allow jumping between multiple forks/versions.
I made sure to install the files referenced in post #1:

RT_Stats v2.00_Beta_11 + CallCmd
GSCript, Grunt
MaskTools2, MvTools2
RemoveGrain

Calling Duplicity2() results in:

RT_DBaseAlloc: Cannot Query Filesystem
([GScript], line 260)
(C:/Users/user/UniversalAvisynthInstaller/AVSPLUS_x86/plugins/Duplicity2.Avsi, line 760)
(New File (1), line 9)

Normally I'm a half-ways intelligent guy and googling the errors I receive (in other scenarios for instance) leads me to certain dependencies I've overlooked. Sadly I can't find much to go on with the above.

If this is something obvious that a noob should know, I'd love to get pointed in the right direction...

[StainlessS]

I've never seen that error message before, can only assume that something is preventing successfull call to QueryFatVolume(fn). [In BLUE in code block]


AVSValue __cdecl RT_DBaseAlloc(AVSValue args, void* user_data, IScriptEnvironment* env) {
const char * myName = "RT_DBaseAlloc: ";
const char * fn = args[0].AsString();
const int records = args[1].AsInt();
const char *typestr = args[2].AsString();
const int StringlenMax = args[3].AsInt(256);
if(*fn=='\0') env->ThrowError("%sEmpty Filename",myName);

__int64 MaxFileSz = 0xFFFFF00000i64 ; // 1TB - 1MB

int fatvol = QueryFatVolume(fn);
if(fatvol < 0) env->ThrowError("%sCannot query Filesystem",myName);

__int64 dfs = QueryDiskFreeSpace(fn) - 0x100000; // minus 1MB
if(dfs < 0) env->ThrowError("%sCannot query DiskFreeSpace",myName);
// dprintf("DiskFreeSpace=$%I64X",dfs);
__int64 maxfs = (fatvol==1)? 0xFFF00000i64 : MaxFileSz; // limit 4GB on FAT32
__int64 maxcurdfs=min(maxfs,dfs); // Max current space, Limit to free space available to user.


I'm wondering if the RT_stats.dll is being partially blocked.
Right click on RT_Stats.dll and look at Properties, does it say something like
"This file came from another computer and has been blocked for your protection".
https://i.postimg.cc/MZDw4x5d/Blocked.jpg (https://postimages.org/)

Just curious, what OS are you using ?

Here AlternateStreamView, to view and delete alternate streams, (Zone Identifier is the blocking thing):- https://www.nirsoft.net/utils/alternate_data_streams.html

AlternateStreamView is a small utility that allows you to scan your NTFS drive, and find all hidden alternate streams stored in the file system. After scanning and finding the alternate streams, you can extract these streams into the specified folder, delete unwanted streams, or save the streams list into text/html/csv/xml file.

And from another post
For plugin problems where dll's have been Blocked when downloading from the net (or any other files eg pdf etc).
(Problem dll's show in file Properties as BLOCKED, as mentioned two posts earlier).

Streams.exe by SysInternals (Micro$oft) for removing alternatve data streams:-
https://docs.microsoft.com/en-us/sysinternals/downloads/streams
Docs say Vista+, but seems to work just fine on XP.

And below referring to above as post #18

Further to previous posts on blocked dlls (or blocked any type files),
If you need to strip the alternate stream data from any files, and do not have net access to download Streams.exe from SysInternals (Microsoft),
then can just copy files to a FAT32 partition, (as FAT32 dont have alternate streams), and then just copy back and overwrite original files on your
NTFS partition.

EDIT: Link to Streams.Exe @ SysInternals in post #18. (XP+, despite web page saying Vista+).

EDIT: Better streams Alternate Streams Viewer, can delete Zone Identifier Alternate data stream.
https://www.nirsoft.net/utils/alternate_data_streams.html
https://i.postimg.cc/sg074tb2/Alternate-Streamview.jpg (https://postimages.org/)

EDIT:
OOps, I thought QueryFatVolume was a system call, its actually one of mine, as here

int __cdecl QueryFatVolume(const char *relname) {
// Return:- 1=FAT. 0 = Not FAT. -1 on error;
int ret = -1;
char FullPath[_MAX_PATH];
if(_fullpath(FullPath, relname, _MAX_PATH ) != NULL ) {
TCHAR RootPathName[_MAX_PATH];
_splitpath(FullPath,RootPathName, NULL,NULL,NULL );
char *p=RootPathName;
while(*p++);
--p;
if(p>RootPathName && p[-1] != '\\') {*p++='\\';*p='\0';}
TCHAR FileSystemNameBuffer[MAX_PATH+1];
BOOL result = GetVolumeInformation(RootPathName,NULL,0,NULL,NULL,NULL,FileSystemNameBuffer,MAX_PATH+1);
if(result) {
ret = (_strnicmp(FileSystemNameBuffer,"FAT",3)==0) ?1:0; // Just the 1st 3 characters (FAT/FAT32)
}
}
return ret;
}

// # Above, I seem to have mixed _MAX_PATH and MAX_PATH, but they equate to the same thing so should not be source of problem


I'm wondering, how long is your filename ?
Show filename path you are using, if relative path then ALSO give full path.
Also, what is fileSystem, ExtFat ?

EDIT: I Suggest scanning your plugins directories (or better still the UniversalAvisynthInstaller directory) with AlternateStreamView, and delete any Zone Identifier streams.

EDIT: A bit more on preventing blocked files using GPEdit.msc (Pro versions OS only):- https://forum.doom9.org/showthread.php?p=1872609#post1872609

EDIT: Also look at properties of your clip, dont know if video clips are ever blocked. [I think but am not sure, that I have seen clips blocked]

[StainlessS]

Try this mod to script,

Line 370, Duplicity2.avsi

DB=(DBase=="") ? RT_GetFullPathName("~@@@_"+RT_LocalTimeString+".DB") : RT_GetFullPathName(DBase)


Move to 629, just before RT_DBaseAlloc(), add line in red

DB=(DBase=="") ? RT_GetFullPathName("~@@@_"+RT_LocalTimeString+".DB") : RT_GetFullPathName(DBase)
RT_DebugF("DBase = '%s'",DB,name="DB_BUGTEST: ") # Cap in DebugView and post if problem
RT_DBaseAlloc(DB,FrameCount,"iiiiii")

[johnmeyer]

I wanted to try DropDeadGorgeous to see if it would work on my test clips. I copy/pasted from posts 46, 47, & 48. When I ran it, I got an error that it needed the DB file and I realized I need to run Duplicity2 first.

I downloaded Duplicity2 and ran it from the client supplied in the zip.

However, I got this error:

Avisynth open failure
I don't know what ThB means

I looked at your client, and you forgot to provide a default value for this variable, yet you pass it in the function call.

I looked at Duplicity2 and found that the default is simply 2x of another variable so I added this line:ThB = ThG*2

That fixed that problem but unfortunately, I then got this error:Avisynth open failure:
System exception - Access Violation
Duplicity2.avsi, line 760

Line 760 in that avsi file is this: HasGScript ? GScript(InstS) : Eval(InstS,"GS_EVAL") # Use GSCript if installed (loaded plugs override builtin)Even though GScript is in my plugins folder, I went ahead and explicity loaded it. I tried both RT_Stats_x86.dll and RT_Stats_25.dll (12/22/2020 versions).

I still get the same error, so I think it is something other than not having RT_Stats installed, or not having the latest version.

I am using AVISynth+ 3.7.0 running under Windows XP Pro 32-bit.

[StainlessS]

Hi John, sorry for the delay in response, been a bit otherwise engaged.

I don't know what ThB means

from post #25, (in blue [I guess I hid it a bit, sorry])

Import(".\Duplicity2.Avsi")

######
FN = "D:\G.AVI" DBase = "D:\G_Det.DB" ThB=2.0
#FN = "D:\GTEST.AVI" DBase = "D:\GTEST_Det.DB" ThB=2.0
#FN = "D:\HAR.AVI" DBase = "D:\HAR_Det.DB" ThB=2.5
#FN = "D:\DeDupTest.AVI" DBase = "D:\DeDupTest_Det.DB" ThB=2.0
######
Avisource(FN)
SRC=Last
###
Mode=2 # 0=Write Frames only : 1=Replace Dupes with Exact Dupes : 2= Interpolate/Blend Duplicates.
ThG=1.0 # Primary Frame Global Detect
ThP=64 # Pixel Count Detect
MaxDupLen=9 # Max detected, if more then is considered static sequence not duplicate)
MaxInterp=9 # Max Interpolated frames when Mode=2 (More than MaxInterp & less or equal to MaxDupLen then uses Blend mode)
InterpFast=true
Show=True
ShowDot=False
ShowBlk=true
VERB=False
###
TITLE=False
STACK=True
STACK4=False
###

Result=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=False,InterpFast=InterpFast,DBase=(STACK4||!SHOW)?DBase:"")

Metrics=SRC.Duplicity2(Mode=Mode,ThG=ThG,ThB=ThB,ThP=ThP,MaxDupLen=MaxDupLen,MaxInterp=MaxInterp,
\ Show=True,Verb=VERB,ShowBlk=ShowBlk,ShowDot=ShowDot,InterpFast=InterpFast,DBase=(STACK4||!SHOW)?"":DBase)

Dif=Subtract(Result,SRC)

SRC = (TITLE) ? SRC.TSub("Src") : SRC
Result = (TITLE) ? Result.TSub("Result") : Result
Metrics = (TITLE) ? Metrics.TSub("Metrics") : Metrics
Dif = (TITLE) ? Dif.TSub("Dif(Result,Src)") : Dif

LFT=StackVertical((STACK4||!SHOW)?Result:Metrics,SRC)
RGT=StackVertical(Metrics,Dif)
STK4=StackHorizontal(LFT,RGT)

(STACK4) ? STK4 : (STACK) ? LFT : (Show)?Metrics:Result

Return Last


# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}


Avisynth open failure:
System exception - Access Violation
Duplicity2.avsi, line 760
I seem to recall a GScript update recent-ish, maybe to smooth out something when using with avs+.
Also, if using AVS+, then GScript not required. [EDIT: remove GScript from plugins and retest with XP version AVS+]
Also, are you using XP version of AVS+ ? [It needs it now, recent test versions of AVS+ do not always contain the XP compat version of AVS+].


EDIT: GScript updated for AVS+ arrays, maybe other mods. [30 Nov 2020].
EDIT: I don't know where I got the 30 Nov 2020 GSCript from, Groucho compiled for x64 May 2020, but that aint it. [EDIT: I think Pinterf maybe did the 30 Nov 2020 ver$]

EDIT: Current AVS+ on XP requires special version because non XP ver$ uses system calls that are non-existing on XP [and so will crash].

[VoodooFX]

Isn't GRunT like replacement for GScript?

[StainlessS]

Isn't GRunT like replacement for GScript?
Nope.
Gscript = If, For, While, etc [for non + AVS]
Grunt = GScriptClip(Args="...",Local=True/False), + GFrameEvaluate, GConditionalFilter, GWriteFile and GWriteFileIf .
[ with Grunt, ScriptClip(Args="...",Local=True/False) etc also works, the 'G' versions just makes Grunt requirement explicit ]

Grunt:- https://forum.doom9.org/showthread.php?t=139337

Not sure if Avs+ implements as below

For added convenience, there is a new variant of ConditionalFilter which takes a single boolean expression instead of three separate parameters as at present. This is useful when the condition to be tested is a compound one or is already available in boolean form. For example,
ConditionalFilter(c, c1, c2, \
"AverageLuma(c1) > AverageLuma() && AverageLuma(c1) > AverageLuma(c2)")
where previously you would have to add (...,"=", "true")


EDIT:
P, I note that in runtime filters you added 'local' arg, so is now similar (I presume same) to Grunt, however you did not add 'Args' arg,
that is very handy addition and would still make it necessary for many scripts to use Grunt in Preference.
If Args arg added, would probably make Grunt an un-necessary and superfluous plugin. [Sorry Gavino :) ]
For a long time a few of us have wanted both GScript & Grunt capability added into Avs source, in avs std the idea was always resisted.

[VoodooFX]

Gscript = If, For, While, etc [for non + AVS]

Then why this example below produces "Script error: syntax error"?


dummy = 1
BlankClip()
if(dummy == 0) {
crop(2,2,-2,-2)
} else {
crop(4,4,-4,-4)
}

[Dogway]

I think you might need a space between the if and the bracket

[VoodooFX]

I think you might need a space between the if and the bracket
Didn't helped. Btw, error is at the first curly bracket.

[Dogway]

Ah yep, well it works for me but I use internal conditionals. Might be a Gscript bug.

[StainlessS]

Gcript requires to be wrapped in GScript("..."), eg


dummy = 1
BlankClip()
GSCript("""
if(dummy == 0) {
crop(2,2,-2,-2)
} else {
crop(4,4,-4,-4)
}
""")

Untested.

Or you can import a GScript script use eg GImport("..."), where script itself dont need no GScript("""...""") wrapper.

EDIT: GScript:- https://forum.doom9.org/showthread.php?t=147846&highlight=GSCript

EDIT: AVS+ implemented GScript language extensions do not require GScript("...") wrappers.
Best avoid loading GScript() plugin when using AVS+. (otherwise if/while etc will require Gscript wrappers, [plugins override builtins] EDIT: Not sure about this, maybe wrong)

EDIT:
Just tried above script, with and without GScript() wrappers, and both worked OK. [avs+ x64 with GScript dll in plugins]
I used to 'belly-ache' about wiki saying something like "avs+ does not need GScript wrappers",
saying that it should read "will not work with Gscript wrapper", so I guess Pinterf got the message and fixed it.
EDIT: Leaving GScript("...") wrapper uncommented, and removing dll from plugins, does produce an error for above script.

[VoodooFX]

Best avoid loading GScript() plugin when using AVS+.
Then to hell with that Avs 2.6 support, time to let it rest in peace. :devil:

[StainlessS]

To hell with ...
But see edits.

EDIT:
Best to advise user that v2.60 std using Gscript plugin, then need to load the script using GScript lang extensions using the GImport()
command, and use simple Import() or Plugins autoload avsi, when using AVS+. [AND STRIP ALL INSTANCES of the GScript Wrappers from the actual script containing GScript extensions].

[VoodooFX]

But see edits.
I see.
I renamed InpaintDelogo.avsi to InpaintDelogo.avs, then called it with:
GImport("c:\Program Files (x86)\AviSynth 2.5\plugins\InpaintDelogo.avs")
And looks like it started working with Avs 2.6. [when it comes to run actual if/else code then same error]

EDIT:
Actually it still does't work, it just squashed that error on the script load stage.

EDIT2:
But that "dummy" example works...

EDIT3:
It doesn't work because if/else is inside eval("""..."""), need to get rid of it.

[StainlessS]

You can only really load script functions containing GScript extensions, [or whatever code allowed in *.avsi scripts],
in main script, must still wrap any GScript extension code.

Not ideal, but there you go :) [Gscript("...") and GImport("...") and GEval("...") only affect code that is wrapped in those G thingies]

where Import() is sorta like

S = RT_ReadTextFromFile(".\SomeScriptFile.avs")
Eval(S)


GImport() is sorta like,

S = RT_ReadTextFromFile(".\SomeIfElseFile.avs")
GEval(S)


GScript() is sorta like,

S = "Some If Else stuff"
GEval(S)


You can use eg


Function FuncNameExists(String Fn) { Try{Eval(Fn+"()")B=True}catch(e){Assert(e.FindStr("syntax")==0,"FuncNameExists: Error in Function Name '"+Fn+"'")B=(e.FindStr("no function named")==0)}Return B}
Function HasGScript() { Return FuncNameExists("GSCript") } # v2.60 std dont have FunctionExist(), so we use above whotsit [which i believe will work ok even in v2.58].

gString =""" Some If Else Stuff"""
(HasGScript) ? GEval(gString) : ? Eval(gString)



EDIT3:
It doesn't work because if/else is inside eval("""..."""), need to get rid of it.
looks like you got it.

[VoodooFX]

Thanks StainlessS, I think I can make it work with few small edits.

So, your DetSub_Extract.avs can be Avs 2.6 compatible too after all, or not?

[flossy_cake]

I'm after something that can blend a duplicate with its neighbours over 4 or 5 frames to hide the jerk.

It seems again you've given me exactly what I want StainlesS, but I need to learn how to use it and my brain isn't working at the moment.

I had previously been trying to achieve it with TDecimate, which can almost do it with its hybrid=3 mode, and you can even specify how many frames to blend the duplicate over with the Cycle=n value.

But it has some kind of bug or intentional behaviour which makes it blend with the wrong frame, eg.

Test video 1080p24 clock: https://drive.google.com/file/d/1jpvkan5bi6ICIpDV28tr_lOTAX7sSVqY/view?usp=sharing


# downscale for speedup
Spline36Resize(854, 480)

# synthesise 1 duplicate frame per second for the first 4 seconds when the clock hand is pointing at 12 (north)
DuplicateFrame(24, 48, 72, 96)

TDecimate(mode=0, /* 0 or 1 makes no difference */
\ cycleR=1, /* must always be 1 */
\ cycle=2, /* set this to the number of frames you want to blend the dupe over */
\ hybrid=3,
\ viddetect=2,
\ vidthresh=0.9, /* framediff < 0.9 = dupe */
\ conCycle=1,
\ dupthresh=0.0,
\ denoise=true,
\ chroma=false,
\ hint=false,
\ display=true)


The first duplicate is done properly, blended with previous (23+24) then blended with next (24-26)

https://i.ibb.co/bNBzSdc/animated1.png

But then the next time the clock hand comes around it blends with 2 frames ahead (51+53 instead of 51+52):

https://i.ibb.co/pK6y0VK/animated2.png

I can increase the Cycle value to 3,4,5 and kind of reduce/delay the issue, but it still eventually does a blend with 2 frames ahead. I think maybe it's following some rule of blending with the next frame with the highest framediff or something, but I don't want that, I just want it to blend with neighbours only. It seems like your filter will do exactly that, just need to figure out how :o

[StainlessS]

Sorry Flossy, I saw this 1 or 2 days ago but had no time. [missed it previously]

Did you get it sorted.
The DropDeadGorgeous thingy is really for situations like webcam cap, where frames may be dropped due to
time constraints and cap driver code speed, and same number of dupes were inserted to make up for it,
so not suitable for your requirement.

Duplicity2, can interpolate/blend a number of consecutive dupes, but using only the two frames from either side of the dupe sequence.

I'm no expert with TDecimate.

[flossy_cake]

Did you get it sorted.

Yep got it sorted with a slight tweak to TDecimate.cpp in Visual Studio.

For my NTSC DVDs containing cadence breaks I will probably do 2 passes of TDecimate - pass 1 converts the TFM'd 30p to 24p by removing 1 in 5 dupes, then a second pass in hybrid=3 mode to blend over any remaining duplicates which occured due to cadence breaks in the source. I found a cycle value of 5-10 seems to look ok, i.e blend the dupe over 5-10 frames. 2 frames is not enough and still can see the jerk. 5 is kind of okay but still can see a slight jerk. Maybe 7 or 8 could be optimal, still experimenting. The hybrid modes have a useful feature where if it sees one of the frames in the cycle appears to be a scenechange frame, it will not blend it, which is good otherwise you would get blended scenechange frames which look a bit weird visually.

[rgr]

I haven't installed it yet (I use Dup1 to detect duplicates), but... wouldn't it be possible to link this plugin (Duplicity) to RIFE (https://github.com/Asd-g/AviSynthPlus-RIFE) to create interpolated frames?

[StainlessS]

possible to link this plugin (Duplicity) to RIFE
I've no idea how RIFE works (never tried it), but Duplicity is a matched pair of scripts for detector/fixer, it is an all or nothing setup.