Log in

View Full Version : Need help substituting frames around a scene change with another source's.


Chainmax
3rd October 2014, 21:44
I just made two encodes out of a ~20m video, the reason for that being that the first encode doesn't look too hot around scene changes, but looks pretty nice elsewhere. The idea would be to use the second encode's frames only around the scene changes of the first, with maybe a a couple neighboring frames just in case (so, if a scene change happens between frames n and n+1, then frames [n-1,n+2] works be substituted by the second encode's). How should I go about doing that?

johnmeyer
4th October 2014, 01:26
StainlessS has been working on some scene change logic as part of his RT_Stats "opus." That package also includes "nicer" conditional logic for the:

if scene change (clip1,clip2)

statement (that is pseudo code) that will be the heart of your script.

StainlessS
4th October 2014, 12:00
Here something I was playing with a couple of weeks ago, took out some testing stuff, think its in good working condition.

You need to remove the "# HACK " comment text from beginning of 5 lines to enable it (Hi-lited).
SceneChangeDump.avs

Function SceneChangeDump(clip clp,Float "dFact",Float "dMinim",Int "dMinLen",Float "ChromaWeight",Bool "Show",
\ String "Fn_Start",String "Fn_Mid",String "Fn_End",String "Fn_Range",Bool "CommaSep") {
/*
SceneChangeDump: by StainlessS. Requires GSCript, Grunt, RT_Stats.
Multi-Instance version, Thanks to Martin53 & Gavino for their development.
NOTE, Output filenames (Fn_start etc) are NOT MULTI-INSTANCE, Must give individual names for each instance.
Can jump about on timeline although output files will likely contain rubbish, also if jumping prior to current position, then
scene start frame may be reset to zero and current scene lenth made relative to frame zero (relevant for dMinLen condition).

Args:-
dFact, Default 1.5 (greater than 1.0)
dMinim, Default 8.0, (greater than 0.1) dFact and dMinim , Control how much bigger difference between Curr->Next has to be compared
to Prev->Curr and Next->Next+1, to detect scene change.
Curr->Next has to be greater than (dFact * 'Prev->Curr' + dMinim), and also greater than (dFact * 'Next->Next+1' + dMinim)
Dfact=1.5, dMinim=8.0 works pretty good.
dMinLen, Default 12. Minimum frame length of current scene that qualifies for scene change detection.
ChromaWeight, Default 0.333333 ChromaWeight range 0.0->1.0. YUV Only, default gives twice as much weight to luma as combined chroma.
Show, Default True. Show metrics.
Row 1 Shows some status info, FrameNo, T Conditions, SceneNo, SceneLen.
T Conditions, Show T (True) or F (False).
T1) Current Scene must be at least dMinLen frames long.
T2) Center Metric (Curr->Next) must be greater than Left Metric value in parenthesis (dFact * 'Prev->Curr' + dMinim).
T3) Center Metric (Curr->Next) must be greater than Right Metric value in parenthesis (dFact * 'Next->Next+1' + dMinim).
Row 2, shows Metrics, Prev->Curr, Curr->Next and Next->Next+1. [resultant adjacent values in parenthesis eg (dFact * Prev->Curr + dMinim)]
Fn_Start, Default "". FileName where scene Start frame numbers will be written.
Fn_Mid, Default "". FileName where scene Middle frame numbers will be written.
Fn_End, Default "". FileName where scene End frame numbers will be written.
Fn_Range, Default "". FileName where scene Start and End frame numbers will be written as range, see CommaSep.
CommaSep, Default False, use SPACE range separator, True use COMMA range Separator in Fn_Range file.

*/
myName="SceneChangeDump: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
clp
dfact = Float(Default(dFact,1.5)) # Lower more false +ve's, Higher fewer scene changes (Dfact=1.5 dMinim=8.0 works pretty good)
dMinim = Float(Default(dMinim,8.0)) # Scene Change, avoid silly numbers (very small) on static scenes causing false +ve
dMinLen = Default(dMinLen,12) # Minimum length of scene for scene change detection to be valid (ignore short scenes)
ChromaWeight = Float(Default(ChromaWeight,1.0/3.0)) # Weighting for combined U + V Channels. Suggest about 1.0/3.0 -> 1.0/2.0
Show = Default(Show,True) # Basic metrics
Fn_Start = Default(Fn_Start,"") #
Fn_Mid = Default(Fn_Mid,"") #
Fn_End = Default(Fn_End,"") #
Fn_Range = Default(Fn_Range,"") #
CommaSep = Default(CommaSep,False) # SPACE speparator default for Range
Assert(dFact>1.0,myName+"dFact MUST be greater than 1.0")
Assert(dMinim>0.1,myName+"dMinim MUST be greater than 0.1")
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,myName+"ChromaWeight 0.0->1.0")
Sep = RT_Ord((CommaSep) ? "," : " ")
(Fn_Start!="" && Exist(Fn_Start)) ? RT_FileDelete(Fn_Start) : NOP
(Fn_Mid!="" && Exist(Fn_Mid)) ? RT_FileDelete(Fn_Mid) : NOP
(Fn_End!="" && Exist(Fn_End)) ? RT_FileDelete(Fn_End) : NOP
(Fn_Range!="" && Exist(Fn_Range)) ? RT_FileDelete(Fn_Range) : NOP
STARTCLIP = SubTitle("START OF SCENE",Align=5,Size=64)
ENDCLIP = SubTitle("END OF SCENE", Align=5,Size=64)
#
#
Fmt="%d ] {\a%c%.1s\a-:\a%c%.1s\a-:\a%c%.1s\a-} %d#%d\n%7.3f(\a%c%7.3f\a-) \a%c%7.3f\a- %7.3f(\a%c%7.3f\a-)"

# HACK RT_FileDelete("Hack.txt")

FuncS="""
Function SceneChangeDump_@@@(clip clp,Float dFact,Float dMinim,Int dMinLen,Float ChromaWeight,Bool Show,
\ clip STARTCLIP,clip ENDCLIP,String Fmt,Int Sep) {
clp
n = current_frame
ReStart = False
If(Prev@@@ == n) { # Cache failure, requested same frame again. Rightshift everything, will be Leftshifted later.
#RT_Debugf("%d ] CACHE FAILURE",n,name="SceneChangeDump: ")
C@@@ = B@@@ Global B@@@ = A@@@
} Else If(Prev@@@ + 1 != n) {
# User jumped about (or, start new at frame 0), get values as if at frame previous to current_frame.
Global B@@@ = RT_FrameDifference(Last,n-1,n+0,ChromaWeight=ChromaWeight) # was B when at current_frame - 1
Global C@@@ = RT_FrameDifference(Last,n+0,n+1,ChromaWeight=ChromaWeight)
ReStart = True
}
A=B@@@ B=C@@@ # Pass the parcel. (re-use metrics from previous iteration)
C = RT_FrameDifference(Last,n+1,n+2,ChromaWeight=ChromaWeight) # New difference

T1 = ((dMinLen<=1) || (n-Start@@@ + 1) >= dMinLen) # Scene length at least dMinlen frames ?
T2 = ((dFact*A + dMinim) < B) # Curr->Next sufficiently bigger than Prev->Curr ?
T3 = ((dFact*C + dMinim) < B) # Curr->Next sufficiently bigger than Next->Next+1 ?

EOS = ((T1 && T2 && T3) || n == Last.FrameCount-1) # END OF SCENE conditions fullfilled ?

if(ReStart) {
Global Start@@@ = Max((Start@@@ > n) ? n : Start@@@,0) # Reset Start frame if scene length would be -ve
Global SceneNo@@@ = (Start@@@==0) ? 0 : SceneNo@@@
}

if(EOS) {
(Fn_Start@@@!="")? RT_WriteFile(Fn_Start@@@,"%d",Start@@@ ,append=true) : NOP
(Fn_Mid@@@!="") ? RT_WriteFile(Fn_Mid@@@ ,"%d",(Start@@@+n)/2 ,append=true) : NOP
(Fn_End@@@!="") ? RT_WriteFile(Fn_End@@@ ,"%d",n ,append=true) : NOP
(Fn_Range@@@!="")? RT_WriteFile(Fn_Range@@@,"%d%c%d",Start@@@,Sep,n,append=true) : NOP

# HACK RT_WriteFile("Hack.txt","1 %d",Start@@@ + 0, Append = True) # Start of Scene + 0 (use alt clip 1 instead of 0)
# HACK RT_WriteFile("Hack.txt","2 %d",Start@@@ + 1, Append = True) # Start of Scene + 1 (use alt clip 2 instead of 0)
# HACK RT_WriteFile("Hack.txt","3 %d",n - 1, Append = True) # End of Scene - 1 (use alt clip 3 instead of 0)
# HACK RT_WriteFile("Hack.txt","4 %d",n - 0, Append = True) # End of Scene - 0 (use alt clip 4 instead of 0)

Last = (Show) ? ENDCLIP : Last
} Else If(Start@@@==n) {
Last = (Show) ? STARTCLIP : Last
}
if(SHOW) {
CC=(EOS)?33:45 # Hilite or Default color ? (Subtitle Last frame before Scene Change Status)
RT_Subtitle(Fmt,n,
\ CC,T1,CC,T2,CC,T3,
\ SceneNo@@@+1,n-Start@@@+1,
\ A,CC,(dFact*A + dMinim),CC,B,C,CC,(dFact*C + dMinim))
}

# Must be at end of scriptclip (and not before Subtitle Select or Metrics)
if(EOS) {
Global SceneNo@@@ = SceneNo@@@ + 1
Global Start@@@ = n + 1
}
Global A@@@=A Global B@@@=B Global C@@@=C # set Globals for next iteration
Global Prev@@@ = current_frame
Return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
Global Fn_Start@@@ = Fn_Start
Global Fn_Mid@@@ = Fn_Mid
Global Fn_End@@@ = Fn_End
Global Fn_Range@@@ = Fn_Range
Global A@@@=0.0 Global B@@@=0.0 Global C@@@=0.0
Global Start@@@=0 Global SceneNo@@@ = 0 Global Prev@@@ = -666
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "dFact,dMinim,dMinLen,ChromaWeight,Show,STARTCLIP,ENDCLIP,Fmt,Sep"
Last.GScriptClip("SceneChangeDump_@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
RT_IncrGlobal("SCD_InstanceNumber")
InstS = RT_StrReplace(FuncS,"@@@","_SCD_"+String(SCD_InstanceNumber))
# RT_WriteFile("DEBUG_SceneChangeDump_"+String(SCD_InstanceNumber)+".TXT","%s",InstS)
Return GScript(InstS)
}


AVISource("test.avi").Killaudio()

#config
dFact = 1.5 # Lower more false +ve's, Higher fewer scene changes detected (Dfact=1.5 dMinim=8.0 works pretty good)
dMinim = 8.0 # Scene Change, avoid silly numbers (very small) on static scenes causing false +ve
dMinLen = 12 # Minimum length of scene for scene change detection to be valid (ignore short scenes)
ChromaWeight= 1.0/3.0 # Weighting for combined U + V Channels. Suggest about 1.0/3.0 -> 1.0/2.0
SHOW = True # basic metrics
#

fn_Start = "Start.Txt"
fn_Mid = "Mid.Txt"
fn_End = "End.Txt"
fn_Range = "Range.Txt"

RoboCrop(Wmod=4) # Optional, Remove letterboxing, better metrics. Requires RoboCrop().
SceneChangeDump(dFact,dMinim,dMinLen,ChromaWeight,Show,Fn_Start,Fn_Mid,Fn_End,Fn_Range)
Assumefps(250.0)
################


And test editing script

AVISource("test.avi").Killaudio()
Clip2=Last

SIZE=120

A=Clip2.Subtitle("1 : Start + 0",Size=SIZE,align=5)
B=Clip2.Subtitle("2 : Start + 1",Size=SIZE,align=5)
C=Clip2.Subtitle("3 : End - 1",Size=SIZE,align=5)
D=Clip2.Subtitle("4 : End - 0",Size=SIZE,align=5)

ClipClop(Last,A,B,C,D,cmd="Hack.txt")

return Last


I've kept as two separate edited clips, you could make both A and B from 2nd encode, perhaps more flexible if kept as separates.

Need very recent RT_Stats for RT_WriteFile(). RoboCrop use preferable but not imperative.
EDIT: Also requires ClipClop() and GScript() and Grunt().

EDIT: Oops, I think I misunderstood, I've just replaced two single frames, you mean you want n-1 -> n+2 replaced, I'll shall change.

EDITED: Fixed to replace 4 frames per scene change.

EDIT: If say you only wanted to replace A,B and C, then just replace D in ClipClop line with Last.
If want to replace all 4 frames from Encode_2 Clip2, then eg

ClipClop(Last,Clip2,Clip2,Clip2,Clip2,cmd="Hack.txt")

ClipClop() is quite efficient, about equivalent to a single script trim or splice, even up to 255 replacement clips in function call
(and with any number of replacements in command file).

colours
5th October 2014, 08:45
First off, the correct way to handle this is to simply use filters that don't break near scene changes. (I'm doing a bit of psychic debugging here and assuming you really mean you have two different filter chains, not two different encodes, because that would literally not be fixable unless you use an intra-only format.)

That said.

normal = [clip to use for non-scene-change frames]
sc = [clip to use around scene changes]
normal
scmask = converttoy8().crop(0,0,1,1).scxvidmask("xxx_keyframes.log")
scmask = mt_logic(scmask,scmask.selectevery(1,2),mode="or")
scmask = mt_logic(scmask,scmask.selectevery(1,-1),mode="or")
conditionalfilter(scmask,normal,sc,"averageluma","<","1")


What you'll need for this is an Xvid stats file, which you can generate with the Xvid CLI, SCXvid or SCXvid-standalone. (To wit, for SCXvid, just create an avs file with 'source.scxvid("xxx_keyframes.log")' and then run an analysis pass in AvsPmod or similar.)

All the required plugins are linked on the Avisynth wiki (http://avisynth.nl/index.php/External_filters), as usual.