interleave with a condition

[mathmax]

Hello

I'm trying to do something very specific.. I wonder if that is possible with avisynth. Let me explain the problem:
I have two different clips.. each of them has values associated to its frames. I would like to interleave the frames of the 2 clips ordering them according to the associated values.

For example:

clip 1:

frame 1 : 10
frame 2 : 20
frame 3 : 35
frame 4 : 40
frame 5 : 50
frame 6 : 55

clip 2:

frame 1 : 5
frame 2 : 15
frame 3 : 25
frame 4 : 30
frame 5 : 45
frame 6 : 60

output clip should be:

clip2 frame 1 : 5
clip1 frame 1 : 10
clip2 frame 2 : 15
clip1 frame 2 : 20
clip2 frame 3 : 25
clip2 frame 4 : 30
clip1 frame 3 : 35
clip1 frame 4 : 40
clip2 frame 5 : 45
clip1 frame 5 : 50
clip1 frame 6 : 55
clip2 frame 6 : 60

I need some kind of smart Interleave() with a condition... I also had a look at ConditionalFilter() and ConditionalSelect() but the expression applies on the frames of a clip...
What I want to do is quite different. I want to select either from source 1 or source 2 the next frame with the smallest associated value... is that possible?

[Gavino]

Can we assume that within each clip, the frames appear in ascending order of 'value'?
Can the value for each frame be obtained from a run-time function call (eg AverageLuma, etc), or at least a combination of such calls?

In that case, I think with clever use of ScriptClip it could be done.

[mathmax]

Can we assume that within each clip, the frames appear in ascending order of 'value'?

yes


Can the value for each frame be obtained from a run-time function call (eg AverageLuma, etc), or at least a combination of such calls?

the values can be obtained from a text file using :

ConditionalReader("mytextfile.txt", "myvar")


In that case, I think with clever use of ScriptClip it could be done.
Great! How would you do? :)

[Gavino]

# set sources:
clip1 = ...
clip2 = ...

# setup for reading values:
clip1 = clip1.ConditionalReader("textfile1.txt", "value1")
clip2 = clip2.ConditionalReader("textfile2.txt", "value2")

BlankClip(clip1, clip1.frameCount+clip2.frameCount) # template for ScriptClip input and result
ScriptClip("""
# force frame access to read values:
AverageLuma(clip1)+AverageLuma(clip2)

# select clip to use for current frame:
choice = current_frame >= clip1.frameCount ? 2
\ : current_frame >= clip2.frameCount ? 1
\ : value1 < value2 ? 1 : 2
result = choice == 1 ? clip1 : clip2

# keep unused frame for next time round:
clip1 = choice == 2 ? clip1.SelectEvery(1,-1) : clip1
clip2 = choice == 1 ? clip2.SelectEvery(1,-1) : clip2

return result
""")
The script must be rendered linearly from start to finish, as each frame depends on the previous history so far.

[mathmax]

# set sources:
clip1 = ...
clip2 = ...

# setup for reading values:
clip1 = clip1.ConditionalReader("textfile1.txt", "value1")
clip2 = clip2.ConditionalReader("textfile2.txt", "value2")

BlankClip(clip1, clip1.frameCount+clip2.frameCount) # template for ScriptClip input and result
ScriptClip("""
# force frame access to read values:
AverageLuma(clip1)+AverageLuma(clip2)

# select clip to use for current frame:
choice = current_frame >= clip1.frameCount ? 2
\ : current_frame >= clip2.frameCount ? 1
\ : value1 < value2 ? 1 : 2
result = choice == 1 ? clip1 : clip2

# keep unused frame for next time round:
clip1 = choice == 2 ? clip1.SelectEvery(1,-1) : clip1
clip2 = choice == 1 ? clip2.SelectEvery(1,-1) : clip2

return result
""")
The script must be rendered linearly from start to finish, as each frame depends on the previous history so far.

Thank you very much for this. It is very logical finally.. :)

However, the script crashes after a certain amount of frames are processed. Do you know why?

[Gavino]

Possibly running out of memory. (How many frames do you have?)
Try this new version - instead of building up a long SelectEvery() filter chain with each new frame added, it simply keeps track of the number of frames used from each clip and uses a single filter to get the right frames from the original clips each time.

# set sources:
clip1 = ...
clip2 = ...

# setup for reading values:
clip1 = clip1.ConditionalReader("textfile1.txt", "value1")
clip2 = clip2.ConditionalReader("textfile2.txt", "value2")

n1 = 0 n2 = 0
BlankClip(clip1, clip1.frameCount+clip2.frameCount) # template for ScriptClip input and result
ScriptClip("""
c1 = clip1.SelectEvery(1,-n1)
c2 = clip2.SelectEvery(1,-n2)

# force frame access to read values:
AverageLuma(c1)+AverageLuma(c2)

# select clip to use for current frame:
choice = current_frame >= c1.frameCount ? 2 \
: current_frame >= c2.frameCount ? 1 \
: value1 < value2 ? 1 : 2

# keep unused frame for next time round:
n1 = choice == 2 ? n1+1 : n1
n2 = choice == 1 ? n2+1 : n2

return (choice == 1 ? c1 : c2)
""")

[mathmax]

Thank you.

This time it work well :)
I have 25 000 frames in each clip.

One question: is it possible to select the next/previous frame inside scriptclip?

for example, if I would like to compare two frames of the current clip, should I write:


clp.ScriptClip("""
prevframe = clp.Trim(clp.Framecount - 1, -1)
nextframe = clp.Trim(clp.Framecount + 1, -1)
compare(prevframe , nextframe )
...
""")


or is there a proper way to write this?

[Gavino]

One question: is it possible to select the next/previous frame inside scriptclip?
Yes, you can use SelectEvery(1,-1) or SelectEvery(1,1).
Also, if using GRunT, you can use things like AverageLuma(-1) to directly get the value for the previous frame.

EDIT: Missed your later edit:
clp.ScriptClip("""
prevframe = clp.Trim(clp.Framecount - 1, -1)
nextframe = clp.Trim(clp.Framecount + 1, -1)
compare(prevframe , nextframe )
...
""")
That code isn't right - you would need to use current_frame instead of clp.Framecount. But as I suggested, the simplest way is
prevframe = SelectEvery(1,-1)
nextframe = SelectEvery(1,1)

[mathmax]

Thank you. Yes I meant current_frame but I missed to type..

I'm playing with scriptclip now and I have another question, although it has not much to do with the original subject:

Why does this code not work? It should add a growing border each two lines of my clip.. but it just does nothing :s

ProcessLine(0)

function ProcessLine(clip clp, int n)
{
line = clp.crop(0, n, -0, 2)

newline = line.ScriptClip("""
return line.crop(n, 0, -0, -0).addborders(n, 0, 0, 0)
""")

return (n < 50) ? StackVertical(newline, ProcessLine(clp, n + 2)) : newline
}

[Gavino]

Why does this code not work? It should add a growing border each two lines of my clip.. but it just does nothing
Your ScriptClip calls do not work because the local variables line and n do not exist at run-time. See here (http://avisynth.org/mediawiki/The_script_execution_model/Scope_and_lifetime_of_variables#Runtime_scripts).

The result is that each call just returns the original 'line' with an overlayed error message - which cannot be seen as the clips are only 2 pixels high.

But why are you using ScriptClip here? It's only needed when you want to do processing that is in some way frame-dependent (depends dynamically on the contents of each frame).
http://avisynth.org/mediawiki/Runtime_environment

In your example, you do the same thing on all frames, so ScriptClip is unnecessary.

[mathmax]

The example was voluntarily simplified because I tried to understand where the error came from. I often do like that until I can isolate the problem.. and then I posted the simplified example.

But I think my full script needs scripclip. It's an attempt to fix damaged scan lines by merging the chroma of neighboring frames. Here is the full code:


clprgb = converttorgb()
ref = clprgb.crop(0, 50, -0, 50)
clp = crop(0, 0, -0, 50)
ProcessLine(ref, 0)

function ProcessLine(clip clp, clip ref, int n)
{
line = clp.crop(0, n, -0, 2)
linergb = line.converttorgb()

newline = line.ScriptClip("""
diffp2 = linergb.SelectEvery(1,-2).RGBDifference(ref.SelectEvery(1,-2))
diffp1 = linergb.SelectEvery(1,-1).RGBDifference(ref.SelectEvery(1,-1))
diff = linergb.RGBDifference(ref)
diffn1 = linergb.SelectEvery(1, 1).RGBDifference(ref.SelectEvery(1, 1))
diffn2 = linergb.SelectEvery(1, 2).RGBDifference(ref.SelectEvery(1, 2))

result = diffp1 < diff && diffp1 <= diffp2 && diffp1 <= diffn1 && diffp1 <= diffn2 ? line.mergechroma(line.SelectEvery(1,-1)) \
: diffn1 < diff && diffn1 <= diffn2 && diffn1 <= diffp1 && diffp1 <= diffp2 ? line.mergechroma(line.SelectEvery(1, 1)) \
: diffp2 < diff && diffp2 < diffn1 && diffp2 < diffp1 && diffp2 <= diffn2 ? line.mergechroma(line.SelectEvery(1,-2)) \
: diffn2 < diff && diffn2 < diffn1 && diffn2 < diffp1 && diffn2 <= diffp2 ? line.mergechroma(line.SelectEvery(1, 2)) \
: line

return result
""")
return (n < 50) ? StackVertical(newline, ProcessLine(clp, ref, n + 2)) : newline
}


I could defind linergb inside scriptclip(), but I don't know how I can make the parameter "ref" exist at run-time.. :-/

And in the last code you sent, the variable local variables n1 = 0 and n2 = 0 were also defined outside scriptclip()... but that didn't seem to cause any problem. Why?

edit: In RGBDifference(clip1, clip2), I guess that the dimensions of clip1 and clip2 must be the same.. don't know if there is still a problem with local variables though :-/

[Gavino]

I could defind linergb inside scriptclip(), but I don't know how I can make the parameter "ref" exist at run-time.
Also, 'line' itself is local to the function, so doesn't exist either.
However, as 'line' is the input argument of ScriptClip, it is available inside the run-time script as 'last'.
'ref' could be made a global variable, although this would not work if ProcessLine was called with different 2nd arguments in the same script.
But the cleanest way to handle variable passing with ScriptClip is to use GRunT's extended version of the filter with its 'args' parameter. As I mentioned in the previous post, GRunT also provides a simpler way to call RGBDifference on other frames. So, putting all this together, your function could be written as
function ProcessLine(clip clp, clip ref, int n)
{
line = clp.crop(0, n, -0, 2)

newline = line.ScriptClip("""
linergb = converttorgb()
diffp2 = linergb.RGBDifference(ref, -2)
diffp1 = linergb.RGBDifference(ref, -1)
diff = linergb.RGBDifference(ref)
diffn1 = linergb.RGBDifference(ref, 1)
diffn2 = linergb.RGBDifference(ref, 2)

result = diffp1 < diff && diffp1 <= diffp2 && diffp1 <= diffn1 && diffp1 <= diffn2 ? mergechroma(SelectEvery(1,-1)) \
: diffn1 < diff && diffn1 <= diffn2 && diffn1 <= diffp1 && diffp1 <= diffp2 ? mergechroma(SelectEvery(1, 1)) \
: diffp2 < diff && diffp2 < diffn1 && diffp2 < diffp1 && diffp2 <= diffn2 ? mergechroma(SelectEvery(1,-2)) \
: diffn2 < diff && diffn2 < diffn1 && diffn2 < diffp1 && diffn2 <= diffp2 ? mergechroma(SelectEvery(1, 2)) \
: last

return result
""", args="ref")
return (n < 50) ? StackVertical(newline, ProcessLine(clp, ref, n + 2)) : newline
}


BTW Shouldn't the initial call (in the script) be ProcessLine(clp, ref, 0)?

And in the last code you sent, the variable local variables n1 = 0 and n2 = 0 were also defined outside scriptclip()... but that didn't seem to cause any problem. Why?
Because n1 and n2 were script-level variables, not inside a function.

EDIT: Just seen your later edit:
In RGBDifference(clip1, clip2), I guess that the dimensions of clip1 and clip2 must be the same.
Yes, so I see now your code as written will not work anyway.
I haven't tried to understand what you're actually doing, but you probably need to crop to make ref and linergb the same size.

[mathmax]

it still doesn't work although I have GRunT.dll in my avisynth plugin folder... :( Should I do something special to call the extended version of scriptclip()?

Btw, I think I'll replace RGBDifference()

by


function ChromaDifference(clip clp1, clip clp2)
{
return Pow(clp1.ChromaUDifference(clp2), 2) + Pow(clp1.ChromaVDifference(clp2), 2)
}

[Gavino]

it still doesn't work although I have GRunT.dll in my avisynth plugin folder... :( Should I do something special to call the extended version of scriptclip()?
Only if running Avisynth 2.57, when you need to call it as GScriptClip.
Did you add args="ref"?
What error do you get?


Btw, I think I'll replace RGBDifference() by

function ChromaDifference(clip clp1, clip clp2)
{
return Pow(clp1.ChromaUDifference(clp2), 2) + Pow(clp1.ChromaVDifference(clp2), 2)
}

More efficient to use x*x instead of Pow(x,2) - Pow is for float exponents and uses exp and log.

BTW Only with GRunT can you call run-time functions (eg ChromaUDifference) in a user function - standard Avisynth run-time doesn't allow it. ;)