Simple Runtime Function

[StainlessS]

This moved here from another thread:
http://forum.doom9.org/showthread.php?p=1582243#post1582243


First off, Avisynth calls the function you specify in AvisynthPluginInit2() when your filter is instantiated. You can't get frames yet at this point (I think? never tried it), but you do have access to VideoInfo and the arguments you were called with

I was a little curious about this as I possibly intend to do something similar in near future, so

#include <windows.h>
#include "Avisynth.h"

// Helper function - exception protected wrapper, From Avisynth source, conditional.cpp
inline AVSValue GetVar(IScriptEnvironment* env, const char* name) {
try {return env->GetVar(name);} catch (IScriptEnvironment::NotFound) {} return AVSValue();
}

AVSValue __cdecl AveLuma(AVSValue args, void* user_data, IScriptEnvironment* env) {
PClip child = args[0].AsClip();
int n;
if(args[1].IsInt()) {n = args[1].AsInt(); }
else {
AVSValue cn = GetVar(env,"current_frame");
if (!cn.IsInt()) env->ThrowError("AveLuma: 'current_frame' only available in runtime scripts");
n = cn.AsInt();
}
const VideoInfo &vi = child->GetVideoInfo();
n = (n<0) ? 0 : (n>=vi.num_frames)?vi.num_frames-1:n; // Range limit frame n
PVideoFrame src = child->GetFrame(n,env);
int h,w;
const int pitch = src->GetPitch(PLANAR_Y); // PLANAR_Y no effect on non-Planar (equates to 0)
const int rowsize = src->GetRowSize(PLANAR_Y);
const int height = src->GetHeight(PLANAR_Y);
const BYTE *srcp = src->GetReadPtr(PLANAR_Y);
const double Pixels = (double)(height * vi.width);
__int64 acc = 0;
unsigned long sum = 0;
if(vi.IsYUV()) {
const int wstep = (vi.IsYUY2()) ? 2 : 1;
for(h=height ; --h >= 0 ; ) { // h is upside down to actual scan
// for (w=0 ; w < rowsize; w += wstep) { // w scans left to right
// Slightly faster on non-optimized compiler, about same on optimised compiler
for (w=rowsize ; (w -= wstep) >= 0; ) { // w scans right to left
sum += srcp[w];
}
if(sum & 0x80000000) {acc += sum;sum=0;} // avoid possiblilty of overflow
srcp += pitch;
}
} else {
// RGB to YUV-Y Conversion
int matrix = args[2].AsInt(0); // Default=0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
if(matrix & 0xFFFFFFFC) env->ThrowError("AveLuma: RGB matrix 0 to 3 Only");
double Kr,Kb;
int Sy,offset_y;
if(matrix & 0x01) {Kr = 0.2126; Kb = 0.0722;} // 709 1 or 3
else {Kr = 0.2990; Kb = 0.1140;} // 601 0 or 2
if(matrix & 0x02) {Sy = 255 ; offset_y = 0;} // PC 2 or 3
else {Sy = 219 ; offset_y = 16;} // TV 0 or 1
const int shift = 15;
const int half = 1 << (shift - 1);
const double mulfac = double(1<<shift);
double Kg = 1.0 - Kr - Kb;
const int Srgb = 255;
const int Yb = int(Sy * Kb * mulfac / Srgb + 0.5); //B
const int Yg = int(Sy * Kg * mulfac / Srgb + 0.5); //G
const int Yr = int(Sy * Kr * mulfac / Srgb + 0.5); //R
const int OffyPlusHalf = (offset_y<<shift) + half;
const int wstep = (vi.IsRGB24()) ? 3 : 4;
for(h=height ; --h >= 0; ) { // h is same as actual RGB scan, bottom to top
// for (w=0 ; w < rowsize; w += wstep) { // w scans left to right
// Seems slightly faster on optimized compiler, nothing in it on non optimized.
for (w=rowsize ; (w -= wstep) >= 0; ) { // w scans right to left
sum += (srcp[w] * Yb + srcp[w+1] * Yg + srcp[w+2] * Yr + OffyPlusHalf) >> shift;
}
if(sum & 0x80000000) {acc += sum;sum=0;} // avoid possiblilty of overflow
srcp += pitch;
}
}
acc += sum;
return (float)((double)acc / Pixels);
}

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
env->AddFunction("AveLuma", "c[n]i[matrix]i", AveLuma, 0);
return "`AveLuma' AveLuma plugin";
}


and test script

AviSource("D:\avs\avi\1.avi")
#Return AveLuma() # Show "AveLuma: 'current_frame' only available in runtime scripts"
V1=ScriptClip(""" Subtitle(String(AveLuma),Align=1) """ ) # Default current_frame
#V1=ScriptClip(""" Subtitle(String(AveLuma(current_frame)),Align=1) """ ) # current_frame explicitly supplied
#V1=ScriptClip(""" Subtitle(String(AveLuma(current_frame-100)),Align=1) """ ) # current_frame relative

V2=ShowChannels() # for comparison
StackVertical(V1,V2)

Takes an optional frame number (else current_frame).
Works just fine.



Another test script: Demos in YV12, YUY2, Y8, RGB24, RGB32, YV16, YV24 etc
(RGB mode, converts and returns RGB to YUV Luma)
Added Matrix arg
The result (Subtitled Average Accumulated Luma) is valid on the final frame.

AviSource("D:\avs\avi\1.avi").trim(0,0999)
# Set to ConvertToRGB matrix
COL = 0 MATRIX = "rec601" # Default 0=REC601
#COL = 1 MATRIX = "rec709"
#COL = 2 MATRIX = "pc601"
#COL = 3 MATRIX = "pc709"

#ConvertToYV12()
#ConvertToYUY2()
#ConvertToY8()
#ConvertToYV16()
#ConvertToYV24()
#ConvertToRGB24(matrix=MATRIX)
#ConvertToRGB32(matrix=MATRIX)
Sum = 0.0
SumHi=0.0 # Hi part Extra precision (need for long scans)
SumLo=0.0 # Lo part Extra precision
GScript("""
for(n=0,FrameCount()-1) {
# Sum = Sum + AveLuma(n,matrix=COL) # Were gonna use extra precision instead
SumLo = SumLo + AveLuma(n,matrix=COL)
SumHi = SumHi + floor(SumLo / 256.0) # Hi part, Extra Precision
SumLo = frac(SumLo / 256.0) * 256.0 # Lo part, Extra Precision
}
""")
# Sum=Sum/Float(FrameCount()) # Were gonna use extra precision instead
Sum = (SumHi / Float(FrameCount()) * 256.0) + (SumLo / Float(FrameCount()))
ConvertToYV12(matrix=MATRIX) # For Display and Showchannels info
s=showchannels() # Show eg Ave Luma for frame and Accumulated Ave luma for all frames visited
s.Subtitle(String(Sum),Align=1)


Found this thread by Gavino and added the GetVar helper function.
http://forum.doom9.org/showthread.php?p=1394689&highlight=current_frame#post1394689

EDIT: Added, YUY2, RGB and v2.6 planar modes. Renamed to AveLuma
EDIT: Added matrix arg: 0=rec601 : 1=rec709 : 2=pc601 : 3=pc709
EDIT: Minor mods
EDIT: Accumulator now 64 bit.

AveLuma(clip , int "n", int "matrix") : n defaults to current_frame : matrix defaults=0=rec601

EDIT: About 10% slower than AverageLuma (in above GScript test) as no SSE, BUT, supports optional frame number
(or eg current_frame-2), YUY2 and RGB with conversion to Luma.

EDIT: ShowChannels() Available via MediaFire in sig

EDIT: Seems that this thread subject is/was almost identical to example from Avisynth Docs:

http://avisynth.org/mediawiki/Filter_SDK/Non-clip_sample

[jmac698]

That just came in handy, thanks :)

How would you reset

const BYTE *srcp = src->GetReadPtr(PLANAR_Y);

Since you've added a bunch of srcp += pitch; to it?

[StainlessS]

If I understand properly, a simple

srcp -= (pitch * height);

Should do it, or is that too obvious.
(You want to process the same frame again ?)

EDIT: If you want to process say n to n+3,
then something like

unsigned long sum_store[4];
frm = n; // rem starting frame
for (z=0 ; z< 4;++z) {
n=frm+z;
n = (n<0) ? 0 : (n>=vi.num_frames)?vi.num_frames-1:n; // Range limit frame n
PVideoFrame src = child->GetFrame(n,env);
const int pitch = src->GetPitch(PLANAR_Y);
const int rowsize = src->GetRowSize(PLANAR_Y);
const int height = src->GetHeight(PLANAR_Y);
const BYTE *srcp = src->GetReadPtr(PLANAR_Y);
const float Pixels = (float)(height * rowsize);
unsigned long sum = 0;
int h,w;
for(h=0; h<height; ++h) {
for (w=0; w<rowsize; ++w) {
sum += srcp[w];
}
srcp += pitch;
}
sum_store[z] = sum;
}


EDIT: Depending upon what you're trying to do, WriteFile might come in handy to log eg
AverageLuma results to a file for every frame, look at MatchFrames, although there are probably
much shorter examples, matchframes would require a fair amount of chopping out of unnecessary
stuff. Tell you what take a peek at the Gavino FindFrames linked by Matchframes., and use WriteFile
instead of WriteFileIf variation (if that's what it uses). Also see ConditionalReader (which I have not
yet used).

[Wilbert]

That just came in handy, thanks :)

How would you reset

const BYTE *srcp = src->GetReadPtr(PLANAR_Y);

Since you've added a bunch of srcp += pitch; to it?

As StainlessS said, or simply


const BYTE *srcp = src->GetReadPtr(PLANAR_Y);
...
srcp = src->GetReadPtr(PLANAR_Y);


Thx StainlessS.

[StainlessS]

One asterisk too many Wilbert.

[StainlessS]

@Anybody interested (I'm sure there are those that know already).

I just tried the above filter but using an 'n' frame number argument, in a GScript script,
I was expecting some kind of problem that did not appear, here:-


AviSource("D:\avs\avi\1.avi").trim(0,999)
sum=0.0
GScript("""
for(n=0,FrameCount()-1) {
sum=sum+AveLumatest(n)
}
""")
sum=sum/Float(FrameCount())
s=showchannels()
s.Subtitle(String(sum),Align=1)


Works just fine. Could be used to prescan some sample area without two passes, eg to detect
autocropping, field order, interlacing, pulldown etc, and then process clip proper, all in single pass
could be incorporated into an automated rendering system, this is just what I was looking for.

It's only the missing 'current_frame' that prevents runtime functions from working properley
outside of the runtime environment, if you supply your own frame number (as the above plug allows),
then GScript supplies the magic dust.

Runtime filters, running either at runtime or during compile time. :)
EDIT: Although, if you assign values to current_frame directly, that also works
for built in runtime filters (I think, cant remember where I've used it though, Oh, of course, MatchFrames).
OK, it's NOT as wonderful as I first thought, being able to supply a frame arg is pretty much the same as
synthesizing the missing current_frame, wish I understood the compile time run time stuff properly,
but at least the automated prescan stuff still holds true.


EDIT: The result (Accumulated Average Luma) is valid on the final frame.

[jmac698]

Not to worry, it was only a calculation over pixels which required two passes. I did multiple frames in taverage. Which reminds me, how did I solve it there?

I think I'm not clear on pointers. It makes perfect sense that a variable has an address in memory that it's stored, and that I can manipulate this address as a form of indexing.
When you define

const BYTE *srcp = src->GetReadPtr(PLANAR_Y);

It means this is a pointer, the * means you're setting the address of it, src-> should mean a reference to an element of a structure, which is a collection of possibly different types of variables. GetReadPtr() should be a function. I don't quite get how a function is part of a structure, though I can understand having a pointer to one.
I also don't get how a pointer can be a constant, wouldn't that mean you can't increment it?
The byte must indicate the way the pointer increments; if it were int (and int were 16bit), then adding 1 to the pointer would really add 2.
So adding pitch to the pointer is going to the next row (which includes some padding for alignment).
srcp[0]=n would place n at the current address, where n would be cast to BYTE.
Is this right?

My problem is, if I reset the pointer do srcp=0 is that setting the address to 0, or is it setting the memory pointed to to 0? If I set *scrp=0 that should mean address 0 which is probably going to crash. If I do const BYTE *scrp=src->Get... again, it will say I redefined a variable. If I set *scrp=src->GET.. it gave me another error.
And strangely enough, in my latest code I didn't reset it at all and it still worked, but I don't know why.

[StainlessS]

srcp is a pointer (*) to type "const BYTE", meaning that what it points to cannot be altered (const), ie read only.
It's just allowing the compiler to do checking to enforce your compliance with the wishes of the developer that
implemented GetReadPtr.
src->GetReadPtr(PLANAR_Y); You are calling the GetReadPtr member function of class PVideoFrame via src.
You access the member functions just like you access struct members. I'm a bit rusty on this but in C, I think you can
also call a function pointer in a struct using the same pointer teminology, eg src->GetReadPtr(PLANAR_Y).
A class can have both member properties (data) and member functions (like constructor, GetFrame etc) and you can
call those member functions via a pointer to an instance of a class, and those member functions can access the
properties (data) for the particular instance of the class on which it was called (really clever stuff). For instance,
you could have two copies of the same filter (class, like a structure with member functions), if you call eg
GetFrame on a particular instance of the filter, there is no confusion about which data members belong to it, they
can be treated like local variable. In the CPP for proggers thing, each instance of the Example class would have
it's very own 'LongLife' property and when Avisynth calls the filter via f->GetFrame(n), GetFrame knows which LongLife
belongs to filter 'f'.
(It's all very clever, with a lot of behind the scenes stuff going on and a secret pointer called a 'this' pointer, which you
dont really need to know anything about, but the 'this' pointer points at the particular instance of a class data, that is
the sort of secret way that member functions know which class data members are local to them [if that makes sense]).

The byte must indicate the way the pointer increments; if it were int (and int were 16bit), then adding 1 to the pointer would really add 2.
So adding pitch to the pointer is going to the next row (which includes some padding for alignment).
srcp[0]=n would place n at the current address, where n would be cast to BYTE.
Is this right?


Yep.

My problem is, if I reset the pointer do srcp=0 is that setting the address to 0, or is it setting the memo"ry pointed to to 0? If I set *scrp=0 that should mean address 0 which is probably going to crash. If I do const BYTE *scrp=src->Get... again, it will say I redefined a variable. If I set *scrp=src->GET.. it gave me another error.
And strangely enough, in my latest code I didn't reset it at all and it still worked, but I don't know why. "

srcp=0; is setting the address to 0 // Probably bad
*srcp = 0; is zeroing what p points at. // Hopefully OK
EDIT: *srcp=0; is same as srcp[0]=0;

As Wilbert said,

const BYTE *srcp = src->GetReadPtr(PLANAR_Y); Can do this the first time
...
srcp = src->GetReadPtr(PLANAR_Y); do this thereafter

The *srcp= src->Get etc was a typo, Wilbert fixed it.

"And strangely enough, in my latest code I didn't reset it at all and it still worked, but I don't know why. "

Your god looks after you. :)

EDIT: By the way you can check out the Avisynth.h header to see some of the Avisynth classes etc.

[jmac698]

I know, since this was planar I must have been reading into one of the chroma planes, in which case I have a bug.

[StainlessS]

Jmac, forgot what it was that you were trying to do, (as posting over multiple threads).
Firstly, now that you've changed operation, (ie now using a sum array in taverage [I assume thats what we're talking about])

You need not a 'const BYTE * srcp =GetReadPtr(PLANAR_Y)' but a BYTE * srcp =GetWritePtr(PLANAR_Y);'
as although you only read on first pass (summing), on second pass (to update the frame itself)
you will write to it, so you might as well get a write pointer from the beginning, and
dont use 'srcp =GetWritePtr(PLANAR_Y)' for the second pass, use the "srcp -= (pitch * height);"
for as we found out, excessive calls to GetWritePtr can cause problems, so why tempt fate.
You would also need to reset sum ie sum = LifeLong (if thats what you used, I have not seen the
taverage source).

EDIT: Seems that this thread subject is almost identical to example from Avisynth Docs:
http://avisynth.org/mediawiki/Filter_SDK/Non-clip_sample
Dont know if thread should be closed. Perhaps does have merit due to example implementation
of Gavinio's suggested "exception protected wrapper", and direct use of optional frame arg.

[Wilbert]

dont use 'srcp =GetWritePtr(PLANAR_Y)' for the second pass, use the "srcp -= (pitch * height);"
for as we found out, excessive calls to GetWritePtr can cause problems, so why tempt fate.
Use MakeWritable before writing the second time: http://avisynth.org/mediawiki/Cplusplus_API#MakeWritable. It's better to use that when writing source frames.

[jmac698]

Umm.. this was for my correlation plugin, which is now released. It needed a two pass calculation on each frame. I fixed the bug of reading into the chroma.

[StainlessS]

Added, YUY2, RGB and v2.6 planar modes. See 1st post.

[StainlessS]

Added matrix arg for RGB -> YUV_Y modes. 0=rec601:1=rec709:2=pc601:3=pc709
See 1st post.

EDIT: If no problems reported, will publish in Usage Forum.

[StainlessS]

Minor Mods. See 1st post.

EDIT: If no problems reported, will publish in Usage Forum.

[TheFluff]

Why on earth have you implemented your own YUV-to-RGB conversion in an Avisynth plugin? Try something like this:
const char *argnames[2] = {0, "matrix"};
AVSValue args[2] = {child, "Rec.601"};
// might want to handle potential exceptions here, but who cares
PVideoFrame yv12_frame = env->Invoke("converttoyv12", AVSValue(args, 2), argnames).AsClip()->GetFrame(n, env);

See http://avisynth.org/mediawiki/Filter_SDK/Env_Invoke for more details.

[jmac698]

Great! We've both been wondering how to call internal avisynth functions.
However, I personally have a use for my own function, as I want to make all my plugins deepcolor compatible, so I want 16 bit colors.

[TheFluff]

Great! We've both been wondering how to call internal avisynth functions.

you can invoke any function (internal, script, plugin) present in the current scope
you can even invoke loadplugin or import to load more functions, if you want to
you can also, like, read the filtersdk documentation if you're wondering about something related to filter development, it's actually pretty good

However, I personally have a use for my own function, as I want to make all my plugins deepcolor compatible, so I want 16 bit colors.

His implementation assumes 8-bit precision, so I don't see how it helps, really. 16-bit YUV-to-RGB is also already implemented in dithertools, so you can either just invoke that or copy the implementation.



unsigned long sum = 0;

Slightly academic, but this can overflow with very large frames (slightly bigger than 4096x4096). If you want a 64-bit integer with MSVC in 32-bit mode use __int64 (or #include <stdint.h> and use int64_t). sizeof(long) is 4 in pretty much all 32-bit compilers.

[StainlessS]

Slightly academic, but this can overflow with very large frames

I'll start worrying about that when Avisynth does. (4096x4096 pixels @ 255 would overflow by 1, I can live with that)


unsigned int accum = 0;


EDIT: Actually NO, 4096x4096 * 255 = 0xFF000000, ie does NOT overflow,* 256 would by 1, or as you say, bigger than 4096x4096.

EDIT: I'll look at the "YUV-to-RGB conversion", thanks,
but dont look hopeful as would require invoke on every frame (runtime filters dont have constructors), from the link you posted:
In general, avoid invoking "Invoke" as much as possible. If it can't be avoided, try not to do it every frame, but do it in the constructor, if the filter parameters doesn't change. Invoking on every frame usually brings down the speed of the filter to a fraction of a static filter, as the invoked filter has to be created and destroyed at every frame.
EDIT: The user always has the option of convert to eg Y8 or YV12 before calling the runtime filter, that would be best all round.

[jmac698]

One of my uses, would be for example in drawing test patterns in deepcolor, there's nothing to convert here, I'd have to supply an INT for each color channel. 10bit colorbars are used professionally, even something that simple would require this. I've also proven that color conversions require at least 10bit precision, namely the conversion of yellow. You can tell if yellow was converted properly if it's Y is 162 instead of 161, as it relies on a 10 bit precision. (I mean, rec601 yellow in 75% colorbars).

[StainlessS]

Slightly academic, but this can overflow with very large frames (slightly bigger than 4096x4096). If you want a 64-bit integer with MSVC in 32-bit mode use __int64 (or #include <stdint.h> and use int64_t). sizeof(long) is 4 in pretty much all 32-bit compilers.

OK, you win, implemented 64 bit accumulator.

[TheFluff]

EDIT: I'll look at the "YUV-to-RGB conversion", thanks,
but dont look hopeful as would require invoke on every frame (runtime filters dont have constructors), from the link you posted:

Avisynth supplies you with the user_data pointer for a reason, stick the clip generated by the env->Invoke() call there. Of course, then you run into problems if your function is called multiple times in the same script but with different clips... Solving those problems is left as an exercise to the reader.

[StainlessS]

Avisynth supplies you with the user_data pointer for a reason, stick the clip generated by the env->Invoke() call there. Of course, then you run into problems if your function is called multiple times in the same script but with different clips... Solving those problems is left as an exercise to the reader.


EDIT: The user always has the option of convert to eg Y8 or YV12 before calling the runtime filter, that would be best all round.

Thanks, think I'll stick with that. It is expected (the way I expect to use it) that args would change, having a fallback if user does not convert to a YUV form is not such a bad thing.

Solving those problems is left as an exercise to the reader.

I have not seen any explanation of how the user_data thing works (other than in the font thing using a malloc string,
no idea how it would fare with Avisynth objects and/or whether they would need destroyed).
Also, thanks but I dont need any exercises, quite happy living in ignorance and if you knew how to
solve the "problem", you would no doubt have said so. :)

[TheFluff]

Why do you even want to support RGB input to a function that is only meaningful for YUV images?

I have not seen any explanation of how the user_data thing works (other than in the font thing using a malloc string,
no idea how it would fare with Avisynth objects and/or whether they would need destroyed).

user_data is void* pointer. That means it can point to anything. That means you can create an arbitrary object or c struct or whatever floats your boat, get a pointer to it and set user_data to that pointer. The environment will pass the pointer back to you each time your function is called, thus neatly solving the problem with arbitrary functions that need to keep some kind of state between calls, without having to resort to ugly hacks like global static objects.

In other words: create some kind of object or struct that stores any state you want to save. Do this in AvisynthPluginInit2, and pass a pointer to the object in question as the last argument to env->AddFunction. This pointer will be passed to the function you just registered, as the user_data argument. Do whatever you want with the object there. If you're worried about memory leaks, register an env->AtExit() hook that deletes the object.

The PClip class acts as a reference counted smart pointer; it behaves much like its relatives in the C++ standard library does and has the same semantics. If you're not familiar with how smart pointers work in C++, I really think you should take the time to read up on it.

Also, thanks but I dont need any exercises, quite happy living in ignorance and if you knew how to
solve the "problem", you would no doubt have said so. :)

I didn't explain it because it's a tedious problem to solve, not a hard one. You just need to make your code handle an arbitrary number of clips, not just one, and find some way of identifying which clip you're currently working with.

[StainlessS]

Why do you even want to support RGB input to a function that is only meaningful for YUV images?


Fallback.