rgba_rpn/y8_rpn - RPN pixel manipulation

[wonkey_monkey]

My new filter, rgba_rpn/y8_rpn, allows the arbitrary manipulation of pixel values from multiple clips, and can... well, it can do a lot.

Download: rgba_rpn0.1.zip (http://horman.net/avisynth/downloads/rgba_rpn0.1.zip)

Documentation (http://horman.net/avisynth/rgba_rpn.html)
RPN guide (http://horman.net/avisynth/rpn.html)


Anti-virus software: it's possible that your anti-virus software may find the DLL suspicious, because it generates executable code on the fly. You'll just have to trust me :)

Using strings in reverse Polish notation (https://en.wikipedia.org/wiki/Reverse_Polish_notation), pixel values can have all kinds of mathematical operations performed on them.

It can use YUV as well as RGB clips as input (although only the Y channel of YUV clips is available), and has several different dithering modes (basically: none, ordered, or random).

Please refer to the documentation and RPN guide, which are probably required reading if you want to understand the following examples.

So what can it do?

This is a hard question to answer, because it can do so much.

You can use it to draw an anti-aliased circle:

blankclip.y8_rpn("255 x w 0.5 * - dup * y h 0.5 * - dup * + sqrt 128 - 255 * -")

http://horman.net/avisynth/circle.png


Or a rotating (animated) anti-aliased square:

blankclip.y8_rpn("
x w 0.5 * - @X^ y h 0.5 * - @Y^ 0.01 n * sincos @S^ @C^
X S * Y C * + @A^
X C * Y S * - @B^
255 A abs 150 + B abs 150 + max - 255 *
")


http://horman.net/avisynth/square.png


A more practical example - this call will desaturate any green pixels (those which have a green value greater than both red and blue):

clip.rgba_rpn("[r0] 0.3 * [g0] 0.59 * [b0] 0.11 * + + [c0] [g0] [r0] [b0] max > ?")

http://horman.net/avisynth/sweets.jpg


It animated-wipes (this one complete with gamma awareness and dithering):

a=colorbars
b=version.pointresize(a.width,a.height).trim(0,50)

a=a.reduceby2
b=b.reduceby2

rgba_rpn("64 @W x W + w W + t * - \ 1 zmax pi * cos 1 + 0.5 * @A [c0] dup * * 1 A - [c1] dup * * + sqrt", 1, b, a)


http://horman.net/avisynth/wipe.png


It sharpens (code omitted for space):

http://horman.net/avisynth/sharpen.png


It blurs:


http://horman.net/avisynth/blur.png


It weird blurs:

http://horman.net/avisynth/weirdblur.png



And to really demonstrate the flexibility of rgba_rpn, the following example (with the help of some basic Avisynth functions) implements the w3fdif deinterlacer (discussed recently here (http://forum.doom9.org/showthread.php?t=172469)), which was the basis for kerneldeint:

mpeg2source("test.d2v") # interlaced source
converttorgb24(interlaced=true)

sep = separatefields
odd = sep.selectodd
even = sep.selecteven
x = odd.blankclip(length=1) + odd.trim(0,framecount-2)
y = even.trim(1,0) + even.blankclip(length=1)

woven = interleave(x,y).weave

rgba_rpn("
[c0] [c1] + 0.170 *
[c0(0,-1)] [c0(0,1)] + 0.526 *
[c0(0,-2)] [c0(0,2)] [c1(0,-2)] [c1(0,2)] + + + -0.116 *
[c0(0,-3)] [c0(0,3)] + -0.026 *
[c0(0,-4)] [c0(0,4)] [c1(0,-4)] [c1(0,4)] + + + 0.031 *
+ + + +
", last, woven,0,1)

separatefields
selectevery(2,1,0)
interleave(last,sep)
weave

assumeframebased


http://horman.net/avisynth/w3fdif.jpg


I appreciate these may be hard to understand; using RPN doesn't help when it comes to clarity! I will go through some examples in more depth in subsequent posts, if anybody wants to see them.

Alternatively, I'll be happy to answer questions of the form "How would I..."

[StainlessS]

My word you've done it again, you truly are the Doctor :)

Love the 'dup' rpn shortcut (duplicate top of stack), pity masktools never implemented this [I think it was requested, perhaps by you].

EDIT: 'Swap' too, Perhaps I steal your rpn code one day :)

[wonkey_monkey]

dup wasn't suggested for masktools by me (I've never been a masktools user). It (along with user variables) was suggested by martin53 (http://forum.doom9.org/member.php?u=125347) in the xyremap (http://forum.doom9.org/showthread.php?t=166087) discussion, though.

[Sparktank]

This looks intriguing!

This will get used to recognizing RPN. In the mean time, I definitely look forward to seeing more examples of anything fun.

[luquinhas0021]

What more can I do with this filter? draw antialiased circles and squares doesn't appear useful to me.
Desaturate green pixel sounds good to me.

[wonkey_monkey]

The shapes were just to demonstrate the wide variety of things it can do. There's not much point asking me what you can do with it, because there are too many things you can do with it.

For example, calculating the Mandelbrot set:

blankclip(width=960,height=640).y8_rpn("
0 255

a 3 * 2 - @A @X^
b 2 * 1 - @B @Y^

X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^

X dup * Y dup * +
4 < ?
",-1,2)


http://horman.net/avisynth/mandelbrot.png

The moiré patterning is, I think, due to FPU overflow exceptions, and is also why the frame rate is so low. I'm working on it, but I thought it was interesting enough by itself to leave in on the screenshot. Add more repetitions (NB: more than 29 reps results in an access violation, probably because the code becomes too large for the allocated buffer, or because I haven't been generous enough with one of my arrays - I will look into this) of the repeated line for greater accuracy:

http://horman.net/avisynth/mandelbrot29.png

A filter which can do things as diverse as desaturating green pixels, compositing chroma key clips, and calculating the Mandelbrot set. Hopefully you can see why asking someone else "what can I do with it" is not a very meaningful question in this situation.

[wonkey_monkey]

Animated zoom into the Mandelbrot set (no more moiré, 25fps on a 2.2GHz quad core i7):

a="X dup * Y dup * - A + @T^ X Y 2 * * B + 2 min @Y^ T 2 min @X^ "
b=a+a
c=b+b

blankclip(width=960,height=640,length=1600).y8_rpn("
0 255

a 3 * 2 - -1.2947627 - 1.01 n ^ / -1.2947627 + @A @X^
b 2 * 1 - 0.4399695 - 1.01 n ^ / 0.4399695 + @B @Y^
"+c+c+c+c+c+b+a+"
X dup * Y dup * +
4 < ?
",-1,2)


I definitely need to update the filter with larger internal buffers and arrays.

(this really wasn't what the filter was intended for!)

David

[foxyshadis]

This is pretty cool. It's like having matlab in your avisynth script.

[jmac698]

Very nice! Would've loved this a few years ago. The only thing I'm missing is, a shift operator, and support for stacked 16 bit format built-in. Defining some kind of macros/functions would make it much easier to use, especially as they can be shared as a library and then built upon. Also the rpn converter of masktools could be used. How are virtual pixels handled? How would you average all pixels on a line? In squares? Finally, some kind of "loops" would be very nice, for example n REP to repeat the last string of operations n times. But how to mark the string? Maybe a new kind of variable, and maybe enclose the macro in [] ? Like [* dup] #SQ x #^SQ do a squaring x^2
[2 *] #SHIFT
x #^SHIFT n REP #x<<n shift up n times the x
or better
[2 *] REP #SHIFTn
x n #^SHIFTN #shift x n times left

Then you could have a library and just pre-pend that to your string, and all these new functions are defined for you. I'd write some.

[jmac698]

There's also my Mandelbrot plugin
http://forum.doom9.org/showthread.php?p=1582742#post1582742

Also an 8-bit Mandelbrot with masktools
http://forum.doom9.org/showthread.php?p=1539509#post1539509

I should rewrite it with yours so I can use floats
Yes, I'd be interested in you fixing the bugs to make a nice Mandelbrot possible.

[feisty2]

right, now you can write some simple plugins like removegrain kinda stuff with this, in a script, if you just don't wanna waste ur time to learn the hardass c/cpp

[jmac698]

Btw, a nice trick for a fill function is xor with the last byte, I'm leaving out some details but it's like the Amiga does in hardware, draw some borders and they get filled in horizontally, would be neat to try to write that.

@feisty
yes exactly! That's why I loved masktools and asked for some mods to it. a48 was a lot inspired by my requests. Also I like that it's in script, and easily exposes the algorithm, so nothing is ever lost by being closed source.

[wonkey_monkey]

The only thing I'm missing is, a shift operator,

It's called "multiply" ;) And it might be quicker than implenting a real shift, which would involve a conversion from double to int to double.

and support for stacked 16 bit format built-in.

Possible. Reading pixels would be easier than writing them, I suspect (reading could probably be impemented with a string function; see next answer)

Defining some kind of macros/functions would make it much easier to use, especially as they can be shared as a library and then built upon.

You could do that with Avisynth strings, as I did with my second Mandelbrot post.

How are virtual pixels handled?

What do you mean by "virtual pixel"?

I'm thinking about loops and jumps and forward ifs, but they all need careful consideration. If you conditionally jump from one point to another, you have to ensure (and the compiler would have to enforce) that the stack will be in exactly the same condition (which is not just a matter of how many items are on the stack, but also which are on the FPU stack and which are on the "overflow" stack) at the end of both branches. And that's easier said than done.

[wonkey_monkey]

Here's an example explained in detail. It simulates the "zebra stripes" that some cameras show in the viewfinder to show out-of-range white values.

avisource("...")
levels(0,1,200,0,255,coring=false) # make some white values illegal for testing purposes

striped=last.y8_rpn("
x y + n + 128 * 1024 %
[y0]
dup 255 >
?")

mergechroma(striped.converttoyv12, last)

The first two lines just load the source and adjust the levels to make some of the luminance values illegal (higher than 235).

Next comes the RPN call. First, there's this line:

x y + n + 128 * 1024 %

This builds the stripey, animated background. It adds the pixel x coordinate, the pixel y coordinate, and the frame number, then multiplies the total by 128. It then takes the modulo of this with 1024 to make a repeating pattern of thin black stripes and thicker white stripes.

[y0]

Next, the luma value of the pixel is put on the stack. By default, y8_rpn expands the luma range from 16-235 to 0-255 on read. It doesn't clamp this value, though, so source pixels greater than 235 will leave a value greater than 255 on the stack.

Next:

dup 255 >

This line duplicates the [y0] value, because the comparison with 255 will remove the value from the stack, but we still want it on there. The comparison itself returns 1 if [y0] > 255, otherwise it returns 0.

Next:

?

Based on the previous result (1 or 0), this operator picks either the result of the first line, or the result of the second line (because these are the remaining two items on the stack; it has nothing to do with the placement of the newlines, which are for clarity only).

Finally a call to mergechroma restores the colour to the clip (rgba_rpn/y8_rpn ignore UV channels completely).

By this method, pixels which have an illegal luma value are replaced with the stripey pattern:

http://horman.net/avisynth/zebra.gif

NB: because y8_rpn clamps output values to the legal range, by default, the output will no longer have any out-of-range luma pixels. I'm not sure this is the best way to behave, but it's always been a minefield.

Coming soon: loops and commenting.

[creaothceann]

Wouldn't it be easier to just use the regular expression format? I.e. instead of

x y + n + 128 * 1024 %

this:

((x + y + n) * 128) % 1024

It's easier to write and easier to read...

[wonkey_monkey]

Easier for a human because it's familiar, sure, though I'm not sure it's fundamentally easier. We should all be using RPN! ;) And writing dates in YYYY-mm-dd format...

The reason for using RPN is because it's much, much easier for a computer to parse and therefore to program for.

An infix-to-RPN converter, such as masktools has, would be the answer, but it's not a big priority for me.

[Reel.Deel]

An infix-to-RPN converter, such as masktools has, would be the answer, but it's not a big priority for me.

There's also ConvertToPostfix (http://forum.doom9.org/showthread.php?t=98985&page=3#post704434) by tsp (download binary here (http://www.avisynth.nl/users/tsp/)).

---

I understand the concept behind RPN, the thing I don't understand is how does one come up with the numbers to do what you wanted it to do (I know is not but to me it just seems arbitrary). Is there any reference material that teaches video processing and RPN? I've looked but have not found anything, in this MaskTools guide (http://efmv.org/articles/masktools/) tp7 mentioned that this type processing is called a “point operation” in all image processing books. Is this true for rgba_rpn/y8_rpn also?

[wonkey_monkey]

There's also ConvertToPostfix (http://forum.doom9.org/showpost.php?p=704201&postcount=38) by tsp (download binary here (http://www.avisynth.nl/users/tsp/)).

It seems that's a very early try and tsp improved it later in the thread. It might be a place to start, but rgba_rpn/y8_rpn has a few odd features which might not translate all that easily to infix notation now.

I understand the concept behind RPN, the thing I don't understand is how does one come up with the numbers to do what you wanted it to do (I know is not but to me it just seems arbitrary). Is there any reference material that teaches video processing and RPN? I've looked but have not found anything, in this MaskTools guide (http://efmv.org/articles/masktools/) tp7 mentioned that this type processing is called a “point operation” in all image processing books. Is this true for rgba_rpn/y8_rpn also?

Not really sure I can help there... it's just maths! You've got your variables - x and y pixel position, frame number, current pixel values - and you just make them do what you want them to do.

Give me an example of something you might want to do (even if it's just a silly example) and I can try to take you through how to do it.

[Reel.Deel]

Not really sure I can help there... it's just maths! You've got your variables - x and y pixel position, frame number, current pixel values - and you just make them do what you want them to do.

Give me an example of something you might want to do (even if it's just a silly example) and I can try to take you through how to do it.

"it's just maths!" - that's my biggest problem! :D

I've been playing around with PhotoDemon's distort filters (http://photodemon.org/features/), it would be nice to able to do such things in AviSynth (especially animated). I like the 'Poke' effect, might be more suitable for xyremap though.

[wonkey_monkey]

It depends on the exact effect you want to replicate. xyremap can do a polar remap (first example on that page), but it can't do a zoom blur by itself (and nor can rgba_rpn). Put the two together and you could just about manage it, but it wouldn't be very efficient.

[cretindesalpes]

Looks like a powerful tool but… no source code again?

[StainlessS]

An infix-to-RPN converter, such as masktools has, would be the answer, but it's not a big priority for me.

Shunting-yard algorithm:- https://en.wikipedia.org/wiki/Shunting-yard_algorithm#The_algorithm_in_detail

Here C source removed from current wikipedia page (was there a couple of years back):

#include <string.h>
#include <stdio.h>
#define bool int
#define false 0
#define true 1

// operators
// precedence operators associativity
// 1 ! right to left
// 2 * / % left to right
// 3 + - left to right
// 4 = right to left
int op_preced(const char c)
{
switch(c) {
case '!':
return 4;
case '*': case '/': case '%':
return 3;
case '+': case '-':
return 2;
case '=':
return 1;
}
return 0;
}

bool op_left_assoc(const char c)
{
switch(c) {
// left to right
case '*': case '/': case '%': case '+': case '-':
return true;
// right to left
case '=': case '!':
return false;
}
return false;
}

unsigned int op_arg_count(const char c)
{
switch(c) {
case '*': case '/': case '%': case '+': case '-': case '=':
return 2;
case '!':
return 1;
default:
return c - 'A';
}
return 0;
}

#define is_operator(c) (c == '+' || c == '-' || c == '/' || c == '*' || c == '!' || c == '%' || c == '=')
#define is_function(c) (c >= 'A' && c <= 'Z')
#define is_ident(c) ((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z'))

bool shunting_yard(const char *input, char *output)
{
const char *strpos = input, *strend = input + strlen(input);
char c, *outpos = output;

char stack[32]; // operator stack
unsigned int sl = 0; // stack length
char sc; // used for record stack element

while(strpos < strend) {
// read one token from the input stream
c = *strpos;
if(c != ' ') {
// If the token is a number (identifier), then add it to the output queue.
if(is_ident(c)) {
*outpos = c; ++outpos;
}
// If the token is a function token, then push it onto the stack.
else if(is_function(c)) {
stack[sl] = c;
++sl;
}
// If the token is a function argument separator (e.g., a comma):
else if(c == ',') {
bool pe = false;
while(sl > 0) {
sc = stack[sl - 1];
if(sc == '(') {
pe = true;
break;
}
else {
// Until the token at the top of the stack is a left parenthesis,
// pop operators off the stack onto the output queue.
*outpos = sc;
++outpos;
sl--;
}
}
// If no left parentheses are encountered, either the separator was misplaced
// or parentheses were mismatched.
if(!pe) {
printf("Error: separator or parentheses mismatched\n");
return false;
}
}
// If the token is an operator, op1, then:
else if(is_operator(c)) {
while(sl > 0) {
sc = stack[sl - 1];
// While there is an operator token, o2, at the top of the stack
// op1 is left-associative and its precedence is less than or equal to that of op2,
// or op1 is right-associative and its precedence is less than that of op2,
if(is_operator(sc) &&
((op_left_assoc(c) && (op_preced(c) <= op_preced(sc))) ||
(!op_left_assoc(c) && (op_preced(c) < op_preced(sc))))) {
// Pop o2 off the stack, onto the output queue;
*outpos = sc;
++outpos;
sl--;
}
else {
break;
}
}
// push op1 onto the stack.
stack[sl] = c;
++sl;
}
// If the token is a left parenthesis, then push it onto the stack.
else if(c == '(') {
stack[sl] = c;
++sl;
}
// If the token is a right parenthesis:
else if(c == ')') {
bool pe = false;
// Until the token at the top of the stack is a left parenthesis,
// pop operators off the stack onto the output queue
while(sl > 0) {
sc = stack[sl - 1];
if(sc == '(') {
pe = true;
break;
}
else {
*outpos = sc;
++outpos;
sl--;
}
}
// If the stack runs out without finding a left parenthesis, then there are mismatched parentheses.
if(!pe) {
printf("Error: parentheses mismatched\n");
return false;
}
// Pop the left parenthesis from the stack, but not onto the output queue.
sl--;
// If the token at the top of the stack is a function token, pop it onto the output queue.
if(sl > 0) {
sc = stack[sl - 1];
if(is_function(sc)) {
*outpos = sc;
++outpos;
sl--;
}
}
}
else {
printf("Unknown token %c\n", c);
return false; // Unknown token
}
}
++strpos;
}
// When there are no more tokens to read:
// While there are still operator tokens in the stack:
while(sl > 0) {
sc = stack[sl - 1];
if(sc == '(' || sc == ')') {
printf("Error: parentheses mismatched\n");
return false;
}
*outpos = sc;
++outpos;
--sl;
}
*outpos = 0; // Null terminator
return true;
}

bool execution_order(const char *input) {
printf("order:\n");
const char *strpos = input, *strend = input + strlen(input);
char c, res[4];
unsigned int sl = 0, sc, stack[32], rn = 0;
// While there are input tokens left
while(strpos < strend) {
// Read the next token from input.
c = *strpos;
// If the token is a value or identifier
if(is_ident(c)) {
// Push it onto the stack.
stack[sl] = c;
++sl;
}
// Otherwise, the token is an operator (operator here includes both operators, and functions).
else if(is_operator(c) || is_function(c)) {
sprintf(res, "_%02d", rn);
printf("%s = ", res);
++rn;
// It is known a priori that the operator takes n arguments.
unsigned int nargs = op_arg_count(c);
// If there are fewer than n values on the stack
if(sl < nargs) {
// (Error) The user has not input sufficient values in the expression.
return false;
}
// Else, Pop the top n values from the stack.
// Evaluate the operator, with the values as arguments.
if(is_function(c)) {
printf("%c(", c);
while(nargs > 0){
sc = stack[sl - nargs]; // to remove reverse order of arguments
if(nargs > 1) {
printf("%s, ", &sc);
}
else {
printf("%s)\n", &sc);
}
--nargs;
}
sl-=op_arg_count(c);
}
else {
if(nargs == 1) {
sc = stack[sl - 1];
sl--;
printf("%c %s;\n", c, &sc);
}
else {
sc = stack[sl - 2];
printf("%s %c ", &sc, c);
sc = stack[sl - 1];
sl--;sl--;
printf("%s;\n",&sc);
}
}
// Push the returned results, if any, back onto the stack.
stack[sl] = *(unsigned int*)res;
++sl;
}
++strpos;
}
// If there is only one value in the stack
// That value is the result of the calculation.
if(sl == 1) {
sc = stack[sl - 1];
sl--;
printf("%s is a result\n", &sc);
return true;
}
// If there are more values in the stack
// (Error) The user input has too many values.
return false;
}

int main() {
// functions: A() B(a) C(a, b), D(a, b, c) ...
// identifiers: 0 1 2 3 ... and a b c d e ...
// operators: = - + / * % !
const char *input = "a = D(f - b * c + d, !e, g)";
char output[128];
printf("input: %s\n", input);
if(shunting_yard(input, output)) {
printf("output: %s\n", output);
if(!execution_order(output))
printf("\nInvalid input\n");
}
return 0;
}



EDIT: here several implementations (incl C, VB, Java, Python and many more):-
http://rosettacode.org/wiki/Parsing/Shunting-yard_algorithm

[foxyshadis]

Defining some kind of macros/functions would make it much easier to use, especially as they can be shared as a library and then built upon.

You should be able to just call regular avisynth functions (like "x y + "+mycrazymacro(x,y,12)+" abs") or whatever, as long as it returns a string.

NB: because y8_rpn clamps output values to the legal range, by default, the output will no longer have any out-of-range luma pixels. I'm not sure this is the best way to behave, but it's always been a minefield.

Ouch! Masktools and other manipulation plugins often want 0-255, especially for the mask/alpha. At least there needs to be a way to get unclipped output.

[wonkey_monkey]

rgba_rpn/y8_rpn isn't only for masks, though, so assuming 16-235 seemed the best default for an unknown input (since it seems most Avisynth functions do so). Whichever I picked for the default, it was going to trip someone up...

As for infix, I'm considering a hybrid format:

a ([c0]+10) *

Anything in (...) will get translated from infix to RPN, everything else is still RPN.

I've now got conditional jumps (which also means loops) and "if...then...else" working, and once-per-row and once-per-frame calculations to speed things up, which has lead to this:

http://horman.net/avisynth/colourbrot.png

[wonkey_monkey]

Well, I tried, but infix notation is just too difficult to combine with the syntax I've already implemented.

[martin53]

David,
thanks for this great work again! Maybe you noticed i'm trying over the years to implement a frame restorer for short camera shakes
(the basic idea of which is, to replace a blurred frame by a motion compensated neighbor frame).
Object occlusion and the sometimes not optimum outputs of MVTools2 gave me the hope I can basically prototype this chain:
- Take original clip, motion compensated previous and next frame
- Use motion mask in rgba_rpn to find object occlusions through untrustable motion
- mask original with unoccluded compensated content
-...

I need relative pixel distances [y0(x,y)] for this, more specifically, x and y come from a MMask produced clip.

Now I have difficulties with the rpn formulas
...some clip...
c = c.Cnv2("YV24")
hor = c.MMask(vf, kind=3).Cnv2("YV24") #kind=3 horizontal
ver = c.MMask(vf, kind=4).Cnv2("YV24") #kind=4 vertical
y8_rpn(c, hor, ver, "[y1] 128 - @X^ [y2] 128 - @Y^ [y0(X,Y)]", -1, -1, 1)

(Cnv2("YV24") is the ConvertTo shorthand I published in the forum, btw y8_rpn threw an an error with YV12 format)
or simpler
y8_rpn(c, hor, ver, "[y0([y1] 128 -,[y2] 128 -)]", -1, -1, 1)

It says
rgba_rpn: image reference parse error

Are relative pixel coordinates currently limited to constants, and do you think you'd be able to allow computed relative pixel coordinates, too?

[wonkey_monkey]

Are relative pixel coordinates currently limited to constants

Yup, fraid so.

and do you think you'd be able to allow computed relative pixel coordinates, too?

Possibly. I did consider it for a while, but it's not a simple fix. The code is pretty tightly optimised for constants, so it'd have to be rewritten for the new case (and would probably have a format something like "... [y1] 128 - [y2] 128 - [y0<] ..." or something weird like that.

... -1, -1, 1)

I've have thought you'd want the last parameter to be 0, otherwise it will expand all pixel values (including the mmask data) from 16-235 to 0-255. No?

I'll probably turn those numerical parameters into a parsed string instead, in a new version.

[martin53]

I've have thought you'd want the last parameter to be 0, otherwise it will expand all pixel values (including the mmask data) from 16-235 to 0-255. No?
Ah, yes, I think I wanted to use 0 - this setting was the reason I entered the other values at all :D

[martin53]

a format something like "... [y1] 128 - [y2] 128 - [y0<] ..."
i.e. the two top entries from the rpn stack? why not.

[pbristow]

Wow! I've been away too long... This looks like an awesome tool. I knew that time spent at Uni learning RPN and Forth wasn't going to be wasted. (See https://en.wikipedia.org/wiki/Forth_(programming_language) )

And you not only referenced my w3fdif thread, you used 'Chunky' Gilmore for the demo. I'm touched. =;o}

I note you're using tricks like separating and re-interleaving fields to do the temporal stuff, there. How hard would it be to add a z (or f, or t) coordinate representing relative time (frame number); i.e 0 meaning "now", -1 meaning one frame ago, etc. This would enable more direct/intuitive implementation of things like w3fdif. You'd maybe have to limit it to two or three frames forward or back, to avoid buffering loads of extra frames, but that would be enough to open up a whole lot of experiments in denoising, de-interlacing, customised motion-detection...

Oh, and a little blob of jam on it, too, please. =;o}

[pbristow]

OK, reading through the (excellent!) documentation, I see you've already got the "f" for frame number and "t" for fractional time within clip (interesting choice!), so those can be used in the RPN expressions.

I think what I'm really looking for is an extension of the "Offset Pixels" syntax - i.e.:


Offset pixels

Pixels at fixed offsets from the current pixel may also be referred to using the following notiation:

[{channel letter}{image number}(x,y)]

For example, to refer to the red channel of the pixel 10 to the left and 3 below the current pixel, in the third clip:

[r2(-10,3)]

... to allow [r2(-10,3,-1)] and [r2(-10,3,+1)] to refer to the previous and next values of that pixel.

[wonkey_monkey]

I note you're using tricks like separating and re-interleaving fields to do the temporal stuff, there. How hard would it be to add a z (or f, or t) coordinate representing relative time (frame number); i.e 0 meaning "now", -1 meaning one frame ago, etc.

The w3fdif example is a bit complex because the odd/even rows in the target image need, respectively, pixels the next/previous frame to do their work, so separating and weaving was an efficient way to put them both in the same frame, and have the same code work for both even and odd rows. In the general case you would just trim or pad a clip to offset the frames, and I think this is going to remain the way to do it.

Temporal blur (untested):


present={source clip}
past=blankclip(present,length=1)+present
future=present.trim(1,0)

rgba_rpn("[c0] dup + [c1] [c2] + + 0.25 *",present,past,future)


Translating all the bounds checking into machine code, and having to prepare all the clip pointers beforehand, would be complicated and I'd probably have to place too many artificial limitations on it.

[pbristow]

OK, so you can supply multiple clips representing the different points in time... Can you apply different calculations to each clip, and then cross-refer the results in the target calculation?

For example, how would you compute, the difference between the target pixel one frame ago and the same pixel one frame ahead, and then use that in calculating (a) how much vertical blur to apply, (b) how much temporal blur to apply?

Maybe I'll just have to knuckle down and develop a translator that converts/"compiles" my co-ordinate-based way of thinking into the necessary combination of Dups/Trims/Interleaves/Weaves etc. =:o}

One approach that occurred to me was to use a stacked format for past, present & future, and then use a vertical offset to reference the required moment in time (n), i.e.:
offset = ( (n-k) * original_height )
where k is the maximum temporal radius, and the total number of moments stacked in the frame is 2k + 1 .

[pbristow]

For example, how would you compute, the difference between the target pixel one frame ago and the same pixel one frame ahead, and then use that in calculating (a) how much vertical blur to apply, (b) how much temporal blur to apply?




I'm being thick, aren't I? That's just


c[0] dup c[1] - swap c[2] -


...to get the forward and backward differences.

OK, as you were, David. I'll see what I can do with it as it stands. ANd thanks again. =:o}

[pbristow]

I seem to be not getting the basics right, so until I do I've boiled my RPN string down to a no-op, thus:


function PB_NOOP(source)
{
return rgba_rpn( "c[0]", 0, 0, 0, source )
}


This gives me "image reference parse error" (the same as I was getting before with a more adventurous script). Any ideas?

Here's the testbed script that's calling it:


Import("Z_PB_de-int.avsi")

Name = "Doctor Who - S07E03 Extra - The Making of the Gunslinger.avi"

AviSource(name) # (Frame based, 25 fps)
ConvertToRGB24
SeparateFields # 50 fps, half-height

Evens = SelectEven() # 25 fps, half-height
Odds = SelectOdd # 25 fps, half-height
NewEvens = Evens.SelectEven # 12.5fps, half-height
NewOdds = Odds.SelectOdd # 12.5fps, half-height
Interleave(NewEvens,NewOdds) # 25fps, half-height
Weave # 12.5fps interlaced.

#return Last.Bob
return Last.Bob.PB_NOOP()

[wonkey_monkey]

This gives me "image reference parse error" (the same as I was getting before with a more adventurous script). Any ideas?

It's [c0], not c[0]. The parser picks out the [...] references first so the error it gives you is perhaps not as helpful as it should be.

For example, how would you compute, the difference between the target pixel one frame ago and the same pixel one frame ahead, and then use that in calculating (a) how much vertical blur to apply, (b) how much temporal blur to apply?

I would use the difference to calculate the value of each of a fixed number of coefficients, then multiply a fixed number of pixel references by these coefficients. If the difference means the blur is to be small, the outer coefficients would need to calculate to 0.

That's probably going to get pretty complicated pretty quickly, but it's the only way to do it. The next version of rgba_rpn (if I get it working well enough) will allow conditional jumps which might make it slightly easier, thought I'm not sure exactly how.

[SSH4]

May be possible add statistical function like median, std, range, min, max with radius parameter?
*_rpn looks more flexible than masktools but without this feature can't beat masktools.

[wonkey_monkey]

May be possible add statistical function like median, std, range, min, max with radius parameter?

Possible, but extremely difficult to implement in machine code.

*_rpn looks more flexible than masktools but without this feature can't beat masktools.

It isn't trying to!

[pbristow]

It's [c0], not c[0].

D'OH!!! =:o} Thankyou, will try again... Then maybe you'll get to see my *actual* first-attempt at doing something with it. =:o}


[EDIT: That seems like an oddly coder-specific form of dyslexia... =:o\ ]

[pbristow]

OK, now baffled again.


function PB_NOOP(source)
{
return rgba_rpn( "[c0]", 0, 0, 0, source )
}


...works, but:




function PB_Deint_attempt (source)
{
bobbed = source.Bob # 25fps, Missing lines are vertically interpolated
woven = source.DoubleWeave # 25fps, Missing lines are copied from previous field
#return bobbed
#return woven

return rgba_rpn( "[c0]", 0, 0, 0, bobbed )

}


...doesn't. Just gives a black screen.

Same calling script as before, except I no longer have "ConvertToRGB24" in there (realised it wasn't necessary). Uncommenting the "return bobbed" line gives the expected output, but passing the same clip through rgpa_rpn, using the same no-op instruction that works fine with a different clip... doesn't.

Are there issues with whether the clip is frame-based or field based?

I should probably start a new thread over in usage, shouldn't I? =:o\

[wonkey_monkey]

Same calling script as before, except I no longer have "ConvertToRGB24" in there (realised it wasn't necessary).

The fact that it now doesn't work should tell you that it is necessary ;)

rgba_rpn returns an RGB (or RGBA) clip. When you refer to "[c0]" that's shorthand for calling for the pixel from the source clip matching the currently-being-processed channel of the output clip (R, G, B, or A). If your input clip doesn't have any of those channels - which your source clip almost certainly doesn't - then you get black.

Try "[y0]" and you should get a grayscale version of your expected result, which is also what you'd get it if you called y8_rpn with [c0] (since in this case both the source and output clips would have a y channel). rgba_rpn/y8_rpn can't process YUV chroma at all - too complicated, though I might add it for YV24 clips - so you do want to convert to RGB24 after all.

[pbristow]

Dagnabbit, you were right. And your explanation is exactly why I put it in the first place. Could've sworn I'd tested taking it out properly; must have had another one left around in the code at the time.

Anyway, big progress now made, thankyou. For prototyping I've started with "traditional techniques" (lots of ugly calls to Overlay, Levels, etc.), then I'm replacing those one-by-one with equivalent rgba_rpn calls, all with nice short easily debuggable RPN strings. So far, performance hasn't gone up or down much, which is unsurprising as the filter calls themselves are probably the bulk of the load at present... But as I gain confidence, I can start combining them into a fewer calls, for optimisation. That'll be interesting. =:o}

[Sparktank]

Will there be a x64 port of this in the future?

[wonkey_monkey]

I've been rewriting it a lot over the last few months, but yes, definitely - in fact it's x64 I'm mainly working on, x86 has been broken for a while and I'll need to look at working out why at some point. It's much faster, has new expressions, and a lot of new tricks either already added or planned, like temporal offsets, support for more colour spaces and depths, integer support, named variables, etc, etc...

[Sparktank]

Awesome! Can't wait to play with it.

[Didée]

Sorry for digging out such an old thread, but hey ...

There you've been hardly away for 15 years, and you've missed the best parts. What a mighty tool is this!?!?

However I am overwhelmed by the complexity of the RPN, and its special characters, and how-to-address-what... I know MaskTools' RPN nomatter sleeping or awake, that's quite easy, but this one is much more difficult!

Simple question: can I do the following: Have a base clip A, have a (mask 0-255) clip B, then make spatial distortion of A in accordance to B's values.

Base - mask - result: (here: just horizontal shift, mask<128 > one direction, mask>128 > the other)

https://thumbs4.imagebam.com/39/0e/3d/MEZJ6A2_t.png (https://www.imagebam.com/view/MEZJ6A2) https://thumbs4.imagebam.com/18/8a/64/MEZJ6A3_t.png (https://www.imagebam.com/view/MEZJ6A3) https://thumbs4.imagebam.com/08/60/2d/MEZJ6A4_t.png (https://www.imagebam.com/view/MEZJ6A4)

In fact I'm trying to do some sort of clip manipulation with manual motion tracking, which needs spatial distortion of image parts to fit things together.
I tried something with your warp/quad plugin, which at first seemed to be of good use. But that filter is not nice when its in a long Animate()-chain over >1000 frames ... with warp/quad I can only process ~100 frames within an Animate()/Animate()/()/Animate()/... sequence until it has eaten all of my memory and the PC explodes.

[wonkey_monkey]

This filter doesn't do remapping. xyremap (https://forum.doom9.org/showthread.php?t=166087) does, but it can't read pixel values. I've rewritten both to be more capable over the years - but too "beta" to bother releasing - but still with no crossover between the two, unfortunately.

Warp has the facility to map a particular set of control points to a certain frame number, and then to transition between several such maps, if that's of any use. Have you watched the tutorial video (https://www.youtube.com/watch?v=SG1eex7HmLM)?

I also have a rough Avisynth GUI for playing around with warp control points, among other things.

I've also been working on a motion tracking/remapping plugin which can take input from files the GUI can create. This can faciliate remapping based on manually selected/tracked points over time.

If you can share about what your project entails, some of these things may actually be of some use to you!

[StainlessS]

@Didée,
See this post with some seriously Wonkey code (pixel_rpn, and xyremap) :- https://forum.doom9.org/showthread.php?p=1792366#post1792366
Converts this source clip:- https://www.youtube.com/watch?v=dPPjUtiAGYs
To this Wonkey clip:- https://www.youtube.com/watch?v=ZkkUNFXaYyk

Outstanding !

EDIT: Attachments can take some long time now, maybe post on file host and insert links into post.
Postimage instructions:- https://forum.doom9.org/showthread.php?p=1959414&highlight=postimage#post1959414