Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th September 2015, 22:22   #1  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
rgba_rpn/y8_rpn - RPN pixel manipulation

My new filter, rgba_rpn/y8_rpn, allows the arbitrary manipulation of pixel values from multiple clips, and can... well, it can do a lot.

Download: rgba_rpn0.1.zip
Anti-virus software: it's possible that your anti-virus software may find the DLL suspicious, because it generates executable code on the fly. You'll just have to trust me

Using strings in reverse Polish notation, pixel values can have all kinds of mathematical operations performed on them.

It can use YUV as well as RGB clips as input (although only the Y channel of YUV clips is available), and has several different dithering modes (basically: none, ordered, or random).

Please refer to the documentation and RPN guide, which are probably required reading if you want to understand the following examples.

So what can it do?

This is a hard question to answer, because it can do so much.

You can use it to draw an anti-aliased circle:

Code:
blankclip.y8_rpn("255 x w 0.5 * - dup * y h 0.5 * - dup * + sqrt 128 - 255 * -")


Or a rotating (animated) anti-aliased square:

Code:
blankclip.y8_rpn("
x w 0.5 * - @X^ y h 0.5 * - @Y^ 0.01 n * sincos @S^ @C^
X S * Y C * + @A^
X C * Y S * - @B^
255 A abs 150 + B abs 150 + max - 255 *
")


A more practical example - this call will desaturate any green pixels (those which have a green value greater than both red and blue):

Code:
clip.rgba_rpn("[r0] 0.3 * [g0] 0.59 * [b0] 0.11 * + + [c0] [g0] [r0] [b0] max > ?")


It animated-wipes (this one complete with gamma awareness and dithering):

Code:
a=colorbars
b=version.pointresize(a.width,a.height).trim(0,50)

a=a.reduceby2
b=b.reduceby2

rgba_rpn("64 @W x W + w W + t * - \ 1 zmax pi * cos 1 + 0.5 * @A [c0] dup * * 1 A - [c1] dup * * + sqrt", 1, b, a)


It sharpens (code omitted for space):



It blurs:




It weird blurs:




And to really demonstrate the flexibility of rgba_rpn, the following example (with the help of some basic Avisynth functions) implements the w3fdif deinterlacer (discussed recently here), which was the basis for kerneldeint:

Code:
mpeg2source("test.d2v") # interlaced source
converttorgb24(interlaced=true)

sep  = separatefields
odd  = sep.selectodd
even = sep.selecteven
x = odd.blankclip(length=1) + odd.trim(0,framecount-2)
y = even.trim(1,0) + even.blankclip(length=1)

woven = interleave(x,y).weave

rgba_rpn("
[c0] [c1] + 0.170 *
[c0(0,-1)] [c0(0,1)] + 0.526 *
[c0(0,-2)] [c0(0,2)] [c1(0,-2)] [c1(0,2)] + + + -0.116 *
[c0(0,-3)] [c0(0,3)] + -0.026 *
[c0(0,-4)] [c0(0,4)] [c1(0,-4)] [c1(0,4)] + + + 0.031 *
+ + + + 
", last, woven,0,1)

separatefields
selectevery(2,1,0)
interleave(last,sep)
weave

assumeframebased


I appreciate these may be hard to understand; using RPN doesn't help when it comes to clarity! I will go through some examples in more depth in subsequent posts, if anybody wants to see them.

Alternatively, I'll be happy to answer questions of the form "How would I..."
__________________
My AviSynth filters / I'm the Doctor

Last edited by davidhorman; 12th September 2015 at 22:40.
davidhorman is offline   Reply With Quote
Old 12th September 2015, 22:42   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,656
My word you've done it again, you truly are the Doctor

Love the 'dup' rpn shortcut (duplicate top of stack), pity masktools never implemented this [I think it was requested, perhaps by you].

EDIT: 'Swap' too, Perhaps I steal your rpn code one day
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 12th September 2015 at 22:46.
StainlessS is offline   Reply With Quote
Old 12th September 2015, 22:52   #3  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
dup wasn't suggested for masktools by me (I've never been a masktools user). It (along with user variables) was suggested by martin53 in the xyremap discussion, though.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 12th September 2015, 23:00   #4  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 823
This looks intriguing!

This will get used to recognizing RPN. In the mean time, I definitely look forward to seeing more examples of anything fun.
__________________
Win10 (x64) | GPU Caps Viewer v1.38.2.1
Crucial M500 240GB SSD | Kingston SSDNow V300 (Marvell) 120GB | NVIDIA GeForce GTX 750 Ti 2GB | 391.01 (2-23-2018)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Old 12th September 2015, 23:31   #5  |  Link
luquinhas0021
The image enthusyast
 
Join Date: Mar 2015
Location: Brazil
Posts: 267
What more can I do with this filter? draw antialiased circles and squares doesn't appear useful to me.
Desaturate green pixel sounds good to me.
__________________
Searching for great solutions
luquinhas0021 is offline   Reply With Quote
Old 13th September 2015, 00:32   #6  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
The shapes were just to demonstrate the wide variety of things it can do. There's not much point asking me what you can do with it, because there are too many things you can do with it.

For example, calculating the Mandelbrot set:

Code:
blankclip(width=960,height=640).y8_rpn("
0 255

a 3 * 2 - @A @X^
b 2 * 1 - @B @Y^

X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^
X dup * Y dup * - A + @T^ X Y 2 * * B + @Y^ T @X^

X dup * Y dup * +
4 < ?
",-1,2)

The moiré patterning is, I think, due to FPU overflow exceptions, and is also why the frame rate is so low. I'm working on it, but I thought it was interesting enough by itself to leave in on the screenshot. Add more repetitions (NB: more than 29 reps results in an access violation, probably because the code becomes too large for the allocated buffer, or because I haven't been generous enough with one of my arrays - I will look into this) of the repeated line for greater accuracy:


A filter which can do things as diverse as desaturating green pixels, compositing chroma key clips, and calculating the Mandelbrot set. Hopefully you can see why asking someone else "what can I do with it" is not a very meaningful question in this situation.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 13th September 2015, 01:02   #7  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
Animated zoom into the Mandelbrot set (no more moiré, 25fps on a 2.2GHz quad core i7):

Code:
a="X dup * Y dup * - A + @T^ X Y 2 * * B + 2 min @Y^ T 2 min @X^ "
b=a+a
c=b+b

blankclip(width=960,height=640,length=1600).y8_rpn("
0 255

a 3 * 2 - -1.2947627  - 1.01 n ^ / -1.2947627 + @A @X^
b 2 * 1 - 0.4399695  - 1.01 n ^ / 0.4399695  + @B @Y^
"+c+c+c+c+c+b+a+"
X dup * Y dup * +
4 < ?
",-1,2)
I definitely need to update the filter with larger internal buffers and arrays.

(this really wasn't what the filter was intended for!)

David
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 13th September 2015, 08:19   #8  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,314
This is pretty cool. It's like having matlab in your avisynth script.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 13th September 2015, 10:44   #9  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,860
Very nice! Would've loved this a few years ago. The only thing I'm missing is, a shift operator, and support for stacked 16 bit format built-in. Defining some kind of macros/functions would make it much easier to use, especially as they can be shared as a library and then built upon. Also the rpn converter of masktools could be used. How are virtual pixels handled? How would you average all pixels on a line? In squares? Finally, some kind of "loops" would be very nice, for example n REP to repeat the last string of operations n times. But how to mark the string? Maybe a new kind of variable, and maybe enclose the macro in [] ? Like [* dup] #SQ x #^SQ do a squaring x^2
[2 *] #SHIFT
x #^SHIFT n REP #x<<n shift up n times the x
or better
[2 *] REP #SHIFTn
x n #^SHIFTN #shift x n times left

Then you could have a library and just pre-pend that to your string, and all these new functions are defined for you. I'd write some.

Last edited by jmac698; 13th September 2015 at 11:02.
jmac698 is offline   Reply With Quote
Old 13th September 2015, 10:52   #10  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,860
There's also my Mandelbrot plugin
http://forum.doom9.org/showthread.ph...42#post1582742

Also an 8-bit Mandelbrot with masktools
http://forum.doom9.org/showthread.ph...09#post1539509

I should rewrite it with yours so I can use floats
Yes, I'd be interested in you fixing the bugs to make a nice Mandelbrot possible.
jmac698 is offline   Reply With Quote
Old 13th September 2015, 11:05   #11  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
right, now you can write some simple plugins like removegrain kinda stuff with this, in a script, if you just don't wanna waste ur time to learn the hardass c/cpp
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 13th September 2015, 11:06   #12  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,860
Btw, a nice trick for a fill function is xor with the last byte, I'm leaving out some details but it's like the Amiga does in hardware, draw some borders and they get filled in horizontally, would be neat to try to write that.

@feisty
yes exactly! That's why I loved masktools and asked for some mods to it. a48 was a lot inspired by my requests. Also I like that it's in script, and easily exposes the algorithm, so nothing is ever lost by being closed source.

Last edited by jmac698; 13th September 2015 at 11:09.
jmac698 is offline   Reply With Quote
Old 13th September 2015, 12:42   #13  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
Quote:
Originally Posted by jmac698 View Post
The only thing I'm missing is, a shift operator,
It's called "multiply" And it might be quicker than implenting a real shift, which would involve a conversion from double to int to double.

Quote:
and support for stacked 16 bit format built-in.
Possible. Reading pixels would be easier than writing them, I suspect (reading could probably be impemented with a string function; see next answer)

Quote:
Defining some kind of macros/functions would make it much easier to use, especially as they can be shared as a library and then built upon.
You could do that with Avisynth strings, as I did with my second Mandelbrot post.

Quote:
How are virtual pixels handled?
What do you mean by "virtual pixel"?

I'm thinking about loops and jumps and forward ifs, but they all need careful consideration. If you conditionally jump from one point to another, you have to ensure (and the compiler would have to enforce) that the stack will be in exactly the same condition (which is not just a matter of how many items are on the stack, but also which are on the FPU stack and which are on the "overflow" stack) at the end of both branches. And that's easier said than done.
__________________
My AviSynth filters / I'm the Doctor

Last edited by davidhorman; 13th September 2015 at 12:49.
davidhorman is offline   Reply With Quote
Old 14th September 2015, 18:01   #14  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
Here's an example explained in detail. It simulates the "zebra stripes" that some cameras show in the viewfinder to show out-of-range white values.

Code:
avisource("...")
levels(0,1,200,0,255,coring=false) # make some white values illegal for testing purposes

striped=last.y8_rpn("
x y + n + 128 * 1024 %
[y0]
dup 255 >
?")

mergechroma(striped.converttoyv12, last)
The first two lines just load the source and adjust the levels to make some of the luminance values illegal (higher than 235).

Next comes the RPN call. First, there's this line:

Code:
x y + n + 128 * 1024 %
This builds the stripey, animated background. It adds the pixel x coordinate, the pixel y coordinate, and the frame number, then multiplies the total by 128. It then takes the modulo of this with 1024 to make a repeating pattern of thin black stripes and thicker white stripes.

Code:
[y0]
Next, the luma value of the pixel is put on the stack. By default, y8_rpn expands the luma range from 16-235 to 0-255 on read. It doesn't clamp this value, though, so source pixels greater than 235 will leave a value greater than 255 on the stack.

Next:

Code:
dup 255 >
This line duplicates the [y0] value, because the comparison with 255 will remove the value from the stack, but we still want it on there. The comparison itself returns 1 if [y0] > 255, otherwise it returns 0.

Next:

Code:
?
Based on the previous result (1 or 0), this operator picks either the result of the first line, or the result of the second line (because these are the remaining two items on the stack; it has nothing to do with the placement of the newlines, which are for clarity only).

Finally a call to mergechroma restores the colour to the clip (rgba_rpn/y8_rpn ignore UV channels completely).

By this method, pixels which have an illegal luma value are replaced with the stripey pattern:


NB: because y8_rpn clamps output values to the legal range, by default, the output will no longer have any out-of-range luma pixels. I'm not sure this is the best way to behave, but it's always been a minefield.

Coming soon: loops and commenting.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 14th September 2015, 18:27   #15  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 360
Wouldn't it be easier to just use the regular expression format? I.e. instead of

Code:
x y + n + 128 * 1024 %
this:

Code:
((x + y + n) * 128) % 1024
It's easier to write and easier to read...
creaothceann is offline   Reply With Quote
Old 14th September 2015, 19:07   #16  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
Easier for a human because it's familiar, sure, though I'm not sure it's fundamentally easier. We should all be using RPN! And writing dates in YYYY-mm-dd format...

The reason for using RPN is because it's much, much easier for a computer to parse and therefore to program for.

An infix-to-RPN converter, such as masktools has, would be the answer, but it's not a big priority for me.
__________________
My AviSynth filters / I'm the Doctor

Last edited by davidhorman; 14th September 2015 at 19:10.
davidhorman is offline   Reply With Quote
Old 14th September 2015, 19:34   #17  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
Quote:
Originally Posted by davidhorman View Post
An infix-to-RPN converter, such as masktools has, would be the answer, but it's not a big priority for me.
There's also ConvertToPostfix by tsp (download binary here).

---

I understand the concept behind RPN, the thing I don't understand is how does one come up with the numbers to do what you wanted it to do (I know is not but to me it just seems arbitrary). Is there any reference material that teaches video processing and RPN? I've looked but have not found anything, in this MaskTools guide tp7 mentioned that this type processing is called a “point operation” in all image processing books. Is this true for rgba_rpn/y8_rpn also?

Last edited by Reel.Deel; 14th September 2015 at 19:59. Reason: update link
Reel.Deel is offline   Reply With Quote
Old 14th September 2015, 19:47   #18  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
Quote:
Originally Posted by Reel.Deel View Post
There's also ConvertToPostfix by tsp (download binary here).
It seems that's a very early try and tsp improved it later in the thread. It might be a place to start, but rgba_rpn/y8_rpn has a few odd features which might not translate all that easily to infix notation now.

Quote:
I understand the concept behind RPN, the thing I don't understand is how does one come up with the numbers to do what you wanted it to do (I know is not but to me it just seems arbitrary). Is there any reference material that teaches video processing and RPN? I've looked but have not found anything, in this MaskTools guide tp7 mentioned that this type processing is called a “point operation” in all image processing books. Is this true for rgba_rpn/y8_rpn also?
Not really sure I can help there... it's just maths! You've got your variables - x and y pixel position, frame number, current pixel values - and you just make them do what you want them to do.

Give me an example of something you might want to do (even if it's just a silly example) and I can try to take you through how to do it.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 14th September 2015, 20:12   #19  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
Quote:
Originally Posted by davidhorman View Post
Not really sure I can help there... it's just maths! You've got your variables - x and y pixel position, frame number, current pixel values - and you just make them do what you want them to do.

Give me an example of something you might want to do (even if it's just a silly example) and I can try to take you through how to do it.
"it's just maths!" - that's my biggest problem!

I've been playing around with PhotoDemon's distort filters, it would be nice to able to do such things in AviSynth (especially animated). I like the 'Poke' effect, might be more suitable for xyremap though.

Last edited by Reel.Deel; 14th September 2015 at 20:22. Reason: typo
Reel.Deel is offline   Reply With Quote
Old 14th September 2015, 20:15   #20  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
It depends on the exact effect you want to replicate. xyremap can do a polar remap (first example on that page), but it can't do a zoom blur by itself (and nor can rgba_rpn). Put the two together and you could just about manage it, but it wouldn't be very efficient.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.