xyremap - reverse Polish pixel remapper (experimental)

[wonkey_monkey]

Download beta: xyremap0.5beta.zip (http://horman.net/avisynth/download/xyremap0.5beta.zip) v0.5 (beta)
version 0.5: some new RPN functions, a bug fix, and (approximately) gamma-aware interpolation (see http://forum.doom9.org/showthread.php?p=1736501#post1736501)
version 0.4: new x86 RPN compiler; new functionality (see http://forum.doom9.org/showthread.php?p=1735605#post1735605)

Download: xyremap0.3.zip (http://horman.net/avisynth/download/xyremap0.3.zip) v0.3
version 0.3: new parameter "static" to speed up non-animating remaps
version 0.2: bug fixes to pixel interpolator, min/max functions, added new RPN parameters

See included xyremap.txt for parameters, etc.
See here (http://en.wikipedia.org/wiki/Reverse_Polish_notation) and here (http://manao4.free.fr/mt_masktools.html#reverse_polish_notation) for explanations of Reverse Polish Notation

Please note that the RPN parser is not very intelligent when it comes to malformed notation, and may simply return an unexpected result without erroring.

Examples

To make some animated waves:

xyremap(version.converttorgb32,y="y x 0.1 * n 5 / + sin 8 * +")

http://pasteboard.s3.amazonaws.com/images/1350170398645789.png

To take a trip into the time vortex:

stackhorizontal(version,version)
stackvertical(last,last)
converttorgb32
xyremap(\
"y h 0.5 * - x w 0.5 * - atan2 n 250 / + TAU + u 1.5 * TAU / * u 0.5 * % u 0.25 * +",\
"1 y h 0.5 * - 2 ^ x w 0.5 * - 2 ^ + sqrt / w 100 * * n + v 0.5 * % v 0.25 * +",\
"y h 0.5 * - 2 ^ x w 0.5 * - 2 ^ + sqrt w 0.5 * /",\
w=640,\
h=360,\
draft=false)

(four "versions" are stacked so the remapper can avoid the image edges, which would otherwise result in some visible edges on the output)

http://imageshack.us/a/img42/5848/tunnelb.jpg

[ajp_anton]

Whoa, this is just what I was requesting a short while ago, but then gave up that project...
Would love to get some time to play around with this, looks like it can do pretty much everything =).


"Note that "-" is used instead of "+" because we are mapping from output to input pixels, not the other way around (which would be more intuitive)."
I'm guessing it's like this so that each output pixel will have an input location (interpolated pixel) to get its data from.

"t: t=n/framecount (so 0<=0<1)"
0<=t<1

[wonkey_monkey]

"Note that "-" is used instead of "+" because we are mapping from output to input pixels, not the other way around (which would be more intuitive)."
I'm guessing it's like this so that each output pixel will have an input location (interpolated pixel) to get its data from.

Exactly.

"t: t=n/framecount (so 0<=0<1)"
0<=t<1

Hmm, well spotted. Although 0<=0<1 is also true ;)

David

[ajp_anton]

Am I the only one who finds this interesting?

Feature requests:
- Ability to choose the color of the padding outside the input image, or extend the existing border pixels. The bicubic interpolation is making my stretched edges dark.
- Boolean to make x and y go from 0 to 1 so I don't have to divide by u and v.
- Is it possible to make the function go the other direction, map input pixels to the output image instead? It's difficult enough to find the function from input to output that then needs to be reverseed/solved with WolframAlpha, but my functions are often non-solvable =). And it would be easier to wrap helper functions around them.

[wonkey_monkey]

Am I the only one who finds this interesting?

Apparently :D

Feature requests:
- Ability to choose the color of the padding outside the input image

This is probably best left outside of xyremap's remit. The alpha channel is available so you can comp in a different background with overlay.

extend the existing border pixels. The bicubic interpolation is making my stretched edges dark.

This can be done either by stacking multiple versions of your input together before remapping (or padding with background colour), then using the % (modulo) operator to constrain the input pixels to within the "real" borders. Or you could try adding:


... 0.5 max w 0.5 - min
... 0.5 max h 0.5 - min


to your x and y expressions respectively. This may or may not solve it though - I can't test it properly because I've just discovered a bug in the interpolation function that seems to be messing up levels.

- Boolean to make x and y go from 0 to 1 so I don't have to divide by u and v.

Rather than a bool, how about two new parameters, a and b, for these values?

- Is it possible to make the function go the other direction, map input pixels to the output image instead?

What happens if two input pixels map to the same output pixel? Also bicubic interpolation that way around would be tricky (if not impossible).



TL;DR: there is a bug in the interpolator that I need to fix which is producing slightly wrong pixel values.

David

[wonkey_monkey]

xyremap v0.2 (http://horman.net/xyremap.zip)

A couple of bugs fixed - min/max weren't working properly, and the pixel interpolation bug is fixed - and some new RPN parameters are added:

a: pixel x coordinate normalised to [0-1]
b: pixel y coordinate normalised to [0-1]
c: pixel y coordinate normalised to [0-1*display_aspect_ratio]


You'll still have to multiply by w/h at the end though - I may look at this again sometime, but at least it saves you the more costly divisions.

Hope that helps!

David

[ajp_anton]

Thanks.
Yes it might be a problem with mapping into the same, but my functions don't do that =). Oh well, I guess I can live without it.

I padded it myself using pointresize (needed 2 pixels so the last pixel wouldn't be slightly dark), but I would've liked for an easier solution. The modulo solution looks interesting though, didn't think of that.

Another request: an "if" statement.

[mastrboy]

Am I the only one who finds this interesting?

The plugin functions in itself is interesting, but as probably many others I only use avisynth to correct "errors" in the video, like f eks rainbowing and dotcrawl.

My guess is that those who want to manipulate video and add effects turn to Adobe AfterEffects or similar.

[wonkey_monkey]

The plugin functions in itself is interesting, but as probably many others I only use avisynth to correct "errors" in the video

Perhaps my examples were a little esoteric, but of course xyremap could be put to use correcting picture distortions and the like. Perhaps a YUV version would be useful for correcting some chroma misalignments.

My guess is that those who want to manipulate video and add effects turn to Adobe AfterEffects or similar.

Which is not, contrary to popular belief, a free download ;) so now (for very simple things) they don't need to.

Anyway, here's another frivolous (and very slow) use of xyremap:

a=blankclip(width=640,height=1280,length=1500).subtitle(\
"It is a period of civil war.\n"+\
"Rebel spaceships, striking\n"+\
"from a hidden base, have won\n"+\
"their first victory against\n"+\
"the evil Galactic Empire. \n\n"+\
"During the battle, Rebel\n"+\
"spies managed to steal secret\n"+\
"plans to the Empire's\n"+\
"ultimate weapon, the DEATH\n"+\
"STAR, an armored space\n"+\
"station with enough power to\n"+\
"destroy an entire planet. \n\n"+\
"Pursued by the Empire's\n"+\
"sinister agents, Princess\n"+\
"Leia races home aboard her\n"+\
"starship, custodian of the\n"+\
"stolen plans that can save\n"+\
"her people and restore\n"+\
"freedom to the galaxy.... "\
,lsp=0,align=8,size=50).converttorgb32

b=a.reduceby2.bicubicresize(320,320)
c=b.reduceby2.bicubicresize(160,80)
d=c.reduceby2.bicubicresize(80,20)

a=a.xyremap("x w 0.5 * - 1 y h / * / u w / * u 0.5 * +","v h y - y / h * n 4 * - v + - v 1280 / *","y 4 - h / sqrt 1 h y - 0.005 * - *",w=640,h=360)
b=b.xyremap("x w 0.5 * - 1 y h / * / u w / * u 0.5 * +","v h y - y / h * n 4 * - v + - v 1280 / *","y h / sqrt h y - 0.005 * 1 180 y - 0.01 * - min *",w=640,h=360)
c=c.xyremap("x w 0.5 * - 1 y h / * / u w / * u 0.5 * +","v h y - y / h * n 4 * - v + - v 1280 / *","y h / sqrt 180 y - 0.01 * 1 90 y - 0.02 * - min *",w=640,h=360)
d=d.xyremap("x w 0.5 * - 1 y h / * / u w / * u 0.5 * +","v h y - y / h * n 4 * - v + - v 1280 / *","y h / sqrt 90 y - 0.02 * *",w=640,h=360)

e=overlay(a,b,mode="add",pc_range=true)
f=overlay(c,d,mode="add",pc_range=true)
overlay(e,f,mode="add",pc_range=true)


http://img716.imageshack.us/img716/6016/crawli.png

Tell me that's not practical ;)

David

[jmac698]

I think this would be useful to remove barrel distortion from a camera lens. Once you do that, it's simple to stitch together scenes into panoramas.
The other problems though, could be vignetting and exposure changes, but in some cases it could work.

[StainlessS]

This is quite spectacular, limited usage as Mastrboy suggests, but impressive, just the same.
Think maybe the fascination with Dr Who may have influenced the vortex-ish type intro.

EDIT: PS, nice work.

[jmac698]

Really? Vortices are a common demo effect
scene.org
I'd like to make another avisynth demo
http://forum.doom9.org/archive/index.php/t-163121.html

[wonkey_monkey]

Think maybe the fascination with Dr Who may have influenced the vortex-ish type intro.
Really? Vortices are a common demo effect

No, StainlessS is right, though how he could possibly know I like Doctor Who is a mystery... :rolleyes:

[martin53]

David,

I'd like to suggest some extensions that I hope are manageable to program, yet extend the capabilities significantly.

A) Extension of RPN

dup, unary operator, duplicates top operand for later
re-use
sign, unary operator for sign (-1 for <0, 0 for 0, 1 for >0)
round, floor, ceil, converting float to integer like AviSynth does
log, exp, logarithm and exponent


B) Extension of input variables
r/g/b/a red/green/blue/alpha pixel values

C) Extension of parameters
r/g/b/a red/green/blue/alpha pixel values additionally to x/y coordinates - i understand that xyremap is made with two cascaded loops, one over the lines and one over the columns? so, with the extensions b) and c), instead of remapping, xyremap could be used as color manipulation function beyond YV12LUT() for RGB format.

D) A background clip, and dx, dy parameters to overlay output with offset over it - considering alpha, of course.

E) new name xyrgbaSwissKnife ;)

[wonkey_monkey]

Hi Martin,

I had considered some extensions. I'm not sure what you mean about dup - how would you "re-use" the duplicated operand? I considered an assignment operator, =, to set your own variables (and perhaps then an operator to reset the current stack as well, if required, leaving assigned variables intact), but that means extra checks to make sure the hard-coded variables aren't overwritten. Perhaps also some comparison operators - lt, gt, eq - and some way to implement if ... then ... else in RPN.

As for B) and C), which r/g/b/a pixel values would these be? The filter is all about calculating x,y coordinates of source pixels, so how can it know which pixel to read to get the RGBA values? I'm also not sure what you mean by "additionally to x/y coordinates" in C).

A separate RGBA filter is certainly possible, though.

D) I thought was best left to the user to use the internal layer filter.

E) Silly :)

David

[ajp_anton]

"some way to implement if ... then ... else in RPN."

See how it's done in mt_masktools. You have the ==, >, <... checks that take two input variables and return true/false.
Then you have the ? that takes three inputs. One bool, one to return if it's true, one if it's false.
if a==b, then T, else F
a b == T F ?

[martin53]

Hi Martin,

I'm not sure what you mean about dup

- You are right: MaskTools also use RPN, and I feel it would be a good idea to use similar operand sets. I only used yv12lut so far. There is no min operator. To subtract a value, but avoid negative results, I write "x 128 - 0 < 0 x 128 - ?", or in human speech, "subtract 128 from x, and if the result is below zero, take zero, else take x minus 128". X-128 must be computed twice, and if I needed to limit a very complicated expression, I have no confidence that I could manage that (bug finding is not easy with RPN). I thought that with dup, an intermediate result could be stored and re-used more easily. But in fact it would be useless because the copy is needed only later and cannot easily handled on the RPN stack.

A different strategy would be needed. A limited set of 'variables'-operands, like 's0' stores top of stack to 'v0' and leaves it also on stack, 'r0' recalls 'v0' to top of stack would help - and stay within the syntax.

As a side thought: To facilitate bug finding, the function could throw an error if the expression does not end with 1 entry on the stack, and dump the stack. What do you think about this?

[martin53]

As for B) and C), which r/g/b/a pixel values would these be? The filter is all about calculating x,y coordinates of source pixels, so how can it know which pixel to read to get the RGBA values?
I see the concept. B) only makes sense in C).
For the RPN formulas of parameters "x" and "y", no extension is useful.
My suggestion meant four more parameters string "r", string "g", string "b", string "a", all with RPN expressions.
With these expressions, also the variables r, g, b, a of the source pixel would be fantastic (instead of copying it 1:1). Imagine the cool color gradients possible with just xyremapping a BlankClip, even x="x", y="y" and using the variables x,y in the brightness RPN expression (shiver). xyremap(x="x",y="y",r="x u 2 / - 2 ^ y v 2 / - 2 ^ + u 2 ^ v 2 ^ + / PI * cos") red = cos( PI * ((x-u/2)²+(y-v/2)²)/(u²+v²) )


A separate RGBA filter is certainly possible, though.

It would be a pity if x and y would not be present as variables to define the brightness. No color gradients then.


D) I thought was best left to the user to use the internal layer filter.

You allow an implicit cropping with the parameters w and h. You are right: even if an alpha value is calculated with xyremap, the output clip can still be forwarded to layer. I tend to create complicated processing. Because long graphs use much memory, I prefer few powerful filters over many simple ones. Only because of that, I phantasized to combine the two steps into one.

[wonkey_monkey]

As a side thought: To facilitate bug finding, the function could throw an error if the expression does not end with 1 entry on the stack, and dump the stack. What do you think about this?

What I think is that it already does this :)

compiler error - check your string (x) # where x is how many values are left on the stack


x 128 - 0 < 0 x 128 - ?

In my RPN machine, this can be written:

x 128 - 0 min

as you know, but with user variables it could possibly be written:

x 128 - q= 0 < 0 q ?

The "q=" would have to be a pseudo-operator, since it doesn't really work like RPN should (and it has no effect on the stack - a stack reset operator could also be useful), but it would be useful.

You allow an implicit cropping with the parameters w and h

True, but that's for speed reasons. No sense doing all those heavy calculations only to crop them away.

Imagine the cool color gradients possible with just xyremapping a BlankClip, even x="x", y="y" and using the variables x,y in the brightness RPN expression (shiver).

I don't see any sensible interpretation for the rgba values except when using x="x" and y="y" - in which case it may as well be a separate filter (maybe in the same DLL, but a separate keyword - rgbaremap). Plus, I already used the variables a and b :(

Warping an image in two dimensions is hard enough - warping it in 6 is crazy! ;)

David

PS x="x" and y="y" are the defaults for those expressions, so don't need to be specified.

[Didée]

Imagine the cool color gradients possible with just xyremapping a BlankClip, even x="x", y="y" and using the variables x,y in the brightness RPN expression (shiver).
MaskTools v2: "mt_lutspa()"

[martin53]

mt_lutspa()
Thanks, Didée - I'm just learning v2...;)

[martin53]

I don't see any sensible interpretation for the rgba values except when using x="x" and y="y" - in which case it may as well be a separate filter (maybe in the same DLL, but a separate keyword - rgbaremap). Plus, I already used the variables a and b :(

Absolutely! you are the author, you decide. Just suggestions and raw ideas.

[wonkey_monkey]

Version 0.3 (http://horman.net/avisynth/download/xyremap0.3.zip) adds a "static" parameter - set this to true to make non-animating remaps much quicker.

[Kisa_AG]

Version 0.3 (http://horman.net/avisynth/download/xyremap0.3.zip) adds a "static" parameter - set this to true to make non-animating remaps much quicker.

Wow, it's incredebly much faster!
My script (2.7K video from GP3BE) now opens little bit slower (22 seconds in comparison to 6 seconds with xyremap v.0.2), but it runs MUCH faster - 6.8 fps in comparison to 0.18 fps with xyremap v.0.2.
It's almost 40 times faster! Incredebly!
Thanks, davidhorman! Perfect filter!

By the way, do you have plans to add another interpolations to existing bicubic and nearest-neighbour. Let's say Lanczos or Spline? I think it can make pictures little bit sharper right after conversion, isn't it?

[wonkey_monkey]

New version, if anyone cares!

Download beta: xyremap0.4beta.zip (http://horman.net/avisynth/download/xyremap0.4beta.zip) v0.4 (beta)

Changes include, but are not limited to, the following:


It's faster - at least twice as fast as before*, and gets faster as the formulae get more complicated (but "static" remaps are no quicker)
You can now set and recall your own variables:

"@A" will store the top value on the stack to variable A
"@A^" will store the value and pop it off the stack
Variables are case-sensitive, and all the built-in ones are lower case, so sticking to upper case will ensure you never overwrite them.

"dup" duplicates the top stack value (use this with * instead of ^ for integer powers, it's much quicker)
"swap" swaps the top two stack values
"round", "floor" and "ceil" do what you'd expect
"neg" - "x neg" returns -x


*it incorporates an RPN-to-x86 compiler, for those (if any) interested in such technical details

It also now has comparison and conditional operators:

"<", ">", "<=", ">=", "==", "!=" each return 1 if true of 0 if false.
"[x] [y] [cond] ?" returns x if cond is non-zero, otherwise it returns y.

Real example: "10 20 x y < ?" - returns 10 if x<y, otherwise 20.

For now there are no boolean operators, but you can hopefully simulate them with mathematical operators.

This is a beta, so use at your own risk.

Any questions?

*tumbleweeds*

[Wilbert]

Your link points to http://horman.net/avisynth/download/xyremap0.3.zip

[wonkey_monkey]

Aha, so, spotted that did you? :o

(fixed)

And just in case:

http://horman.net/avisynth/download/xyremap0.4beta.zip

[wonkey_monkey]

I've uploaded yet another new version (0.5). It fixes a minor bug (one of the variables was being set to an incorrect value), adds a couple of new RPN operations (which I've now detailed properly here (http://horman.net/avisynth/rpn.php)) and the bicubic interpolation is now (approximately) gamma-aware.

http://horman.net/avisynth/download/xyremap0.5beta.zip

[StainlessS]

Thank you Mr David, Have I ever told you that you are such a wonderful fellow ?

[wonkey_monkey]

Oh, now, stop it, you'll make me blush. It's just nice to know someone other than me is getting some use out of this.

I still feel the whole RPN thing may be a little opaque, so if anyone does have any "how would I do this" or "can I do this" questions...

[raffriff42]

A "gallery" of visual effects, with source code (like your first post), would help get us started.

A infix-to-postfix converter (https://www.google.com/search?q=infix-to-postfix+converter) (like Masktools' mt_polish) would help code readability a lot, at least for those of us who do not think in RPN.

A GUI with realtime preview and code generation would be great, but that's too much to ask :(

[Reel.Deel]

A "gallery" of visual effects, with source code (like your first post), would help get us started.

Good idea! Here's 'waves' from the OP:
http://s17.postimg.org/yombnp14v/Av_Synth_Pixel_Art.png http://s24.postimg.org/npifnr0lx/xyremap_waves.gif

I'll try to make a vortex GIF tomorrow :).

[ajp_anton]

Bump for a 64-bit version request.

[wonkey_monkey]

Easier said than done, since it's not a simple case of just recompiling with a different target architecture. It's on my list to do soon, though.

[StainlessS]

Good idea! Here's 'waves' from the OP:
http://s17.postimg.cc/yombnp14v/Av_Synth_Pixel_Art.png http://s24.postimg.cc/npifnr0lx/xyremap_waves.gif

I'll try to make a vortex GIF tomorrow :).

Fixed Reel.Deel image posts (Reel.Deel, just need change PostImage.org to PostImage.cc)

[wonkey_monkey]

Well they worked four years ago when he first posted 'em.

An x64 version is still on my list, but unfortunately I've messed around with my RPN library so much that xyremap won't currently compile a working version.

[StainlessS]

At least one of your images (2nd post in fusion thread) has broken Imageshack link, dont know if fixable like PostImage links.

Its enticing to use common library, but any change can kill stuff, so I generally dont use, I usually copy paste relevant stuff and make changes
where required. Neither way is ideal.

[LigH]

I believe one could use this filter to simulate a kind of "fisheye scaler" as used in some "smart" TV sets to anamorphically scale 4:3 video up to a 16:9 canvas, where the center remains mostly undistorted, and the borders are pulled to the sides, as if you are inside a cylinder the video gets projected on.

I just need to calculate a convenient formula to resample a 704 or 720 pixels wide centered line to a length of 1024 pixels (I guess a "viewing angle" parameter could control the amount of distortion here). And then transform it into UPN.

[hello_hello]

I believe one could use this filter to simulate a kind of "fisheye scaler" as used in some "smart" TV sets to anamorphically scale 4:3 video up to a 16:9 canvas, where the center remains mostly undistorted, and the borders are pulled to the sides, as if you are inside a cylinder the video gets projected on.

Unless you need something more complicated, have you come across WarpedResize?
https://forum.videohelp.com/threads/363684-4-3-to-16-9-avisynth#post2314433

[LigH]

:o Looks even better.

[wonkey_monkey]

You could use XYRemap, but I also already wrote Widen (http://horman.net/avisynth/download/widen.zip) for this exact purpose, which works in all 8-bit colour spaces (but is 64-bit only).

If you're looking for a formula, it involves tan or inverse tan, can't remember which way round I did it.

You could also try messing around with Defish (https://forum.doom9.org/showthread.php?t=152860) which may mitigate the ugliness of edge distortion on pans by introducing some vertical distortion as well.

[LigH]

I will prefer the specialist plugin, its precision is probably higher than an 8-bit LUT. And even better it's available for x86-64.

:thanks:

[wonkey_monkey]

its precision is probably higher than an 8-bit LUT.

Not sure what LUT you are referring to.

[LigH]

If I use an RGB24 image as distortion map (a.k.a. look-up table), the resolution of the distortion would be 8 bit at most. I guess it might be coarse enough to see some "motion banding" in panning scenes?!

[wonkey_monkey]

That was a suggestion by someone that never got implemented. xyremap works purely on calculations of target coordinates to source coordinates. Widen doesn't use a LUT either.

[LigH]

:o Apparently I mixed that up with an earlier idea what I was looking for... one day I notice the UPN in the description, the other day I completely forget about it.

[StainlessS]

Maybe of interest to some Wonkey_Dalek:
Doctor Who launching immersive theatre experience with Daleks and Time Lords

https://metro.co.uk/2020/08/18/doctor-who-immersive-theatre-daleks-time-lords-13144487/

EDIT: Maybe one step closer to being a Cyberman:
Cyborg’ technology just took a step closer to reality
https://metro.co.uk/2020/08/18/cyborg-technology-just-took-step-closer-reality-13142896/

[LigH]

@StainlessS: Your PMs are full, I wanted to ask you about plugin development (https://forum.doom9.org/showthread.php?p=2007311#post2007311)...

But maybe @wonkey_monkey too. Not in this thread, of course. Just because you both may get notified when I mention it here.