Extreme Pre-Keystone + Brightness Graident Correction For Steep Angled Projectors?

[mdrejhon]

Hello.

(My Avisynth questions are at the bottom)

I want to shine a 5-figure lumens projector steeply angled upwards on the side of an apartment building.

I need extreme keystone correction far beyond the projector's adjustment range, and so I will need to use an external tool such as Avisynth or other video processor. How can I pull it off? I'm good at geometry but I'm new to Avisynth scripts.

Here are my diagrams.







EDIT: Corrected image.


On the "screen", the top edge stretches a lot more than the bottom edge, and the top edge is much dimmer. (Fortunately the super excess lumens of the projector compensates). So I want to build compensating corrections into a custom-processed video file purposely generated for playback on this steeply-angled-upwards projector. I have to intentionally play a portrait video distorted into a 16:9 signal because the 16:9 projector is vertically stretching due to extreme angle into something more suitable for embedding a portrait video. So there's some weird aspect ratio remapping. Focus is not a problem (top and bottom is almost the same, the lenses are really good at the distances used), so I'll live with any imperfections.

--> Everything works perfectly, I now just need to pre-distort a video file <--

My Questions:

Since this is a simpler projection mapping (Wikipedia) task than the complex paid projection mappers that professionals use, but too complex for a projector's built-in keystone. So I'm stuck in this lala land here, and I'm now seeing there are already some features in Avisynth scripts that might actually do what I need, in a satisfactory "simpler" projection mapping task. Since I've been turned off by the complex projection mappers, I'm looking at a simpler "Extreme Keystone" task that just barely veers into low-end projection mapping.

With Avisynth scripting, has anyone pulled off off keystone correction with (A) brightness gradient correction, (B) vertical stretch nonlinearity correction (C) major aspect ratio distortion since I'm embedding a squat portrait video in a 16:9 video file, (D) keystone with intentional corner clipping? (E) Black fill for outside boundaries (to avoid stray light into sky)

I think this now requires a custom script, in some scriptable video processor, so that's why I'm now experimenting with Avisynth. Has anyone ever made this extreme-keystone script before? I have programming/scripting knowledge, but I am combining many complex steps together.

By the way, this is a community weekend project on a loaner lumen cannon and family-budget only. That's why we're fiddling with free tools.

Thank you!

[poisondeathray]

Are you pre-manipulating , then encoding a video, then playing that video in the projector ? Or is this "live" as in realtime ? The reason I ask is I'm not sure how optimized some of the filters are, you might not get realtime fps

You can do this with david horman's quad . Overlay your portrait onto a black background (using overlay(a,b, x=whatever) , then enter the quad x,y coordinates for the desired distortion . It goes upper left, and clockwise (UL, UR, LR, LL)
https://forum.doom9.org/showthread.php?t=165978

other options - vdub has a perspective filter that you can manipulate with a GUI (just drag the corners) , and natron has a corner pin, blender has projection mapping

The gradient is easy to do; you can just apply either a darkened or brightened version through a gradient mask using the same quad coordinates

If you need more help, just ask

[mdrejhon]

Originally Posted by poisondeathray

Are you pre-manipulating , then encoding a video, then playing that video in the projector ? Or is this "live" as in realtime ? The reason I ask is I'm not sure how optimized some of the filters are, you might not get realtime fps

You can do this with david horman's quad . Overlay your portrait onto a black background (using overlay(a,b, x=whatever) , then enter the quad x,y coordinates for the desired distortion . It goes upper left, and clockwise (UL, UR, LR, LL)
https://forum.doom9.org/showthread.php?t=165978

other options - vdub has a perspective filter that you can manipulate with a GUI (just drag the corners) , and natron has a corner pin, blender has projection mapping

The gradient is easy to do; you can just apply either a darkened or brightened version through a gradient mask using the same quad coordinates

If you need more help, just ask

Pre-manipulating, yes. Real time is not necessary.

Also....
The common perspective filters don't work correctly because they compress the narrow dimension, rather than invert it which is what I really need, as a lineary stretch compensation. Basically the "distant perspective" edge needs to be most vertically stretched, not the least stretched.

[wonkey_monkey]

Originally Posted by mdrejhon

Pre-manipulating, yes. Real time is not necessary.

Also....
The common perspective filters don't work correctly because they compress the narrow dimension, rather than invert it which is what I really need, as a lineary stretch compensation. Basically the "distant perspective" edge needs to be most vertically stretched, not the least stretched.

If I understand you correctly, and judging from the example image you've given, I don't see how that would work.

Just a normal perspective transform, as with quad, should be exactly the right thing for this.

If you've done any tests, can you show the results of projecting a perspective-transformed image with an angled projector?

The narrow end will be stretched out more by projection, both vertically and horizontally, because it will travel further before hitting the building - so it should be vertically (and horizontally) compressed before projection.

[poisondeathray]

Originally Posted by davidhorman

The narrow end will be stretched out more by projection, both vertically and horizontally, because it will travel further before hitting the building - so it should be vertically (and horizontally) compressed before projection.

This makes more logical sense to me.




It sounds to me like mdrejhon is suggesting that the top edge of the predistorted compensatory video should be "wider" ; if that were true, then the diagram is a bit misleading - the edge at the equivalent horizontal level beyond the text should also have be wider.

I like to think of the original smartphone video as a "flat" 2D plane conceptually . So you're projecting a flat plane onto a perpendicular (mostly) flat surface of the building .

In the diagram, the "pre-distortion" goal looks more curved at the top of the frame edge, at least if you only look at the letters and ignore the edges. ie. It's no longer planar. ie. the words "Portrait" and "Smartphone" are not coplanar with the other text. Then quad won't work for that type of projection , assuming that diagram is accurate




So, like David asked - Do you have any reference pictures, or measurements, or calibration videos ? Because something doesn't quite add up here . Perhaps there is something to do with the projector or lens ? Or the building isn't flat ?

[mdrejhon]

Yes, it's a trick question that tricks the logical senses, but you are wrong.
EDIT: Oh, you mean the compensatory stretch algorithm? Yes, I now see the error in the original diagram. Fixing now.

I got a 98% mark in mathematics/geometry, so I guarantee you that I'm 100% correct on this particular item. *wink*

Here's photo proof...

(For this post, this is not the projector I plan to use. I just right now grabbed a random color-misadjusted projector (malfunctioning) and random crap smartphone to quickly take photos for geometry's sake -- but regardless, this is universal for all projectors).




(Crap picture, only 300 lumens.... The other light cannon I have available is 5-figure lumens, and still plenty bright at top edge. But now you know why I need a brightness-correction gradient!)

That's because a 1-degree angle covers a LARGER vertical distance in the farther surface than the closer surface, and the farthest part of image STRETCHES vertically as a result.

Okay, still don't believe me?

Here's a Geometry 101 lesson:



Finally, now that I've convinced you.. HELP Avisynth? Does this filter exist? <--- <--- <---

P.S. Thank you for you help; much appreciated. I understand many people are just simply accidentally imperfect on the keystone correction formula, simply due to the sheer 'trick question' nature of keystone correction!. But, still, one of you may be excellent shader programming experts (which alas, I am not) and probably can whippy-together the correct inverse-perspective correction algorithm in a mere minutes of ShaderToy work and then easily converted to an Avisynth script.
P.P.S. On a semi-related topic, I am the founder of Blur Busters, and the creator of TestUFO Motion Tests. Gaming monitor manufacters pay me to work for them and improve their displays. For those unfamiliar with Blur Busters / TestUFO, I am the creator of several motion tests at TestUFO such as Eye Tracking Motion Blur Animation / Black Frame Insertion Demo / Persistence of Vision Animation / Software-Interpolated Demo of Variable Refresh Rate, and many other animation demos that are normally not seen anywhere else on the Internet.

[poisondeathray]

Originally Posted by mdrejhon

Finally, now that I've convinced you..

Sorry, not convinced

That's because a 1-degree angle covers a LARGER vertical distance in the farther surface than the closer surface, and the farthest part of image STRETCHES vertically as a result.

The top projection is wider in your photo , stretches at top and is dimmer. But that's what you said earlier

So why would you want to make the compensatory video even wider ? at the top ? Wouldn't that stretch the end result even more ?

I got a 98% mark in mathematics/geometry, so I guarantee you that I'm 100% correct on this particular item. *wink*

Where did the 2% go ? :wink:

[mdrejhon]

The compensatory algorithm, yes. Now, I see the error in the original diagram. While I was correct on the projector behaviours, I certainly erred in that diagram.

Stand by, I will now post the corrected algorithm. Yes, A+ for the catch.

EDIT: Here's the corrected image doublechecking compensatory behaviours.

[poisondeathray]

I'm just trying to help. My math is rusty

That's because a 1-degree angle covers a LARGER vertical distance in the farther surface than the closer surface, and the farthest part of image STRETCHES vertically as a result.

I understand, this makes perfect sense, and is in line with what David is saying . The top of the normally projected image is stretched vertically and horizontally more, as shown in your picture . All good. All agree.

I'm just using basic logic here, no equations yet - so the compensation video should be more narrow and compressed at the top, but wider at the bottom . It should be darker at the bottom, brighter at the top. ie. Opposite to what you see on that normal sample projection . The projection will serve to undistort the compensation video , so it's flat on the vertical building surface

There is a light brightness falloff equation , something like inverse square (not sure) . But that' s in a perfect environment. We have things like pollution, air particles that scatter , so the equation needs some fudge factor. It's probably faster just to make a gradient ramp and a few tests


What are you actually starting with for the phone ? Is it 9:16 and what dimension ?

[mdrejhon]

Originally Posted by poisondeathray

I'm just using basic logic here, no equations yet - so the compensation video should be more narrow and compressed at the top, but wider at the bottom . It should be darker at the bottom, brighter at the top. ie. Opposite to what you see on that normal sample projection . The projection will serve to undistort the compensation video , so it's flat on the vertical building surface

Ok, correct, we're on the same page now.

Originally Posted by poisondeathray

What are you actually starting with for the phone ? Is it 9:16 and what dimension ?

While the source video is 16:9 rotated to 9:16 -- the horizontal/vertical stretch is less important for this particular application. Any imperfection in this will just be part of the artistic impression -- simple aspect ratio distortion. Basically squeeze the 16:9 (but with consistent linearity) onto whatever the aspect ratio the side of building.

If I want it correct square pixel aspect ratio instead, obviously I'll need to crop the video to a roughly 3:1 aspect preserving the center vertical strip of the portrait smartphone video corresponding to the aspect ratio of the side of the building side (above roughly the 2nd story). But the exactness of the pixel aspect ratio is not an essential detail for this specific situation, as long as the final resulting stretch is correctly uniform (e.g. lines are straight). As observed, you notice I horizontally compress in the resulting image, while preserving stretch linearity.

Originally Posted by poisondeathray

There is a light brightness falloff equation , something like inverse square (not sure) . But that' s in a perfect environment. We have things like pollution, air particles that scatter , so the equation needs some fudge factor. It's probably faster just to make a gradient ramp and a few tests

Good point about pollution and air attenuation effects.

The simple gradient ramp will be sufficient as the human vision is logarithmic in brightness perception (without A/B tests, human perception of gradient imperfections is poor with busy images. It often mainly shows up in light field test patterns -- e.g. hotspotting effects on medium-high-gain (1.3 gain) screens in home theater often much perceptually more uniform with busy material such as forests and patterns than with more-solid color fills such as sky or bright clouds). So it's possible the boilerplate formula is 'good enough' if the air attenuation is within that error margin, but obviously tests will be needed. Even imperfect gradient ramp would probably be sufficient. Focus issue is probably more detectable nonunformity than air-attenuation-related nonuniformity, but projector focus is not a problem given sufficient excess distance & sufficient excess resolution (both of which is true in this case). If I had to, I'd pre-blur the in-focal-plane portion to compensate, but focus is sufficiently uniform enough in tests for desired clarity needed.

[poisondeathray]

Originally Posted by mdrejhon

While the source video is 16:9 rotated to 9:16 -- the horizontal/vertical stretch is less important. Part of the artistic impression is an aspect ratio distortion to squeeze the 16:9 (but as a correctly mapped rectangle) onto whatever the aspect ratio the side of building (and if necessary, simply crop-off any lightspillage).

If I want it correct square pixel, obviously I'll need to crop the video to a roughly 3:1 aspect preserving the center vertical strip of the portrait smartphone video corresponding to the aspect ratio of the side of the building side (above roughly the 2nd story). But the exactness of the pixel aspect ratio is not an essential detail for this specific situation, as long as the final resulting stretch is correctly uniform (e.g. lines are straight).

What are the actual dimensions of the source video ? 1920x1080 ? 3840x2160 ?

What 16:9 actual dimensions are you feeding into the projector ? And how are you feeding it ? Because some projector have limitations, and those dimensions are what you have to embed/overlay the source clip into

"lines are straight" - you just want it to look like a "rectangle", instead of a "trapezoid" , right ?

[wonkey_monkey]

Originally Posted by mdrejhon

Finally, now that I've convinced you.. HELP Avisynth? Does this filter exist? <--- <--- <---

Yes. Quad, as already mentioned

Just keep modifying the target quadrilateral parameters until the projection is rectangular and the rest of the geometry will be correct.

xyremap can take care of the brightness gradient, although that's not really what it's designed for. Something like

# your source filter goes here

converttorgb32
xyremap(m="1 b ** - 0.4 * 0.6 +") # adjust these two numbers but keep them totalling one to avoid blowing highlights
spline16resize(int(16.0/9.0*height),height) # resize to 16:9

# you may want to double or even quadruple the size in the resize above (by changing it to "spline16resize(int(16.0/9.0*height)*X,height*X)" where X is the scale...

t=0.4 # increase to bring top corners inwards
b=-0.1 # decrease (make more negative) to push bottom corners outwards
quad(t,0, 1-t,0, 1-b,1, b,1, normal=true)

# ...and then reduce it here ("spline16resize(width/X,height/X)" or whatever dimensions you want to give your projector) to mitigate aliasing

should do it.

The gradient may not be technically correct. I'll have a think about it.

[mdrejhon]

That's an excellent first-go script!

Originally Posted by poisondeathray

What are the actual dimensions of the source video ? 1920x1080 ? 3840x2160 ?

Irrelevant to this discussion. It's simply geometry.

Originally Posted by poisondeathray

What 16:9 actual dimensions are you feeding into the projector ? And how are you feeding it ? Because some projector have limitations, and those dimensions are what you have to embed/overlay the source clip into

The script goal is to overcome the limitations.

The above script actually, I believe, solves my problem in a good-enough way. I have to test, however.

[wonkey_monkey]

If the final project has or will have a webpage or a video or anything, I'd love to see it in action.

[StainlessS]

Originally Posted by poisondeathray

What are the actual dimensions of the source video ? 1920x1080 ? 3840x2160 ?

Originally Posted by mdrejhon

Irrelevant to this discussion. It's simply geometry.
The script goal is to overcome the limitations.

Additional info is not always superfluous, PDR enquiry although seemingly of no obvious immediate use to you, is likely not without reason, and your response is definitely not going to assist. [EDIT: it dont help you to blow off the big guns when you want assistance, thats what you just did, althoug DH is also a huge gun, guess you decided which enormous gun you like best and sod the rest]