Transposing a 2D videostream into a 3D object (frame count as a depth)

[Seedmanc]

I got a strange idea once, what if you could represent your average videostream that has width, height and duration as a box with the same dimensions, but having its z-axis along the video length? Such a "box" would have all video frames inside of it. Imagine that when you're watching a video, you're looking at this box from its front side, going deeper into it as you're progressing along the time axis.
http://puu.sh/nFo2G/4aba703b4f.png

With me so far?

Now, what if we rotated the box 90 degrees around its width or height axis? The video length would become the new video height or width respectively and vice-versa. You would be looking at a collection of every frame's side pixel columns, and watching the video would move you along the original video frame pixel lines (sorry if my wording is confusing).
http://puu.sh/nFp3W/77a43c33a2.jpg

In order to implement this I wrote the following script (Avisynth+)

AVISource(“C:\source.avi”)
converttorgb32

original = last
last.blankclip(length = original.width, width = original.framecount, height = original.height)

scriptclip("""

for ( i= 0, original.framecount - 1) {
selected = original.freezeframe(0, original.framecount - 1, i)
layer(selected.crop(original.width - 1 - current_frame, 0, 1, 0), x = i)
}

""")
(this one transposes video length to width).

You can see how the video looks like after such transformation here: https://www.youtube.com/watch?v=Ut00lonRqVk (left to right is beginning to end of the original)
Here's also the same method used to transpose length to height: https://www.youtube.com/watch?v=-K5b2BKI0ew
Compare to the original video: https://www.youtube.com/watch?v=-YKuxZgDamM (mine was trimmed to speed up processing).

The catch is though that despite the simplicity of the script it runs extremely slowly (originally at 0.2 fps for a 672x384 video, later sped up to ~1.5-3 fps by replacing overlay with layer and switching from Avisynth to AVS+). Speed is also inversely proportional to the original video length, because in order to construct one frame of the output the script has to access every frame of the input.

Interesting thing is, this conversion is reversible, you can get the original video back by applying the transformation again (although it will be reversed and mirrored, but those filters are lossless as well).
This opens such possibilities as applying a spatial only filter to the temporal domain and vice-versa. I already tried upsampling FPS with a neural network upscaler as well as upscaling with MFlowFPS. Results weren't that much impressive, although in the first case it was curious to notice that parts of the frame were moving independently of each other.

Another point to make is how switching dimensions, temporal with spatial, lets you see the whole timeline of the video at once, even though only for a column of pixels at the moment. To remove this limitation we need to add up a dimension instead of replacing one with another - to go 3D.

I took the already transposed width-to-length video and processed it with another script, overlaying each frame with a diagonal shift to achieve 3D looks.
AviSource("D:\width-transposed.avi").converttorgb32.trim(0, 639)

a = last
a.blankclip(source_framecount, a.width + 640, a.height + 640/2, "RGB32", color=$888888)
scriptclip("""
for (i = 0, 639) {
frozen = a .freezeframe(0, a.framecount - 1, i).crop(current_frame, 0, 0, 0)
layer(frozen , x = i, y = i / 2, level = round(256.0 * (640 - i) / 640))
}
""")
Now the video actually looks like a 3D box, with the original playing on its left side, while you can see all the frames at once: https://www.youtube.com/watch?v=2vUihdaMbwk
Well, could, you can't see through an object, so I added some transparency to its front edge.

Why am I writing all this? To share my experience and discoveries made thanks to Avisynth and to give my insight on the concept of observing N-dimensional objects from a N+1 dimensional space. Remember 4D tesseract in Interstellar, seeing 3D space in all timeframes at once? Here we see all instances of a 2D video by going 3D.

Since this is more of a "share thoughts" topic than a "get help" one I'm not asking anything by it, but perhaps you can share some cool non-conventional things you did with video by Avisynth too, or maybe give ideas on speeding up such transformations.

[Sparktank]

i knew looking in this thread would keep me awake. :)

you got me with the right keywords: z-axis, transpose, tesseract, Interstellar, 3D, the!

my little folder of art grows every year.

Really great stuff to play with.

perhaps you can share some cool non-conventional things you did with video by Avisynth too

Well, not by me, but I did ask so things can get rolling and a lot of fun stuff did happen.
Usually it starts out with someone doing something on youtube where other software was used first.

Your tesseract project reminds me of a version of the "video fingerprint" I wanted to replicate.
http://forum.doom9.org/showthread.php?t=167769
I need to really sit down one day and sort out examples and update the first post with the different scripts and plugins, coplete with examples.

Some weird video thing I forgot I asked about. :lol:
http://forum.doom9.org/showthread.php?t=172475
Layering with only maxY.

There's still plenty of other things out here, I have subscribed to.
Some fun with Spline100 mod, iirc.

I am definitely going to have to spend some time with this over teh weekend. :D

[junh1024]

If you want to increase the FPS, install SVP for avs, use

super=SVSuper("{gpu:0}")
vectors=SVAnalyse(super, "{}")
SVSmoothFps(super, vectors, "{ algo: 23 }", url="www.svp-team.com", mt=2)

and be done with it. Better than Interframe, Mflow, and that russian one.

[Sparktank]

Yeah, I thought about adding SVp to it. I really like how they put the script together for you.
You can customize with InterFrame, same libraries. But you have to know how first.

Right now, I'm looking at converting a scene from Interstellar to a bunch of PNG's.
Crop the borders for scenes in 1920x804 (2.388:1) WMOD=32 HMOD=4.
Then add a transparent/alpha border to those frames.
Keep the IMAX scenes intact.
Run it through something like ffmpeg to concat all the images, maintain alpha transparency, and make a lossless RGB32 intermediate file to run through this.

It would be nice to see the side view of the Interstellar clip. Showing absolute changes in height for different aspect ratios present in the bluray.
Give an excellent illustration. But the only way I can see that working is to show transparency for the non-IMAX frames instead of black or colored bars.
The black bars would look like it's just part of a very dark IMAX scene, as there are a lot of dark frames/scenes.

I could pick a flashy magenta or neon green that isn't present anywhere in the film as a last resort.

I managed to get ImageMagick to create transparent borders.
https://i.imgur.com/02MoE9Tl.png (https://imgur.com/02MoE9T)
# South, image moves south to create transparent border for TOP=138
"C:\Apps\Image\ImageMagick\convert.exe" "Interstellar RGB32 [278].png" -background none -gravity south -extent 1920x942
\ "Alpha-south [278].png"

Borders should be:
a) gone
b) black or white or grey (depending "what" is viewing it.)
Saving it and looking at it differently will show you the borders indeed transparent.

In XnView, I made the backgrounds all green.
Then when I select/highlight the pictures, I can definitely confirm transparent borders.
la la:
http://i.imgur.com/R1YDDy2.png (http://imgur.com/R1YDDy2)

So somehow...
take all the cropped with alpha borders....mix with full framed IMAX...turn it on its side and make a pretty wallpaper...
or something to that effect.

It would actually be great for making custom soundtrack.
Take clips from the bluray, work it up like "tesseract" as a video to match time length of a track from the OST and then start listening on MPC-HC instead of Foobar2000.
This would be a hit at any gatherings or parties. :laugh: If anyone at them listened to soundtracks, that is.

[Seedmanc]

Sparktank, yeah I remember that thread about video footprint, I recall I was trying to do a similar thing in runtime, but it ground to a halt as it progressed further into video, haha. As for the lighter color thing with SW, does it have any practical uses, or it's just an effect? I reckon, you can't recover two streams once you mixed them this way, or can you? I wonder if there was a way to mux several video streams into one without increasing size or framecount, with the ability to demux them after compression (perhaps lossily). Increasing density, so to say.
nh1024, I need to increase speed of my scripts, this has nothing to do with frame interpolation. If only there was a way to make Avisynth store entire videostream in memory, perhaps that would have made it faster.

Speaking about colors and dimensions, I remember doing sort of image recognition through combination of masks. I was able to detect eyes on the picture with a good enough success rate.
In short, I profiled (http://40.media.tumblr.com/1424161e3e6dc68407bcbec11b7a0e29/tumblr_inline_npbhkdxMul1t2vg9v_1280.png) the eye (http://40.media.tumblr.com/55b172a77437815d76d6cc966efce7f9/tumblr_inline_npbhjazPCe1t2vg9v_1280.jpg) image, then limited first in the brightness (http://41.media.tumblr.com/e329780b85e3f7ddb2df98a05b14b56f/tumblr_inline_npbhphdEHE1t2vg9v_1280.png) domain with mt_binarize, then in the frequency (http://41.media.tumblr.com/c76927436b7d36cd1c97f35a6174a637/tumblr_inline_npbhu2uksE1t2vg9v_1280.png) one with mt_edge and finally in colors (http://40.media.tumblr.com/10c8228784b6053dc109536b93a9d9d9/tumblr_inline_npbiad4Sv51t2vg9v_1280.jpg) with MaskHS.
Combined masks provided decent detection rate:
http://41.media.tumblr.com/bcd1651f79f18d86aad516a38104baf3/tumblr_inline_npbiepHlMf1t2vg9v_500.jpghttp://41.media.tumblr.com/5db2c8df170d9fd17341f17f200ade8b/tumblr_inline_npbih7QRdk1t2vg9v_500.jpghttp://41.media.tumblr.com/1619b0f15ab282d2aedcba098ca901cb/tumblr_inline_npbihqOi2F1t2vg9v_500.jpg
if not for some false positives. Still I think it was pretty cool for just a bunch of masks in Avisynth, given how you would usually require neural networks for image recognition.
Full article (http://seedmanc.tumblr.com/post/120523659350/speaking-about-xanga-and-encoding-heres-that-one) with scripts, if you're interested.

[raffriff42]

Stuff like the Eadweard Muybridge motion studies (https://en.wikipedia.org/wiki/Eadweard_Muybridge#Later_motion_studies) might look cool in 4D, if you could see through the frames to get the "human centipede" effect.

[Sparktank]

or it's just an effect?

That's entirely for effect. If you encode it like that, it's all pretty much what you see.

A friend of mine started streaming to twitch her gameplays. So at one point I told her to stare at something for 5 seconds without moving. Haha. I just need to dig up a war movie now and plaster the two together. :D

I just need to catch a few streams of her doing insane things before making the video.

[Seedmanc]

The other time I decided to put a test to Avisynth's runtime capabilities and made a script that analyzed a gameplay recording, recognizing certain situations and generating a subtitle template with timestamps, character names and screenshots so that the translator could later just fill it with text. That was one massive script, however it ran at around 450 fps for 720p, bottleneck being the decoder I think. Script's still here http://pastebin.com/rtJi5cD9

[Seedmanc]

I was messing around trying to improve my script that overlays all frames of a video to achieve 3D look, when I realized that this is basically the inverse of how MRI scanners work.

So I went to youtube, got myself an MRI sequence (https://www.youtube.com/watch?v=Pj17eBYS7s8) video and built a volumeric body model using its frames: (NSFW-ish image (http://i.imgur.com/VYEQihg.jpg))

The script is as follows, you don't have to export the whole video to still images as an intermediate step anymore, although it would have improved the responsiveness.

dss2("E:\01 MRI Scan Male Body (480p_30fps_H264-128kbit_AAC).mp4").trim(0,1500)

converttorgb32
last.blankclip(height=round(last.height*1.9)).overlay(last)
turnright
skewrows(1) #rotate the model to achieve isometric 3D look
turnleft
crop(0,height-width,0,0)
flipvertical


m= converttoyv12.mt_binarize(76) # build a mask to remove the background
m=mt_logic(m,converttoyv24.maskhs(170,100,coring=false) .converttoyv12 .mt_inpand ,mode="min").converttoy8
mask(last, m.converttorgb32) # put it to the alpha channel

original=last

last.BlankClip(width=last.framecount-1+last.width, color=$888888)

scriptclip("""
for (i = original.framecount-1, current_frame, -1) {
frozen = original.freezeframe(0, original.framecount - 1, i)
layer(frozen , x = i,level=128)
}
""")
crop(300,0,0,0)

By playing with the mask radius and layering opacity you can even see the underlaying tissues behind the skin, kinda like in that scene in Hollow Man.
Unfortunately it takes a good minute for the script to reload and it renders at a fraction of FPS, so the interactivity is rather limited. If only there was a way to massively accelerate this processing, it would have became a very useful tool.

[wonkey_monkey]

I'm not sure whether you'll really care to know, but that video wasn't made from an MRI scan. If you're currently eating and/or are squeamish, you may not wish to continue reading. You have been warned!

A donated cadaver was frozen in ice/gelatin, then images were taken. Between each image, a 1mm slice was removed from the ice. All those blue flashes are, I believe, cracks and other things in the ice.

It's called the Visible Human (https://en.wikipedia.org/wiki/Visible_Human_Project) project.

I'd guess the problem with your script is that, each time, it's looping from frame 0 to (current_frame). I may have an experimental plugin that can greatly improve this.

[Seedmanc]

Yeah right I kinda saw that in a description of a similar video but that eluded me. Should have taken notice however considering that the images were colored, something an MRI can't do.
Regardless, the idea is the same, as MRI can produce similar kind of images minus the coloring.

[wonkey_monkey]

I got a scriptclip error when I tried your script (line 2, column 60), but in any case I can't figure out where, in your clip, the frames are shifted along to make the "stack". Hint?

[Seedmanc]

davidhorman, this is a script for Avisynth+, which has the "for" construction incorporated instead of being separate in GScript. You can probably replicate the functionality by installing GScript and putting it inside of ScriptClip.

As for stacking, that's what the For script is, well, for. For every requested output frame it accesses all frames of the source starting with the end of the source clip and up to the requested frame number. Then it overlays every frame with a shift equal to the iterator (the frame number in the original).

Thus the exceptionally low fps, since to construct a single frame it needs to process all frames. It gets faster close to the end though.

[Gavino]

...

original=last

last.BlankClip(width=last.framecount-1+last.width, color=$888888)

scriptclip("""
for (i = original.framecount-1, current_frame, -1) {
frozen = original.freezeframe(0, original.framecount - 1, i)
layer(frozen , x = i,level=128)
}
""")
crop(300,0,0,0)

...
If only there was a way to massively accelerate this processing, it would have became a very useful tool.
A variant of the trick from the 2009 thread "Sticking" overlay might help.
The idea is to reuse each partial stack in the construction of the next rather than building each one from scratch.
For this to work, you will have to generate the clip frames in reverse order, so run the following script, save the result and either play it backwards (if possible) or run another reverse script to generate the final result.

For the script, replace your scriptclip section with the following code: (warning - untested!)
original = original.Reverse()
stack = last # the blank clip
ScriptClip("""
frozen = original.freezeframe(0, original.framecount - 1, current_frame)
stack = layer(stack.SelectEvery(-1), frozen, x=current_frame, level=128)
return stack
""")

The performance gain comes from the expectation that at each frame, the previously generated frame (obtained via SelectEvery(-1)) will be preserved in the Avisynth cache.

The frames have to be generated in strict linear order, so it probably needs to be run single-threaded.

[wonkey_monkey]

Dammit, Gavino, ruining my big reveal...

...because the plugin I've been developing has just such a "sticking overlay" thing built-in. As with the above, it means the script as to be run strictly in frame order to give the correct result (although I might make this optional as I've now discovered it can lead to interesting results if misused in just the right way).

I'll post the script and the (beta) plugin if anyone's interested, but I'll leave it out for now because it's long and complicated, and so is the plugin, for which I haven't finished the documentation. I also have no idea if it's compatible with Avisynth+.

Here, though, is the result (source was https://www.youtube.com/watch?v=dPPjUtiAGYs)

https://youtu.be/ZkkUNFXaYyk

It runs at about 20fps on my computer (i7-2720QM, 2.20GHz Quad Core). He came out a bit green because I didn't bother inpanding the mask.

[poisondeathray]

...because the plugin I've been developing has just such a "sticking overlay" thing built-in. As with the above, it means the script as to be run strictly in frame order to give the correct result (although I might make this optional as I've now discovered it can lead to interesting results if misused in just the right way).

I'll post the script and the (beta) plugin if anyone's interested, but I'll leave it out for now because it's long and complicated, and so is the plugin, for which I haven't finished the documentation. I also have no idea if it's compatible with Avisynth+.

Here, though, is the result (source was https://www.youtube.com/watch?v=dPPjUtiAGYs)

https://youtu.be/ZkkUNFXaYyk

It runs at about 20fps on my computer (i7-2720QM, 2.20GHz Quad Core). He came out a bit green because I didn't bother inpanding the mask.


That's is pure awesomeness!!!! :D


I'd love if you posted the plugin

[Sparktank]

I'll post the script and the (beta) plugin if anyone's interested,

Yes, yes, yes, please! :goodpost::thanks::devil:

[smok3]

Would that be bunch of 2d planes with pictures mapped to them? (I have some blender skilz)

[Seedmanc]

Gavino, this is most impressive. I've been thinking about reusing the previous frame, but I could never figure out how. The SelectEvery(-1) is some black magick, it doesn't even work on its own alone (crashes Avspmod), only in this script. Interesting that both in the editor and in Avsmeter the framerate is highly erratic and often hang up, but when actually encoding the result it's a smooth fast ride.

smok3 I wish I had experience in 3D editor. My actual goal after all these experiments in Avisynth is to be able to create a full 3D model out of all video frames to be able to rotate and look through it freely, unhindered by a video editor which is not even meant to work with 3D.

[StainlessS]

The SelectEvery(-1) is some black magick, it doesn't even work on its own alone (crashes Avspmod)

I think the SelectEvery(-1) is mistake, should be SelectEvery(1,-1)
Have not tried that script.

But this works as expected [with the (-1) I get 1 black frame].

Colorbars.KillAudio.ShowFrameNumber
SelectEvery(1,-1)
SSS="""
subtitle(String(current_frame)+"]")
"""
ScriptClip(SSS)
return last


EDIT: From docs
SelectEvery (clip, int step_size, int offset1 [, int offset2 [, ...]])
Both step_size and offset1 are non optional.

EDIT: This also works as expected

Colorbars.KillAudio.ShowFrameNumber
SSS="""
SelectEvery(1,-1)
subtitle(String(current_frame)+"]")
"""
ScriptClip(SSS)
return last

but without the 1,-1, fails in Scriptclip.
[Shows static ShowFrameNumber as 107891 on entire clip of 107892 frames, ie last frame of clip]
Is it supposed to do that ?, Maybe it is black magic :eek:

EDIT: Checked source args string, "SelectEvery " = "cii*"
Actually 1 compulsory clip, 1 compulsory int, and zero or more optional int, so maybe it is truly dark magic by the dark lord Gavino :)
EDIT: So the docs are strictly speaking incorrect compared to implementation.

[Gavino]

I think the SelectEvery(-1) is mistake, should be SelectEvery(1,-1)
Yes, my mistake! (told you it was untested :D)

The script in the old thread had Loop(2,0,0), but I thought SelectEvery(1,-1) was a better way of expressing it.
Unfortunately, I wrote it wrong. :(

From docs

Both step_size and offset1 are non optional.

Checked source args string, "SelectEvery " = "cii*"
Actually 1 compulsory clip, 1 compulsory int, and zero or more optional int, so maybe it is truly dark magic by the dark lord Gavino :)
So the docs are strictly speaking incorrect compared to implementation.
If offset1 is omitted, it defaults to zero.
So SelectEvery(-1) is equivalent to SelectEvery(-1, 0), which leads to a divide-by-zero in the filter constructor.
Maybe it should check for rogue input parameters and insist step_size > 0.

[wonkey_monkey]

Okay, here's the beta plugin: pixel_rpn (http://horman.net/avisynth/download/pixel_rpn.zip). It is x86 only!
And you'll also need xyremap (http://horman.net/avisynth/download/xyremap0.5beta.zip) (this is the fast beta version, the old version may be okay)

And here is the very long script:

dgsource("vis.dgi") # ffmpeg source will hopefully work

rgb=converttorgb24

################################################################################
################################################################################
## first call to pixel_rpn: fix shifts and broken frames
################################################################################
################################################################################

rgb=rgb.pixel_rpn("
$rgb:

################################################################################
# first replace broken frames with blends of their good neighbours
################################################################################
n 505 == ?: [c0(-1,6,0)] 0.75 * [c0(3,6,0)] 0.25 * + :
n 506 == ?: [c0(-2,6,0)] 0.5 * [c0(2,6,0)] 0.5 * + :
n 507 == ?: [c0(-3,6,0)] 0.25 * [c0(1,6,0)] 0.75 * + :
n 1015 == ?: [c0(-1,9,0)] 0.89 * [c0(8,9,0)] 0.11 * + :
n 1016 == ?: [c0(-2,9,0)] 0.78 * [c0(7,9,0)] 0.22 * + :
n 1017 == ?: [c0(-3,9,0)] 0.67 * [c0(6,9,0)] 0.33 * + :
n 1018 == ?: [c0(-4,9,0)] 0.56 * [c0(5,9,0)] 0.44 * + :
n 1019 == ?: [c0(-5,9,0)] 0.44 * [c0(4,9,0)] 0.56 * + :
n 1020 == ?: [c0(-6,9,0)] 0.33 * [c0(3,9,0)] 0.67 * + :
n 1021 == ?: [c0(-7,9,0)] 0.22 * [c0(2,9,0)] 0.78 * + :
n 1022 == ?: [c0(-8,9,0)] 0.11 * [c0(1,9,0)] 0.89 * + :
n 1422 == ?: [c0(-1,9,0)] 0.67 * [c0(2,9,0)] 0.33 * + :
n 1423 == ?: [c0(-2,9,0)] 0.33 * [c0(1,9,0)] 0.67 * + :

################################################################################
# fix horizontal shifts in clip
################################################################################
n 230 <= ?: [c0] :
n 252 <= ?: [c0(5,0)] :
n 360 <= ?: [c0(-2,0)] :
n 390 <= ?: [c0(6,0)] :
n 416 <= ?: [c0(-2,0)] :
n 729 <= ?: [c0(6,0)] :
[c0(9,0)]

################################################################################
# the statements above are nested if...then...else structures.
# this next operation endifs them in one go:
################################################################################
. repeat
")

################################################################################
################################################################################
## second call to pixel_rpn: generate mask and also mask some parts manually
################################################################################
################################################################################

masked=rgb.pixel_rpn("
$rgb:
[c0] # just copy the RGB channels as is

$a:
################################################################################
# what follows are a bunch of conditions which calculate areas to mask
################################################################################

# bottom line
n 1016 <= ?:
590
590 n 550 - 0.4 * +
590 n 1000 - 0.1 * -
590 n 690 - 0.5 * +
720 n 770 - 0.9 * -
min max min max
:
515 n 1813 - 0 max 5 * +
650 min
.

y <=

# top line
150 n 73 - 0 max - 45 max
n 1424 >= ?: 18 + .
n 1756 >= ?: 10 + .
y >

# right
1075
n 291 >= ?: 15 + .
n 305 >= ?: 15 + .
n 320 >= ?: 20 + .
n 354 >= ?: 20 + .
n 421 >= ?: 15 + .
n 649 >= ?: 15 - .
n 675 >= ?: 20 - .
n 708 >= ?: 30 - .
n 775 >= ?: 50 - .

x <

# left
95
n 1430 >= ?: 80 + .
x >

# corner
x 1100 >
y 100 <
*

# corners
n 430 >=
n 506 <
x y 1.5 * - -390 <
* *

x y - -300 <

# another corner
n 908 >=
n 933 <
x 960 >
y 460 >
* * *

# foot bit
n 1755 ==
x y - 567 <
x 640 >
*
y 100 <
+ *

n 1753 ==
y 100 <
*

# post-it
n 759 >=
y 600 >
n 773 <
* *

# more bits
n 1424 >=
y 155 x 0.25 * - <
*

n 1511 >=
y 80 <
*

n 1558 >=
n 1567 <
y x 1.25 * + 734 >
y x 0.66 * - -310 >
y x 1.25 * + 1485 >
+ * * *

n 1847 >=
x 500 >
y 465 >
x 915 <
* * *

n 1865 >=
x 500 >
*

n 1866 >=
x 300 >
*

################################################################################
# all the above conditions are added (acts as a logical or)
################################################################################
+ repeat

?:
0 # if true (non-zero), output 0
:
[b0] [g0] [r0] min < # if not mostly blue...
# [g0] 2 * [r0] > * # or mostly red... (uncomment this to remove red tissue)
?:
1 # then solid
:
0 # else clear
.
.
")

################################################################################
# the following call generates a clip with an overlaid mask for debugging
################################################################################

debug=masked.pixel_rpn("
$g: [a0] 255 *
$rb: [c0]
")

#return debug

################################################################################
# applies a 5x5 gaussian blur to the mask, and premultiplies RGB by alpha
################################################################################

masked=pixel_rpn(masked,"
$a:
[a0(-2,-2)] [a0(-1,-2)] dup + dup + [a0(0,-2)] dup + dup + [a0(1,-2)] dup + dup + [a0(2,-2)] + + + +
[a0(-2,-1)] [a0(-1,-1)] dup + dup + [a0(0,-1)] dup dup + + dup + [a0(1,-1)] dup + dup + [a0(2,-1)] + + + + dup + dup +
[a0(-2, 0)] [a0(-1, 0)] dup + dup + [a0(0, 0)] dup dup + + dup + [a0(1, 0)] dup + dup + [a0(2, 0)] + + + + dup dup + + dup +
[a0(-2, 1)] [a0(-1, 1)] dup + dup + [a0(0, 1)] dup dup + + dup + [a0(1, 1)] dup + dup + [a0(2, 1)] + + + + dup + dup +
[a0(-2, 2)] [a0(-1, 2)] dup + dup + [a0(0, 2)] dup + dup + [a0(1, 2)] dup + dup + [a0(2, 2)] + + + +
+ repeat
dup 0.00390625 * @A^ # store normalised alpha for premultiplication
$rgb:
[c0] A * # premultiply by alpha (this only works because pixel_rpn calculates in the order ABGR, allowing us to set A and use it here)
")

################################################################################
# increase to full HD resolution and remap to a moving isometric projection
################################################################################

isometric=masked.addborders(320,180,320,180).xyremap( \
x=" x 1080 - t 1500 * + 2 t 0.0 * - * 0.666666 * ", \
y="h y 1150 + t 1500 * - 2 t 0.0 * - * x - 0.666666 * -" \
).trim(1,0) # trim first frame, which is a dupe

################################################################################
# render as a stack, using pixel_rpn's [<.c] trick to copy the previous frame
################################################################################

stack=isometric.pixel_rpn("
$settings:
frames:2000 # pad out to 2000 frames
offset:25 # add 25 frames to the start

$rgb:
[c0] [a0] 0.0039215686274509803921568627451 * 0.5 * @A *
# the '0.5 *' in the line above renders at half opacity for a nicer result - try 0.1 and 0.01 for different transparencies
[<.c] 1 A - * +
")

################################################################################
# because we rendered the stack with half opacity slices,
# re-overlay full opacity slices for better animation.
# if you remove the "0.5 *" above, you can skip this call
################################################################################

pixel_rpn(stack,isometric,"
$rgb:
[a1(-25)] 0.0039215686274509803921568627451 * @A [c1(-25)] * # the (-25) offsets the offset added above, putting stack and isometric back in sync
1 A - [c0] *
+
")

converttoyv12 # convert to yv12 for encoding
assumefps(60)

I've annotated it as best I can, but I know it's not very readable. Any questions, let me know.

[wonkey_monkey]

And a semi-transparent visualisation from the same script:

http://i.imgur.com/Jt3U069.jpg

[StainlessS]

Any questions, let me know.

What is all that ? :)

EDIT: WOW, impressive image.

EDIT: @DavidHorman, rpn.html, typo, Maximmum, max x y min, Returns the highest of x or y

[poisondeathray]

thanks david

Looking at the script, it's way above my programming knowledge but I'll play with the script/plugins and try to figure it out .



maybe I'll fiddle with some 3d approaches as smok3 suggested too . But mapping to 2d planes would still need the planes to be extruded at some point in the workflow for geometry

[wonkey_monkey]

It's curious how the end result appears to be shaded, despite there being no such shading in the source images. I can't quite figure out the maths behind that. Maybe it's something to do with many semi-transparent dark pixels being stacked in the same place.

[Seedmanc]

davidhorman, yes, that's exactly what it is. Gives the depth impression. However it's hard to tune, sometimes even giving the smallest possible weight to the background instead of removing it entirely still results in all-dark picture because many frames add up.

[wonkey_monkey]

EDIT: @DavidHorman, rpn.html, typo, Maximmum, max x y min, Returns the highest of x or y

What's funny is that you spotted the spelling error, but didn't spot that I actually put "min" in the example code :D

[StainlessS]

but didn't spot that I actually put "min" in the example code

Yeh, it was a test to see if you noticed once I had drawn attention to that line.

I'm getting a bit sick of repeatedly viewing the output clip, tis addictively fascinating.
At about the belly button, you can see sort of structures inside the larger cavities. (rendered as supplied script).

Those plugz of yours are quite fantastic, you truly are the Doctor/Surgeon.

EDIT: Structures
https://s20.postimg.cc/xcn153sl9/structures_zpsg2mpn3kn.jpg (https://postimg.cc/image/6et43d7y1/)

EDIT: Just using Avs standard here.
EDIT: Is there an anatomist in the house?
Pieces of the jigsaw, here:- https://www.google.com/search?q=organs&biw=1280&bih=839&tbm=isch&imgil=kOMsBd_TZ30FSM%253A%253ByulQqrdreMtU4M%253Bhttp%25253A%25252F%25252Fwww.oxfordlearnersdictionaries.com%25252Fdefinition%25252Fenglish%25252Forgan&source=iu&pf=m&fir=kOMsBd_TZ30FSM%253A%252CyulQqrdreMtU4M%252C_&usg=__guT19hYbVq3BZ5OtLKXFK1ckhJQ%3D&ved=0ahUKEwjgsOuDubHRAhXF1ywKHVZLCaoQyjcIUw&ei=YpxxWOD3EcWvswHWlqXQCg#imgrc=kOMsBd_TZ30FSM%3A

EDIT: The shiny structure (not shown above) at high end of marked cavity is I think the Liver, blue marked structures may be somewhere about the Gall Bladder.

[wonkey_monkey]

Is there an anatomist in the house?

Are you using Avisynth+? It'd be good to know my plugin works with it. It's internally multi-threaded, and I wasn't sure how well it would work. It needs more work though; this script only gave me 60% CPU utilisation. I think I can do better than that.

[poisondeathray]

Did some googling today and there are a bunch of open source (and some paid programs) that do image stacking and 3d model construction , some are medically oriented e.g/ slicer4, but some are versatile generic tools that can be used in many different ways eg. ImageJ / fiji . Some of them are just 2d to 3d viewers but others allow exporting 3d formats such as obj , stl . Some are geared towards 3d printing, but once you have a 3d mesh you can do lots of interesting manipulations in a 3d program such as blender . I got some fun ideas for next halloween

[StainlessS]

Info Update, from YouTube source poster, comments section [ https://www.youtube.com/watch?v=dPPjUtiAGYs ]
@zuhra911
3 years ago
Hi,
I’m a copyright producer on RT, Anastasia.
Are you the owner of this video?


And a reply to above
@arseniyzefirov7765
3 years ago
Hi, I created this channel in 2009. I don't have access to it now. I have compiled this video from separate images from the Visible Human Project CD. I also added an audio track from game "Land of the dead - Road to fiddler's green". As of 2019, a license is no longer required to access the VHP datasets of Visible Human Project

EDIT: A later instance of similar source images is available from VHP, is of adult female @ higher resolution and finer slice data.