fauxD: 2D to stereo-3D conversion in real-time

[eslave]

Wow, it feels like I've been on the road for over a week...oh that's right, I have. The 'complimentary' wi-fi at most hotels seems more like an insult.

Jeff,

I also couldn't figure-out what was wrong at first, but I think I may know now. Also, if that's not a business phone number I'd remove it immediately.

• Please avoid using "fauxD_for_testers.AVS" unless you really like getting your hands dirty. The current version makes things much easier, IMHO. The old version is provided for historical and comparative purposes and I'd rather not address any of its numerous issues.
• You should have MVTools v2.3.1 (or newer) named "mvtools.dll" (or whatever your LoadPlugin line says in 'fauxD.AVSI'). Oops! The name of the mvtools DLL as provided by Fizick is actually 'mvtools2.dll' (My error, please adjust accordingly. See next 2 posts).
• By default FFDshow will not intercept the directshow filter-chain for MPEG1. Check the "Codecs" configuration tab in FFDshow.
• Don't play "fauxD_ffdshowsample.AVS" directly. Just paste its lines into FFDshow (or Import the AVS if that's easier for you) and then play the video, not the AVS file. You were trying to create 2 instances of fauxD in the filter-chain. One by playing the AVS, and another within FFDshow. I imagine that's why you saw only 2 flat frames before it crashed. I have no idea how memory is shared with in-place filtering under such conditions, but its probably bad. I never foresaw a situation where this would be necessary and took no precautions for it.
• The dialog was primarily for debugging purposes and I had ultimately intended to remove it, but some really wanted to see it. All the text fields are read-only. Shift and Skew were never fully implemented in the code so I removed the useless slider controls. I think you may be referring to this obsolete image regarding a temporary test-script which was inadvertently quoted.

Please check these items against a freshly configured copy of fauxD.

While testing your sample I noticed the video is 30fps (according to GSpot), presumably a 24fps pullup(down?) as every 5th frame or so appears to be a duplicate (frequently results in a rather distracting flat image). You may want to try a basic pulldown (see your AVIsynth docs) or possibly Restore24 (I've never tried this), or just use the DVD source if you have it. (Enable "Raw Video" in the 'Codecs' tab of FFDshow. Somewhat slower in my experience, and you'd probably need to deinterlace it.) Being a movie trailer the cuts are rapid and intrusive, many of the panning sequences are too fast (eye-strain inducing), and the still scenes are - well - very still (I'm guessing that's audience psychology for intensifying the perception of drama with the alternating high-action, ho-hum sequences).

Here are some of my results: See the FFDshow dialogs in the last image. Ignore MPC's reported frame-rates as I was pausing and frame-stepping.

fauxD_sample.AVS (simplest form)
Drag-N-Drop/SendTo (fauxD.CMD + fauxD_batchsample.AVS)
FFDshow The upper FFDshow dialog is from another screen-shot.

Thanks to everyone who attempted to field this question as all the replies I saw were generally correct, but again this seems to have been a configuration issue (of sorts). I wish I could have responded sooner.

-eslave

edit:
I probably should have mentioned this months ago, but you need to disable MPC's internal Transform Filters for MPEG or FFDshow probably won't be used for those codecs.

[Fizick]

MVTools v2.3.1 (or newer) is named "mvtools2.dll"

[eslave]

LOL! My mistake!

I've been working with a renamed copy for so long that I had completely forgotten. Thanks for catching that, Fizick! I even have it mis-named in the current release of fauxD.AVSI. I guess thats one more thing I'll need to fix for next release.

It looks like I'm living in the past again as well. I've just noticed your update (v2.4.2) and I'm looking forward to testing it when I have some genuine spare time.

-eslave

[travolter]

I was testing fauxd and works great!!!

I hope more updates soon.. 3D is a nice boost for make low quality vids more interesting

anyone know more projects like fauxd?

I know about 3dfier.. and I dont know how good compared with faudx.. there is a demo version available here: http://3dfier.blogspot.com/

[wonkey_monkey]

To save me downloading a slew of plugins and fiddling with the script, and also because I don't have any anaglyph glasses, can anyone explain, roughly, what this script is doing to create the faux-3D effect?

David

[AnnaFan777]

any chance of a 3d full color version ? (for shutter glasses)
doesn't have to be realtime.

[rrrrob]

Here's some cool converted 3D clips from my3d2u:

http://www.youtube.com/user/3dgeek2009

[freemanvn]

Perfect movie,can you to send for me this script?
I try test the file fauxD_for_testers in FauxD_v012.zip but it's bad,movie no depth...please guide me with...thank you
(sorry my bad english)

[wonkey_monkey]

Originally Posted by rrrrob

Here's some cool converted 3D clips from my3d2u:

http://www.youtube.com/user/3dgeek2009

Those aren't converted - they were always 3D.

I'd very surprised if a script could achieve those results!

David

[eslave]

Hmm, I guess I shouldn't be too surprised about the source of this recent resurgence in 3D interest. Alas, I still have not made any code improvements since v0.12.

travolter,
Thank you for your kind remarks! Making 2D videos 'more interesting' was my intent with fauxD. Surprisingly enough I also hadn't heard of 3Dfier until bumping into these posts a few months ago. If I were to hazard a guess (judging purely from 3Dfier's filesize and speed) I would surmise that 3Dfier uses the same pulfrich technique as fauxD, possibly with some additional shifting or skewing. I haven't yet taken the time to perform a rigorous frame-by-frame comparison or disassembly, though I see strings like "ObjectDepth/Prop", "PulfrichOn/Prop", "TrapezoidEffect" and a lot of vector messages probably related to feature-matching/tracking for determining global motion (pan direction) or possibly some sort of Tri-Def-like mathematical wizardry.


davidhorman,
I don't like downloading plugins either, but fauxD's only dependency is mvtools2. There isn't really much script fiddling required except the two LoadPlugin lines and maybe changing your 3D frame-type. There are three sample scripts included in the archive, and this post has a pretty good installation explanation as well. The fake-3D effect is called the pulfrich effect/illusion and my implementation is explained with greater detail in the documentation as well as in these posts with morsa. I was also mildly impressed by rrrrob's clips until I realized what they were. You could probably get results close to that with fauxD and its unused shift/skew functions, but it would almost certainly require painstaking effort on a per-scene basis. Maybe if I ever get around to incorporating OpenCV or some kind of horopter manipulation into fauxD then it might produce results like that without user intervention, albeit at tremendous CPU cost.


AnnaFan777,
Waaay ahead of you, buddy. Change the stereo_mode parameter in your fauxDit call for interlaced shutter-glasses. (Whether you specify true or false the frames will be interleaved with the AVSI script, not with the plugin.)

Import("D:\somepath\fauxD.avsi")
fauxDit(someclip, 3, 40, false)

My "logic" was to double the frame height so that each eye will have a full-resolution view (instead of the effective vertical resolution being halved). You can horizontally scale the final output to correct its aspect ratio, but do not vertically scale the final output or the stereo effect will be ruined. This worked fine for my shutter-glasses in interleaved/interlaced mode when I had a CRT monitor. I have never seen my LCD shutter-glasses work in either the "Page-Flip" or "Over/Under" modes. Unfortunately, my new LCD monitor is too slow (cheap) to work properly with shutter-glasses and I haven't experimented with them for quite some time now.


freemanvn,
Sample scripts are provided in the archive (like this, see more here). Please DO NOT use 'fauxD_for_testers.avs'. Including it in the archive has proven to be a mistake as it only seems to add to the confusion. It was only included for comparison with the current version and for anyone needing a 'top-down' perspective of the process before it was broken-up into functions. After configuring the LoadPlugin lines in fauxD.avsi for your system a two line script should suffice:

Import("D:\somepath\fauxD.avsi")
fauxDit(DirectShowSource("D:\somepath\somefile.ext"), 0, 40, true)

-eslave

[freemanvn]

-eslave
thank you so much

[freemanvn]

I think Red Cyan effective than,script of Yellow-Blue no apparent effect(almost never see the 3D effect)
Can you update the new version of FauxD(full script)with new features such as:
- Yellow(Amber) Blue
- Magenta Green
ect...
once again,thank you much

[freemanvn]

Mr.-eslave!can you help me now?
How make a script with Green-Magenta glasses?
Thank you first...

[eslave]

Certainly! Using the same process we can easily create fauxD anaglyph functions for any combination of primary colors.
(Sorry for the delay. I've been juggling terabytes of data around and re-shuffling OS partitions once again. It's been several weeks since I've had the time to boot into my 'OldDev' partition to play with this.)

Copy only the blue lines into fauxD.avsi as shown:

 ...function AnaglyphRedCyan(){}...
#///////////////////////////////////////////////////////////////////
# Create GM anaglyph from 2 clips. Used as a 'mode' function by FauxD()
function AnaglyphGreenMagenta(clip left, clip right)
{
# GREEN filter over left eye, MAGENTA (RB) filter over right eye (for Journey, Monsters v Aliens, Coraline ?)
# http://nzphoto.tripod.com/sterea/anaglyphs.htm      "magenta / green - produces bad ghosts on computer screens."

#CopyChannel(MAGENTAtarget, GREENsource)
return CopyChannel(left, right, 1)      #use plugin (should be faster?), green is channel offset 1 (BGRA)

return  MergeRGB(left.ShowRed, right.ShowGreen, left.ShowBlue)      #AVS method
}
 ...function InterleavedShutter(){}...

 ...within function fauxDit()...
global reverse_stereo = false   #for the sake of completeness, affects AVS functions only
global avs_mode = (stereo_mode == 1) ? "OverUnder" : (stereo_mode == 2) ? "SideBySide"  :  (stereo_mode == 3) ? "InterleavedShutter" : (stereo_mode == 4) ? "AnaglyphYellowBlue" : (stereo_mode == 5) ? "AnaglyphGreenMagenta" : "AnaglyphRedCyan"  # convert integer to string (function name) for fauxD/ScriptClip
use_plugin = (stereo_mode == 3) ? false : use_plugin      #'native' shutterglasses are only supported through the AVS function
 ...

If your glasses have Magenta over the LEFT eye then enable reverse-stereo or swap the left/right variable names in the function's parameter list (at the top of the function).

After updating fauxD.avsi you should be able to produce Green/Magenta anaglyphs with an AVS script like this:

Import("fauxD.avsi")
fauxDit(AVISource("somevideo.ext"), 5, 40, false)

-eslave

[lolacroft]

Mr.-eslave!
Is it possible to make a script:
1. Output depth map only (in order to correct it)?
2. Output Left/Right with the new external (corrected) depth map?

Thank you first...

[SkilledAbbot]

Arise from the Grave FauxD!! Any chance there is a 64 bit version of the DLL floating around?