fauxD: 2D to stereo-3D conversion in real-time

[eslave]

Yes, that's right, fauxD will pulfrich your videos into pseudo-stereo-3D in real-time (at most SD and sub/near-SD resolutions). I know there are sophisticated stereo-3D buffs out there just waiting to flame me for offering this, but I think fauxD is far too entertaining to keep to myself any longer.

The sense of 'presence' offered by this technique can be quite impressive under the right conditions. Imagery in the frame needs to be moving horizontally for the best effect (just a few pixels can often be enough). Rapid camera and subject motions will defeat this technique, and so will a stationary camera staring at an unchanging scene.

By default fauxD will produce red-cyan anaglyphs from your videos, but by changing one argument this can easily support other stereo frame-types; such as Over-Under, Side-By-Side, or Row-Interlaced (for shutter-glasses). See the documentation or script comments for more details.

So dust-off those old paper or LCD glasses and enjoy something in 3D!

-eslave

I finally got around to gathering some samples (red-cyan anaglyph only). Admittedly most are 'best-case' images. If they look a little washed-out its because I accidentally left the brightness Tweak() on.




I've been asked if this works with 3D games or in-game videos. This will work with the videos, but only when imagery is moving laterally. If you want to actually play or record your 3D games in stereo-3D then check www.iz3d.com/t-dcdriver.aspx and use FRAPS to record it.

If you need something for still images I've had decent results with 'redblue3D' at sglx.web.ur.ru/converter/ and funny results with make3d.stanford.edu. Bear in mind its been years since I've tried 3DCombine and that I haven't tested every still image conversion program.

v0.12 re-posted 1/15/2009 This time with the correct source-code. That's what I get for rushing.

[setarip_old]

Hi!

Naive question perhaps but, can you convert this to a VirtualDub filter?

[eslave]

setarip_old,

I'm certain a vdub port of fauxD itself is possible, and it should be relatively trivial, but its not on my to-do list this week. I'll look into it more, but I've never explored the vdub API - so don't hold your breath. (I'm not much for developing UI elements either.)

The only foreseeable stumbling block is fauxD's dependency on MVTools. I would need to get motion data from a depan logfile, or by accessing AviSynth+MVTools programmatically, or some other means of informing fauxD as to which way the camera is panning for each frame.

In the meantime fauxD's source is available, and from what I recall vdub/mod will accept AVS scripts, correct? (Maybe you need ffdshow? I just dragged&dropped fauxD.avs onto vdubmod and it opened for me.) Of course, my current plugin doesn't always like random (re-)seeks, being rapidly re-opened, and other things one might be prone to do while editing within vdub.

I'll play with the idea for a few days and see what I can do.

-eslave

Update: I think I'm going to table this idea for a while. FauxD is not stand-alone, nor is it mature enough to be hacked into a vdub filter at this point. Even if I did force it into the form of a vdub filter it would probably crash your editor every 5 minutes - hardly what one would consider a 'contribution to the community'. Similarly, there would be very little point in creating an AviSynth front-end filter for vdub, as vdubmod already supports AVS scripts. I'm not giving up on this idea, its just going to take a different approach than I had originally thought.

[setarip_old]

I'll play with the idea for a few days and see what I can do.

Thank you ;>}

[rfmmars]

Indeed I am very interested in your work, can't wait to play with it.

Richard

photorecall.net

[leeperry]

looks great, thanks!

but how can I use it in ffdshow ?
I've renamed fauxD.avs to .avsi and I've tried fauxD() but ffdshow won't load at all(I've got the latest MVtools and I've copied the content of the archive in the /plugins/ dir)

[eslave]

leeperry,

Please read the html document under "Script Usage"; look for "Input method". You should not need to rename or move any files to use fauxD within ffdshow.

Just un/comment one or two lines in the PROLOGUE section (see the "Method #X" comments), then paste 2 lines into ffdshow's avisynth config dialog (and enable it's checkbox):

import("d:\somepath\fauxD.AVS")  #extension doesn't really matter
return last  #(optional) just in case

After making these changes every file I open in MPC gets processed through fauxD (and anything else enabled in ffdshow's filter chain). Did you install ffdshow with avisynth support? I'm not sure if every build includes that capability, and I think the installer asks you as well.

If you really want both methods to be available at all times (without re-editing) you could make two copies of fauxD.avs - one for ffdshow and another for Drag&Drop/SendTo.

Be careful not to open fauxD.AVS with an ffdshow enabled application when ffdshow's internal avisynth is configured for fauxD. The application would attempt to fauxD() your content twice and either crash, throw an error, or run incredibly slowly.

Let me know if you still have difficulties.

edit1: Sorry, I've just re-read this post (with the benefit of sleep) and realized I should have consulted my reference first as the doc doesn't offer any real info in its present state. I'll need to update it soon.

edit2: New documentation posted 1/4/2009.

-eslave

[leeperry]

so what I've done so far :

-fix the "D:\av\fauxD\" paths in fauxD.avs to my own path
-comment "Import("fname.txt")"
-uncomment "source=last"
-I've got mvtools2.dll v2.3.1.0

I get an error msg that "MVSuper" is unknown(line 74 in fauxD.avs)...so I've taken it out as you said...but then it says "MVAnalyze" is unknown

[mikeytown2]

leeperry, you have mvtools 2 correct?
http://avisynth.org.ru/mvtools/mvtools2.html

edit...
remove the V from MVSuper so its MSuper. Do the same for the rest of the MVTool function calls.

[eslave]

Ok, I've revamped the documentation and made it as clear as I possibly can with respect to the input methods. (There is a picture, too.)

Leeperry, if you're still getting errors with the names of MVTools functions (MVSuper, MVAnalyse) then I think you may have a bad filename or incomplete path specified somewhere, preventing the plugin from loading. To the best of my knowledge those are the correct function names. (I'm looking at the MVToolsV2 doc right now.) I rename my copies of the MVTools DLLs with their version number so I can keep track of what I'm doing. Please double-check the paths and filenames used on the LoadPlugin and Import lines of your script.

You may also want to hit CTRL-ALT-DEL and check the 'Processes' list (show all) to see if any stalled instances of your video player are still 'running'. If so kill all instances of the player and wait a few seconds before trying again. I have seen numerous flashes of the fauxD vector dialog occur, sometimes for a up to a minute, after killing stalled player instances.

To clarify the "input method" issue I should say your input can come from anywhere, you just need to get it into a clip variable named source near the top of the PROLOGUE section. The rest of the fauxD script (whether 'opened' or 'imported') will then do all the work and return the final clip. You should not need to create or modify any explicit calls to fauxD() yourself in the script's normal 'configuration' (as released) for it work properly. Conceivably issues with color-spaces and pixel formats with respect to ffdshow may arise, but I rather doubt that's the case unless you went randomly checking boxes in ffdshow at some point.

Let me know how it goes.

-eslave

[rfmmars]

Even with the last update, it still wouldn't work. I had to change all of the MVtools call to Mtool~. Still didn't see any results, but exported into Vdub, all is well. below is my simplified AVS so people can get started.
http://img136.imageshack.us/my.php?i...8004728bk5.jpg

A dialog box appears but with no information????? EDIT: I must be going blind, I could have sworn there was no data when I did the screen grab.

Code
...........................................

source = DirectShowSource("c:\3D-test.avi")

LoadPlugin("c:\AVS\Plugins\AGC.dll")
LoadPlugin("c:\AVS\plugins\removegrain.dll")
LoadPlugin("c:\AVS\Plugins\mt_masktools.dll")
LoadPlugin("c:\AVS\Plugins\warpsharp.dll")
Loadplugin("c:\avs\plugins\gradfun2db.dll")
loadplugin("c:\avs\plugins\DeGrainMedian.dll")
loadplugin("c:\avs\plugins\Smoothdeinterlacer.dll")
LoadPlugin("c:\avs\plugins\Convolution3d.dll")
LoadPlugin("c:\avs\plugins\FFT3DFilter.dll")
LoadPlugin("c:\avs\plugins\VagueDenoiser.dll")
LoadPlugin("c:\avs\plugins\depan.dll")
LoadCPlugin("C:\avs\plugins\AVSCurveFlow.dll")
LoadPlugin("C:\avs\plugins\MVTools2210.dll")
LoadPlugin("C:\avs\plugins\fauxD.dll")

Import("C:\avs\fauxD\fauxD_util.avsi")

####################################################################

scth = 40 #scene change threshold (on a scale of 255, I believe) lower=less tolerance
use_plugin = true #use fauxD function from plugin or AVS
stereo_mode = 0 # 0 = Red-Cyan Anaglyph, 1 = Over-Under (Right Above, Left Below), 2 = Side-By-Side (Left, Right), 3 = shutterglasses (pure AVS, Left=Field1)

flat = source.ConvertToYV12





#----- PREP WORK (Analysis) ---------------------------------

analyze = (source.height%2==0) ? source : source.ConvertToRGB32.CropBottom(1) #adjust to make height even
analyze = (analyze.height%2==0 && !(analyze.height%4==0) && !(analyze.height%8==0)) ? analyze.CropBottom(2) : analyze #try to make height mod4/8
analyze = (analyze.height%2==0 && !(analyze.height%4==0) && !(analyze.height%8==0)) ? analyze.CropBottom(2) : analyze #again if needed to make height mod4/8
analyze = (analyze.height%8==0) ? analyze.verticalreduceby2.verticalreduceby2 : analyze #quarter height
analyze = (analyze.height%4==0) ? analyze.verticalreduceby2 : analyze #half height

analyze = analyze.ConvertToYV12 # SHOULD be YV12 at this point for MVAnalyse


vectors = MSuper(analyze).MAnalyse( blksize = 16 , search = 5 , searchparam = 8 ) #BETTER SEARCH
global depan = MDepan(analyze, vectors) #convert motion vectors into DEPAN clip format (add zoom and rot options for complete vectors)

#----- PULFRICH (Interpret GM & fuse frame-pairs as stereo) ---------------------------------
framestep = 1
#TESTING previous frame (1 frame late)
global current = flat.KillAudio #current frame (now)
global previous = UnalignedSplice(BlankClip(current,framestep), current).KillAudio #previous frame (1 frame late)
global mvthresh = 0.0 #disabled - failed attempt at ignoring 'false-positive' motion
global reverse_stereo = false #for the sake of completeness, affects AVS functions only

global avs_mode = (stereo_mode == 1) ? "OverUnder" : (stereo_mode == 2) ? "SideBySide" : (stereo_mode == 3) ? "InterleavedShutter" : "AnaglyphRedCyan" # convert integer to string (function name) for fauxD/ScriptClip
use_plugin = (stereo_mode == 3) ? false : true #'native' shutterglasses are only supported through the AVS function
stereo = (use_plugin && (stereo_mode!=3)) ? fauxD(flat.converttoRGB32, depan, stereo_mode) : AudioDubEx( ScriptClip(flat, """ FauxD(GetPan(depan), avs_mode) """) , flat.KillVideo )





#----- EPILOGUE (Show me already!) ---------------------------------



flat = flat.ConvertToYV12 #for YDifferenceFromPrevious
stereo = stereo.MatchPixelFormat(flat)
out = ConditionalFilter(flat, flat, stereo, "YDifferenceFromPrevious()", ">", String(scth), false ) #faster than RGB?

# do any FINAL post-processing here
out = (stereo_mode == 3) ? out.Tweak(Bright=30) : out #brighten final output (for shutterglasses) AUTO

out = out.ConvertToRGB24 #for TMPGEnc, VFApi, or any other application limited to 24-bit RGB (8-bits per color/channel)
return out #the END (if everything worked)


#----- SANITY CHECK (Yes, I know this looks stupid.) ---------------------------------

return MessageClip("fauxD Error: Sanity Check!", 320, 240, shrink=false, text_color=$FF0000)

...............................................................................

Richard

photorecall.net

[eslave]

At the risk of repeating myself, please double-check your filenames and paths. I see you still have "MVTools2210.dll" on your LoadPlugin line. Unless you've renamed your file to match then AviSynth will not load the plugin. However, if you're seeing vector data in fauxD's dialog then things should be functioning properly.

I'll admit that I've never really tried using fauxD.AVS together with other scripts (such as LSF/SeeSaw). There may be unforeseen issues with all the global/ScriptClip stuff (its still relatively new to me). I had considered stream-lining the main script for this release and leaving a heavily commented copy for testers, but I felt that would only add to the confusion.

I'll take a swack at trying fauxD in Vdubmod later today and see what happens.

I apologize for all the unnecessary configuration problems. My original intent was to make it easier for casual testers to modify, but the results appear to be quite different from my expectations.
I'll bet you can't tell this is my first release!

-eslave

[rfmmars]

Originally Posted by eslave

At the risk of repeating myself, please double-check your filenames and paths. I see you still have "MVTools2210.dll" on your LoadPlugin line. Unless you've renamed your file to match then AviSynth will not load the plugin. However, if you're seeing vector data in fauxD's dialog then things should be functioning properly.

I'll admit that I've never really tried using fauxD.AVS together with other scripts (such as LSF/SeeSaw). There may be unforeseen issues with all the global/ScriptClip stuff (its still relatively new to me). I had considered stream-lining the main script for this release and leaving a heavily commented copy for testers, but I felt that would only add to the confusion.

I'll take a swack at trying fauxD in Vdubmod later today and see what happens.

I apologize for all the unnecessary configuration problems. My original intent was to make it easier for casual testers to modify, but the results appear to be quite different from my expectations.
I'll bet you can't tell this is my first release!

-eslave

What I did was rename a copy of my MVtools.dll "Version 2.31 to match your "loadplugins script" , I could not find the version you had listed.

I am very happy with the first results, just wanted to make it as easy for others to get started.

Many thanks,

Richard

[eslave]

I'm glad to hear its working for you!

I'm just sorry it didn't work for you 'out-of-the-box' on the first day, and I'm fixated on finding a resolution for the configuration problems some seem to be having. When I initially released fauxD I had hoped that people could start having fun right away, without struggling to find a working configuration.

-eslave

The functionality of the file formerly posted here has been incorporated into the current version.
Disregard the 'quoted' image two posts down.

[leeperry]

ok cool, something that works with a simple call in ffd would be really awesome

[rfmmars]

Originally Posted by eslave

I'm glad to hear its working for you!

I'm just sorry it didn't work for you 'out-of-the-box' on the first day, and I'm fixated on finding a resolution for the configuration problems some seem to be having. When I initially released fauxD I had hoped that people could start having fun right away, without struggling to find a working configuration.

-eslave

I'm still not sure this is a good idea, but I'm posting a stream-lined edition of fauxD.avs suitable for import into ffdshow's AviSynth dialog or other scripts. Don't forget to change the LoadPlugin and Import lines for your system.

Its me again,

Even with your revised script, Mvtools~ calls must be changed to Mtools~. After the change, the script works but there are problems using it in AVSp. Can you post a link to your MVtools DDL file?

Problems in AVSp using the above script.

1. The data window appears but is blank, no info
2. No 3D process is seen at all.
3. There is no way to use sliders in AVSp because of problems 1 & 2

3D effect and data readout is only seen after export or frameserved to Virtualdub, but no adjustment can be made at that point, most needed is to be able see the process and adjust in AVSp.

EDIT: If I move the timeline slider in AVSp with the mouse, I don't see the 3D conversion but if I hold down the "right" arrow key, the effect is seen, but still do not see any info in the data readout window, if would be nice if this could be fixed but no biggie now if not possible.

EDIT 2: Data readout doesn't stay on top when pressing "right" arrow key, but if AVSp is made not full screen, then it can be seen and there is data in the display. Anyway to force the data readout to stay on top of the AVSp display?

Richard

photorecall.net

[eslave]

In its current version (v0.11) the fauxD data window will remain blank (or unchanged) until at least two consecutive frames are processed. Seeking (with the sliders) or cutting directly to a frame is interpreted as a discontinuity by fauxD. (Thanks for pointing that out, as I had never really thought much about it until now. It'll be fixed in the next version.) Presently there is no real work-around for this (even if you create and use a logfile for the video in question). However, if you frame-step with the right arrow (or play the video) then fauxD should perform as expected. Incidentally, I primarily use AvsP as an editor. Until now I hadn't often used the built-in video player because it keeps the DLL locked - a nuisance when I'm trying to update it. I normally test my scripts with MPC (by double-clicking the AVS, or dragging and dropping the AVS onto the player window, or through a batch CMD on my SendTo menu).

Also, the YDifferenceFromPrevious scene-change test can completely ignore fauxD's stereo output, resulting in a flat image even if you see data updating in the dialog. (Fusion across scenes is a real eye-sore.) Since AviSynth doesn't like 'looking-ahead' in the video stream my only safe choice is to use the flat image for that frame.

No 3D will be produced until both conditions are satisfied.

I am using MVToolsV2 version 2.21. The latest version at the time of this writing (2.31) works as well, but can cause a hang-on-exit with some apps on my system.
(Fizick, let me know if you want these direct links removed.)

I'll add an option to keep the dialog 'always-on-top' if it works as intended. Although the dialog is more of a debugging tool than a genuine necessity considering only the X values are reasonably close to accurate (because the analysis clip was vertically reduced).

I still have no explanation for your Mvtools~/Mtools~ issue. I don't know how to help you with that.


Right now I'm trying to implement some kind of 'look-ahead' to maintain 3D at scene changes and preserve the effect for that additional 1/24 or 1/30 of a second. If I can get it working then I'll use it to solve the flat-on-start (first frame) issue you also seem to be describing. I've just finished restructuring and consolidating the script files as well. Now everything is in the form of a function and all of it is contained in a single file for easy import. I'll post the new version once I've finished updating the doc and succeed with (or temporarily give-up on) this look-ahead thing. After that I'll nuke the attachment in my previous post.

-eslave

P.S.: Please stop quoting my posts in their entirety.

[eslave]

I would have edited my previous post, but it still hasn't been 'approved'. This wasn't the case before. Maybe I've done something to upset a mod, I don't know.

I honestly don't know how I failed to notice this before, but the MVTools function names WERE changed between the two versions in question. So yes, all of you were correct to question that. I was just too stubborn to investigate it thoroughly. For some reason Fizick decided to remove all the 'V's from the function names in the current version, but strangely the error messages contained within still have all their 'V's intact.

Aha! Reading can help. From section "V) Revisions": (Lol, section 'V', how appropriate!)

2.3.0 (6.12.2008 by Fizick)
Renamed all functions from MVxxx to Mxxx (some renaming was requested by LaTo, Mystery Keeper, Sagekilla). Error messages is still not updated.

Anyway, I've just finished getting that look-ahead thing working, so some 3D effect can be seen on every frame (unless your scene-change threshold is too low). I'll likely be posting the new version in a less than a day or two, and I'll probably change the all the calls to support the current MVTools version (remove all the 'V's). Hopefully that will settle alot of confusion with respect to configuration.

-eslave

[Shinobu]

Thanks for this great plugin.
Works fine here (if i remove the "v" from the function ^^).

Too bad a doesn't have any 3d red/blue glasses (ordered yesterday) to test it.

as soon as i got the 3d glasses i'll test on my 720p dlp projector and post my impressions.

"Anyway, I've just finished getting that look-ahead thing working, so some 3D effect can be seen on every frame"
=>i'll wait for this version too ^^

[rfmmars]

Just want to say a big thank you for working with all of us to work out those pesty bugs in your great plugin, keep all your works coming.

Richard