Deanaglyph green-magenta with full colored 2D Version

[MvB]

Hi there,

I found the following script (by oliver amato) to deanaglyph green-magenta movies into side-by-side:

# Load up our anaglyph file
vidr = avisource("anaglyph.avi")
vidlOrig = avisource("anaglyph.avi")
vidsound = avisource("anaglyph.avi")

# We need these for later
vidr = ConvertToRGB(vidr)
vidr = MergeRGB(vidr.ShowGreen("YV12"), vidr.ShowGreen("YV12"), vidr.ShowGreen("YV12"))
vidr = Greyscale(vidr)
vidr = ConvertToYV12(vidr)
vidr = ConvertToRGB(vidr)
vidlOrig = ConvertToRGB(vidlOrig)
vidl = vidlOrig
vidl = MergeRGB(vidl.ShowRed("YV12"), vidl.ShowRed("YV12"), vidl.ShowBlue("YV12"))
vidL = Greyscale(vidl)

# Create a 'colour map' by resizing the image down to 64x64 (makes a blurry image)
vidColor = avisource("video.avi")
vidColor = BilinearResize(VidColor, 64, 64)
vidColor = BilinearResize(VidColor, width(vidr), height(vidr))

# Make our grayscale images
vidl = MergeRGB(vidl.ShowRed("YV12"), vidl.ShowGreen("YV12"), vidl.ShowBlue("YV12"))
vidl = ConvertToYV12(vidl)
vidColor = ConvertToYV12(vidColor)

# Use our colour map
vidl = mergechroma(vidl, vidColor)

# Resplice the correct channels
vidl = ConvertToRGB(vidl)
vidl = MergeRGB(vidl.showRed("YV12"), vidl.ShowGreen("YV12"), vidl.ShowBlue("YV12"))

# Create a 'colour map' by resizing the image down to 64x64 (makes a blurry image)
vidColor2 = avisource("video.avi")
vidColor2 = BilinearResize(vidColor2, 64, 64)
vidColor2 = BilinearResize(vidColor2, width(vidr), height(vidr))

# Make our grayscale images
# vidr = MergeRGB(vidr.ShowRed("YV12"), vidr.ShowBlue("YV12"), vidr.ShowGreen("YV12"))
vidr ConvertToYV12(vidr)
vidColor2 = ConvertToYV12(vidColor2)

# Use our colour map
vidr = MergeChroma(vidr, vidColor2)

# Resplice the correct channels
vidr = ConvertToRGB(vidr)
vidr = MergeRGB(vidr.showRed("YV12"), vidr.ShowGreen("YV12"), vidr.ShowBlue("YV12"))

# show the Crosseye image
StackHorizontal(vidr,vidl)
AudioDub(vidsound)

But there are problems:

- at the rb channels there are visible halo-like artefacts from the green channel in high contrast areas (maybe from the optimizing). Any chance or filter to get rid of them? If you're lucky your 2d version is the picture of the rb part of your anaglyph. Then you don't have to use the rb channels for the picture, and so you don't get much halos because the green channel does have much less than the rb channels. that works with journey to the center of the earth very well with a litte modification of the script.

- let's say you use the green channel from the anaglyph version as the right pictures and rb channels or the greyscale version of the 2d movie as source of the left pictures. Obviously the greyscale from rb or 2d source are much brighter than the green only channel. But setting the gamma for the brighter picture helps only a little, because different colors in the left pictures have different brightness in the right picture (e.g. a bright, green coat would bei quite bright in the green channel, but quite dark in the rb channels). Such things are very painful to watch in 3d, because the comes the flicker again which is very disturbing and can cause headaches. So i thought of the only solution that i could think of as avisynth-noob: I take the green channel from the 2D source, adjust brightness or left and right image, which is a green channel, too (the brightness of the colors should be the same then, is there a plugin to normalize the two sides so they have the same brightness?) and map the colormap of the 2d version over the two images. The result is very good, no eye flickering anymore, but:

- the colormap of the 2d movie only aligns to one the pictures from the 3d view, the other will have grey borders and the colors at the wrong place if there is a parallax, that becomes very noticeable when objects are are not at screen height or popping out of the screen. the color in the right picture of the moving object move in the other direction like the object in the left image of the screen. This is a problem i was not able to solve so im asking: is there a possibility in avisynth to do some pattern matching: find similar blocks in the 2d version and the 3d version and transferring the color information of these matching blocks to the grey-scale images of the deanaglyphed images + adjusting the brightness correctly per block so that no flicker might occur?
That is the most challenging part as i don't know anything about pattern matching, breaking images in blocks and copying color information from one block to the other... i don't even know if it's possible with the actual available avisynth plugins. Maybe someone from the community has an idea? I'm no programmer, maybe someone has written something that could accomplish this?

I modified the script a litte for much better results:

# Originally Written by Olivier Amato, modified by MvB
# Producing 2x720p size, 1080p doesn't seem to work

# Loading both videos
vidr = directshowsource("anaglyph.m2ts").bicubicresize(1280,720)
vidlOrig = directshowsource("normal.m2ts").bicubicresize(1280,720)

# anglyph video -> rgb, filtering green channel and convert to grey
vidr = ConvertToRGB(vidr)
vidr = MergeRGB(vidr.Showgreen("YV12"), vidr.showgreen("YV12"), vidr.showgreen("YV12")) # Green channel of anaglyph
vidr = Greyscale(vidr)

# The same with the left image of the 2d video
vidl = ConvertToRGB(vidlOrig)
vidl = MergeRGB(vidl.Showgreen("YV12"), vidl.Showgreen("YV12"), vidl.Showgreen("YV12")) # Green channel of 2d video
vidl = Greyscale(vidl)

# adjust gamma, a normalization would be nice if somehow available for both green channels. Should give the same brightness for each of the picture pairs.
#vidl = vidl.Levels(0, 0.95, 255, 0, 255)

# Create a 'colour map' with the original resolution for the 2d greyscale picture vidl
vidColor = vidlorig

# Make our grayscale images
vidl = ConvertToYV12(vidl)
vidColor = ConvertToYV12(vidColor)

# Use our colour map
vidl = mergechroma(vidl, vidColor)

# Resplice the correct channels
vidl = ConvertToRGB(vidl)
vidl = MergeRGB(vidl.showRed("YV12"), vidl.ShowGreen("YV12"), vidl.ShowBlue("YV12")) # Is this necessary?

# Resizing the colormap of the 2d movie to 128x128
vidColor = BilinearResize(vidColor, 128, 128)
vidColor = BilinearResize(vidColor, width(vidr), height(vidr))

# Make our grayscale images
vidr = ConvertToYV12(vidr)
# vidColor = ConvertToYV12(vidColor)

# Use our colour map
vidr = MergeChroma(vidr, vidColor)

# Resplice the correct channels
vidr = ConvertToRGB(vidr)
vidr = MergeRGB(vidr.showRed("YV12"), vidr.ShowGreen("YV12"), vidr.ShowBlue("YV12"))

# Show crosseye or parallel image
StackHorizontal(vidl, vidr)
ConvertToYV12()


If the 2d version is not the left or right image (center or something) and the 3d doesn't really work, i'm forced to get the right and left picture out of the anaglyph stream resulting in these halos where you can see a bit of the picture from the the other side through -> problem nr. 1.
I don't know if there could be anything done in this case against the halos (transparent visible piture from the other side).

But the most interesting part for sure is to get a working pattern matching algorithm so the the colors from the 2d version could be placed in the right spot of the two images. Peter Wimmer wrote some program called "deanaglyph"

(http://www.3dtv.at/knowhow/DeAnaglyph_de.aspx) that works this way but he has not much time and nobody knows, if he ever has time to write an avisynth filter. And you can see the halos i'm speaking of in the reconstruced frame of spy kids on this site. But it's in German only.

So i thought maybe the tools are already there in avisynth to do something similar?

thanks
MvB

PS: Sorry if this post sound confusing, but i'm not a native english speaker.

[scharfis_brain]

some time ago, I made an attempt to restore anaglyph 3D to stereo pairs:
http://forum.gleitz.info/showthread.php?t=40349&highlight=anaglyph

[Didée]

Posting a sample would definetly help. ;)

After a glance at the script, it seems it'll have serious problems when there's any true "chroma detail" in the source. The re-coloring is done by just using chroma from the heavily-downsized helper clip. Now, say you have tablecloth with a fine-patterned blue-yellow checkerboard....

[MvB]

Hi Schafi,

thanks for your answer. If i understand correctly, this script of your's tries to create a side-by-side version out of the anaglyph version without the color information stored in the 2d version, am i right?

i don't know what the mvtools do there, is it possible to get the full colors from the 2d version in your script?

does the script do pattern matching?

I'll try it now.

thanks
MvB

[scharfis_brain]

what do you mean by '2d version'?

the script does 'pattern matching' by abusing mvtools motion estimation and compensation.
It will reconstruct both - left AND right - images in full colour out of one anaglyphic 3D image.

[MvB]

Posting a sample would definetly help. ;)

After a glance at the script, it seems it'll have serious problems when there's any true "chroma detail" in the source. The re-coloring is done by just using chroma from the heavily-downsized helper clip. Now, say you have tablecloth with a fine-patterned blue-yellow checkerboard....

Hi Dideé,

i made some screenshots of the problems, so that everone can see what i mean. Hope i'm not to stupid to use imageshack

http://img52.imageshack.us/img52/3597/halos.jpg
http://img411.imageshack.us/img411/2303/haloandcolor.jpg
http://img35.imageshack.us/img35/9396/halocolor.jpg
http://img532.imageshack.us/img532/8980/nearlyperfectlittlecolo.jpg

especially the 2nd one has everything in it i described above :(

[Didée]

What I meant was - a representative video sample to play with. Or perhaps even more than one.

From those screenshots, I can only confirm that the problems are like they're expected to be. Which doesn't help too much, probably. :p

[MvB]

thanks schafi, i tried your script but it gives me green and magenta color tones over whole picture and much more halos and blocks

http://img31.imageshack.us/img31/5236/schafi3.jpg
http://img202.imageshack.us/img202/8703/schafi2.jpg
http://img59.imageshack.us/img59/8100/schafi1.jpg
http://img691.imageshack.us/img691/7323/schafi4.jpg

in comparison to above script:
http://img256.imageshack.us/img256/1728/mvb2.jpg
http://img687.imageshack.us/img687/333/mvb1.jpg


@Dideé

I try to find webspace where i can place a sample of 2d and anaglyph video. Stay tuned. What resolution would be sufficient?

[scharfis_brain]

wait am moment.

do I understand correctly, that youÄve got a full colour 2d video as well as a anaglyph 3d video to start with?

this will make restoration a LOT easier. But I second Didée's request for samples, in order to be able to create a working filter chain.

[Didée]

Doesn't matter. Whatever resolution your sources are in. (In case of doubt, I'd prefer standard resolution over HD clips, for easier processing.)

MediaFire.com is a nice free file hosting service.


From the technical side, it's a problem whichever way you turn it. The plain spatial approach (as in the script you first posted) is problematic because you can't restore original color resolution from a downsized small resolution. Scharfis' approach is a very interesting one (we had talked about it in the past, though only little), but has a major problem from the fact that you can't safely make a "motion" estimation between different color planes ... the "same" place actually may look very different in any two planes, and MVTools is just searching for same-looking blocks.

Whichever way you turn it: nothing but problems. ;)

[MvB]

yes, that's it all about. What i try in the above script is to apply the color information from the 2d full color movie to the anaglpyh movie.
at least the newer titles have the 2d and 3d version bundled.
i don't know why, but my virtualdubmod only saves black frames. try to figure out, why.
maybe make a sample of the first 2 min or so. Let's see how big this will be.

[MvB]

I uploaded 2 mkvs to try with:

testn.mkv is the 2d full color file
testana.mkv is the 3d anaglyph file (green-magenta)

http://www.mediafire.com/?n2zzh0fjytc
http://www.mediafire.com/?4itwkjzmhmo

[MvB]

From the technical side, it's a problem whichever way you turn it. The plain spatial approach (as in the script you first posted) is problematic because you can't restore original color resolution from a downsized small resolution. Scharfis' approach is a very interesting one (we had talked about it in the past, though only little), but has a major problem from the fact that you can't safely make a "motion" estimation between different color planes ... the "same" place actually may look very different in any two planes, and MVTools is just searching for same-looking blocks.

Whichever way you turn it: nothing but problems. ;)

that's why i took the green plane from the 2d version and the green plane from the 3d version and tried to color those both planes. if there was a pattern matching algorithm that could find similar blocks in both green planes and colors those blocks from the color information of the 2d clip, the results would be nearly perfect. There would be no need to resize the colormap and, as you can see with the screenshots, it's possible to live with the halos. The worst problem, i think, is the color in the wrong position of the picture. Motion estimation between different color planes wouldn't be necessary.
Look at deanaglyph from peter wimmer. the only thing there to solve is the ghosting. but first things first. the color is more important. That problem with the ghosting is more important when you can't use the 2d video as source for one eye.



if you need some more clips, please tell.

[Didée]

Well, when there is a plain 2d variant of the source available, this should ease things by much. When both formats are there, it should be possible to make the "motion"-estimation work much more properly.

if you need some more clips, please tell.
Frankly, I was hoping that someone else would look into this, once that a sample is available. I can't solve all problems people come up with ... Throwing ideas is so much faster than producing a complete workflow.
(Read: if it's left up to me, then it may be next week, or next month, or next year ...)

[MvB]

@dideé

thats fair. i understand.

[scharfis_brain]

basically you need to extract the green channel out of both clips.
then generate the vectors between both green clips.
finally compensate the blue and red channels of the the 2D clip using the previously generated vectors to generate new red and blue clips,
which will be merged with the original green.

this way you'll get the missing channel.

[MvB]

basically you need to extract the green channel out of both clips.
then generate the vectors between both green clips.
finally compensate the blue and red channels of the the 2D clip using the previously generated vectors to generate new red and blue clips,
which will be merged with the original green.

this way you'll get the missing channel.

I'm trying to find out what mvtools are doing... sorry, as i told you, i'm a noob. I don't understand how mvtools are working.
mvanalyse creates the vectors, but which ones in what direction and how to use them for r and b of the 2d stream i don't even have an idea.

But your idea sounds very good. Maybe someone with knowlegde of mvtools can try this.

MvB

[MvB]

So, i tried to make a suiteable script, which brings absolutley perfect results in slow moving scenes (including NO GHOSTS and HALOS), but unbelievable lot of blocks in other scenes.

this is the script (maybe i've gotten all wrong, but it seems to work)

blk=8
tm=true
dct=4
thSAD=1000000


vidl = directshowsource("testn.mkv").converttorgb()
vidr = directshowsource("testana.mkv").converttorgb()

r = vidl.showred()
gn = vidl.showgreen()
b = vidl.showblue()
ga = vidr.showgreen()

gnga = interleave(gn,ga).converttoyv12()
ri = interleave(r,r).converttoyv12()
bi = interleave(b,b).converttoyv12()

mv_gnga = gnga.mvanalyse(blksize=blk, overlap=blk/2, chroma=false,truemotion=tm, dct=dct)
mc_g = gnga.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000, thSAD =thSAD)
mc_r = ri.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000,thSAD=thSAD)
mc_b = bi.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000,thSAD=thSAD)


vidl = vidl.converttorgb()
rvidr = mc_r.selectodd().converttorgb()
bvidr = mc_b.selectodd().converttorgb()
gvidr = mc_g.selectodd().converttorgb()

#vidr = mergergb(rvidr,ga.converttorgb(),bvidr) # slightly green borders
vidr = mergergb(rvidr,gvidr,bvidr) # this looks sometimes perfect, often full of blocks, but no ghosts
stackhorizontal(vidl,vidr)
converttoyv12()



Does anybody know what happens here?

Thanks

PS: that's the resultvideo
http://www.mediafire.com/?emmileigmyt

[MvB]

Hi there,

now i have four versions of the script.

1) the first version see first posting, but for red-blue
(it's the same, just have to make the motion-compensation for red channel than for green channel, i'm working now with a red-cyan video (final destination 4))

- pros: Faster than mvtools
- cons: strong resized color map, colors sometimes at the
wrong place in the reconstructed frame, red is pink

Script Nr. 2 (post above):
- pros: no halos or ghosts, exact color
- cons: the motioncompensated picture is very blocky and instable

Script Nr. 3
- pros: ghosts, blocks are nearly not visible, quite good picture, most eyefriendly, i think.
- cons: colors mostly in the right place, but red ist pink.

the third version ist here:
blk=8
tm=true
dct=4
thSAD=1000000


vidl = directshowsource("normal.m2ts", audio=false).converttorgb().bicubicresize(1280,720)
vidr = directshowsource("anaglyph.m2ts", audio=false).converttorgb().bicubicresize(1280,720)

colormap = vidl

rn = vidl.showred()
g = vidl.showgreen()
b = vidl.showblue()
ra = vidr.showred()

vidr = vidr.converttoyv12()
rnra = interleave(rn,ra).converttoyv12()
gi = interleave(g,g).converttoyv12()
bi = interleave(b,b).converttoyv12()

mv_rnra = rnra.mvanalyse(blksize=blk, overlap=blk/2, chroma=false,truemotion=tm, dct=dct)
mc_g = gi.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000, thSAD =thSAD)
mc_r = rnra.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000,thSAD=thSAD)
mc_b = bi.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000,thSAD=thSAD)


vidl = vidl.converttoyv12()
rvidr = mc_r.selectodd()
bvidr = mc_b.selectodd()
gvidr = mc_g.selectodd()


vidr = mergechroma(ra.converttoyv12(),mergergb(rvidr,gvidr,bvidr).converttoyv12())
vidl = mergechroma(rn.converttoyv12(),colormap.converttoyv12())
vidr = vidr.levels(0,0.7,255,0,255)
stackhorizontal(vidl,vidr)
converttoyv12()

The fourth version:

- Pros: Color red is right again, little ghosting in high contrast areas
- Cons: sometimes there are cyan color artefacts in areas of the picture, blocks somtetimes visible

blk=8
tm=true
dct=4
thSAD=1000000


vidl = directshowsource("normal.m2ts", audio=false).converttorgb().bicubicresize(1280,720)
vidr = directshowsource("anaglyph.m2ts", audio=false).converttorgb().bicubicresize(1280,720)

colormap = vidl

rn = vidl.showred()
g = vidl.showgreen()
b = vidl.showblue()
ra = vidr.showred()

vidr = vidr.converttoyv12()
rnra = interleave(rn,ra).converttoyv12()
gi = interleave(g,g).converttoyv12()
bi = interleave(b,b).converttoyv12()

mv_rnra = rnra.mvanalyse(blksize=blk, overlap=blk/2, chroma=false,truemotion=tm, dct=dct)
mc_g = gi.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000, thSAD =thSAD)
mc_r = rnra.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000,thSAD=thSAD)
mc_b = bi.mvcompensate(mv_rnra, thSCD2=255,thSCD1=10000,thSAD=thSAD)


vidl = vidl.converttoyv12()
rvidr = ra.converttorgb()
bvidr = mc_b.selectodd().converttorgb()
gvidr = mc_g.selectodd().converttorgb()


vidr = mergergb(rvidr,gvidr,bvidr).converttoyv12()

stackhorizontal(vidl,vidr)





So, there are 3 questions left for me:

1) Is there a way to let mvtools not produce blocks, that would be perfect. Maybe a deblocking filter would help? Which one would be good to try?
2) Is it possible in script 3 to do something that the red channel does not end up in pink (maybe stretch the histogramm of the merged red channel)? I think that comes vom mixing the colormap over the greyscale image from the red channel.
3) I think without blocking version 2 would bring the perfect picture. After that comes version 4, and if it would be possible to remove the cyan artifacts it would be very good.

MvB

PS: rewriting now for green-magenta

[Jack-Bauer]

@MvB:
Stupid question: Why are you using mvtools instead of mvtools2 (2.5.10) ??

[MvB]

@MvB:
Stupid question: Why are you using mvtools instead of mvtools2 (2.5.10) ??

@Jack

I used mvtools because schafis script used this version and i didn't know if mvtools2 work better or worse and if they have the same parameters. So i tried with schafis version. Do you think mvtools2 would work better?

[Jack-Bauer]

I just dont know but mvtools2 is newer than mvtools (one year ahead).
So there may be some improvements (hopefully, the macroblock issue....).
The syntax is slighly different though.

[AnnaFan777]

have you tried this script?

http://forum.doom9.org/showthread.php?t=145122

[Jack-Bauer]

After having done a little work on the subject, I came to the conclusion that deghosting is the more important part of the issue.

Think about it:
Either two methods:
- 64x64 colormap: The extracted green channel is used for the image. Hence it must be ghost free.

- MVtools: The extracted green channel is used as a target for the vectors.
To be able to detect matching blocks of pixels, movement compensation needs to be fed with two clean images.
Ideally, they should be coloroured images, which they are not (we cannot do anything about it).
Secondly, if one of the images has a lot of ghosting, halos, whatever you call it, retrieving matching blocks is likely to fail.

In both cases we badly need a ghost free image.

Take Journey to the center of the earth as a good example:
I suspect that the movie has been encoded using some sort of pre-compensation (ghost busting), as RealD is currently using to predict the unavoidable ghosting because of the circular polarizing filters. This is likely to be mandatory with anaglyphs..!

That means, in the encoded file, the left image is using a little of the inverted right image and vice-versa.

Our challenge is to eliminate this compensation in the anaglyph extracted image (ga).
Unfortunately the compensation does not seem to be linear and a lot of experiments have driven me to the following deghosting script.
Unfortunately again, there seems to have a cutoff action in the dark pixels (remember the "inverted" word above? you cannot have blacker than black...)

Hence the script (the output compares the non-deghosted image to the deghosted one):
----------------------------------------------------------------------------------------------------------------------
offset=30 #compensation black level
comp=0.28 #compensation weight

vidl = AVISource("2D.avi",false).converttorgb()
vidr = AVISource("3D-ana.avi",false).converttorgb()

gn = greyscale(vidl)
ga = showgreen(vidr)

gn = converttoYV12(gn)
ga = converttoYV12(ga)
#--------------
mask=gn.levels(offset,1,255,0,255-offset)
ga2=levels(ga,0,0.7,255,0,255) #deghosting is non linear
ga2=merge(ga,mask,comp)
ga2=levels(ga2,5,1.3,255,0,255) #restoring gamma and adjusting picture black level
#--------------

stackhorizontal(ga,ga2) #before / after

reduceby2()
----------------------------------------------------------------------------------------------------------------------
This script works well in the mid-range tones.
Still, there is too much compensation when the backgroung is dark, as "expected".

My test frames are:
397 deghosting too strong on black background
1464 not enough deghosting of black (this image seems to be more challenging... forget it for the moment)
1814 ok
2913 ok
4200 again too much compensation in dark areas
4660 same thing
5453 acceptable deghosting on both sides of white shirt (could be reduced)
11654 ok
102091 ok
123898 again, too much compensation on black background

(Be careful if using virtualdub, both images are not always synchronized when seeking)

The ghosting left seem to be circumscribed to the dark areas. Some kind of threshold has to be applied here.
The idea is to split the tone range into two different processes. One for the mid-tones (the one above) and one specially devoted to dark areas.
Currently, I am trying to split the tone range:
ga1 = ga.levels(0,1,30,0,30)
ga2 = ga.levels(30,1,255,0,255-30)
out = overlay(ga1,ga2,mode="add")
But... out should rebuild the image but does not. Anyone an idea ?

EDIT: out = overlay(ga1,ga2,mode="add",pc_range=true) works.
Last minute: This deghosting is not enough to solve the macroblock issue when using mvtools. I quit.


Jack

[MvB]

@AnnaFan777

I've seen this thread, but the script does not use a full colored 2d version of the movie to restore colors of 3d version.

@Jack-Bauer

Thank you very much for your efforts. Even if this helps to reduce the ghosting it's a big step forward. As soon as i have time i'll try it.

[Jack-Bauer]

New script. Works much better.
Unfortunately the pixels which have been cut off in the dark areas were lost (negative values!) and cannot be restored. Introducing a cutoff mask to average the problem in the 0-10 grey range.
I am afraid thats the best I can do... I didnt check the whole movie but I think 60-70% of the ghosts are gone. Better than nothing.
Jack

comp = 0.22 # pre-compensation weight %
cut = 10 # pre-compensation cutoff

vidl = DirectshowSource("2D.mkv",audio=false)
vidr = DirectshowSource("3D-ana.mkv",audio=false)
#------------------------------------
vidl = vidl.converttorgb()
vidr = vidr.converttorgb()

gn = vidl.greyscale()
ga = vidr.showgreen()
#------------------------------------
# create cutoff mask to prevent overcompensation in dark areas
cutoff = ga.levels(0,1,cut,127,255)

# create deghost mask (applying cutoff)
mask = overlay(gn,cutoff,mode="multiply",pc_range=true)

ga2 = levels(ga,0,0.7,255,0,255) # deghosting is non linear
ga2 = merge(ga2,mask,comp) # deghost !
ga2 = levels(ga2,0,1.3,255,0,255) # restoring gamma
#------------------------------------

stackhorizontal(ga,ga2) # before / after

reduceby2()

[MvB]

Thank you! I'll try asap.

[MvB]

gn = vidl.greyscale()
ga = vidr.showgreen()


Is there a reason you didn't use the green channel of vidl but all the colors?

[MvB]

Hi to all that are interested:
This is a preliminary version of the deanaglyph script (in this case green magenta) as long as there is no better idea to deghost the image. But the results are quite good, in fact the best i've ever seen till now. Thanks to Schafi and jack-bauer.

# Deanaglyph with color information of 2d version (green-magenta)

blk=16
tm=true
dct=4
thSAD=1000000

# somehow, i don't know why, there is only a black picture if i set the resolution to full hd for both movies

vidl = directshowsource("normal.m2ts",audio=false).converttorgb().bicubicresize(1280,720)
vidr = directshowsource("anaglyph.m2ts",audio=false).converttorgb().bicubicresize(1280,720)

# this is the part of Jack-Bauer for deghosting. It's much better than no deghosting (thank you, jack)
# but some ghosting remains which throws mvtools of the track,
# that's why i have to merge the green channel of the anaglyph video
# with the mvtools-created colormask (only merging
# with blue and red would result in magenta artifacts).
# The same goes for the normal video, merging the green
# channel with a colormask of the left video to keep colors constant. That's the best
# idea i have at the moment and the result is quite good.
#
# A theorie: if there was a ghost-free green channel from the anaglyph video mvtools maybe could create
# a perfect motion compensation without artifacts. In this case we would be able to replace the right video
# completely with the motion compensated original left video resulting in perfect colors. Could this be correct?

comp = 0.22 # pre-compensation weight %
cut = 10 # pre-compensation cutoff

gn = vidl.greyscale()
ga = vidr.showgreen()

#------------------------------------
# create cutoff mask to prevent overcompensation in dark areas
cutoff = ga.levels(0,1,cut,127,255)

# create deghost mask (applying cutoff)
mask = overlay(gn,cutoff,mode="multiply",pc_range=true)

ga2 = levels(ga,0,0.7,255,0,255) # deghosting is non linear
ga2 = merge(ga2,mask,comp) # deghost !
ga2 = levels(ga2,0,1.3,255,0,255) # restoring gamma
#------------------------------------

# here ends the deghosting part that works for 60% to 70% of the movie. That's much better than nothing.

# Creating a colormap of vidl
colormap = vidl

# This is the magenta channel of vidl in rgb
rb = mergergb(vidl.showred(),vidl.blankclip(),vidl.showblue())

# and the green channel of normal video, that's what we need in addition to ga
gn = vidl.showgreen()

# then interleave the normal green channel with the deghosted green channel
gnga = interleave(gn,ga2).converttoyv12()

# and the maganta channel(s) with themselves
rbi = interleave(rb,rb).converttoyv12()

# doing motionanalysing of both green channels and applying the resulting motion vectors to vidl
mv_gnga = gnga.mvanalyse(blksize=blk, overlap=blk/2, chroma=false,truemotion=tm, dct=dct)
mc_g = gnga.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000, thSAD =thSAD)
mc_rb = rbi.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000,thSAD=thSAD)

# assigning the motion compensated magenta channel(s) to rbvidr
rbvidr = mc_rb.selectodd().converttorgb()

# and the motion compensated green channel to gvidr
gvidr = mc_g.selectodd().converttorgb()

# Applying the motion compensated color mask to the green channel of vidr
vidr = mergechroma(ga2.converttoyv12(),mergergb(rbvidr.showred(),gvidr,rbvidr.showblue()).converttoyv12())

# Applying the original color mask to the green channel of vidl
vidl = mergechroma(gn.converttoyv12(),colormap.converttoyv12())

# This seems necessary
vidr = vidr.levels(0,0.9,255,0,255)

# Stacking images
stackhorizontal(vidl,vidr)






PS: oh yeah, i forgot to mention, it's sloooow:
core2duo, 2.9 GHz , 0.5 fps!
When i have time again i try to investigate if there's a possibility to use mutithreaded avisynth for the job.

[Jack-Bauer]

gn = vidl.greyscale()
ga = vidr.showgreen()
Is there a reason you didn't use the green channel of vidl but all the colors?

Absolutely. I am not sure about coloured anaglyphs (or half-color, optimized...). But for greyscale anaglyphs, if you only take the green channel, pure red will become black. As well as pure blue. In fact you wont be able to make a visual distinction between red (medium grey) and blue (dark grey). Both would be black.
For coloured anaglyph, I think it could be even worse and I wouldnt take the chance.

I am improving the deghosting part as it seems that the white end also suffers from some kind of cutoff (it only matters for a couple of scenes). Although I understand why there is a cutoff action in the greyscale lower black part (to preserve the black level), it could have been avoided in the higher whites (you can have whiter-than-white if you only use luma{0-230} for example).
To be continued...

Your right MvB, its sloooow (But when the encoding is done, its done)
I understand that the mvtools approach is theoritically the right one, but if it cant be improved, choice will have to be done beetween mvtools and 64x64 colormap which is an order of magnitude faster to encode...

Maybe somme decision could be taken inside the script as to automatically swap methods depending upon the result for a given scene ???
Jack

[MvB]

The good thing is: with your actual deghosting algorithm, would say, nearly half the movie could be taken on with mvtools only (no need to merge). I tried some scenes and they were perfect, some others unfortunately so bad that this method cannot be used yet.
Did you try this script? the results aren't so bad. I try to encode this movie now with this version of the script and see it with my h5360 and nvidia shutter glasses. Just have to wait ..... 52 hours :)

PS: after combining the red and blue channels in one "magenta" channel the script runs in 1280x720 with 0.66 fps! YES! Saved 10 hours.

[Jack-Bauer]

The good thing is: with your actual deghosting algorithm, would say, nearly half the movie could be taken on with mvtools only (no need to merge).
You mean creating the whole image with mvcompensate OR using the deghosted one for luma and mvtools for chroma?
Saved 10 hours.
Good luck and tell us what it looks like with your setup.
Jack

[MvB]

@jack
the whole image with mvcompensate

I'll post some screenshots when it's done.
Did you have some luck with the new deghosting idea?

[MvB]

32% till now. I was able to take screenshots from the movie while encoding. The result is very pleasing. Judge for yourself. Nearly the whole encoded movie has not ghosting except some parts i've taken screenshots of, too.

http://www.mediafire.com/file/nqzni4vynkn/car.jpg

This is the problematic dino, look how that has improved!
http://www.mediafire.com/file/qdf4znvhgqw/dino.jpg

There were ghosts at the red jacket and the colorplane was way of, now gone :)
http://www.mediafire.com/file/mvz3m4mtnon/home.jpg

The entrace of the home, absolutly perfect
http://www.mediafire.com/file/mjuht40zmtq/home2.jpg

There were big ghosts visible at the black alarm clock, now gone :):)
http://www.mediafire.com/file/mzdmxn0z50y/home3.jpg

Look at that wonderful picture in the institute, perfect!
http://www.mediafire.com/file/0mrkzz3kkvm/institute.jpg

Here we see black ghosting? Before it was white ghosting.
http://www.mediafire.com/file/om3eanyzg4z/introghosts.jpg

picture at the labor. the ghosting at the white paper in the background ist gone :)
http://www.mediafire.com/file/dzdjizeemmf/labor.jpg

do you remember how that looked before?
http://www.mediafire.com/file/0mjziljetzd/labor2.jpg

This looked as bad as the dino before, look now:
http://www.mediafire.com/file/jy22nmjwzod/mines.jpg

Perfect
http://www.mediafire.com/file/nyrxwtzonta/mines2.jpg

This picture on the plane looks perfect
http://www.mediafire.com/file/tlmgmmtymux/plane.jpg
while this picture seems to have som ghosting at the blue seat:
http://www.mediafire.com/file/on5yzx3oezm/planeproblem.jpg

You're totally right, the ghosting is the problem for mvtools. Definitly. the improvement speaks for itself.

[Jack-Bauer]

Fantastic! What I like best is applying the same rebuilt color map to both channels. Not the original color map though but this eliminates eye rivalry. Nice trick.
introghost.jpg: That's exactly what I was refering to when speaking of white end cutoff. A couple of scenes may benefit from a change in my script. I will post it tomorow.
Jack.

[MvB]

If your deghosting is working very good, it should be possible to use the l/r greyscale from the anaglyph video and map the complete picture of the 2d source via motion compensation on each side (double motion compensation for l and r from normal). That's interesting for titles like caroline where the 2d version doesn't seem to be ether l or r. But i think blocks would be much more visible then if they appear.

If there is no 2d version available even schafis script should benefit enormly from deghosting before usage of mvtools.

[Jack-Bauer]

Below is new deghost script. It is not much of an improvement, but frames 520 and 27500 look much better in the whites for example.
Also, the linear main compensation mask give slightly better results.
Please check if it is worth it. I am afraid its the best I can do...
-------------------------------------------------------------------------------------------------------------------------
vidl = DirectshowSource("2D.mkv",audio=false)
vidr = DirectshowSource("3D-ana.mkv",audio=false)
#------------------------------------
gn = vidl.greyscale()
ga = vidr.showgreen()
############################ NEW SCRIPT taking care of white cutoff ######################
# create linear main compensation mask
cutoff_W = ga.levels(0,1,255,255,50)

# create cutoff mask to prevent overcompensation in dark areas
cutoff_B = ga.levels(0,1,10,127,255)

cutoff = overlay(cutoff_W,cutoff_B,mode="multiply",pc_range=true)

# create deghost mask (applying cutoff)
mask3 = overlay(gn,cutoff,mode="multiply",pc_range=true)

# tweaking mask3
mask3 = mask3.levels(0,0.85,255,100,0)

# prevent white saturation
ga3 = ga.levels(0,1,255,0,255-50)

# deghost! and restore black level
ga3 = subtract(ga3,mask3).levels(35,1,255,0,255)

########################### END of new script. Below is old script in order to compare #########

comp = 0.22 # pre-compensation weight %
cut = 10 # pre-compensation cutoff

# create cutoff mask to prevent overcompensation in dark areas
cutoff = ga.levels(0,1,cut,127,255)

# create deghost mask (applying cutoff)
mask = overlay(gn,cutoff,mode="multiply",pc_range=true)

ga2 = levels(ga,0,0.7,255,0,255) # deghosting is non linear
ga2 = merge(ga2,mask,comp) # deghost !
ga2 = levels(ga2,0,1.3,255,0,255) # restoring gamma
#####################################################################
#comparing old / new scripts
stackhorizontal(ga2,ga3)

reduceby2() # if virtualdub

[MvB]

@Jack

So, i did some comparison:

it seems that the new algorithm is able to remove these little grey ghosts in white background, but it introduces more artifacts at the other end. Otherwise both algorithms are very similar as expected. The old one gives a more "normal" picture.

I made some screenshots. The pictures without red marks are quite similar and good, the black artifacts are not as dark with the old algorithm than with the new one.

The dino has less ghosting with the old one
http://www.mediafire.com/file/dnxtlzqyobj/comp10.jpg

Here's the output quite similar
http://www.mediafire.com/file/ymzomkiyfyz/comp4.jpg

less ghosting with old one
http://www.mediafire.com/file/y3nmyoygtk0/comp5.jpg
http://www.mediafire.com/file/m0omn3tzjyj/comp6.jpg
http://www.mediafire.com/file/nmynq4f2ud1/comp7.jpg

The black hosts are not as black as with the new one, in every case the black artifacts with the new one are much more visible
http://www.mediafire.com/file/tqwdvnmwubj/comp8.jpg

Here the white ghosts are more visible with the new one
http://www.mediafire.com/file/nmymjuxfmzm/comp9.jpg

This one is great with both
http://www.mediafire.com/file/zwwgewkd3ay/compclock.jpg

as you can see, the black artifacts are less dominat with the old one
http://www.mediafire.com/file/0tjwnonjwjz/compdino.jpg
http://www.mediafire.com/file/lmnh4ymn1iy/compface.jpg

As i see it, the old one is a little bit better. Would you agree?
Thank you very much for this.

[Jack-Bauer]

As usual, the simpler the better...
I agree with your comparisons, the old algorithm is better.

Too bad for the white end. But it only matters for a few scenes anyway.
Probably we can live with that. Actual visual experience will tell when the encoding is over.
Jack

[wonkey_monkey]

If one leaves more white ghosts, and one leaves more black ghosts, would mixing the two outputs help? Or also processing an inverted clip and mixing that in too...

David

[scharfis_brain]

simple masking methode just won't work properly to recreate a full coloured stereo pair.
you'll have to compensate the image areas individually. (that's why I suggested mvtools)

[MvB]

Encoding finished. I will look at the movie when i have time again, maybe tomorrow and we'll see...

@schafi
I'm using mvtools for the colorplanes. What we tried was to get rid of the ghosts that mvtools can to better pattern matching.

PS: What i've seen so far is that if the distance of one object in l/r gets to far (e.g. popping far out of the screen) mvtools won't locate this object correctly.

[MvB]

As far as i can speak by watching parts crosseyed at my 19'' work monitor the picture is perfect. I was not able to see major glitches (that does not mean there couldn't be some), it feels like watching avatar qualitywise :). Most of it i like that there's no eye rivalry anywhere, everythings great!

More to come when i see it on the big screen with shutter equipment...

[Jack-Bauer]

I am eager to hear about it.
Jack

[MvB]

My date was canceled yesterday so i tried the movie with my parents. And i have to say, great! Everbody was very excited because:

- picture was nearly artifact-free
- there was absolutly NO eye rivalry
- nearly the whole movie felt like 3d-cinema qualitywise, but with less ghosting than polarisation :)
- there were only very few neglegtible moments where something was not exactly right. But only a few seconds in the whole movie so that didn't really matter. One Example is in the clip where Trevor wants to throw that luminescent thingy into the abyss.


Here an example clip, heavily downsized :)

http://www.mediafire.com/file/mmj3wmedumg/testconv.AVI

So i have to thank everybody involved with this script a lot. The result is absolutly great and i think it's not possible to do much better without programming special software for the purpose.

Thank you all very much.

if that script works with red/cyan as good as with green /magenta has to be tested..., but i think the red channel has so much ghosting that it should be very very difficult.
MvB

@mods: if that audioless clip would be copyright infringement please delete all the links at once. this was not intended.

[Jack-Bauer]

Great MvB !! I am proud I have taken part in this achievement!

Is the script you used exactly the same as the last one you posted here? (I also intend to encode)

Dont forget that the end credits (starting at the beginning of the two feather scene) has reverse left-right eye. Also, the title at the beginning should take both channels from the anaglyph.

Jack

[MvB]

@jack

yes, it's the same script.

For the end credits i would have to reprogram the script to use the magenta channel(s) for motioncompensation. I didn't have the time to do that, i don't know if it's worth it. Is there the possibility to tell avisynth something like this:

if framenr < 100000 then
use this algorithm,
else
use this one.....
end if

Than this change could be incorporated in the script. Even to automatically switch to the anaglyph version for the left channel could be done so.

I tried polar express with the deghosting but there it didn't work. (red/cyan). That's more difficult.

[Jack-Bauer]

A simple B/W script could be, for example:

normal = normal.showgreen()
green = ana.showgreen()
red = ana.showred()

movie = stackhorizontal(normal,green)
title = stackhorizontal(green,red) # not checked, maybe reverse
credit = stackhorizontal(red,normal)

part1 = movie.trim(0,620)
title = title.trim(621,997)
part2 = movie.trim(998,123910)
credit = credit.trim(123911,0)

part1 + title + part2 + credit

(also, the title appears in the credits, I forgot... Change has to be made here also)

Jack

[MvB]

Hi jack,

how's the result? Better than your tries before?

[Jack-Bauer]

To improve the process, I'm afraid the only thing I thought of was "manually" tweaking critical scenes.

I have not encoded the movie yet. But when it is done, I intend to watch the result in order to pick up

some (half a dozen?, a dozen?) critical scenes which need deghosting improvement.

The example below improves 3 scenes:


LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")

vidl = DirectshowSource("normal.mkv",audio=false).converttorgb()
vidr = DirectshowSource("ana.mkv",audio=false).converttorgb()

comp = 0.22 # pre-compensation weight %
cut = 10 # pre-compensation cutoff

v1 = deana(vidl,vidr,comp,cut).trim(0,17962) # use defaults
v2 = deana(vidl,vidr,0.15,cut).trim(17963,18053) # custom : plane blue seat
v3 = deana(vidl,vidr,comp,cut).trim(18054,27329) # use defaults
v4 = deana(vidl,vidr,0.1,cut).trim(27330,27762) # custom : skyline
v5 = deana(vidl,vidr,comp,cut).trim(27763,35757) # use defaults
v6 = deana(vidl,vidr,0.1,50).trim(35758,35897) # custom : light flare
v7 = deana(vidl,vidr,comp,cut).trim(35898,0) # use defaults

v1+v2+v3+v4+v5+v6+v7

#########################################################################
Function notused(vidl,vidr,comp,cut){
gn = vidl.greyscale()
ga = vidr.showgreen()
#------------------------------------
# create cutoff mask to prevent overcompensation in dark areas
cutoff = ga.levels(0,1,cut,127,255)

# create deghost mask (applying cutoff)
mask = overlay(gn,cutoff,mode="multiply",pc_range=true)
ga2 = levels(ga,0,0.7,255,0,255) # deghosting is non linear
ga2 = merge(ga2,mask,comp) # deghost !
ga2 = levels(ga2,0,1.3,255,0,255) # restoring gamma

#------------------------------------
stack = stackhorizontal(ga,ga2)
return(stack)
}
#########################################################################
Function deana(vidl,vidr,comp,cut){

blk=16
#blk=8
tm=true
dct=4
thSAD=1000000

gn = vidl.greyscale()
ga = vidr.showgreen()

#------------------------------------
# create cutoff mask to prevent overcompensation in dark areas
cutoff = ga.levels(0,1,cut,127,255)

# create deghost mask (applying cutoff)
mask = overlay(gn,cutoff,mode="multiply",pc_range=true)

ga2 = levels(ga,0,0.7,255,0,255) # deghosting is non linear
ga2 = merge(ga2,mask,comp) # deghost !
ga2 = levels(ga2,0,1.3,255,0,255) # restoring gamma
#------------------------------------

# here ends the deghosting part that works for 60% to 70% of the movie. That's much better than nothing.

# Creating a colormap of vidl
colormap = vidl

# This is the magenta channel of vidl in rgb
rb = mergergb(vidl.showred(),vidl.blankclip(),vidl.showblue())

# and the green channel of normal video, that's what we need in addition to ga
gn = vidl.showgreen()

# then interleave the normal green channel with the deghosted green channel
gnga = interleave(gn,ga2).converttoyv12()

# and the maganta channel(s) with themselves
rbi = interleave(rb,rb).converttoyv12()

# doing motionanalysing of both green channels and applying the resulting motion vectors to vidl
mv_gnga = gnga.mvanalyse(blksize=blk, overlap=blk/2, chroma=false,truemotion=tm, dct=dct)
mc_g = gnga.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000, thSAD =thSAD)
mc_rb = rbi.mvcompensate(mv_gnga, thSCD2=255,thSCD1=10000,thSAD=thSAD)

# assigning the motion compensated magenta channel(s) to rbvidr
rbvidr = mc_rb.selectodd().converttorgb()

# and the motion compensated green channel to gvidr
gvidr = mc_g.selectodd().converttorgb()

# Applying the motion compensated color mask to the green channel of vidr
vidr = mergechroma(ga2.converttoyv12(),mergergb(rbvidr.showred(),gvidr,rbvidr.showblue()).converttoyv12())

# Applying the original color mask to the green channel of vidl
vidl = mergechroma(gn.converttoyv12(),colormap.converttoyv12())

# This seems necessary
vidr = vidr.levels(0,0.9,255,0,255)

# Stacking images
stack = stackhorizontal(vidl,vidr)
return(stack)
}
#########################################################################

Although the blue seat scene improvement is not obvious (just practicing the method...), the other two

look much better.

MvB, you already have watched the encoded movie. Should you have a scene list to improve, I would be glad to generate the lines with the best parameters.

EDIT: My mistake: I tried to include several scenes in v4, but that wont do. Instead please read:
v3 = deana(comp,cut).trim(18054,27459) # use defaults
v4 = deana(0.1,cut).trim(27460,27762) # custom : skyline

Seems that more detailled manual tweaking would be necessary. Difficult to use for the whole movie. Still looking for an alternate automated method...

Jack