Demosaic

[Archimedes]

Lately i was searching for a debayer filter for AviSynth. I only found a demosaic plugin ( http://forum.doom9.org/showpost.php?p=1070720). As the interpolation methods are very simple (nearest and bilinear), the quality are not really good (aliasing, color artifacts and so on). May be good enough for previewing.

So i wrote my own debayer function in AviSynth. More precisely, a script for Fritz Photo (http://forum.doom9.org/showthread.php?t=150291). May be, it’s not perfect now and we have to live with the 8 bit limitations, but the results seems to be quite stable. The function needs the NNEDI3 plugin.

Input is a RGB24 or RGB32 clip and output is a RGB32 clip. The function has two modes. Mode 1 is a little bit softer, but gives good results on portraits and so on. Mode 2 keeps the most details, while color artifacts are minimized. As the bayer pattern can be implemented in different ways, the parameters xoffset and yoffset take care about. For the parameters nsize, nns, qual and pscrn, please read the documentation of NNEDI3.

function Demosaic(clip input, int "mode", int "xoffset", int "yoffset", int "nsize", int "nns", int "qual", bool "pscrn") {
mode = Default(mode, 2)
xoffset = Default(xoffset, 0)
yoffset = Default(yoffset, 0)

nsize = Default(nsize, 6)
nns = Default(nns, 1)
qual = Default(qual, 2)
pscrn = Default(pscrn, True)

clip = input.Crop(0, 0, -(input.Width % 4), -(input.Height % 4)).ShowGreen("YV12")

last = clip
yoffset == 0 ? AssumeFrameBased().SeparateFields().SelectOdd() : AssumeFrameBased().SeparateFields().SelectEven()
TurnRight()
xoffset == 0 ? AssumeFrameBased().SeparateFields().SelectEven() : AssumeFrameBased().SeparateFields().SelectOdd()
TurnLeft()
yoffset == 0 ? NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnRight()
xoffset == 0 ? NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnLeft()
R = last

last = clip
yoffset == 0 ? AssumeFrameBased().SeparateFields().SelectEven() : AssumeFrameBased().SeparateFields().SelectOdd()
TurnRight()
xoffset == 0 ? AssumeFrameBased().SeparateFields().SelectEven() : AssumeFrameBased().SeparateFields().SelectOdd()
TurnLeft()
yoffset == 0 ? NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnRight()
xoffset == 0 ? NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnLeft()
G1 = last

last = clip
yoffset == 0 ? AssumeFrameBased().SeparateFields().SelectOdd() : AssumeFrameBased().SeparateFields().SelectEven()
TurnRight()
xoffset == 0 ? AssumeFrameBased().SeparateFields().SelectOdd() : AssumeFrameBased().SeparateFields().SelectEven()
TurnLeft()
yoffset == 0 ? NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnRight()
xoffset == 0 ? NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnLeft()
G2 = last

mode == 1 ? Eval("""
G = G1.MergeLuma(G2, 0.5)
""") : Eval("""
NOP()
""")

mode == 2 ? Eval("""
G1.TurnRight()
g11 = AssumeFrameBased().SeparateFields().SelectEven()
g12 = AssumeFrameBased().SeparateFields().SelectOdd()

G2.TurnRight()
g21 = AssumeFrameBased().SeparateFields().SelectEven()
g22 = AssumeFrameBased().SeparateFields().SelectOdd()

xoffset == 0 ? Interleave(g11, g12.MergeLuma(g22, 0.5)) : Interleave(g11.MergeLuma(g21, 0.5), g12)
AssumeFieldBased()
Weave()
TurnLeft()
G1 = last

xoffset == 0 ? Interleave(g21.MergeLuma(g11, 0.5), g22) : Interleave(g21, g22.MergeLuma(g12, 0.5))
AssumeFieldBased()
Weave()
TurnLeft()
G2 = last

yoffset == 0 ? Interleave(G1.AssumeFrameBased().SeparateFields().SelectEven(), G2.AssumeFrameBased().SeparateFields().SelectOdd()) : Interleave(G2.AssumeFrameBased().SeparateFields().SelectEven(), G1.AssumeFrameBased().SeparateFields().SelectOdd())
AssumeFieldBased()
Weave()

nnedi3_rpow2(rfactor=2, nsize=0, nns=1, qual=2, pscrn=True, cshift="Spline36Resize", fwidth=last.Width, fheight=last.Height)

G = last
""") : Eval("""
NOP()
""")

(mode < 1) || (mode > 2) ? Eval("""
G = G1.MergeLuma(G2, 0.5)
""") : Eval("""
NOP()
""")

last = clip
yoffset == 0 ? AssumeFrameBased().SeparateFields().SelectEven() : AssumeFrameBased().SeparateFields().SelectOdd()
TurnRight()
xoffset == 0 ? AssumeFrameBased().SeparateFields().SelectOdd() : AssumeFrameBased().SeparateFields().SelectEven()
TurnLeft()
yoffset == 0 ? NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnRight()
xoffset == 0 ? NNEDI3(dh=True, field=1, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn) : NNEDI3(dh=True, field=0, Y=True, U=False, V=False, nsize=nsize, nns=nns, qual=qual, pscrn=pscrn)
TurnLeft()
B = last

MergeRGB(R, G, B)
}

[Comatose]

Can you show a before and after please?

[Sagekilla]

This script only seems to output a grayscale image, regardless of whether you give it regular YV12 input or grayscale YV12 input.

And it shouldn't matter whether I give it input from a non-debayerized source and a faked source like grayscale YV12, since the two should look identical to your script.

[Archimedes]

Can you show a before and after please?
Before and after is something like this:

Before:
http://img149.imageshack.us/img149/1149/rawsonya100.png (http://img149.imageshack.us/my.php?image=rawsonya100.png)

After (not white balanced):
http://img18.imageshack.us/img18/5856/rawsonya1000640x0480.png (http://img18.imageshack.us/my.php?image=rawsonya1000640x0480.png)

And it's not a greyscale output. :rolleyes:

[scharfis_brain]

can you show a sample that is near to (or exceeds) the nyquist frequency?

ie, a sample that is prone to aliasing or false colour in highly detailed areas.

[Archimedes]

Here are some examples (without white balance). The original raw file has only 8 bit.

1.1 Demosaic (method=nearest):
http://www.bilderupload.de/bild.php/2,14989,0109071116371024x0768nearestZY8OU.png (http://www.bilderupload.de/bild.php/14989,0109071116371024x0768nearestZY8OU.png)

1.2 Demosaic (method=bilinear):
http://www.bilderupload.de/bild.php/2,14990,0109071116371024x0768bilinearU9UIJ.png (http://www.bilderupload.de/bild.php/14990,0109071116371024x0768bilinearU9UIJ.png)

1.3 FritzPhotoDemosaic (Mode=1):
http://www.bilderupload.de/bild.php/2,14992,0109071116371024x0768mode1GPHA0.png (http://www.bilderupload.de/bild.php/14992,0109071116371024x0768mode1GPHA0.png)

1.4 FritzPhotoDemosaic (Mode=2):
http://www.bilderupload.de/bild.php/2,14993,0109071116371024x0768mode2L0TQI.png (http://www.bilderupload.de/bild.php/14993,0109071116371024x0768mode2L0TQI.png)

2.1 Demosaic (method=nearest):
http://www.bilderupload.de/bild.php/2,14994,0209071700511200x0900nearestELHYH.png (http://www.bilderupload.de/bild.php/14994,0209071700511200x0900nearestELHYH.png)

2.2 Demosaic (method=bilinear):
http://www.bilderupload.de/bild.php/2,14995,0209071700511200x0900bilinearJVL0Q.png (http://www.bilderupload.de/bild.php/14995,0209071700511200x0900bilinearJVL0Q.png)

2.3 FritzPhotoDemosaic (Mode=1):
http://www.bilderupload.de/bild.php/2,14997,0209071700511200x0896mode1KARQD.png (http://www.bilderupload.de/bild.php/14997,0209071700511200x0896mode1KARQD.png)

2.4 FritzPhotoDemosaic (Mode=2):
http://www.bilderupload.de/bild.php/2,14999,0209071700511200x0896mode2FIRY4.png (http://www.bilderupload.de/bild.php/14999,0209071700511200x0896mode2FIRY4.png)

[Archimedes]

Update the script for use with NNEDI2.

[AnnaFan777]

how does it compare to DCdraw ?

http://www.cybercom.net/~dcoffin/dcraw/

[Archimedes]

I've realized, that in mode 2 the image center shift of channel G was not corrected. This is fixed now.

how does it compare to DCdraw ?
How we can compare? dcraw can handle 16 bit inputs, Demosaic, as a proof of concept, can only work in 8 bit mode. Demosaic filters each channel (R, G1, G2 and B) separately, dcraw works per default an all channels. However, it has an option (-f), to work on each channel separately.

In default mode, the results from dcraw looks much sharper (may be more detailed), but you have to deal with aliasing, color artefacts and labyrinth patterns (depending on the sources). One solution against such color artefacts and labyrinth patterns is, to interpolate the RGGB pattern as four colors (option –f). But the results looks less sharpen now (similar to the results of the demosaic function). However, a strong sharpening filter afterwards produces more aliasing on the dcraw source.

[Mug Funky]

i wonder if at some future point we could see a tritical build of dcraw? :)

or a nnedi2'ed hack of the redcode codec?

that's way too much to put on the poor guy though.

[scharfis_brain]

have you guys tried kolev raw so far?
it seems to implement some kind of edge directed interpolation.

I never use it though for other reasons.

[Archimedes]

have you guys tried kolev raw so far?
it seems to implement some kind of edge directed interpolation.
Not tested. Another alternative raw converter is RAWHide (http://www.my-spot.com/RHC/) which offers AAC Demosaicing.

Advanced Chroma Corrective Demosaicing. This may be the one of most accurate and advanced routines available. This routine extracts the highest resolution and most detail with the lowest instance of moiré and color artifacts. Where the routines used in other programs fail, ACC excels. Trade-Offs: Best resolution and color artifact resistance vs. Lower point data accuracy then EWC and less natural grain in images with high ISO and extreme white balance. Red's can be "noisy".
But i haven't tested it so far.

[markanini]

Dear Archimedes,

I'm intrigued by the NNEDI2 demosiac. I found a thread in the german dslrforum http://www.dslr-forum.de/showthread.php?t=467892&page=9 your script was discussed and you posted a build of dcraw, what was the point of that build(my german is poor). Was it a patch to add 8-bit linear gamma output?

[Archimedes]

I try to recapitulate it. :)

There was a problem with a handy photo. The handy supported an 8 bit raw format, but saved it as a non linear grayscale tiff format (with the bayer pattern). Most raw converter interpreted this format as a normal graphic format, but not as a raw. First, i tried the Demosic-Plugin ( http://forum.doom9.org/showthread.php?p=1108152#post1108152), which works, but the results was not very good. The author of dcraw then make some patches, where the tiff format was interpreted as a raw format. At the same time i have the idea with NNEDI to do the demosaicing.

The answers of the author of dcraw regarding the different patches in chronological order:

Pretty good image for a camera phone. See the patch
below.

Serious problems remain. First, the patch at 5403 makes
dcraw attempt to decode _all_ 8-bit grayscale TIFFs, which is
not good.

Secondly, the image is not white-balanced, and cannot be
without first linearizing it. The variable "use_gamma" should
be eliminated. Instead, all raw images should be linearized
upon loading.

Source >> ( http://www.dslr-forum.de/showpost.php?p=4752485&postcount=65)

The 8-bit TIFF data is gamma adjusted, and white
balance will not work until the data gets correctly
linearized. I'm updating the gamma logic to generate
inverse gamma curves for this purpose, assuming that
the camera used BT.709 or sRGB gamma.

Source >> ( http://www.dslr-forum.de/showpost.php?p=4776089&postcount=74)


After he has received some test charts of the handy owner, he answered:

This is interesting. I expected the linearization
curve to be a power function, but it's actually an
exponential with a tangent to the origin. I've updated
dcraw.c to generate such curves.

Both graphs plot calibrated patch brightness across
the bottom, and the RGB values reported by the camera in
the vertical direction.

Before ( http://img8.abload.de/img/beforeu34j.gif)
After ( http://img8.abload.de/img/after73jq.gif)

Source >> ( http://www.dslr-forum.de/showpost.php?p=4927673&postcount=84)

Was it a patch to add 8-bit linear gamma output?
So it was a patch from the author of dcraw (i only compiled the source code) to make a correct linearisation of the non linear input data (used for white balancing).