Deinterlace two fields source

[otherman]

I'm making some experiment for a magic lantern project, and I need to find a way to deinterlace this kind of source

interlaced source (http://s18.postimg.org/6taw4c5nr/binterlaced001.png)

it has 2 field from the first frame and 2 from the second frame, instead of classic 1 and 1.

Any solution or suggestion?

[Guest]

Please provide an unprocessed source sample video clip. Screenshots are not useful.

Based on the screenshot, it has been resized before deinterlacing, but I will wait for your sample.

[otherman]

the source is a normal progressive video without any artifact, I made this frame using rows 0,1,4,5,8,9... from first frame and rows 2,3,6,7,10,11... from the next frame (a couple from first and a couple from second). I made this to emulate a possible new way to record interlaced raw-video for Canon camera using Magic Lanter firmware.
For the nature of Bayer sensor used by Canon Camera, this must be this way (2 rows from first, 2 rows from the next).
I can get also classic interlacing (1 field from the first, 1 field from the next), but with first field lacking blue channel (RG and not B), and the second field lacking the red one (GB and not R) like this frame (http://s30.postimg.org/ime2knjov/rgb.png).
Any good solution for one of this case would be amazing!

[otherman]

ok, this is it (https://mega.co.nz/#!4okjyIYY!Vv3qL9Txhg2a7bBHBJsaUk47Wn41NONvDTPYIm7aE_4)

[Guest]

Seems like you could do multiple separatefields() and then an appropriate selectevery().

An Avisynth guru like Gavino will come soon with the script, I'm sure. :)

[otherman]

:)

well, I'm making some test, thanks.

[foxyshadis]

Interleave(A,B)

That simple!

Looking at your sample, it's not unprocessed, is it? The fourcc is divx, and it's been pointresized up to 1080i; no camera would record in that. Can you hack off the first 50MB or so of the raw camera output?

[otherman]

Actualy, the camera can NOT record this way at the moment. This is a proof of concept, a simulation. It's not pointresized up to 1080i; there are two DIFFERENT (and real) rows from first frame, than two row of the second frame, other two row of the first frame, and so on. There is not any resize!
So, if I can get a good deinterlace out of it, a devoloper is ready to make this feature for next firmware version. It can mean (fake)4k raw recording on some camera :)

[Audionut]

You beat me to it otherman. :)

Interleave(A,B)

Would this not interleave every other line? The source frames (raw feed from sensor), will be like so.

A
A
B
B
A
A
B
B

Instead of the classic,

A
B
A
B

I can work on uploading unprocessed source frames.

[Guest]

It would be fairly easy to make a custom avisynth filter to undo this.

[otherman]

Can you explain a little more? My experiment are not going well!

[Guest]

I'm saying that a relatively minor modification to an existing Avisynth deinterlacer could be done to accomplish your result. I don't have available free time to do that for you, but I would consider doing it as paid consultation. You were wondering if it is theoretically possible to do this variant deinterlacing. It's clear to me that it is possible. Maybe it is enough for you to know that.

Are you a programmer capable of writing Avisynth filters?

[otherman]

A programmer? Not at all!
Otherwise, I think this article (http://www.wisdom.weizmann.ac.il/~vision/SingleVideoSR.html) can get your interest. I'll definitely pay for this!

[bxyhxyh]

I think custom filter for 2 fields interlaced video is needed for this video.

But I tried normal deinterlacer.
AVISource("C:\Users\win8\Desktop\virtua1080i - 2linee.avi")

SeparateFields()

F13=SelectEven()
F24=SelectOdd()

F1=F13.AssumeFrameBased().SeparateFields().SelectEven()
F3=F13.AssumeFrameBased().SeparateFields().Selectodd()
F2=F24.AssumeFrameBased().SeparateFields().SelectEven()
F4=F24.AssumeFrameBased().SeparateFields().Selectodd()

F12=Interleave(F1,F2).Weave()
F34=Interleave(F3,F4).Weave().SwapFields()

Interleave(F12,F34).AssumeFieldBased().Weave()
QTGMC()


Now, I assume it is not pure interlaced video. Even if it is, something gone wrong when you interlace it.

I got 8 completely different frames from these 4 interlaced frames.
Result is not so handsome as you see.
Because it lacks 4 more interlaced frames.

[colours]

Someone somewhere just has got to make interlacing even worse than it already is.

And did someone say… custom filter? ( ¬‿¬)

This one does cubic interpolation. Nothing too fancy, but it doesn't look too bad.

function twolinebob(clip src, bool tff)
{
assert(src.isyuv(), "twolinebob: This only works with YUV because I'm lazy.")
src.converttoy8()
w = width()
h = height()
separaterows(4)
a = selectevery(4, 0)
b = selectevery(4, 1)
c = selectevery(4, 2)
d = selectevery(4, 3)
ab = selectevery(4, 0, 1).weaverows(2)
cd = selectevery(4, 2, 3).weaverows(2)
c_ = ab.mt_convolution("1", "0 0 0 -3 10 5 -2").separaterows(2).selecteven()
d_ = ab.mt_convolution("1", "0 -2 5 10 -3").separaterows(2).selectodd()
a_ = cd.mt_convolution("1", "-3 10 5 -2 0").separaterows(2).selecteven()
b_ = cd.mt_convolution("1", "-2 5 10 -3 0 0 0").separaterows(2).selectodd()
interleave(interleave(a, b, c_, d_).weaverows(4),\
interleave(a_, b_, c, d).weaverows(4))
tff ? last : selectevery(2, 1, 0)
chroma = src.isy8() ? last : ytouv(twolinebob(src.utoy8(), tff),\
twolinebob(src.vtoy8(), tff),\
last)
src.isyv12() ? converttoyv12().mergechroma(src.nnedi3(y=false,field=tff?3:2)) : last
(src.isyv16()||src.isyv411()||src.isyv24()) ? chroma : last
}

The provided sample seems to be BFF so use twolinebob(false) to fix that up. This will not work with Avisynth 2.5 and people shouldn't still be using 2.5 anyway.

There's some aliasing, but QTGMC doesn't really clean it up very well (or at all) so you may be SOL on that front.

neuron2: I strongly suspect it's not easy to just modify a sufficiently sophisticated deinterlacer (i.e. something nonlinear, e.g. eedi3) to make it work with such sources.

Edit: Considering that the source is a Bayer mosaic of some sort, exactly how it's converted to YCbCr will affect the correct way of handling the chroma channels; this script correctly handles the chroma in the provided sample, but it's hard to tell if it'll work on a real capture. Also, the RGB one is almost impossible to fix due to entire colour channels being omitted in every other field; temporal interpolation is pretty much the only recourse, but that's rather unreliable.

[Guest]

neuron2: I strongly suspect it's not easy to just modify a sufficiently sophisticated deinterlacer (i.e. something nonlinear, e.g. eedi3) to make it work with such sources. Believe what you want but I have extensive experience with deinterlace filter development and will respectfully disagree. Anyway, what place does "strong suspicions" have in a technical matter? Even your script above would perform much better if implemented as a native filter. Let's continue that debate in PM if you need to, so as not to clutter this thread.

[foxyshadis]

Now that I know this is Magic Lantern, I can tell you that your idea of how this deinterlacer needs to work is way off. The sample files you've generated are nothing like the raw cr2/dng files the camera would generate. Canons are all basic 2x2 RGGB Bayer sensors, nothing exotic like Sony's RCGB, but that's still much different than what you created. A good solution to your file would not work for what you're looking for. (Smaller sRAW files are already converted to YUV 4:2:2, or 4:2:0 for sRAW1, with top-left chroma positioning, but they're not a "typical" YUV<->RGB matrix, it's a custom one.)

There are plenty of cr2 files out there people can play with, although I'm not aware of any burst samples that would be suitable for deinterlacing. I can try creating some, I have a Canon G1X, if I can get the hosting space. They're about 20mb a pop, but I can try sRAW too. Never used that myself.

[Audionut]

Magic Lantern now has access to 14 bit raw live view data (the same data as CR2 raw), simply line skipped and/or pixel binned (by Canon) down to HD resolutions.

ML also has access to line scanning the raw data at different ISO. Since the RGGB pattern is contained over 2 lines, and interpolating only the RG data or the GB data would be rather complicated, a dual line process is employed to capture the full RGGB data.

As the bandwidth of the storage system is rather limited on most cameras, the idea is to create interlaced content. Here, the first frame can be captured by only scanning and outputting the even dual line feed, and the next frame the odd dual line feed.

Color space issues aside (I'm not sure how that effects a deinterlacer), the real issue IMO, is first determining if this dual line content can be sufficiently deinterlaced, without artifacting that would make the content unusable.

Of course, the option is there to output single line interlaced content, with each line only containing either RG, or GB data. And the developer has provided an application to output this content. I have created a short 21 frame output.

https://www.dropbox.com/sh/e9jyoryn77xog2i/AACE0eEK75xNJpOFnCeRlrN-a

These frames contain valid RG or GB data, with the other lines in the frame containing 0's. Separatefields() will remove the 0's, leaving just the valid data.

[otherman]

Audionut, your explenations are so better than mine! :thanks: