Can you encode video that hasnt been debayered?

[ExperimentalAnimations]

Im speaking of raw data thats streamed out of a camera onto a computer, where it has to be debayered after its captured. Can you encode this data (using lagarith for example)? Technically this data has no colorspace, so im wondering if that might conflict with encoders.

[Sagekilla]

That video would be stored as simply a grayscale video (YUV with U/V set to 128). I don't know how well that would compress but it should be perfectly legal to say "Hey, here's some YV12 video, encode this." You just need to force the encoder to assume it's YV12 with no U or V component.

[ExperimentalAnimations]

Would i still be able to debayer it after i encode it to lagarith?

[Sagekilla]

Well, I don't know how debayering is generally done, whether through a dedicated piece of software that accepts some arbitrary grayscale input that's assumed to be from a bayer sensor or something other means.

If there was an avisynth filter for doing this, I'd say go for that. Unfortunately, I don't know of any filters that can actually do this. If you ask -real- nicely in the avisynth development forum, someone might do it. Especially considering something like this could be a nice step towards support raw processing on avisynth.

[Hellworm]

I don't know how you feed the raw data to avisynth or how you retrive it, but it should be simple to write an avisynth filter that losslessy converts from raw bayer to YUY2 and vice versa.
Normal bayer pattern has two pixel of green per each red and blue, so the filter would only need to reorder the data to use green as luma and red and green as chroma.

[Sagekilla]

I don't think you would be able to do quite that. Yes, luma is predominantly green but that would give incorrect luminance levels. Likewise, for chroma you'd get bad color since chroma is based off RGB and not just red or green. But, it would be trivial to create a filter that would debayer your video.


As for feeding raw data into avisynth, if it's encoded with some codec, you need to find a way to force that codec to output it in grayscale form (no interpolation performed). Otherwise, you'd need to write a custom codec to force this.

[Wilbert]

@ExperimentalAnimations,

What kind of formats are supported by your camera?

@others,

Normal bayer pattern has two pixel of green per each red and blue, so the filter would only need to reorder the data to use green as luma and red and green as chroma.
How would you encode that? I.e. with a codec that encodes to RGB, or something else?

[ExperimentalAnimations]

This is the camera

http://www.sumix.com/products/cameras/smx-12a2c.html

I only know that it can output RAW, or cineform RAW (lossy) if you buy the program, but either way its bayered and needs to be debayered in post, which i understand goes directly into RGB colorspace (which is 3x the size), which is ideally what i would want since my entire pipeline from start to finish would be in RGB.

So I just wanted to know if one could encode bayered RAW video into lossless compressed format which could then be debayered at a later time.

[Sagekilla]

@Wilbert: No, A bayer pattern has a filter that only allows R, G, or B values. When a sensor on any bayer camera (pretty much every sensor out there, with some exceptions) captures an image, the raw image is composed of a grayscale representation of the scene where alternating pixels are the values for R/G/B. The data is usually stored as a grayscale image because it only has one component for each pixel. The other components need to be interpolated in software.


@ExperimentalAnimations: Yes it should be perfectly legal to encode that bayered RAW video into something like lagarith. As I said before, it should come out as nothing but a grayscale image to most programs, except with a higher bit depth than usual (most of the time 10 or 12 bit). If I knew how to work with individual color channels in avisynth, I could try writing a primitve debayer filter for you. Unfortunately, I don't know any decent debayering algorithms either. It's tricky to design a good one that doesn't give you artifacting.


Edit: The proper term for anyone curious is "Demosaicing"

[Hellworm]

I don't think you would be able to do quite that. Yes, luma is predominantly green but that would give incorrect luminance levels. Likewise, for chroma you'd get bad color since chroma is based off RGB and not just red or green. But, it would be trivial to create a filter that would debayer your video.


No, only for compression, the grayscale image would contain very uncorrelated informatik (rgb pixels next to each other) which is basically incompressible.
But if the encoder sees green as luma and red and blue as chroma it can efficiently compress it even if it's not correct. And as the filter would only reorder pixels the whole thing is losslessy reversible.
Of couse if you view it directly you only get strange colors, but the point is to compress the video and demosaic at a later time.

[Wilbert]

@ExperimentalAnimation,
You should be able to convert this stuff (to a pnm sequence for example) with dcraw (http://cybercom.net/~dcoffin/dcraw/), and you can import your pnm sequence in AviSynth.

Also could you upload a sample clip (say 10 frames) in raw format?