Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th May 2013, 19:35   #1  |  Link
Biggiesized
Registered User
 
Biggiesized's Avatar
 
Join Date: Apr 2007
Location: Pittsburgh, PA
Posts: 225
Optimal Bayer CFA codecs?

Given the growing abundance of digital cinema style cameras on the market, there has been a growing need to capture and record RAW sensor data onto high speed solid-state media. Compressed R'G'B' or Y'U'V' data in the form of (10-bit/8-bit) H.264 Intra or ProRes or DNxHD hasn't been a problem for some time, but 12-/14-/16-bit linear RAW can represent a lot of bandwidth.

If you want to go the mathematically lossless route, what are some the best codecs applicable today in terms of compression ratio and hardware requirements? I've done some research and here's what I've turned out:

1) JPEG-LS (HP Labs LOCO-I) - not that effective on CFA but perhaps it can be further optimized
2) JPEG2000 - popularized by RED even though they've gone the lossy route (3~12:1 compression ratio); there is a 5/3 CDF wavelet transform for lossless compression
3) Various vector prediction techniques - one example
4) Libima - experimental; not sure if it's abandonware
5) CinemaDNG - I believe it supports a HuffYUV style lossless compression mode, at least on the upcoming BMPC

Any others to consider?
Biggiesized is offline   Reply With Quote
Old 8th May 2013, 21:21   #2  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
As it comes to codecs- Cineform, ffv1, but none has a hardware implementation.
kolak is offline   Reply With Quote
Old 8th May 2013, 22:03   #3  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by kolak View Post
As it comes to codecs- Cineform, ffv1, but none has a hardware implementation.
cineform isn't "mathematically lossless" as per his requirements

(Also cineform does have a hardware implementation on cinedeck recorders)
poisondeathray is offline   Reply With Quote
Old 8th May 2013, 22:22   #4  |  Link
Biggiesized
Registered User
 
Biggiesized's Avatar
 
Join Date: Apr 2007
Location: Pittsburgh, PA
Posts: 225
Quote:
Originally Posted by poisondeathray View Post
cineform isn't "mathematically lossless" as per his requirements

(Also cineform does have a hardware implementation on cinedeck recorders)
I wonder if David Newman could re-optimize the codec for lossless coding.

Does FFV1 handle RAW bayer data?
Biggiesized is offline   Reply With Quote
Old 8th May 2013, 22:25   #5  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
I spoken with him about it- he seems to be not interested. Lossless coding requires different approach, no?
Also 3:1 mode gives in real world way good enough source to start with- there is really no need for lossless mode.
RED users use 5:1 or 7:1 even for 100M$ productions.

I think 3:1 is a nice compromise.

No ffv1 does not understand RAW data, but I managed to unpack RAW data into 16bit TIFF and than encode it.
No idea if I broken 12bit DNG or not
After debayering it looks fine, so maybe not

Last edited by kolak; 8th May 2013 at 23:31.
kolak is offline   Reply With Quote
Old 8th May 2013, 22:30   #6  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
Quote:
Originally Posted by poisondeathray View Post
cineform isn't "mathematically lossless" as per his requirements

(Also cineform does have a hardware implementation on cinedeck recorders)
Cinedeck recorders run on WIN
kolak is offline   Reply With Quote
Old 8th May 2013, 23:08   #7  |  Link
Biggiesized
Registered User
 
Biggiesized's Avatar
 
Join Date: Apr 2007
Location: Pittsburgh, PA
Posts: 225
Quote:
Originally Posted by kolak View Post
I spoken with him about it- he seems to be not interested. Lossless coding requires different approach, no?
Also 3:1 mode gives in real world way good enough source to start with- there is really no need for lossless mode.
RED users use 5:1 or 7:1 even for 100M$ productions.

I think 3:1 is nice compromise.

No ffv1 does not understand RAW data, but I managed to unpack RAW data into 16bit TIFF and than encode it.
No idea if I broken 12bit DNG or not
After debayering it looks fine, so maybe not
5:1 is okay because it hides the artifacts well enough when downsampling from 5K to 2K. Even seen the native resolution compressed that much? There are tons of picture artifacts like rampant softening and poor chroma resolution.
Biggiesized is offline   Reply With Quote
Old 8th May 2013, 23:24   #8  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
Seen few posts on RED forum which say that 5:1 and 7:1 mode is used a lot. 5:1 seems to be prefered one.
kolak is offline   Reply With Quote
Old 25th March 2014, 16:27   #9  |  Link
zcream
Registered User
 
Join Date: Sep 2007
Location: Sydney Australia
Posts: 33
Does anyone know how a CFA Bayer data is encoded via lossless jpeg ?
The code in public libraries only has examples for RGB or 4-channel debayered data.
I could not find any sample code of lossless jpeg used for RAW bayer compression.

DNG SDK does it, but its hard to comprehend.
zcream is offline   Reply With Quote
Old 6th April 2014, 17:53   #10  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
I dimly recall that JPEG-XR was to have Bayer support. I don't know if that ever made it into an implementation though.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Tags
bayer, cfa, codec, color filter array, lossless

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.