Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th April 2009, 00:22   #1  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
Shifting Histograms

Hi,

Recently I was comparing codecs for the 100th time, and I made a surprise discovery. Some codecs shift the entire histograms of image up or down by 1-2%. Xvid and Ffdshow (trying several compressions) both exhibited clear histogram shifting. X264 and Morgan Multimedia Mj2k held the histograms in place.

I was wondering if this is well known? I expect lossy compression to lose image detail, but not to make entire red, green or blue channels lighter/darker.

Cheers,
A
a uio is offline   Reply With Quote
Old 21st April 2009, 20:05   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,416
You sure you're decoding the outputs correctly? The TV / PC levels issue is pretty well known and causes lots of issues.

~MiSfit
Blue_MiSfit is offline   Reply With Quote
Old 22nd April 2009, 00:30   #3  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
Quote:
Originally Posted by Blue_MiSfit View Post
You sure you're decoding the outputs correctly? The TV / PC levels issue is pretty well known and causes lots of issues.

~MiSfit
Yes. Those typically cause errors in the range of 5-6% and are easy to see. These 1% shifts are quite subtle unless you look at the histograms and toggle back and forth.

My first guess would be that there is truncation instead of rounding going on in calculations that consistently shifts all values in some direction.
a uio is offline   Reply With Quote
Old 24th April 2009, 16:54   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,639
Quote:
Originally Posted by a uio View Post
Hi,

Recently I was comparing codecs for the 100th time, and I made a surprise discovery. Some codecs shift the entire histograms of image up or down by 1-2%. Xvid and Ffdshow (trying several compressions) both exhibited clear histogram shifting. X264 and Morgan Multimedia Mj2k held the histograms in place.

I was wondering if this is well known? I expect lossy compression to lose image detail, but not to make entire red, green or blue channels lighter/darker.
Well, since you're comparing in RGB, you're likely comparing differences in Y'CbCr > RGB algorithms. For it to be a codec difference, you should compare the decoded YV12/IYUV output.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Instant Video

My Compression Book

Amazon Instant Video is hiring! PM me if you're interested.
benwaggoner is offline   Reply With Quote
Old 30th April 2009, 21:08   #5  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
Quote:
It might be in the YUV/RGB conversion. You could try directly comparing YV12 if you haven't done that already.
Indeed, directly encoding YV12 sources and decoding to YV12 eliminated the histogram shifting. So it seems that between VDub, Adobe Premiere, and the codecs, I'm getting different YUV/RGB conversions. If I find out more, I'll post it.

Thanks,
A
a uio is offline   Reply With Quote
Old 30th April 2009, 22:02   #6  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,312
Like Ben said. Have a look at: http://avisynth.org/mediawiki/Colorimetry
Wilbert is offline   Reply With Quote
Old 4th May 2009, 06:04   #7  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
I did some more controlled testing. Using only VirtualDub, I encoded a sample clip and then decoded it. To test the YUV conversions that were occurring inside the codecs, I set VirtualDub to output RGB24 to codecs during encoding, and to request RGB24 from codecs when decoding. The histogram shifting reappeared, in contrast to tests that directly encoded/decoded YV12 instead, where there was no shifting.

Morgan MJPEG2K, X264, and Lagarith (YV12 mode) all had no histogram shifting.

Xvid made all three RGB channels 1% darker or so.

Ffdshow Mjpg made the Red channel 1% darker or so, the Blue channel a much smaller about darker, maybe 0.2%.

To make sure this wasn't a 601/709 issue, I repeated the tests on a solid 50% gray frame. The same shifting still occurs.

I iterated the procedure for Xvid and Ffdshow, and the shifting accumulates. Xvid frames get darker with each iteration. Ffdshow frames turn green/cyan.

After five iterations, the differences are very clear to the naked eye. If anybody else cares to try these simple tests, I'd love to hear if they get similar results.

Regards,
A
a uio is offline   Reply With Quote
Old 13th May 2009, 19:11   #8  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
Update: The issue has been confirmed by the Xvid team and was indeed a rounding problem in the RGB/YUV conversion code. A fix is in the works.

Cheers,
A
a uio is offline   Reply With Quote
Old 14th May 2009, 15:04   #9  |  Link
Carl Sundbom
Registered User
 
Join Date: Apr 2009
Posts: 1
Ah, this explains what I've been seeing too. Thought I was going crazy. Thanks for figuring this out, looking forward to the fix.
Carl Sundbom is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.