Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
2nd January 2011, 08:36 | #1 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
10bit quicktime sample requested v210 format
Hello,
In order to test and complete my Deep Color Tools, I need a sample Quicktime movie in v210, yuv format. Ideally, I just need a few frames of real 10bit colorbars, also a short clip of real video would be nice. Better yet, how can I create such files with free software. I have no way to acquire such video, so I'd have to export colorbars or some kind of 10bit yuv paintprogram. Last edited by jmac698; 2nd January 2011 at 10:05. |
3rd January 2011, 02:58 | #2 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Here are a few frames:
For both samples, I exported AJA v210 mov using "trillions of colors" (which indicates 10bit+) Sample 1: 1280x720p29.97 Belle Nuit Test Chart Method used: imported 8-bit RGB TIFF into After Effects in 32-bit float mode http://www.mediafire.com/?m2nc3n920kui5ws#1 Sample 2: 1920x1080p50 Park Joy Method used: imported 3840x2160,16-bit RGB SGI sequence into AE, scaled to 1920x1080p50 in 32-bit float mode http://www.mediafire.com/?by46kau3h677e9j PS. You can encode to v210 in avi or mov container using ffmpeg, but I'm not sure of the precision or if it's a 10-bit+ workflow for RGB<=>Y'CbCr conversions e.g ffmpeg -i input.avi -vcodec v210 -an output.mov Let me know if you have other requests Cheers |
3rd January 2011, 08:22 | #3 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
That's great, thanks! Of course if you have After Effects, it's not a big deal, but I'm making a free program. What sorts of filters do you think would be useful in Avisynth that you can't do in your tools?
I was thinking of some of the cleanup you do in old film restoration. |
3rd January 2011, 13:47 | #4 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Quote:
but regarding your deep color tools - any sort of color or levels adjustments in 8-bit will introduce more banding than when used in 10-bit+ mode, even if the final output was intended to be 8-bit Y'CbCr so something like curves (hard to do in avisynth in a user friendly manner) , levels , etc... |
|
9th January 2011, 15:10 | #5 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
If you were following this thread, I've made working 48bit levels management in my deep color utilities, and also was able to write a new file importer for v210 files - thanks to your samples!
And yes, I could use more samples, specifically made ones for testing. I need these: 480p, 480i, and then all resolutions with sound. Really, it's up to you - what files would you use in production? And some colorbars like these http://avisynth.org/mediawiki/HDColorBars I can soon let you use many avisynth plugins with full 48bit processing - for example IVTC can be done on upper 8 bits as a proxy and then applied to 48bits. I can easily do blur/sharpen/gamma correct resize/color conversion etc. I think this is exciting And where did that parkjoy come from? It's definitely using all the bits. Last edited by jmac698; 9th January 2011 at 15:12. |
9th January 2011, 16:11 | #6 | Link | |||
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Quote:
if it's generated from avisynth, you can encode with ffmpeg (accepts avs) e.g. ffmpeg -i input.avs -vcodec v210 -an output.mov But HDColorBars returns YV12 according to info() , so I'm not sure how good the v210 sample will be. I think you would want a 10-bit or higher Y'CbCr source, or at least 8-bit RGB => 10-bit v210 Y'CbCr 4:2:2 Quote:
Quote:
You can find lower res versions here http://media.xiph.org/video/derf/ But the original SGI scans can be found here http://www.its.bldrdoc.gov/vqeg/ Last edited by poisondeathray; 9th January 2011 at 16:54. |
|||
9th January 2011, 17:19 | #7 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
Yes, avisynth only uses 8bit, but I invented a pseudo-format which stores 16bits as two video clips, then functions to treat them as one.
And I didn't explaiin, I'm wondering if you have software that creates 10bit color bars originally. Any pro equipment should send this over SDI and you can just cap it. Actually SD ones are good too. Just helps me verify if I generate it in script in 10bit someday, or use as is for now. Even with ffmpeg, I have no way to generate real 10bit files. I can transcode 8bit source to 10bit files, but where is my 10bit source generator? ffmpeg gives you a color generating function but it's only specified in RGB. Update: very cool! I've seen those images many times, never knew where they came from. Great link! I can use that rgb2yuv program, but how to create 16bit SGI files? Even so, I still can't create YUV directly (but should be enough for 10bit yuv). If there's an output plugin for povray, that's pretty easy to script in. Hmm... Update2: yep, povray can create 16bit linear rgb in PNG and PPM. With some conversions I could finally get 10bit yuv, what a pain though. Last edited by jmac698; 9th January 2011 at 17:40. |
9th January 2011, 18:03 | #8 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
The 16-bit SGI files were scanned from film , you can use other utilities to manipulate them e.g. xnview
You can generate RGB test patterns in software in 16 or 32bit mode, export using 16-bit tiff of png, or 32bit openexr . Then do the conversion to v210 in 32-bit linear light. I'm not familiar with povray, but if it can't do this then I'll generate some SMPTE bars for you |
13th January 2011, 17:39 | #10 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
How do you know it's a "valid" 10bit source? The answer is you don't know unless you have something like a gradient to look at. Color bars , SMPTE bars , etc... are less useful for this testing , because there are no gradients - they show discrete solid uniform colors (ie. already quantized, so 8bit will look the same as 10bit). While you can get gradients in live footage (like skies, shadows), it's easier to see with computer generated ramps, and you can more easily start with higher precision (like a 16 or 32bit RGB format). Not all cameras output a 10bit signal through HDMI or HD-SDI (many are actually output 8bit even though the A/D signal processing upstream may be higher)
v210 derived from a true 10-bit or higher source should show a smooth histogram for gradients, not quantized values or banding. (if you try to generate a v210 sample from an 8bit greyscale ramp for example, many programs will warn you that the bit depth is exceeded. If you do it anyways, and re-import, you will notice banding and aberations in the histogram) So even if you're working in a 16 or 32-bit workspace with 10-bit v210, if that asset was derived from an 8-bit source (computer generated or not), you may show banding. The higher precision workspace will help to reduce new banding during manipuations, but not help "baked in" values from an 8-bit source. For example , in 16bpc , RGB values can be 0-32768 in 16bit instead of 0-255 in 8bit mode. But the values are already quantized as soon as it touches avisynth, even before you do any adjustments or color corrections. v210 decoded properly same v210 clip decoded in avisynth ffms2, direct avs importer But ffmpeg does support 10-bit input and output directly; -if you feed true v210 source , encode v210 => no banding -if you feed true v210 source through .avs, encode v210 => banding (and ffmpeg give warning source does not equal 10-bit) So it appears avisynth is indeed the culprit I would recommend making testclips derived from 16bit RGB generated ramps, where defects can be more easily seen. Any curves or levels adjustments made will be very sensitive to show breaking up in the histogram, even if the visual change is subtle in the actual image (for example a 6-bit LCD panel might display differently than a 10-bit panel, but the histogram will be sensitive to the banding eitherway) I'll test your "deep color tools" when I get a chance , if 9bits were salvaged or it was dithered as you mentioned in the other thread, it should make a difference For now here are some other clips: Radial Ramps, Computer generated 16bit RGB ramps => v210 http://www.mediafire.com/?cxc8bmw5haceclo SMPTE NTSC Bars 10bit RGB => v210 http://www.mediafire.com/?bclge7uam81a787 5sec clip shot with Arri Alexa in ProRes422 => v210 (mediafire only accepts 200MB clips, so this is split rar archive) http://www.mediafire.com/?fzxi5p7f6kikg74 http://www.mediafire.com/?pqrydie75vmv9bv |
13th January 2011, 18:32 | #11 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Ah, POV-Ray! Haven't worked with it for many years, but oh yeah, how I love(d) it!
(This scenery I did back in 1999(!) - and so is the used hardware. The original task was to build a room-fitting table in realworld. It's a tragedy the script got lost in a HD crash.) (The table, however, still does serve well.) You know, for "2D drawing" in POV-Ray, you need to switch to another camera model: Code:
camera { orthographic location <0,0,0> direction <0,0,1> right 1.33*x up y sky <0,1,0> }
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
23rd January 2011, 05:08 | #13 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
If you're referring to the "little spikes" on the histogram, I think they are rounding errors from 16bit RGB => 10bit Y'CbCr . You don't see as many "little spikes" in 16bit RGB => 10bit RGB for example (and of course none when going 16bit RGB=>16bit RGB) The "little spikes" don't show up on watching the footage, only the "big banding" in the histogram correlates with perceived "banding", on live action (real) footage. I suspect you would see the "little spikes" on the histogram even with a 16bit Y'CbCr format too (if there was such a thing), because Y'CbCr doesn't overlap RGB completely The real difference is when you grade or make adjustments like curves, levels; 8bit just falls apart Last edited by poisondeathray; 23rd January 2011 at 05:15. |
23rd January 2011, 11:28 | #14 | Link |
Avisynth language lover
Join Date: Dec 2007
Location: Spain
Posts: 3,431
|
I think what's going on is this.
There are only 256 histogram bins, each covering a range of 128 32-bit values. The V210 input has (I think) 880 (220*4) different values, ie the usual [16, 235] expanded from 8 bits to 10. So some of the histogram bins will cover 3 input values and some (roughly 7 in 16) will cover 4. Hence the spikes. In the other histogram, there are only 220 input values, so we get 35 missing RGB values, ie the black gaps in the histogram. |
23rd January 2011, 14:16 | #15 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Thanks poisondeathray and Gavino.
This is what I expected. When I export your 16btr RGB ramp file to 10bit 4:4:4 BM codec all spikes gets soften- AE show smooth histogram, so I think you're right Does it mean codec is 4:4:4 capable? New Canopus HQX 10bit (4:2:2) codec also smooth them, so does it mean it's actually 4:4:4 capable? Is this histogram test 100% accurate to say if converted video is 10bit or not (eg. for codec, workflow testing)? Yes- 8bit falls very quickly during grading. Thanks, Andrew Last edited by kolak; 23rd January 2011 at 14:22. |
23rd January 2011, 15:42 | #16 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
kolak - the ramps I uploaded were all v210 , but derived from 16bit RGB (so the name might be misleading) - so none of them were 4:4:4 (I don't think the sampling would matter much for greyscale anyway) . If you want I can upload some other patterns for testing, like color ramps with 10-bit 4:4:4 RGB. If you have AE, this is pretty simple to make also
I'm not sure what you mean by "Does it mean codec is 4:4:4 capable? " - can you explain ? There are 10-bit 4:4:4 Y'CbCr and RGB variants, I think BM has both Quote:
Even if you convert 8-bit Y'CbCr footage to v210 , and work in a 16-bit or 32-bit workspace - there are benefits over working in an 8-bit workspace with native 8-bit footage. You can apply 2 instances of levels filter (the 2nd one is to monitor changes in the histogram) , If you make adjustments to the 1st instance, you will see much more banding than if you used native 8-bit footage (instead of transcoding to v210) But the avisynth import definitely indicates a problem - because there is quantization even before you 've made any manipulations. Even if you started out with 8bit gradient, it wouldn't be that bad as in that screenshot What gavino says above is also true when any Y'CbCr is exported from AE - it's not using full range <a bit off topic here> If we were to use "full range" Y'CbCr in 8-bit distribution format , there would be a lot less banding (a lot of the banding can be attributed to the truncated range) , but of course this isn't feasible for stuff like blu-ray , dvd e.g. Flash supports full range flag . Here are 2 examples, one is full range, one is standard range (both Y'CbCr data, and "flagged" using x264) So the full range is decoded to RGB for display properly (blip.tv hosts original file, so you couldn't do this with youtube for example, which would re-encode and display as Rec.709) ConvertToYV12() No flags http://www.blip.tv/file/4409086 ConvertToYV12(matrix="PC.709") Fullrange flag http://www.blip.tv/file/4409087 Last edited by poisondeathray; 23rd January 2011 at 16:02. |
|
23rd January 2011, 16:37 | #17 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
I know about avisynth being 8bit- wish for more
I know that your file is 10bit 4:2:2 (just converted form 16bit RGB), but once exported through BM 4:4:4 codec all spikes are gone- gradient is smooth. Based on you explanation, I assume that when codec is capable of 4:4:4 there won't be spikes or there will be much less. This seams to be true- any 4:4:4 codec smooths spikes during my tests. Any 4:2:2 kept them or make even more. The only problem was with HQX codec (oficially 4:2:2), but it also removes spikes- so maybe it's internally 4:4:4. This was my question- are removed spikes prove that codec is 4:4:4? With full range you have 255 levels with YUV 220- not? Thx, Andrew Last edited by kolak; 23rd January 2011 at 16:49. |
23rd January 2011, 17:02 | #18 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
I see spikes with BM4:4:4 if the source is v210 (i.e. taken from that sample) - it won't "fix" the spikes or fill in the missing data . I tested with BM 10bit 4:4:4 RGB fourcc is "r210" . Can you upload sample where "spikes" are fixed ? I can't replicate your results
4:4:4 shouldn't be much different than 4:2:0 for a greyscale image . I think the difference is Y'CbCr <=>RGB conversions not in full range and colorspaces not overlapping completely, not an issue sampling . For example if you use sheervideo, it has a 10-bit 4:4:4 Y'CbCr format, and you still see "spikes" Quote:
BTW, I'm not worried about the "spikes", the banding is the real problem that manifests as real banding when viewed normally Last edited by poisondeathray; 23rd January 2011 at 17:07. |
|
23rd January 2011, 17:15 | #19 | Link | |
Avisynth language lover
Join Date: Dec 2007
Location: Spain
Posts: 3,431
|
Quote:
As I said, the spikes come from the reduced range in Y'CbCr, and the chroma sampling is irrelevant for grayscale. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|