VDub2 Canopus HQX 16-bit negotiation issue

[poisondeathray]

Originally Posted by kolak

So this is basically v210 encoder and ProRes encoder?

Yes, ffmpeg v210 and ffmpeg prores encoder(s) (native, and prores_ks, and prores_aw)

[kolak]

DNxHR as well (at least in ffmpeg 4.3.2+)
Good to know- thanks.

update: it's Resolve decoder, not ffmpeg encoder.

[poisondeathray]

Originally Posted by kolak

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out

I heard apparently there are some bugs in newest resolve waveform...I'm still using older version as I have a few projects in progress and I never upgrade in between

Temporarily assign clip attributes to "full" range, and you will see difference between original v210 (or resolve v210) and ffmpeg v210 . As you already know, resolve is working in RGB float. When "auto" is set and clip is unflagged, or flagged limited, it takes Y 64-940 to "map" to RGB 0-1023 . But internally nothing is clipped at that stage, it's working in float RGB. It's just the display at that point or RGB interpretation, not the actual file

For premiere you can see in lumetri scopes too set to float, you might have to increase the window size, but there is clipping at low and high level with ffmpeg v210 version, but not "original v210", or "resolve v210". But I agree it is a "crap" display compared to ffmpeg waveform, or avisynth histogram(10)

DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.

It didn't in the past and it doesn't for me (just checked)

Exactly what are you doing for input and for measuring output ? What decoder is being used

[kolak]

Yes, in Premiere I sort of saw it when setting to float.

I used Resolve to check, so problem is probably there.

[poisondeathray]

Yes, resolve's "waveform" is not a true Y' waveform . It's actually RGB converted values - so that depends on how you convert to RGB

ffmpeg -vf waveform or avs's histogram(10) are true Y' waveforms , they are measuring Y' values directly

And it's not a "knock" against resolve. For grading software, that's what the scopes should be doing

[kolak]

I see it now in Resolve as well (with forced full range). It shows it in form of missing data, so your line basically ends at 1019. At the bottom it shows it in some strange way, but it's also visible (Resolve 17.1.1.).

[kolak]

It's Resolve DNxHR decoder is doing it. ffmepg waveform is fine.
Looks like anything "official" (Apple, AVID etc.) is now skipping those reserved bits and rest of the codecs/tools not. More mess to the already messy world of codecs etc.

[poisondeathray]

Originally Posted by kolak

It's Resolve DNxHR decoder is doing it. ffmepg waveform is fine.
Looks like anything "official" (Apple, AVID etc.) is now skipping those reserved bits and rest of the codecs/tools not. More mess to the already messy world of codecs etc.

Yes

I see it in resolve import of ffmpeg dnxhr_hqx too, but not with libavcodec decoding (0-1023)

If you use resolve to encode v210 (0-1023) to dnxhr_hqx, it's 0-1023 with libavcodec decoding, but reimporting to resolve also some clipping (again, temporarily set to full range clip attribute)

So that suggests it's resolve's dnxhr_hqx decoder behaviour

Cineform is full scale.

But you said cineform is 0-1023 ? In resolve ? Was that go pro cineform official, or libavcodec sdk implementation, or resolve cineform, or something else
So not all "official"

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

[poisondeathray]

Canopus/GV HQX exported from resolve is also 0-1023 in both libavcodec and re-import back into resolve is 0-1023

[kolak]

ffmpeg Cineform.
I don't think David was that paranoid with all those SMPTE specs

[poisondeathray]

Another thing maybe shekh can look at is vdub2's prores encoding show gaps when encoding v210 . Not just the clipping, but almost like 8bit conversion. ffmpeg's equivalent v210 to prores does not (4-1019, but smooth)

vdub2_v210_to_prores


ffmpeg_v210_to_prores
(see earlier post)
https://forum.doom9.org/showthread.p...46#post1940546

[kolak]

Originally Posted by poisondeathray

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

Yes, but they should leave files world.
Those reserved bits are actually coming from SDI I think.
I don't think HDMI actually needs them at all, but I may be wrong.

Yes, there may be bug in Vdub and data may be going over 8bit at some point (or maybe some auto format choice bug). It quite clearly shows 4 levels pattern.
Vdub2 is not the same as old Vdub. New features, but new mess related to whole ffmpeg etc as well.

[poisondeathray]

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

[Hotte]

Thanks poisondeathray for all your answers.

Originally Posted by poisondeathray

Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

The detail differences in the jpegs are not visible in the original TIFFs => JPEG issue.

You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

No idea how that works. Did not find anything like "z_convert_format". Could you give a little code example ?

You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

Yes if you think that this is acceptable for a preview I am with you.

If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end

Could you give me a good link where I can download test material ?

Just another question to add: What if I record with standard luminance range instead of full ? You said I should go for standard in the end anyway and during the process there is risk of IQ loss if conversion is not being done properly. My subjective feeling is always, that I loose dynamic range if I record with standard range. Is that correct or not ?

[kolak]

Originally Posted by poisondeathray

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

Yep. And after quick search I'm almost sure that those reserved bits are coming from SDI and HDMI doesn't actually need them, but for "compatibility" they preserved them in HDMI spec. Now they force them to files world. Don't like it.
Sometimes 4-1019 is called SDI Full

[kolak]

Originally Posted by Hotte

Just another question to add: What if I record with standard luminance Range instead of Full ? You said I should to go for standard in the end anyway and during the process there is risk of IQ loss if it is not done properly. My subjective feeling is, that I loose dynamic range if I record with standard range. Is that correct or not ?

This is somehow important for 8bit recording. You simply have less range: 235-16=219 vs 0-254=255, so actually quite a big difference (more banding etc.). For 10bit (or more) this is less relevant.

[poisondeathray]

Originally Posted by Hotte

Could you give me a good link where I can download test material ?

I posted a 0-1023 10bit greyscale gradient earlier

All NLE's should have 75% colorbars. I'd imagine Edius has them. Free version of resolve can generate them too. If you have problems I can upload bars later

"official" 8bit, 10bit, 12bit YUV values are listed in rp219

A proper 10bit YUV to 8bit RGB conversion should give you the exact numbers for 75% colors. e.g. 75% "red" should be 191,0,0 in 8bit RGB using limited range equations (rec matrix in avisynth). Or 180,0,0 using full range equations (pc matrix in avisynth)

(Subsampling is not an issue with "large" bars, only at the borders between colors)

Did not find anything like "z_convert_format". Could you give a little code example ?

This is avsresize, the avisynth implemenation of the zimg library
http://avisynth.nl/index.php/Avsresize


eg. avsresize. "RGBP" is 8bit RGB planar

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

"point" is just nearest neighbor algorithm. It just doubles the chroma samples in the width. e.g. If you had 1920x1080 4:2:2, the U,V samples are 960x1080. It just duplicates them to 1920x1080. So on a pattern, the color borders will look "blocky"

"Bilinear" samples a 2x2 grid (4 pixels)
"Bicubic" samples a 4x4 grid (16 pixels)

There are dozens (or more like 100's) of resampling algorithms.

So you can get different pixel colors, depending on the chroma upscampling algorithm for RGB conversion, but a large "bar" such as in color bar patters would be independent of chroma upsampling. But this is one of the reason why one program might do things differently tha another program

If you get "wrong" colors on large colorbars, then something is ... wrong

Chroma location interpretation can also affect results (one of the reasons for suggesting avsresize, one possible explanation for some of the differences you see), but more for 4:2:0 material because it's 1/2 sampled width and height, 4:2:2 is "only" 1/2 sampled width, the errors are less drastic. When you have different chroma interpretations ("center" or "mpeg1") vs. ("mpeg2" or "left") , the color borders can shift when do subsequent conversions. It's usually not that noticable with 1 or 2 generations. It's more of an academic issue, or if you deal with RGB content and graphics. Avisynth is a bit non standard, because "mpeg2" or "left" is the standard chroma siting in all programs

[Hotte]

Originally Posted by poisondeathray

All NLE's should have 75% colorbars. I'd imagine Edius has them.

Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

I tried

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:f=>rgb:709:709:f")

(exchanged the first l with an f). This shows only minor changes on a pixel level in comparison to ConvertToRGB24, no matter if I go for bilinear, bicubic or whatever. The overall very slight shift vs. yellow in the grass which the VDub2-TIFF holds against the Edius-TIFF (and this time I was using uncompressed V210 instead of HQX) is still there, but hey, differences are very slight and should be just fine for preview purpose.

[kolak]

Originally Posted by poisondeathray

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

Side note- ProRes444 modes do pass those reserved bits (in official Apple encoder).

[poisondeathray]

Originally Posted by Hotte

Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?

Yes. You have known colors. In YUV and RGB. In 8,10,12 bit . And rp219 lists the exact values for Y,U,V in 8,10,12 bit

For 8bit YUV, the colors in 8bit RGB will be slightly off, and it's expected to be +/-3. For example "75% Red" might be 192,1,0 or something (when using standard range equations). But 10bit YUV bars should yield perfect 8bit RGB values.

It's common to get slight color shifting when improper conversion is performed. A common scenario is a "fast" 8bit conversion to YUV before RGB. You're just ruling out if something was done incorrectly somewhere , maybe some setting, some switch - that's what all these low level tests are used for - so when you're doing your actual projects, there aren't any unexpected surprises (like clipping)