VDub2 Canopus HQX 16-bit negotiation issue [Archive]

kolak

6th April 2021, 20:18

Hi,
I've recently started to pass HBD-Footage (i.e. 10 bit) through AVS+ with VDub2 de- and recompressing with the Canopus HQX codec. Everything worked fine with 8-bit. The clip is export from Edius-X NLE using the Canopus (Grassvalley) HQX Codec which is capable of up to 8K, 10bit, 4:2:2 all-intra in AVI-Containers.

They arrive as YUV422P16 in AVS+ (due to info() and ffmpeg -i) loaded with FFmpegSource2(FILNAM, atrack=-1), further processing works fine. Results appear correctly in VDub2. The trouble starts with "Save as AVI" with constant "Video format negoation error". Output codec again is HQX, mode ist Fast Recompress.

Menu > Video > Decode Format... shows P216 as current format. Whatever I select as at least 10bit-output format: Error persists. Compression menu shows: "Valid pixel formats RGB RGBA YUY2" although the codec is listed under "Similar to Source: P216".

Strange enough: VDub2 has absolutely no problem to take any 10/16bit "hqxin.avi" and save it as "hqxout.avi" without AVS+ as Fast recompress with the same codec.

Any ideas ?

There must be some false detection/reporting.
VFW/DirectShow etc. version of Canopus HQX doesn't support anything above 8bit. It never did. Only Edius and Resolve can export/read 8bit+ data for HQX.

Hotte

6th April 2021, 20:32

Kolak, meanwhile I have the same impression: In vfw the HQX-codec is somehow false-detected:

VDub2 reports the codec with: Valid pixel formats: rgb rgba yuy2. This was true for its predecessors (Canopus-HQ/HQ lossless) but not for HQX.

Of course the producer is asked but they won´t probably give a damn about vfw and VDub...

So is there a way to override the check in VDub2 and just let compression go ?

kolak

6th April 2021, 22:51

Force YUY2 as pixel format. This will work.
There is no way to do any 8bit+ pixel formats with HQX- nothing you can do about it. It's the way how VFW/directshow codec is written.
Resolve uses native support over low level integration- it doesn't touch VFW/directshow.

rgb rgba yuy2 are still only valid pixels formats for HQX. Well- last time I touched it was was years ago, but I doubt anything changed. VFW/directshow are not very good technologies and pro companies don't touch them anymore.
As I said- HQX outside Edius (Resolve) is 8bit only.

Hotte

7th April 2021, 00:03

Thanks kolak.

I have had a look at other high-quality 10-bit all-intra codecs and there we have:

huffyuv, FV1
not compatible with Edius

cineform
compatible but very bad timeline performance

ProRes
compatible, relatively good timeline performance, very fast encoding, 422HQ has IQ pretty close to HQX at highest quality setting (to split hair: HQX superfine has a close to invisible advantage in preserving detail)

DNxHD
compatible but not available

Any other suggestions ?
HEVC / H.264 are not all intra so probably not the best intermediate choice...

My questions:
ProRes could be an candidate. I wonder why this proprietary codec is available in vfw from a technical pov (I thought vfw could not manage >8 bit ?) and from a license pov ? I mean 422HQ is pretty good and XQ freely available - how come ?
Are there any other hq-codecs available and installable - how does that work ?

Thanks for some elucidation

poisondeathray

7th April 2021, 00:25

cineform
compatible but very bad timeline performance

Interesting, it's faster in other programs than HQX. Probably the fastest CPU based high quality intermediate . (There are faster GPU ones, like Daniel2, NotchLC)

DNxHD
compatible but not available

You can use ffmpeg as alternative (unless you require something specific in vdub2 ?)

Any other suggestions ?
HEVC / H.264 are not all intra so probably not the best intermediate choice...

HEVC will be too slow to decode.

h264 can be set intra, but even with the fastest decode settings (tune fastdecode, intra) , is typically more sluggish on the timeline (high latency) compared to prores, cineform. (However, some editors can use GPU accelerated decoding of h264, but usually not 10bit422, only the 8bit variants)

But x264 can be significantly higher quality at qp 1 than prores xq4444 or HQX - PSNR is typically ~8-15 db higher)

My questions:
ProRes could be an candidate. I wonder why this proprietary codec is available in vfw from a technical pov (I thought vfw could not manage >8 bit ?) and from a license pov ? I mean 422HQ is pretty good and XQ freely available - how come ?
Are there any other hq-codecs available and installable - how does that work ?

The prores version in vdub2 is the libavcodec/ffmpeg version . It's not VFW.

Richard1485

7th April 2021, 02:01

I'd go with ProRes over DNxHD, which can cause problems with levels/gamma. It's a shame that Cineform gives you slow performance with Edius. Normally, I'd have picked that.

shekh

7th April 2021, 02:25

Hotte

7th April 2021, 02:32

Thanks posiondeathray and Richard.

Tried cineform again with other settings - doesn`t help: Too bad: Horribly slow on the timeline. Btw. Cineform is either interleaved YUV oder RGB - right ?

X264 wouldn't play at all if settings are very HQ / all-Intra but that might need more testing in detail.

DNxHD (generated with ffmpeg) is by far the fastest on the timeline - of course, it is the native codec of Grassvalley. Cannot see levels / gamma issues at first sight, but I will watch out for that, also I will be doing more detail comparison with ProRes - thanks Richard.

My question to @poisondeathray: With ffmpeg you mean I should call ffmpeg from outside taking the avs-script as input and output to DNxHD - right ?
For some setup reason I`d prefer to continue using VDub2 as host. So the question is if there is a way to integrate ffmpeg-DNxHD into the compression list of VDub2 ?

Hotte

7th April 2021, 02:36

Hi,
maybe the issue with "Similar to Source: P216" was because the codec was selected before choosing "Similar to source"? This filtering of codecs does not remove the selected one even if it appears inappropriate.

Yes that could pretty much have been the case. It does not appear anymore if I do things right.

poisondeathray

7th April 2021, 02:49

My question to @poisondeathray: With ffmpeg you mean I should call ffmpeg from outside taking the avs-script as input and output to DNxHD - right ?
For some setup reason I`d prefer to continue using VDub2 as host. So the question is if there is a way to integrate ffmpeg-DNxHD into the compression list of VDub2 ?

Yes, ffmpeg directly, with avs input

It should be possible to implement DNxHD in avlib-1.vdplugin (I don't know how to write vdub plugin - it would have to be shekh or someone else)

Hotte

7th April 2021, 07:54

Yeah! ***** The DNxHD would be a great addition to the compression menu as it seems to play some not unimportant role in professional video exchange and looks like being an excellent codec, also encoding is fully supported by ffmpeg.

I'd be ready to support that in any way that makes sense. I`d be happy to hear Shekh`s evaluation of the matter. I have no idea if that is a lot of work to do.

What I haven't quite understood is how you guys work with lossless codecs that are not supported by our NLEs ? I mean its nice to have encoders like huffyuv, FV1,Daniel2(?), NotchLC(?)... But how do you use these ? VDub2 is a great host but a very basic editor - isn`t it ?

Could somebody try to explain that to me?

poisondeathray

7th April 2021, 16:45

What I haven't quite understood is how you guys work with lossless codecs that are not supported by our NLEs ? I mean its nice to have encoders like huffyuv, FV1,Daniel2(?), NotchLC(?)... But how do you use these ? VDub2 is a great host but a very basic editor - isn`t it ?

Could somebody try to explain that to me?

(Daniel2 and NotchLC are not "lossless")

It depends on which ones - what codec, what NLE host, and what versions of the NLE host. Each editor has little quirks, and sometimes certain point releases of each editor might change behaviour

Many "lossless" codecs, especially "YUV" lossless, are not lossless in some NLE's. Many get converted to RGB for example - Huffyuv is a prime example. I don't know of any professional NLE that treats huffyuv as lossless (some ffmpeg based ones do, such as shotcut)

eg. Edius has a YUV capable timeline (some editors do not, such as resolve, vegas) , but some formats are converted to RGB in Edius.

e.g Premiere has a YUV capable timeline too, but all "lossless YUV" codecs do not work as lossless - they get converted to RGB. The exception is x264 lossless in certain point releases
for certain pixel formats

Float RGB can technically be lossless for anything (including conversion to/from YUV), but only if the program handles it correctly everywhere. For example, Resolve uses float RGB internally, but it does not do the timeline RGB to YUV conversion correctly, so exported YUV assets lose precision. You have to use full float formats, such as EXR image sequences to maintain true lossless in/out

What you should do is perform some tests in the editor that you're using

StainlessS

7th April 2021, 17:57

Many "lossless" codecs, especially "YUV" lossless, are not lossless in some NLE's. Many get converted to RGB for example - Huffyuv is a prime example. I don't know of any professional NLE that treats huffyuv as lossless (some ffmpeg based ones do, such as shotcut)

I'm currently not able to use HuffYUV on W7, [tried install but not showing up in VD2/VDMod].
But cant you just select "Fast Recompress" and in HuFFYUV select "Convert To YUV", "Predict Median", [or I think also 2 other modes]
or something like that [for lossless].

poisondeathray

7th April 2021, 18:24

I'm currently not able to use HuffYUV on W7, [tried install but not showing up in VD2/VDMod].

Was it for x64 ? There is a procedure to install it, I can't recall the exact steps

I think something like this
https://forum.videohelp.com/threads/381944-unable-to-install-huffy-codec-on-windows-7-%5Bsolved%5D

But cant you just select "Fast Recompress" and in HuFFYUV select "Convert To YUV", "Predict Median", [or I think also 2 other modes]
or something like that [for lossless].

The problem is not the codec. It's not just huffyuv, it's all VFW based "lossless" codecs

eg. They work fine in ffmpeg, avisynth , or libavcodec/ffmpeg based programs like shotcut

The problem is how other programs treat UT Video, huffyuv, lagarith, magicyuv, etc... in YUV mode

Hotte

7th April 2021, 19:37

Thanks for all your contributions.
Just a short Question: DNxHR (which is the successor of DNxHD) is not supported by ffmpeg. If you buy that codec is there a way to use ffmpeg as host with it ?

poisondeathray

7th April 2021, 19:46

DNxHR (which is the successor of DNxHD) is not supported by ffmpeg. If you buy that codec is there a way to use ffmpeg as host with it ?

ffmpeg supports DNxHR, you have to set the profile

Encoder dnxhd [VC3/DNxHD]:
General capabilities: threads
Threading capabilities: frame and slice
Supported pixel formats: yuv422p yuv422p10le yuv444p10le gbrp10le
dnxhd AVOptions:
-nitris_compat <boolean> E..V....... encode with Avid Nitris compatibility (default false)
-ibias <int> E..V....... intra quant bias (from INT_MIN to INT_MAX) (default 0)
-profile <int> E..V....... (from 0 to 5) (default dnxhd)
dnxhd 0 E..V.......
dnxhr_444 5 E..V.......
dnxhr_hqx 4 E..V.......
dnxhr_hq 3 E..V.......
dnxhr_sq 2 E..V.......
dnxhr_lb 1 E..V.......

But there are some differences that differ from official Avid DNxHR vs. ffmpeg DNxHR - e.g. adaptive color transform is not implemented in ffmpeg version

Hotte

7th April 2021, 20:47

Yes, I found the post again where sbd explained how to apply HR modes to DNxHD to also achive the high resolutions.

I just tried this and out and compared the results to native Edius-X DNxHR exports:

ffmpeg -i F:\Temp\inputhqx.avi -c:v dnxhd -profile:v dnxhr_hqx -c:a pcm_s16le output.avi

Sorry to tell that the quality is not outstanding and noticeably less sharp than native HR or ProRes high quality. Although file sizes are all in the same range.

So probably less interesting to integrate as ffmpeg VDub2 Encoder...

kolak

7th April 2021, 21:01

Hmmm....
DNxHR is about as efficient as GV HQX if you manage to encode to around same file size. ProRes is maybe 1-2dB in PSNR better, but this is totally meaningless in real world.

If you doing such a test make sure source was not encoded earlier at all (I mean it doesn't come from eg. Alexa which was set to ProRes encoding). You will get so wrong results.
Use REAL uncompressed source or one which did not go through any DCT codec (if you testing DCT codec). Other solution is shifting image few pixels horizontally and vertically, so you break old block boundaries.

kolak

7th April 2021, 21:09

cineform
compatible but very bad timeline performance

Where?

Cineform is top choice for editing. In app which properly supports it (Premiere, Resolve, Scratch, Vegas if I'm correct) you have amazing performance and at any time you can increase it by miles just by swapping decoding to 1/2 (or further) resolution.

Hotte

7th April 2021, 21:18

kolak

7th April 2021, 21:21

Sounds like issue unrelated to codec itself, but its implementation (or just preview?)
Edius use AVID libraries. Ffmpeg uses own DNxHR encoder, but I tested it some time ago (year or more) and it was actually slightly better than AVID (and better than DNxHD itself). Maybe ffmpeg people messed things.

You also have to be aware than DNxHR is slightly different than ProRes and both are very different than HQX or Cineform.
DNxHD/R are strict CBR codecs, so each frame is exactly same size (easy or difficult scene = sam bitrate), which has its good and bad sides. ProRes is VBR, but restricted VBR. It can go low with bitrate for very easy scenes, but it can peak only 10% above predefined target bitrate. HQX and Cineform are "very VBR" (specially Cineform). They are design to keep relatively same quality between frames, but this means some frames can be 2x bigger than others, so bitrate is very VBR.
Side note- Cineform was design more for high-end finish/film and film scanning. It behaves differently when you feed it with uncompressed (eg. film scan with all grain etc.) source compared to something already encoded (specially DCT encoded). FS3 mode was introduced specially for anime where Cineform tend to use "too low" bitrate (no noise, grain etc) which companies like Disney did not like. For "normal" footage FS3 is overkill- you getting quality improvements, but for the cost of relatively high bitrate raise.

In real world all of those codecs are good and there is not much difference between them.
HQX is only 10bit 4:2:2 (+alpha). ProRes is always YUV internally (for 4:4:4 12bit YUV+ losslessly compressed alpha). Cineform and DNxHR can be both 10bit 4:2:2 YUV and 12bit RGB+uncompressed alpha.

Hotte

7th April 2021, 21:36

Where?

Cineform is top choice for editing. In app which properly supports it (Premiere, Resolve, Scratch, Vegas if I'm correct) you have amazing performance and at any time you can increase it by miles just by swapping decoding to 1/2 (or further) resolution.

Horribly slow (1-2 Frames/s at 4K 10 bit) timline performance in Edius-X. But I am happy to learn.

So this is what I do:

- I do not have any raw video file but just a 4K 10bit 4.2.2 25bit mov out of a Panasonic G9
- I export this from Edius-X with Canopus HQX 10 bit. Normally this is perfect input for AVS+, but today I just load it into VDub2
- In VDub2 this becomes YUV422P16
- Select Cineform as compressor, 10 bit, YUV422, Fast Recompress: Negotiation Error
- Select Cineeform 16-bit, YUV422, Fast Recompress: Negotiation Error
- So what do you recommend to do next ? Normal / Decode Format - Which ?

kolak

7th April 2021, 21:41

Edius has no native support for Cineform. It all goes over VFW/directshow which is not the way to do it.
In Ends 7 Cineform was fine with performance. Edius X has new engine, so things must changed.
Cineform in Edius is also 8bit only.
You need a codec with native support, so in case of Edius: ProRes, DNxHR or HQX.

kolak

7th April 2021, 21:47

So this is what I do:

- In VDub2 this becomes YUV422P16
- Select Cineform as compressor, 10 bit, YUV422, Fast Recompress: Negotiation Error
- Select Cineeform 16-bit, YUV422, Fast Recompress: Negotiation Error
- So what do you recommend to do next ? Normal / Decode Format - Which ?

You get YUV422P16 because you import over ffmpeg source plugin. Then when you try exporting you use VFW system HQX codec, which only supports 8bit.

Try before you hit import using options button- this may let you force VFW HQX decoder not ffmpeg.
If not when you go to Compression and choose GV HQX then go to Pixel Formats and switch to YUY2. This should let you export to GV HQX. If not try also forcing Decode format to YUY2. You will loose 10bit precision.

Instead all of this use virtual AVI file system in AVS. Mount your AVS script as fake AVI and then import that AVI back to Edius. Not sure if AVS+ supports it but in vapoursynth you can mount final script result as v210 format, so this way you can preserve 10bit. Edius should be fine with 10bit (v210 ) AVI. If not use YUY2. No need for any intermediate files. You can use "AVS filters" directly in Edius.

Hotte

7th April 2021, 21:52

Thanks Kolak, you are incredibly qualified!

I start to love DNxHR. Its HQXmode is some 70% the size of Canopus HQX and the quality is really, really good.

My setup is:
- I export with Canopus HQX into AVS+ and do some 16-bit filtering
- I need to come back from AVS+ into Edius to do the final things
- I want to keep 10-bit and loose as little IQ as possible

So export Canopus HQX 10-bit is fine. It goes on very well with AVS+. The trouble is to come back. I'd love to go for Canopus HQX which is part of VDub2 but it is only 8-bit. Ouch!

ProRes422HQ could do. But has less timeline performance than DNx.
Is there a way for DNxHR ?
Or any other ?

EDIT: We`re overtaking each other with questions and hints - funny, sorry :) . I remebered to work with mounting some releases ago and I had a not so good experience. Everything became so horribly slow. But maybe I will come back to that later.

kolak

7th April 2021, 22:02

I left Edius forum long time ago. I think I recognise your name :P

Learn to use avisynth virtual file system as I said. Then no need to use any intermediate files and rendering.
You load source to avs , do your filtering and then present result as fake AVI file which you can load to Edius. If you don't do crazy processing in avs all should still work realtime. With avs+ all should work fine I think.

Other way. Instead of using VDub download ffmpeg and just convert your avs directly with ffmpeg back to desired codec. Can be DNxHR or ProRes.
Ffmpeg support avs script as source, so you convert it like any file: ffmpeg -i source.avs ......

Hotte

7th April 2021, 22:06

Thank you so much, Kolak.

poisondeathray

7th April 2021, 22:18

Let's be precise: There is no real world difference between Prores HQ in the best setting and native DNxHR out of Edius.

But there is a clearly visble difference between native DNxHR exports (which I just realized are also significantly smaller) and DNxHD with DNxHR_HQX profile only.

It is difficult to describe: I would not say there is less resolution but contrast is lower like a very slight haze covering the scene and also some ringing in the original file at very large zoom levels become less obstrusive. I would instinctively push microcontrast a tiny bit which could cure it - don`t know really.

To summarize - You're saying no problem with official DNxHR exports from Edius; the problem is with ffmpeg DNxHR encoding variant

"contrast is lower" , "haze covering the scene" sounds suspiciously like the dreaded gamma shifts of old.

Did you use MOV or MXF wrapper ?

Hotte

7th April 2021, 22:45

Did you use MOV or MXF wrapper ?

AVI. I will redo with MXF.

EDIT: MXF is rejected bei Edius (although it wraps DNxHR into MXF itselves ?!), MOV works.
Same effect. If i manipulate the gamma curve a tiny bit I am not getting closer to the original. I'd now guess that I am losing a whiff of microcontrast whereas resolution could be quite close. But we tend to perceive a loss of microcontrast as a loss of resolution. Yes it could be resolution also.

poisondeathray

7th April 2021, 23:07

- I do not have any raw video file but just a 4K 10bit 4.2.2 25bit mov out of a Panasonic G9
- I export this from Edius-X with Canopus HQX 10 bit. Normally this is perfect input for AVS+, but today I just load it into VDub2
- In VDub2 this becomes YUV422P16
- Select Cineform as compressor, 10 bit, YUV422, Fast Recompress: Negotiation Error
- Select Cineeform 16-bit, YUV422, Fast Recompress: Negotiation Error
- So what do you recommend to do next ? Normal / Decode Format - Which ?

For a native 10bit file (not stacked), Cineform in vdub2 requires normal or full recompress. It might be the same for stacked , not sure. I would avoid stacked all together if possible.

You can verify output with other tools, such as 10bit waveforms. ffmpeg has one, and avisynth does too now. If you test a 0-1023 gradient, you will see if there were problems somewhere , such as an 8bit intermediate step somewhere, which will manifest as gaps

One bizarre issue with some DCT based codecs like DNxHR/DNxHD and prores to a lesser extent is severe noise on some types of patterns like small checkers. It's not subsampling related (4:4:4 is affected too). Official Avid and ffmpeg implementations are affected. I've reproduced it on other patterns, other colors, other grid sizes. You can see some of the disussion here starting around post 672 . x264 (also dct based) , cineform (wavelet based) are unaffected
https://forum.doom9.org/showthread.php?p=1855173

kolak

7th April 2021, 23:26

I just done quick test and ffmpeg DNXHR is fine.
BM RAW 12K file converted to v210 in Resolve and this to DNxHR. ffmpeg one look as good (bit better) as Resolve direct encode to DNxHR. Direct ProRes encode is just slightly better (2dB in PSNR). HQX and Cineform are also very close. All codecs around 60dB PSNR (source is clean and god quality), so all looks fine.

Only issue is DNxHR encode out of Resolve which has strangely lower Y PSNR value. BM recently had some bug in DNxHR encoder, so looks like there may be still some left over.

Hotte

7th April 2021, 23:45

I just done quick test and ffmpeg DNXHR is fine. Only issue is DNxHR encode out of Resolve which has strangely lower Y PSNR value.

With Edius-X it is the other way round. Direct DNxHR export quality is excellent whilst filesizes are only 70% compared to the rest of the bunch whereas DNxHD reimport (compared in Edius) is sth Edius seems to be suspicious of...
CineForm needs larger filesizes to look almost like ProRes422HQ at highest quality level. So cineform appears to be very marginally weaker compared to ProRes.

Cineform needs normal recompress in VDub2 to convert from YUV422P16 to V210. Reimported into Edius-X it carries on having bad TL-performance of about 3 fps. It does not look like I lose the 10-bit with Cineform, I have a very fine sky gradient in my sample, but I will check that.
Thanks all!

kolak

7th April 2021, 23:52

Cineform is slightly less efficient than other codecs, but this is fine thanks its other features, like partial resolution decoding. Small bitrate raise to be "the same" is not a big deal.
GV HQX, DNxHR are similar if you match bitrate. ProRes is most efficient, but by margin (but also always).
Best feature of ProRes is fact that all tools which have an official encoder have it implemented well. Apple doesn't allow for bad implementations and this is easily visible when you work with many apps (performance is good, no level issues, alpha works etc.). with allotter codes it's not always true.

Your issue may be due to fact that you're encoding from HQX (bug may be in ffmpeg HQX decoder). Export v210 uncompressed and use this as a start.

kolak

8th April 2021, 00:02

It does not look like I lose the 10-bit with Cineform, I have a very fine sky gradient in my sample, but I will check that.
Thanks all!

If you import to Edius you will (unless GV added ability to read 10bit formats in VFW which I don't believe in). In order to prove it, looking is not enough as Edius preview surface is 8bit anyway, so you never will see proper 10bit data in its GUI preview.
Try applying very heavy curve and then real 10bit data and 8bit will show difference.

Hotte

8th April 2021, 00:09

Best feature of ProRes is fact that all tools which have an official encoder have it implemented well.

That sounds coherent. Edius does all sorts of ProRes exports.

Your issue may be due to fact that you're encoding from HQX (bug may be in ffmpeg HQX decoder). Export v210 uncompressed and use this as a start.

I tried that just for fun. 4K V210 uncompressed export is nothing less than crazy, not feasible with larger projects. No improvement, TL-perf is bad. I suspect Edius hates the cineform codec ;-)

Hotte

8th April 2021, 00:12

Try applying very heavy curve and then real 10bit data and 8bit will show difference.

Thanks. Will check that some day. But because off the timeline issue cineform is out of the race anyway.

Hotte

8th April 2021, 00:14

Edius preview surface is 8bit anyway, so you never will see proper 10bit data in its GUI preview.

Are you sure ? This is the workgroup version where you have the 8bit/10bit Preview option. Cant imagine that the 10bit is pure fake preview.

Hotte

8th April 2021, 00:25

One bizarre issue with some DCT based codecs like DNxHR/DNxHD and prores to a lesser extent is severe noise on some types of patterns like small checkers. It's not subsampling related (4:4:4 is affected too). Official Avid and ffmpeg implementations are affected. I've reproduced it on other patterns, other colors, other grid sizes. You can see some of the disussion here starting around post 672 . x264 (also dct based) , cineform (wavelet based) are unaffected
https://forum.doom9.org/showthread.php?p=1855173

I fear if I dive into this, no codec will be left for me. So I am probably doing better to keep my eyes a bit shut :cool:

kolak

8th April 2021, 00:39

That sounds coherent. Edius does all sorts of ProRes exports.

I tried that just for fun. 4K V210 uncompressed export is nothing less than crazy, not feasible with larger projects. No improvement, TL-perf is bad. I suspect Edius hates the cineform codec ;-)

I meant to encode DNxHR from it in ffmpeg.

kolak

8th April 2021, 00:41

Are you sure ? This is the workgroup version where you have the 8bit/10bit Preview option. Cant imagine that the 10bit is pure fake preview.

It's not really preview. It's switching whole engine to work in 8bit to gain performance.
In order to have 10bit preview you need some card- BM/AJA or GV one. GUI is always 8bit. 99% sure :)

Hotte

8th April 2021, 01:01

I meant to encode DNxHR from it in ffmpeg.

That sounded like a good idea.
Sorry to tell you, that there is practically no difference between an uncompressed V210 > DNxHD
and a Canopus HQX > DNxHD

Both where generated using ffmpeg and the fine Haze is in both. Strange - isn't it ?
It also shows how good Canopus HQX is. It is such a pity it is an 8-bit compressor outside!

poisondeathray

8th April 2021, 01:10

Are "fine haze" and "lower contrast" Edius specific problems ? Maybe the decoder, or some setting ?

Is there an actual problem with the file ? If you open the ffmpeg DNxHR in another application (e.g. mpv, ffplay, avisynth, or resolve) , how does it look ?

Did you explore proxy workflows ? In general, they can offer significantly faster timeline performance for complex, multiple layer projects. (Higher quality too, depending on how you set it up)

kolak

8th April 2021, 10:28

That sounded like a good idea.
Sorry to tell you, that there is practically no difference between an uncompressed V210 > DNxHD
and a Canopus HQX > DNxHD

Both where generated using ffmpeg and the fine Haze is in both. Strange - isn't it ?
It also shows how good Canopus HQX is. It is such a pity it is an 8-bit compressor outside!

What ffmpeg version is it?

Hotte

8th April 2021, 22:30

I have made the recommended outside comparison and I chose VDub2 as viewer (and frame TIFF exporter).
This are probably also some technical transients, there is codecs, tiff compression, website compression... I know.
But the relative findings I had all point into the same direction: COMPLETELY different insights.

Samples now! Unfortunately they are heavily jpeg'ed on imgur and rich in artifacts so will give you only an idea of what I am taking about. But you can pretty much trust on the evaluations below. If interested and sbd knows a better dowload source (no google or other sniffer please) where I can drop 8MB tiffs I`d be happy to upload them.

1. Out of camera MOV H.264 4K 10bit 4.2.2
https://imgur.com/LCWcIHe

2. Edius Export Canopus HQX superfine
https://imgur.com/cX8YC01

3. Edius Export V210 uncompressed
https://imgur.com/jK34YI9

4. Edius Export DNxHR HQX native MXF
https://imgur.com/fo16rfQ

5. ffmpeg Edius V210 uncompressed > DNxHD profile dnxhr_hqx
https://imgur.com/ZZs3ixW

6. ffmpeg Edius V210 uncompressed > ProRes HQ 4.2.2 10-bit Highest Quality
https://imgur.com/QfgHtYd

My findings after pic-on-pic comparisons with 1000% magnification:

Edius Canopus HQX
Edius HQX and V210 uncompressed are practically identical down to the extreme detail. No contrast, saturation or sharpness shifts. This is an excellent, neutral codec.

Edius Export DNxHR HQX native MXF
increases micro contrast (fakes more resolution) and overall contrast/saturation. Nice look for people who like it and are ready to accept e.g. more pronounced ringing artifacts (that were already there). Apart from that it preserves resolution very well.

ffmpeg DNxHD profile dnxhr_hqx
in this setup has to be compared to its source file uncompressed V210 or to Canopus HQX to see which codec does better. There is NO HAZE. Resolution is kept very well, about as well as HQX. Like its native brother it introduces a very small amount of additional microcontast but way less than Edius DNxHR - it's ok. What I do not like is the slight increase of overall contrast/saturation to about the same amount as Edius DNxHR although it appears less evident because of less push in microcontrast. Overall an excellent performer for a free codec if you play with levels to tame it. poisondeathray was right!

ffmpeg ProRes HQ
this is an excellent neutral codec, no shifts, no sharpness push, very good detail preservation (although Canopus HQX has an ever so tiny advance in this respect). But you have to push the VDub2 compressor quality slider to the very left to get there. This increases file size by some 10% compared to Canopus HQX. ffmpeg ProRes is relatively slow on an Edius-X timeline and buffer will run out in seconds with 4K 10bit full res. Its not as bad as Cineform in this respect, but also not great.

Bottom line: I was tricked by some Edius internal transients. Sorry for that.

But finally it was you to help me find my 10-bit codec: ffmpeg DNxHD. DNxHD would have well-deserved to be implemented in VDub2 but I will be able to work around it.

Kolak, poisondeathray and all the others: Thank you very much.

poisondeathray

8th April 2021, 23:13

There are some issues with the color in the presented screenshots - there are 601/709 mismatches. It's most easily visible on the grass

edius v210 and canopus hqx are similar in one group in terms of color;

original, v210 to (ffmpeg) dnxhr_hqx, and edius direct dnxhr_hqx are similar in the other group.

If you correct for the mismatch (one group to the other) they all become the similar in terms of color

For example, vdub uses 601 to convert from YUV to RGB for display, instead of 709 . If you used vdub for all of them, they should all be the same color. So there is something else going on with the procedure you've used that is not explained

Hotte

8th April 2021, 23:49

edius v210 and canopus hqx are similar in one group in terms of color
All pics were created manually with VDub2 in the following way: Load Clip, Direct stream copy, Decode Format=AutoSelect,Interpret YCbCr none/none, export single TIFF
These two are AVI-export directly from Edius. All Edius exports in 4.2.2 CSS like the original, project properties are YCbCr 10-bit BT.709

original, v210 to (ffmpeg) dnxhr_hqx, and edius direct dnxhr_hqx are similar in the other group
Same procedure and settings with VDub2:
- Original loaded unchanged
- Edius dnxhr was direct MXF export from built-in codec
- dnxhd was converted from the uncompressed V210 of above with ffmpeg latest release

How can I do better ?

poisondeathray

9th April 2021, 00:57

All pics were created manually with VDub2 in the following way: Load Clip, Direct stream copy, Decode Format=AutoSelect,Interpret YCbCr none/none, export single TIFF
These two are AVI-export directly from Edius. All Edius exports in 4.2.2 CSS like the original, project properties are YCbCr 10-bit BT.709

Same procedure and settings with VDub2:
- Original loaded unchanged
- Edius dnxhr was direct MXF export from built-in codec
- dnxhd was converted from the uncompressed V210 of above with ffmpeg latest release

How can I do better ?

In terms of the 601/709 mismatch - it doesn't add up.

All video versions are YUV. You used same procedure in vdub for screenshot creation for all of them. If one has the "wrong" color, they should all have same the "wrong" color . None of the jpg's have color profiles or metadata .

Even stranger is the ffmpeg dnxhd_dnxhr was created from the v210 version and yet has different color...

Something else is going on, there might be some flags or metadata interfering , but afaik vdub ignores them

Since the "AVI" files are in one group, there might be some local configuration difference, some VFW filter or vdub config difference

Another way is to control the conversion in avisynth explicitly

#load YUV source, use LWLibavVideoSource to be consistent
LWLibavVideoSource("video.ext")
ConvertToRGB24(matrix="rec709")

Or more important to your workflow, would be to take the screenshots in Edius, assuming what procedure Edius uses for screenshots is what it actually uses on the timeline

For example if you wanted to change the edius "v210" to "look" like the "original" colors

ImageSource("jK34YI9_edius_v210.jpg")
ConvertToYV24(matrix="rec601")
ConvertToRGB24(matrix="rec709")

or "original" to "v210"

ImageSource("LCWcIHe_orig.jpg")
ConvertToYV24(matrix="rec709")
ConvertToRGB24(matrix="rec601")

So that "prooves" that it's a "pure" 601/709 mismatch somewhere in the workflow

Hotte

9th April 2021, 20:50

I have redone all my samples to be sure I have not made any mistake - because there are lots of opportunities for errors when you fiddle with different codecs. And this time I also exported TIFFs out of Edius (which are 8bit only but that does not matter for this comparison).

The Edius-TIFFs show a very different but consistent result:

1. All exports and the original brought back into Edius have the exact same color and contrast if you compare them. This is what I would have expected from professional software.

2. Resolution of the codecs does not differ a lot. The same is true for sharpness except for ffmpeg DNxHD-HQX (derived from V210) which has some slight softness, so the haze is back. But the haze is not as bad as before. The reason could be that I was on an old 2017-version of ffmpeg which I updated now - Thanks Kolak!

3. And now comes what really suprises me: The contrast profile of all Edius-Tiffs (like in the preview) is visibly flatter then any of the VDub-TIFFs. I had a look into the viewfinder of my camera and found it is flatter than in the camera also. But there is no primary color-mapping applied in Edius.

Samples in better quality now:

Edius-TIFF: V210 uncompressed
https://imgur.com/ILFz7hv

Edius TIFF: V210> ffmpeg DNxHD HR_HQX. Same color/contrast now but slightly less sharp:
https://imgur.com/fUjkNXh

VDUB TIFF: V210 uncompressed: VDub color space is different, more contrast
https://imgur.com/HhBw4OH

I am a bit confused now: What is the correct contrast profile ? Even more confusion is this: I tried your avisynth/lwlibavvideosource tip but...
lwlibavvideosource("F:\Temp\Export_V210.avi")
info()
Return last
...returns interleaved (?) rubbish. How is this to be read with avs+? It is misinterpreted. It defintely is 10bit.
https://imgur.com/dulokJH

I can`t correctly interpret any of the codec exports with libav, only with ffm2.

Btw ffplayer shows the same false colors as VDub does.

Thanks for any advice.

poisondeathray

9th April 2021, 23:08

The contrast difference is full vs. limited range

The color difference is 601 vs. 709 matrix

If you adjust for them, you can make one group look like the other or vice-versa

10bit - Possibly you're using an old lsmash build, or old avisynth build

https://github.com/HolyWu/L-SMASH-Works/releases

v210 should open up as yuv422p10

MOV and MP4 container does not require indexing if you use LSmashVideoSource - ISO base media formats, (unlike ffms2 or LWLibavVideoSource)

shekh

10th April 2021, 10:49

Yeah! ***** The DNxHD would be a great addition to the compression menu as it seems to play some not unimportant role in professional video exchange and looks like being an excellent codec, also encoding is fully supported by ffmpeg.

I'd be ready to support that in any way that makes sense. I`d be happy to hear Shekh`s evaluation of the matter. I have no idea if that is a lot of work to do.

When I evaluated what codecs are good as intermediate I also saw DNxHD. However it is quite special as it does not work with arbitrary resolutions. I think it is not big problem to make some basic implementation, I just didnt like it.

Hotte

10th April 2021, 11:18

:thanks:
My god, 6 tips that turn my video conversion world upside down! I write my findings down here for Edius-X and Avisynth/VDub2 user generations to come (who are as ingenuous as myself). Be nice and honour Mr./Mrs. Poisondeathray for his/her generous help.

1. Luminance range: YES, this the first time I understand why Edius marks my original Input clip as BT.709 [Full] instead of BT.709 only. In my camera I activated Rec.709 full luminance range (0-1023 with 10-bit) instead of TV range (64-1023). Edius obeys that and shows flat full range. The camera viewfinder/monitor doesn't care and always displays TV (bad!). VDub2 picks TV as well because it doesn't know or care and the AVI-container doesn't tell either.

2. Version of LSMASH: YES, I was on a 2017 built. This one cannot read "YUV422P10" correctly. Upgrading to the latest version solves it. And this is the first time when info() shows the correct colorspace and bit-depth and that is:
Color Space: YUV422P10 Bits per Component: 10. ffmpeg will per default convert to YUV422P16 which is not bad but also not fully correct. You should go for this:
LSmashVideoSource("F:\Temp\Export_HQX.mov",format="YUV422P10")
info()
Return last

3. Pick the right video-container: YES. Edius-X (at least the workgroup version) allows Canopus HQX-super-fine 10bit Exports with AVI and MOV (see Quicktime-Export Menu). I've always tended to avoid Apple proprietary stuff. But in times of ffmpeg and libav this seems nothing to be concerned about. And this time it pays off to pick MOV instead of AVI, because then you can use LSmash and be rewarded with faster load times, no interim index files and a less stupid container that shows and transports more media information. Hence you can see the code above to import a Canopus HQX10bit MOV which is a dignified and adequate replacement for cumbersome Uncompressed AVI as far as my resolution tests yielded.

4. Color space: YES, I managed to tell VDub2 how to display this. At least I am getting very close (not 100% but must be around 99,8%) with the following extension LSmashVideoSource("F:\Temp\Export_HQX.mov",format="YUV422P10")
ConvertToRGB24(matrix="pc.709")
info()
Return last
To display a YUV color space on a monitor in sth that makes sense for our eyes it needs to be converted to RGB anywhere. To make sure that colorspace is not subject to a random default (like oldish Rec.601) in VDub, avs+ or anywhere else it is a good idea to talk Turkey: This is Rec.709. And to make clear that we don't want the PC (aka full) luminance range to be changed we choose "PC.709" instead of forcing gamma down to the more contrasty TV-Range which would be "Rec709".

@Poisondeathray, did I get it get it right or do you see anything to be improved with this little code snippet ?

I am getting almost correct colors and contrast with this. There are very subtle color differences, only viewable at very large zoomlevels:

Edius TIFF: HQX-10bit superfine
https://imgur.com/3bWpRUR

VDub2 TIFF: HQX-10bit superfine converted to RGB24 pc.709 as above
https://imgur.com/YZiFJAv

To round up the whole matter, there are a few questions left and I´d be very, very happy if you could answer these too:

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

2. I am not sure I fully understood the luminance range matter.

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

If 3 is correct:
I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

Hotte

10th April 2021, 12:00

When I evaluated what codecs are good as intermediate I also saw DNxHD. However it is quite special as it does not work with arbitrary resolutions. I think it is not big problem to make some basic implementation, I just didnt like it.

Shekh, that sounds ever so great!

DNxHR is used in the AVID broadcast world (and some other manufacturers) as intraframe interchange/intermediate codec. It is well supported by Edius and is pretty fast on its timeline.
DNxHD was the name of its predecessor which was only used for up to 1920x1080 10bits 4.2.2

There is an ffmpeg implementation of DNxHD which can run in some hqx_hr profile to manage e.g. 4K in 10 bits 4.2.2 and there is a hqx_444 profile for even more.

The implementation of these two profiles would be of great use. The ffmpeg version is a very good codec. I only noticed a very tiny hint of softness - or let`s say defensive micro contrast. Others obviously did not notice that. Anyway this could be cured easily in post if ever necessary since you will never notice it at regular zoom levels in 4K.

For Edius-X users it seems to me the best alternative for 10-bit content to be filtered with avs+ and VDub2 to bring back the filtered clip into sth which Edius can handle. The reasons are:

- Edius' native codec Canopus HQX is only 8-bit in ffmpeg hence VDub2 (it is 10 bit in Edius)
- reimported ProRes is 10bit and has great IQ but it is very slow on an EdiusX-Timeline
- reimported CineForm is 10 bit and has great IQ as well, but is extremely slow on an EdiusX-Timeline - can even make the Editor crash

I know, you are not Edius support but I am sure it would well appreciate VDub2 to offer
ffmpeg DNxHD profile dnxhr_hqx
ffmpeg DNxHD profile dnxhr_444

as new compression options.

kolak

10th April 2021, 12:28

DNxHD an DNxHR in ffmpeg is same thing. They are all under dnxhd codec, just different profiles. You should today use DNxHR and forget DNxHD. All current ffmpeg builds have DNxHR.

Edius' native codec Canopus HQX is only 8-bit in ffmpeg

Wrong. HQX is fine ffmpeg and works at 10bit.
HQX is only 8bit in its VFW/Directshow codec installed by Edius (not in ffmpeg).
In Vdub2 you are using ffmpeg module to decode HQX (which I fine and 10bit), but VFW codec to encode, which only supports 8bit. Plain simple. Your only other tool which can encode HQX at 10bit is Resolve.

Hotte

10th April 2021, 12:47

DNxHD an DNxHR in ffmpeg is same thing. They are all under dnxhd codec, just different profiles. You should today use DNxHR and forget DNxHD. All current ffmpeg builds have DNxHR.
Are you are saying (ffmpeg DNxHD profile=dnxhr_hqx) = DNxHR ? So that's even better!
Still I can see a difference (the slightly reduced microcontrast I was talking about) between ffmpeg DNxHR profile=dnxhr_hqx and native DNxHR Exports from Edius. But that could be a difference between the professional and the ffmpeg implementation - couldn`t it ?

Wrong. HQX is fine ffmpeg and works at 10bit.
HQX is only 8bit in its VFW/Directshow codec installed by Edius (not in ffmpeg).
In Vdub2 you are using ffmpeg module to decode HQX (which I fine and 10bit), but VFW codec to encode, which only supports 8bit. Plain simple. Your only other tool which can encode HQX at 10bit is Resolve.
Ah! Does that mean that if VDub2 Compress-Dialog would implement HQX as "ffmpeg HQX" (rather than "HQX" without ffmpeg as today) it would compress 10 bit ? Now I understood, that HQX is not native VDub2 but came with Edius... Sometimes it takes time to understand - sorry. But then ffmpeg HQX would be another great implementation candidate...

kolak

10th April 2021, 13:04

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

2. I am not sure I fully understood the luminance range matter.

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

If 3 is correct:
I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

If you have HD file then you use Rec.709. Anything which goes through 601 is plain wrong and needs fixing. This conversion is probably near lossless (don't think it can be lossless)- point is that it's totally not needed and all should be handled over 709.

Luminance range is basically meaningless (specially for 10bit + formats) if it's handled properly by your tools. Conversion from one to another is nothing to worry much about unless it's done badly.
It's end delivery point/spec (before this it can be both) which may mandate certain levels, eg. YUV for broadcast is limited range. Although nothing stops you in your own environment use full range for YUV before final export. It all depends on workflows, your tools, monitoring etc. If you don't use properly calibrated monitor over video card (but Edius GUI) then you are not very accurate anyway. Edius GUI preview is not a reference preview and should not be used for anything color critical. You need to buy BM card and feed calibrated monitor with signal from it. For Edius this is a proper way of monitoring video.

3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

Yes, overall this is the best practice. Once your video hits YUV at some point then it's good to stay in YUV (if possible), specially when you are going to deliver only to formats which require YUV. This is exactly why Edius uses YUV engine as it's tool for broadcast not really high-end finishing/grading. Staying with YUV has its own issues as well- you can easily produce out of gamut files (so such a YUV combinations which converting to final display RGB produce illegal (negating, or above max level) values). We know this is real issue in Edius and it has no native gamut legaliser (Premiere does now). If you work in RGB and convert to final YUV master out of gamut errors can't happen (can only as codecs overshoots).

From other hand when you start with high-end recording formats (RAW, film scans etc) then you rather want to work in RGB on GPU (with 32-bit floating point processing) and only during final exports you may convert to YUV.

This is a typical master chain for high quality production: RAW recording, grading/finishing in RGB (in tools like Resolve, Baselight, Mistika, Flame etc), export- quite often at this point master goes to YUV domain. This is the point where you may receive such a master, eg. ProRes, DNxHR and do editing, versioning etc. Here you ideally want to stay in YUV (as we already left RGB during "main" export), but in today's app this is not that essential anymore as RGB<->YUV conversion if not lossless (apparently with 32bit float it can be) is so high quality that this is your last worry. There are 1000x more important aspect of your video- editing, grading etc which impact its final perception.

Stop chasing some ideal paths- focus on artistic value as as long as you stay technically correct with main principals then this is easily good enough. For example your lack of proper monitoring in Edius is 100x more important than whole YUV<->RGB conversion "issue".

With tools like avs etc you have to test and validate path to make sure bit depth, levels, etc are preserved properly. Most NLes are rather closed apps and things work there fine, but don't take this as 100% ideal either. There are problems in Resolve, Premiere as well. I don't think I've seen some critical issues in Edius engine though (except maybe whole new primary color filter, which works in RGB). It seems to be well tested. Problems can still happen so any new export (new major app version) needs new validation and checks.

kolak

10th April 2021, 13:07

Are you are saying (ffmpeg DNxHD profile=dnxhr_hqx) = DNxHR ? So that's even better!
Still I can see a difference (the slightly reduced microcontrast I was talking about) between ffmpeg DNxHR profile=dnxhr_hqx and native DNxHR Exports from Edius. But that could be a difference between the professional and the ffmpeg implementation - couldn`t it ?

Ah! Does that mean that if VDub2 Compress-Dialog would implement HQX as "ffmpeg HQX" (rather than "HQX" without ffmpeg as today) it would compress 10 bit ? Now I understood, that HQX is not native VDub2 but came with Edius... Sometimes it takes time to understand - sorry. But then ffmpeg HQX would be another great implementation candidate...

DNxHR differences can be down to implementations. FFmpeg code is different than AVID reference one (it's the same as you have 10 different h264 encoders- some codec but you get slightly\very different results). You also have to make sure your avs etc path is correct.
For me ffmpeg is actually slightly better than AVID encoder. And because I use plain simple test (v210 source directly encoded ) I have no place for any error (except actually ffmpeg issue if exist). And because ffmpeg gives slightly higher PSNR and also when I visually compare in Resolve ffmpeg DNxHR show less difference I simply don't believe in your tests (your path is full of possible issues- it involves so many "engines").

Yes, but atm. ffmpeg has no HQX encoder- just decoder, so it won't happen.

Hotte

10th April 2021, 14:49

Thanks a lot Kolak for your extensive and well understandable explanations!

With your and poisondeathray's help I was able to understand the traps of the workflow and how I can handle them. With respect to my intended workflow Camera > Edius > AVS+ > Edius would you be so kind to check if these conclusion sounds basically consistent to you:

- My camera delivers H.264 YUV422 10bit. I should always record in rec709 with full luminance range
- I learned my NLE has a YUV engine. I can use the Primary Color Correction filter, no need to be concerned about that as possible conversion loss is not relevant in the amateur field
- Export HQX superfine 10bit as MOV is ok. I am staying in YUV
- Import into avs+ with LSMASH in YUV422P10. Check that rec709 arrives. If not, convert it.
- whenever possible use 16-bit AVS+ YUV422 capable filters
- to monitor interim results in VDub2 convertToRGB24(matrix="pc.709"). (monitor only, not for export)
- Compress the result with appropriate YUV422 10bit codec to import back into Edius (both of us seem to like ffmpeg DNxHD :))

And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

Thank you so much.

poisondeathray

10th April 2021, 15:33

@Poisondeathray, did I get it get it right or do you see anything to be improved with this little code snippet ?

I am getting almost correct colors and contrast with this. There are very subtle color differences, only viewable at very large zoomlevels:

Edius TIFF: HQX-10bit superfine
https://imgur.com/3bWpRUR

VDub2 TIFF: HQX-10bit superfine converted to RGB24 pc.709 as above
https://imgur.com/YZiFJAv

Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

-There are slightly different algorithms to convert YUV422P10 to RGB . For example, chroma upsampling algorithm. (one might use bicubic, one might use slightly different coefficients of bicubic, or bilinear etc...). Chroma center location interpretation. Avisynth internal resizers use "center" instead of "mpeg2". You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")

2. I am not sure I fully understood the luminance range matter.

You did. The flag in the original recording signalled Edius to convert to RGB (for preview) using full range equation. You can simulate the same using "PC" matrix in avs

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

Pros/cons to acquisition format . Full range is probably better if the tools used can manipulate in float . If not, there is more loss because you have to adjust the ranges to a larger extent to make them "standard range"

But final version should always be "standard" range. So fast turn around situations (direct to TV, broadcast ENG) never use "full range" for acquistion

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).

Vapoursynth can do this correctly. In order to be 100% lossless the chroma up/down sampling has to use nearest neighbor algorithm, and float used on all calculations

But these are relatively minor issues. Don't loose sight of the "big picture" as Kolak said

I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

If it works in RGB48, not float, then potentially there is a problem. You'd have to "legalize" YUV beforehand, or risk significant clipping

And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

PSNR is a measure of "quality" compared to a "source".
One way can use ffmpeg to measure
https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio
https://ffmpeg.org/ffmpeg-filters.html#psnr

It's not a good measure for subjective testing, but it's excellent to determine if something is lossless or not (infinity dB)

kolak

10th April 2021, 15:43

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

poisondeathray

10th April 2021, 15:50

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped

PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

kolak

10th April 2021, 15:59

ffmpeg -i ref_source -i source -filter_complex psnr -f null -

this how you do it with ffmpeg. At the end you get stats. Anything within +-3dB is not a very meaningful difference. Eg ProRes may show average PSNR of 56dB and DNxHR 54dB. For clean good quality sources (like my BRAW 12K sample downscaled to HD) PSNR can go above 60dB which means loos is very small. For difficult scenes PSNR for intermediate codecs can go <50dB.

You can read about HQX here in its whitepaper:

https://www.syntex.sk/media/document/1107/75640_grass-valley-hqx-whitepaper.pdf

you also have Apple white paper about ProRes which has very similar results: https://www.apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf )

you have PSNR graphs against other codecs. GV uses same source as Apple (STEM footage), which is specially prepared footage for digital cinema validation. It's never compressed (shot on 35M film) source which is very important. I use to have access to it- very demanding source.
If you take some source from the internet (which you have no idea how it was shot) your results can be very misleading. You may get some nice source which was shot on eg. ALEXA in ProRes natively. Then if you convert that to ProRes then it's 2nd generation fo the same codec and your PSNR will be very high. It won't be true for other codecs as they use different math. You have to really know what you are doing in those tests as you can easily produce something very false :) Cineform behaves quite differently on uncompressed sources with original noise/grain compared to source which are already went through DCT based compression (so ProRes, DNxHR etc). Internet is full of fundamentally flawed tests.

kolak

10th April 2021, 16:01

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped

Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 601) in Rec.601 and then get it back?

poisondeathray

10th April 2021, 16:17

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 620) in Rec.601 and then get it back?

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

kolak

10th April 2021, 16:18

YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).

Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels :)
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

kolak

10th April 2021, 16:23

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

poisondeathray

10th April 2021, 16:24

Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels :)
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range

It's a bit inconsistent handling, because ffmpeg dnxhr does not limit the range. Neither do official implementations of v210 or dnxhr

kolak

10th April 2021, 16:27

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range

This comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion :)

poisondeathray

10th April 2021, 16:33

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

You're talking about slightly different things -

Think about it - you're not producing a file in float. It's for intermediate. You're not clamping anything

Recall , the topic was question was changing 709/601 in YUV. A colormatrix transform, as if the file in YUV had used 709 instead (or 601 instead)

The background info was intermediate RGB preview (recall vdub uses 601) , was different than using 709

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")

And this is correct - You can apply the colormatrix transorm in YUV444PS . It's reversible. You can get the original file's code values back

kolak

10th April 2021, 16:39

You're talking about slightly different things -

Yep, different things.

poisondeathray

10th April 2021, 16:39

This is comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion :)

Yes, but "v210" exported from Adobe, Resolve , etc, retain 0-1023 code values. ffmpeg v210 does not.

Do you see the problem ?

Yeah, it's not important in the big picture; but you better believe that "little detail" becomes important when testing decoders, psnr . Details matter. I reported it before on ffmpeg bug tracker. Nobody seems to care :(

At the very least there is inconsitent behaviour within ffmpeg, and between professional apps

kolak

10th April 2021, 16:41

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.

PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.

poisondeathray

10th April 2021, 17:15

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.

PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.

Check if your original code values had Y <4, >1019 to begin with. Make sure export had "retain sub-black...superwhite" enabled if using resolve

Verify in other programs that the code values are there with the export. (e.g. vapoursynth, vsedit colorpicker, or avisynth can now with histogram(bits=10))

I use a wrapper function to turn right , left

Thisto(10)

function THisto(clip c, int "bits")
{
bits = Default(bits, 8)
c.TurnRight().Histogram(bits=bits).TurnLeft()
}

ffmpeg v210 encoder is definitely not "infinity" on a ramp, and you can see it on the waveform in multiple programs. You don't need PSNR to know it's not lossless. You can see it. Even ffmpeg's own -vf waveform detects the discrepancy.

To be clear, ffmpeg can still DEcode 0-1023 from other formats e.g. if you test 10bit422 AVC input, it still outputs 0-1023 raw values. But if you encode a v210 version , that will only contain 4-1019 values.

Let me know if me to upload a demo package

(When using ffmpeg psnr, it's good idea to reset timebase - different containers use different timebases and this can cause ffmpeg to compare different frames (not important for 1 frame
obviously, but when testing "video"))

This is a 0-1023 ramp test.

Resolve v210 vs. original v210
ffmpeg -r 24 -i 1_resolve_v210.mov -r 24 -i 0_1023x720_0-1023.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
[Parsed_psnr_4 @ 00000027ffcb73c0] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

FFmpeg v210 vs. Resolve v210
ffmpeg -r 24 -i 2_ffmpeg_v210.mov -r 24 -i 1_resolve_v210.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

[Parsed_psnr_4 @ 0000001488292ec0] PSNR y:72.519000 u:inf v:inf average:75.529300 min:75.529300 max:75.529300

kolak

10th April 2021, 17:26

I think Resolve ramp generator also skips them, so this is why I see no issue.

poisondeathray

10th April 2021, 17:35

Here is a 10bit ramp package. 1024x720 perfect ramp. Each column corresponds to pixel value in Y'
https://www.mediafire.com/file/cvulp8cougjm4nb/10bit_ramp_tests.zip/file

0_1023x720_0-1023.mov is "original"

1_resolve_v210.mov is original exported from resolve

2_ffmpeg_v210.mov is generated from
ffmpeg -i 1_resolve_v210.mov -c:v v210 -an 2_ffmpeg_v210.mov

e.g ffmpeg -vf waveform
ffmpeg -i 1_resolve_v210.mov -vf waveform=g=green 1_resolve_v210.png
https://i.postimg.cc/pdnmxW6C/1-resolve-v210.png (https://postimg.cc/WDTNwvjq)

ffmpeg -i 2_ffmpeg_v210.mov -vf waveform=g=green 2_ffmpeg_v210.png
https://i.postimg.cc/mgPDNZB4/2-ffmpeg-v210.png (https://postimg.cc/G41dbC4g)

kolak

10th April 2021, 17:58

Got it now- those ramp generators in Resolve/Premiere are crap :)

So this is basically v210 encoder and ProRes encoder (and decoders are passing whatever is there)?

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out :)

DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.

poisondeathray

10th April 2021, 18:02

So this is basically v210 encoder and ProRes encoder?

Yes, ffmpeg v210 and ffmpeg prores encoder(s) (native, and prores_ks, and prores_aw)

kolak

10th April 2021, 18:33

DNxHR as well (at least in ffmpeg 4.3.2+)
Good to know- thanks.

update: it's Resolve decoder, not ffmpeg encoder.

poisondeathray

10th April 2021, 18:34

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out :)

I heard apparently there are some bugs in newest resolve waveform...I'm still using older version as I have a few projects in progress and I never upgrade in between

Temporarily assign clip attributes to "full" range, and you will see difference between original v210 (or resolve v210) and ffmpeg v210 . As you already know, resolve is working in RGB float. When "auto" is set and clip is unflagged, or flagged limited, it takes Y 64-940 to "map" to RGB 0-1023 . But internally nothing is clipped at that stage, it's working in float RGB. It's just the display at that point or RGB interpretation, not the actual file

For premiere you can see in lumetri scopes too set to float, you might have to increase the window size, but there is clipping at low and high level with ffmpeg v210 version, but not "original v210", or "resolve v210". But I agree it is a "crap" display compared to ffmpeg waveform, or avisynth histogram(10)

DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.

It didn't in the past and it doesn't for me (just checked)

Exactly what are you doing for input and for measuring output ? What decoder is being used

kolak

10th April 2021, 18:37

Yes, in Premiere I sort of saw it when setting to float.

I used Resolve to check, so problem is probably there.

poisondeathray

10th April 2021, 18:44

Yes, resolve's "waveform" is not a true Y' waveform . It's actually RGB converted values - so that depends on how you convert to RGB

ffmpeg -vf waveform or avs's histogram(10) are true Y' waveforms , they are measuring Y' values directly

And it's not a "knock" against resolve. For grading software, that's what the scopes should be doing

kolak

10th April 2021, 18:48

I see it now in Resolve as well (with forced full range). It shows it in form of missing data, so your line basically ends at 1019. At the bottom it shows it in some strange way, but it's also visible (Resolve 17.1.1.).

kolak

10th April 2021, 18:55

poisondeathray

10th April 2021, 19:02

It's Resolve DNxHR decoder is doing it. ffmepg waveform is fine.
Looks like anything "official" (Apple, AVID etc.) is now skipping those reserved bits and rest of the codecs/tools not. More mess to the already messy world of codecs etc. :)

Yes

I see it in resolve import of ffmpeg dnxhr_hqx too, but not with libavcodec decoding (0-1023)

If you use resolve to encode v210 (0-1023) to dnxhr_hqx, it's 0-1023 with libavcodec decoding, but reimporting to resolve also some clipping (again, temporarily set to full range clip attribute)

So that suggests it's resolve's dnxhr_hqx decoder behaviour

Cineform is full scale.

But you said cineform is 0-1023 ? In resolve ? Was that go pro cineform official, or libavcodec sdk implementation, or resolve cineform, or something else
So not all "official" :D

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

poisondeathray

10th April 2021, 19:12

Canopus/GV HQX exported from resolve is also 0-1023 in both libavcodec and re-import back into resolve is 0-1023

kolak

10th April 2021, 19:27

ffmpeg Cineform.
I don't think David was that paranoid with all those SMPTE specs :)

poisondeathray

10th April 2021, 19:29

Another thing maybe shekh can look at is vdub2's prores encoding show gaps when encoding v210 . Not just the clipping, but almost like 8bit conversion. ffmpeg's equivalent v210 to prores does not (4-1019, but smooth)

vdub2_v210_to_prores
https://i.postimg.cc/1RNb6fgP/vdub2-v210-to-prores.png (https://postimg.cc/vDbPw8LS)

ffmpeg_v210_to_prores
(see earlier post)
https://forum.doom9.org/showthread.php?p=1940546#post1940546

kolak

10th April 2021, 19:31

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

Yes, but they should leave files world.
Those reserved bits are actually coming from SDI I think.
I don't think HDMI actually needs them at all, but I may be wrong.

Yes, there may be bug in Vdub and data may be going over 8bit at some point (or maybe some auto format choice bug). It quite clearly shows 4 levels pattern.
Vdub2 is not the same as old Vdub. New features, but new mess related to whole ffmpeg etc as well.

poisondeathray

10th April 2021, 19:33

Hotte

10th April 2021, 19:34

Thanks poisondeathray for all your answers.

Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

The detail differences in the jpegs are not visible in the original TIFFs => JPEG issue.

You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

No idea how that works. Did not find anything like "z_convert_format". Could you give a little code example ?

You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

Yes if you think that this is acceptable for a preview I am with you.

If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end

Could you give me a good link where I can download test material ?

Just another question to add: What if I record with standard luminance range instead of full ? You said I should go for standard in the end anyway and during the process there is risk of IQ loss if conversion is not being done properly. My subjective feeling is always, that I loose dynamic range if I record with standard range. Is that correct or not ?

kolak

10th April 2021, 19:44

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

Yep. And after quick search I'm almost sure that those reserved bits are coming from SDI and HDMI doesn't actually need them, but for "compatibility" they preserved them in HDMI spec. Now they force them to files world. Don't like it.
Sometimes 4-1019 is called SDI Full :)

kolak

10th April 2021, 19:47

Just another question to add: What if I record with standard luminance Range instead of Full ? You said I should to go for standard in the end anyway and during the process there is risk of IQ loss if it is not done properly. My subjective feeling is, that I loose dynamic range if I record with standard range. Is that correct or not ?

This is somehow important for 8bit recording. You simply have less range: 235-16=219 vs 0-254=255, so actually quite a big difference (more banding etc.). For 10bit (or more) this is less relevant.

poisondeathray

10th April 2021, 21:21

Could you give me a good link where I can download test material ?

I posted a 0-1023 10bit greyscale gradient earlier

All NLE's should have 75% colorbars. I'd imagine Edius has them. Free version of resolve can generate them too. If you have problems I can upload bars later

"official" 8bit, 10bit, 12bit YUV values are listed in rp219

A proper 10bit YUV to 8bit RGB conversion should give you the exact numbers for 75% colors. e.g. 75% "red" should be 191,0,0 in 8bit RGB using limited range equations (rec matrix in avisynth). Or 180,0,0 using full range equations (pc matrix in avisynth)

(Subsampling is not an issue with "large" bars, only at the borders between colors)

Did not find anything like "z_convert_format". Could you give a little code example ?

This is avsresize, the avisynth implemenation of the zimg library
http://avisynth.nl/index.php/Avsresize

eg. avsresize. "RGBP" is 8bit RGB planar

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

"point" is just nearest neighbor algorithm. It just doubles the chroma samples in the width. e.g. If you had 1920x1080 4:2:2, the U,V samples are 960x1080. It just duplicates them to 1920x1080. So on a pattern, the color borders will look "blocky"

"Bilinear" samples a 2x2 grid (4 pixels)
"Bicubic" samples a 4x4 grid (16 pixels)

There are dozens (or more like 100's) of resampling algorithms.

So you can get different pixel colors, depending on the chroma upscampling algorithm for RGB conversion, but a large "bar" such as in color bar patters would be independent of chroma upsampling. But this is one of the reason why one program might do things differently tha another program

If you get "wrong" colors on large colorbars, then something is ... wrong

Chroma location interpretation can also affect results (one of the reasons for suggesting avsresize, one possible explanation for some of the differences you see), but more for 4:2:0 material because it's 1/2 sampled width and height, 4:2:2 is "only" 1/2 sampled width, the errors are less drastic. When you have different chroma interpretations ("center" or "mpeg1") vs. ("mpeg2" or "left") , the color borders can shift when do subsequent conversions. It's usually not that noticable with 1 or 2 generations. It's more of an academic issue, or if you deal with RGB content and graphics. Avisynth is a bit non standard, because "mpeg2" or "left" is the standard chroma siting in all programs

Hotte

11th April 2021, 07:53

All NLE's should have 75% colorbars. I'd imagine Edius has them.
Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

I tried
z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:f=>rgb:709:709:f")
(exchanged the first l with an f). This shows only minor changes on a pixel level in comparison to ConvertToRGB24, no matter if I go for bilinear, bicubic or whatever. The overall very slight shift vs. yellow in the grass which the VDub2-TIFF holds against the Edius-TIFF (and this time I was using uncompressed V210 instead of HQX) is still there, but hey, differences are very slight and should be just fine for preview purpose.

kolak

11th April 2021, 13:49

poisondeathray

11th April 2021, 16:40

Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?

Yes. You have known colors. In YUV and RGB. In 8,10,12 bit . And rp219 lists the exact values for Y,U,V in 8,10,12 bit

For 8bit YUV, the colors in 8bit RGB will be slightly off, and it's expected to be +/-3. For example "75% Red" might be 192,1,0 or something (when using standard range equations). But 10bit YUV bars should yield perfect 8bit RGB values.

It's common to get slight color shifting when improper conversion is performed. A common scenario is a "fast" 8bit conversion to YUV before RGB. You're just ruling out if something was done incorrectly somewhere , maybe some setting, some switch - that's what all these low level tests are used for - so when you're doing your actual projects, there aren't any unexpected surprises (like clipping)

poisondeathray

11th April 2021, 16:49

Side note- ProRes444 modes do pass those reserved bits (in official Apple encoder).

Good to know;. But you would expect it to store 0-1023 for 10bit and 0-4095 for 12bit variants because the 4444 variants need to store the alpha channel. You can't have clipping in an alpha channel, or it would be completely useless

kolak

11th April 2021, 18:11

Well, alpha is a separate "stream" inside ProRes444. It's just losslessly compressed bits, more like a data than video.
444 is always encoded as 12bit. Alpha can be processed as 8 or 16bit.

poisondeathray

11th April 2021, 22:34

kolak did you use resolve to encode Pr444 or something else ? Did you use that test pattern ? Can you upload the Pr444 output ?

ffmpeg pr444 variant encode results in 4-4091 in 12bit, with the expected gaps at 12bit

If you down convert to 10bit, you get 1-1023, but smooth, basically perfect except for the missing zero value. PSNR y:90.300512

And if you feed native 12bit src, the ffmpeg pr4444 variant is still 4-4091 (and with gaps at 12bpc , not smooth)

kolak

13th April 2021, 18:13

It was done in Resolve on Mac.

ffmpeg ProRes444 encoder is 'broken'- 10bit only. They fixed only decoder.

poisondeathray

14th April 2021, 20:32

Thanks kolak, I don't have handy access to a Mac

It's ok if resolve processes it - and when downconverted and exported from resolve as v210 it's the correct 0-1023

But if you take the pr444 converted from ffmpeg imported into resolve then export as v210 it's 0-1022. So there are differences in either the up conversion and encoding of pr444 as detected by resolve. Yes, ffmpeg pr444 is only 10bit, but theoretically you should get same results since input source was 10bit anyways

But both mac/resolve pr444 and ffmpeg pr444 samples are 4-4091 according to libavcodec decoder. Both 1-1023 in downconverted 10bit. There must be differences in the way they decode and/or downconvert to 10bit as well. I'm looking into it

(BTW - There is a bug in the ffmpeg -c:v prores version with profile:v 4 alpha channel . It's been patched for prores_ks , but not prores (ticket 8509) . When decoded by libavcodec at 12bit, it's supposed to be 4095 in 12bit, but it's 4092. In other compositing programs, it's not 1.0, it's 0.99. This results in partial transparency)

12bit tests
Here is a 12bit444 HEVC perfect gradient. 4K DCI Y 0-4095 . Older Resolve version decodes it ok, so v17.x should too. Can you run it through a 4K/DCI timeline and export a Pr4444 frame and upload it when you get a chance?

https://www.mediafire.com/file/73cx5yq6kc1lzlk/hevc_12bit444_perfect_greyscale_ramp_Y(0-4095)_4K_DCI.mp4/file

kolak

14th April 2021, 21:17

Free Resolve doesn't decode it (neither you can do DCI frame size in free Resolve).
If you want to pass 12bit then I assume best format for Resolve will be DPX 12 bit or eg. 16bit TIFF. Use UHD size not DCI 4K.

poisondeathray

14th April 2021, 21:43

I thought you had studio version ? It decodes ok. I just wanted to test Prores444 12bit Y levels

You can check if resolve is decoding 12bit correctly by temporarily setting full range attributes, exporting 12bit DPX, then checking levels in vapoursynth or ffmpeg . Full range Y to RGB export is 1:1 mapping for Y to R,G,B

The reason for 4K choice is perfect gradient 4096 values. Same reason for 1024 width for 10bit test. Easier to identify workflow problems. If you scale or change width resolution you will create duplicate values or gaps.

For example, if you check dnxhr 12bit 444 or 422, in YUV, the range is correct 0-4095, but there are slight gap abnormalities in the Y waveform, with some intermediate regular values missing. How "solid" is official pr444 ?

kolak

14th April 2021, 22:02

I done ProRes444 test long time ago in AE with steps for 12bit inside 16bit TIFF. Cineform/DNxHR (RGB mode) were fine and ProRes was generating some strange artefacts in lowest levels (dither alike). It could be just AE etc. No idea. RGB based codecs were absolutely fine.
ProRes is always YUV internally, so RGB data which may go to it as RGB64A gets converted to YUV. If you look at ProRes private headers then there is an info what pixel format was on input. In many cases it's RGB64A, but Apple is using 32bit float YUV as well (FCPX does it I think). Problem is that this header may not be set at all or simply lying.

kolak

16th April 2021, 15:38

Apparently 12bit mode on SDI doesn't use those reserved bits, so maybe this is why ProRes444 which is 12bit passes all values.

poisondeathray

16th April 2021, 19:59

Apparently 12bit mode on SDI doesn't use those reserved bits, so maybe this is why ProRes444 which is 12bit passes all values.

According to SMPTE ST 372 and 403 for dual SDI - values 0-15 and 4080-4095 are reserved for 12bit

What is the equivalent v210, v410 fourcc for 12bit? v212, v412 ? I don't see 12bit uncompressed 4:2:2 or 4:4:4 options anywhere

kolak

16th April 2021, 21:42

Never seen "v210" for 12bit.
Here is explanation which I've seen:

RGB 4:4:4 (and RGBA 4:4:4:4) 10-bit can't use the full 10-bit range when carried on SDI interfaces. If they do, there is a risk of certain patterns of pixels causing the receiver to lose synchronization.

This is because RGB 4:4:4 10-bit uses the same bit layout in transmission as RGBA 4:4:4:4 10-bit, and so uses all of the available 40 bits per pixel. This means it would be possible to send illegal synchronization bit sequences if the full 0-1023 range of levels is allowed in the pixel values.

RGB 4:4:4 12-bit does not have this problem, as it only uses 36 of the available 40 bits per pixel, and they are arranged such that no 36-bit value can represent a synchronization bit sequence. So the full 0-4095 levels can be used.

poisondeathray

16th April 2021, 22:35

Not for SDI...

Full range, SDI Range, SMPTE / Legal range

https://professionalsupport.dolby.com/s/article/Dolby-Vision-Legal-Range-workflows-for-home-distribution?language=en_US

Video Range standards

10-bit Video Range Options (code values)

Full Range 0-1023

SDI Range 4-1019

SMPTE/Video/Legal Range 64-940

12-bit Video Range Options (code values)

Full Range 0-4095

SDI Range 16-4079

SMPTE/Video/Legal Range 256-3760

https://www.dcimovies.com/drafts/DCI-DRAFT-HDR-D-Cinema-Addendum_v09_2018-1116.pdf

Note: If the data is transported over SMPTE ST 372 (SDI dual link), code values 0-15 and 4080-4095 are reserved
(illegal) code values and these code values will be clipped.

kolak

16th April 2021, 23:03

No idea.
We would need SDI spec. I may be able to get confirmation from the actual spec, but at the end it's not that big deal.

Hotte

9th June 2021, 10:31

poisondeathray

9th June 2021, 22:34

Hi,

I am coming back to the TV/Full-Range discussion, because still I am not 100% sure how to handle this correctly. I have got much more 8-bit full-footage, but also TV-range stuff.

So a maybe silly question at the beginning: Why does full-range footage on a tv-range player look so much more contrasty ?

I thought, that full uses the lower and upper ends of the range and TV doesn't.

I'd understand if these ends are just simply being cut off and you miss some blacks and whites.

I'd also understand that - if it is smarter - the full contrast range is being mapped proportionally to fit into a tighter range and you end up with the same contrast appearance but less contrast variation within the range.

What I do not understand is why the whole contrast range looks completely different as if sbd boosted the gamma curve. Is this a compatibility issue and so interpreted the wrong way ?

I was told in this thread that it is ok to record and filter with full (since this gives more headroom especially in 8-bit) but the final result should be in tv-range. However I do not want the contrast to be completely spoiled and I do want to make use of the full contrast range that I recorded.

How is that to be achieved in AVS+ ?

Normal range video takes Y 16-235 (Y 16-235 black to white) and applies a contrast stretch to RGB 0-255 (black to white) on a computer monitor. Every thing looks normal. Black and white point match and are correct

If you take full range 8bit video (Y 0-255 black to white) , and display it incorrectly by "mapping" the same Y=16-235 to RGB 0-255, the contrast increases, and you lose the ends Y 0-15 and 236-255. Black and white point do not match.

Correct full range display would be Y 0-255 mapping to RGB 0-255. Full range video is not common for end delivery. Things like DVD, Blu-ray, web video, are all typically standard range

But modern TV's and players can handle full range video that is created and flagged correctly - but there are millions of devices, older TV sets , setups that do not handle full range correctly - Hence the earlier recommendation to deliver standard range

Hotte

9th June 2021, 23:25

Thanks poisondeathray.

I can identify all my clips that are full range.

(How) can I convert them from full to TV range with AVS ?

poisondeathray

10th June 2021, 02:50

(How) can I convert them from full to TV range with AVS ?

One way is to use levels or similar . This "squishes" full range to limited range

e.g. 8bit full to limited
levels(0,1,255,16,235,false)

10bit full to limited
levels(0,1,1023,64,940,false)

You can use ExtractY, ExtractU, ExtractV if you wanted to control Y,Cb,Cr separately along with CombinePlanes

There are other ways - such as coloryuv pc->tv presets, avsresize, smoothlevels (preset pc2tv) , a few others... but fundamentally it can be a problem to express 256 levels in 219 "slots", or 1024 levels in 876 "slots". If there are problems like banding, you can grade and work at higher bit depths and dither the down conversion to help reduce the issue

There is nothing wrong with full range video, if you know your playback scenario handles it correctly. But full range video is not recommended for general use.

Hotte

10th June 2021, 21:04

poisondeathray, I picked ColorYUV() from your list of recommendations as it is an avs+ native implementation, HBD-friendly and easy to use. My tests hat very good conversion results (full>tv). No visible color or contrast shifts, banding or whatever even with my very critical 10-bit blue sky gradient.

I tried out different older full-video sources of mine and found that each source type behaves differently: Some need tv-conversion to look the same, some not (although export- and codec paths where the same in all cases). Don`t know really. I think I have to test each source and implement source-specific conversion paths.

If you were asked for a player software for Windows which represents some standard oder "reference" concerning correct playback of color range , which one would you pick ? VLC ? Mediaplayer classic ?...

I read through most of this thread again - so much knowledge... but still I am not able to say wether I now should keep on recording my videos in full mode or wether I should switch the camera to tv-mode after all you said about it.

I still have in mind, that "full" must in some way give more headroom for contrast which I find important, especially for 8-bit footage. Or would you say this adantage is less relevant than the probable loss of quality I might envoke by the obvious need to remap using tools like ColorYUV() in most of the playback use cases today ?

poisondeathray

10th June 2021, 21:21

I tried out different older full-video sources of mine and found that each source type behaves differently: Some need tv-conversion to look the same, some not (although export- and codec paths where the same in all cases). Don`t know really. I think I have to test each source and implement source-specific conversion paths.

Some might not be flagged correctly, or incorrect metadata

Often intermediate file types won't have correct metadata eg. uncompressed v210 won't have any, and it's up to you to interpret it correctly in the editor

Final formats should always have correct metadata, so the player/hardware/software knows how to display it correctly

If you were asked for a player software for Windows which represents some standard oder "reference" concerning correct playback of color range , which one would you pick ? VLC ? Mediaplayer classic ?...

Probably MPV , definitely not VLC . MPC can be ok if it's configured correctly

I still have in mind, that "full" must in some way give more headroom for contrast which I find important, especially for 8-bit footage. Or would you say this adantage is less relevant than the probable loss of quality I might envoke by the obvious need to remap using tools like ColorYUV() in most of the playback use cases today ?

Full range is generally better if you're doing something to it in other programs , such as grading manipulations, because the range is spread out

If you're not doing much of anything else, and the final goal was limited range - then limited range would generally be the better acquisition choice

Hotte

11th June 2021, 07:47

poisondeathray

11th June 2021, 14:18

This was a helpful excursion that allows me to draw the following exclusions:

Older 8-bit "familiy and people" footage which was recorded in full range (because I just simply didn't know better) should have been better recorded in TV range, because this is sth you do not want to grade a lot but just simply "quick cut and watch" on different PC-Monitors, TV sets or projectors.

Lately high quality landscape filming comes to the fore. I am recording in 10-bit with flat profiles (maybe Log-profiles later or HDR). The goal is to pull the very best quality out of it so grading in most cases is mandatory.

I understood that full range is probably the better choice for the latter. However the price might be to add ColorYUV(levels="PC->TV") at the end of the process if the video is intended for non-full range players/monitors and in this case to take care that ColorYUV() is not doing any harm to the colors and creates no banding.

And generally the final encoder should be one that tells the color range to the player. I will be trying this out with mpv.

Do this sound coherent to you ?

Thanks!

Yes, but for the RGB to YUV step, (from a full range YUV source initially), if you're grading , or working in resolve, you're working in RGB

It would be faster (and better quality) to go directly to limited range YUV from RGB; instead of RGB to full range YUV then to limited range YUV in 8 or 10 bit (using a coloryuv or similar step) - because the latter involves an extra quantization step. You can demonstrate that 2 steps introduces more banding. Basically, fewer steps in integer math (resolve works internally in float), results in higher quality, fewer rounding losses

Hotte

11th June 2021, 16:08

Well, I do not grade with Resolve. I do the grading in Edius usually in YUV (I might be using the Edius-internal Primary Color Correction Filter which is suspected to work in RGB, but let's leave that out).

If this is not what you were refering to, then I could understand that it might be better to record tv-range rather than full-range (which applies to sensor-RBG > YUV) with having to apply ColorYUV() for tv-range at the end of the yuv-chain because quality loss of the additional ColorYUV() quantisation is likely to be more relevant than the narrower range. To put it short: Only record Full if you are sure to also display Full.

This way ?

poisondeathray

11th June 2021, 19:33

In general, you'd only use full if you're going to be grading it more than some minor adjustments (if you're still going to limited, full range acquisition can help when grading) , or if you final output is full

How are you adjusting the log footage ? If you're using LUTs , there is going to be an RGB step somewhere (at least internally or hidden)

Hotte

11th June 2021, 20:16

Ok, so I need to weigh the decision according due to the level of grading needed. Since I always fight with high contrast with my landscape footage I will probably stay with Full while recording and maybe I will have a projector being capable of displaying Full as well. Seems more future-proof to me and ColorYUV() did quite a good job I found.

Currently I am not shooting LOG because it has to be paid extra with the Panasonic G9. I am using a profile called Cinelike-D which is pretty soft. I grade it to personal gusto using standard YUV-Tools in Edius. But if the GH6 comes out I will probably go for either YUV422 10-bit 50p with HLG2100 or - if it is available - VLOG but with V-Gamma (very wide gamut).

I want to move to wide gamut about which I had an interesting discussion with Frank.

So I start to shoot HLG.2010/2100 which is also available on my G9 (10-bit but only 25p). There is a grading option in the Primary Color Correction Filter in Edius, where you can also apply LUTs to transate HLG2100 to Rec709 until I am equipped with an HLG capable display or projector (might take some time).
The conversion results are not very satisfying. The reds and greens do not look very natural. Just started...