Chroma sample location. What should I do?

[bokeron2020]

Hi!
I'm transcoding a 4K HDR10 bluray to 1080 HDR10 H.265. Input & outpuput are YUV420p10. I'm using ffmpeg for the whole process.
Chroma sample location for the bluray should be top-left... but it's center-left. So... what should I do?

1. Keep it center-left?
2. zscale it to top-left and flag it in x265 to get a within-the-specs output?

Would this *trans-locating degrade the image further considering I am downscaling at the same time?
Any cons/pros I should be aware before doing/not doing this?

[Asmodian]

Originally Posted by bokeron2020

Would this *trans-locating degrade the image further considering I am downscaling at the same time?
Any cons/pros I should be aware before doing/not doing this?

If you don't get the chroma location correct it will cause lower image quality. Moving chroma at the same time as downscaling would be better than doing it as two steps.

Trying to figure out the correct offsets for the chroma planes while downscaling has always been oddly confusing for me.

[bokeron2020]

As I'm using ffmpeg for this, looking at the docs there's two filters, chromashift and zscale with chromalin and chromal options.
Do they perform the same operation? Which should I use?


PS: I'm beggining to learn/understand the theoretical part of all of this... so please excuse me if I'm asking stupid questions.

[Asmodian]

I am not sure what to use in ffmpeg.

[bokeron2020]

Just to check if I know what I'm doing (I'm not too confident about that):

Chromashift apparently moves chroma horizontally and/or vertically.
zscale chromalin > chromal sets the chroma location for the input video and then for the output video... and I suppose that, being a scaler filter, it will scale it down taking the chroma position I've set into account automagically.

As I understand it now, I am dealing with an already subsampled video, and I cannot change that. I am encoding again, so I can only make sure this time the location is done properly spec-wise... so I'm telling the encoder "pick the sample HERE" and don't look back.

What I don't understand is wether starting with an already subsampled video instead of an original RGB or YUV444 makes the process different and something has to be done to lessen any undesirable side-effect like, say, some king of moiré-ish chroma-bomb.

I don't even know if what I've just wrote makes sense. Damn lack of education o'mine.

[Asmodian]

Ah, that makes sense then. chromalin and chromal sounds correct.

During the scaling process the scaler can move the chroma planes slightly, relative to the luma. They are separate already, and being scaled already, so there aren't any downsides compared to starting with full resolution chroma. If you needed to move the chroma while keeping it the same resolution there would be a minor quality drop starting with subsampled chroma, but nothing like moiré, just minor blurring and/or ringing depending on the scaling method used.

[bokeron2020]

Re-reading my posts and your answers I'm noticing that I may be missing something.
When we talk about "chroma sample location" I understand this term refers to where, in a 3x3 grid, we pick the sample to perform the chroma subsampling to 4:2:0.
You seem to be talking about moving chroma planes.

Is it the same thing and I'm failing to grasp it or am I poorly explaining what I mean (due to my lack of knowledge)?

[Asmodian]

It is tricky to think about.

For chroma placement, once you are working with the 4:2:0 data, don't think of it as "picking the sample to perform the chroma subsample to 4:2:0". Instead it is aligning chroma samples with the luma samples. The subsampling process has already been done. Subsampling chroma to 4:2:0 is simply downscaling the chroma data to half the resolution horizontally and vertically; hopefully this involves more than picking samples (e.g. a bicubic resize). This results in three planes of pixels; Y (at full resolution) and Cb and Cr (both at half resolution). This downscaling can be done with the output pixels aligned with different luma pixels (different chroma locations), so the player has to pick the correct chroma pixels to combine with the luma pixels when converting it all back to RGB.

If you need to "move" the chroma plane half a pixel without scaling then you end up blending pixels together in the process of figuring out what the "correct" chroma value is for the new pixel that is half way between the two original chroma pixels. However, if you do this while also downscaling you are inventing all new pixels anyway, so you just need to align the resize kernel so the new pixels are shifted that half pixel in the final output (which represents a whole pixel in the input when doing a 50% scaled output). This kind of shift during the scaling process does not increase the mixing/bluring of neighboring chroma pixels compared to the same downscaling ratio without the shift.

Did this help, or make it worse?

[bokeron2020]

Both!

It helps a lot. I need to shift paradigms to understand what I'm really doing. I'm still on it, though, so much of what you've explained makes sense but still in a fuzzy way.

At the same time there's a lot to learn and work on to just reach "working knowledge" and do it properly, so this is another one bit of data I need to muster up the brains for.

Thank you!

[benwaggoner]

To add complexity, since shifting chroma a half pixel is invisible in most cases at 4K resolution, it's unclear how much UHD 4K encoding actually uses a source with chromaloc 2 versus using a chromaloc 0 source that then gets flagged at 2 for UHD BD spec compatibility. I don't recall ever seeing a mezzanine elementary stream with chromaloc specified.

[nevcairiel]

Mezzanine streams would often be 422, no? That difference would only apply for the final 420 encode. As long as they are actually encoding for UHD BD from a proper master that is likely not 420 itself, there is no reason to not downsample the chroma in the right position, instead of lie about it.

[bokeron2020]

Is there any tool (free, if possible) or procedure to determine real chroma location instead of trusting the chromaloc flag ?


EDIT: Also, a question regarding how important all of this is outside of the professional field.
>>What are the "real world" consequences of just flagging it topleft no matter what and be done with it?
I mean, any differences for a home user now or in the future?

I'm finding a smorgasbord of BluRays and downloadable content with flags all over the place and I'm beggining to question if I'm not being a victim of diminishing returns, because correcting what others did wrong without knowing how wrong they really did it is getting on my nerves. I want to do it properly if I'm not gonna die trying... and right now it looks like I would...

[Balling]

Please wait untill this patch series is applied. https://patchwork.ffmpeg.org/project...bjp@gmail.com/

"transcoding a 4K HDR10 bluray to 1080"

I do not understand this part. HDR Blu-rays must all use top-left. Did you check what is printed in Mediainfo?

As for your question: early ITU-T H.265 specs did not care about chroma siting at all. Now, in latest spec used for Blu-rays, including Dolby Vision for BT.2020 and BT.2100 (last one is HDR, first is SDR) top left location is mandated, you can see that in Mediainfo (4:2:0 (Type 2)). When you encode for Profile 5 Dolby Vision you use default AVC left chroma (just Type 0 in mediainfo), even though is BT.2100. That is not used on Blu-ray though.

Please note that the actual value is in VUI in SPS of HEVC bitstream itself. It is not usually tagged in container, though mkv does have such metadata.


"mezzanine elementary stream with chromaloc specified."

Are you kidding me? Mezzanine is 16 bit DCI-P3 (no BT.2020) with PQ transfer RGB. RGB does not have chroma. And does not have chroma siting. Same about siting in YCbCr 422 and 444 (422 technically does have it but it is only 1 used, not 6 (with 3 used) in 420).
"Is there any tool (free, if possible) or procedure to determine real chroma location instead of trusting the chromaloc flag ?"

No. Though there are test patterns.

As for how you do it... @nev said it:

-vf 'scale=out_color_matrix=bt709:out_h_chr_pos=0:out_v_chr_pos=128'

The position is specified in the luma grid / 256, so for a 4x4 grid in 4:2:0, 0:0 is center of the first pixel, 256:256 the last, and 0:128 is in the first column, between both rows - i.e. mpeg2 position.

HEVC type 2 should be 0:0 from what I can tell (pixfmt.h calls this AVCHROMA_LOC_TOPLEFT)

[FranceBB]

Originally Posted by benwaggoner

I don't recall ever seeing a mezzanine elementary stream with chromaloc specified.

Neither did I, but that's because I've never received a 4:2:0 mezzanine file to be fair...

Originally Posted by nevcairiel

Mezzanine streams would often be 422, no?

Yep. Every time I get a masterfile is 4:2:2 either 10bit or 12bit planar, same goes for our internal mezzanine files which are 4:2:2 planar 10bit, so the chroma location thing doesn't apply for masterfiles as it's always in the classic MPEG-2 position.

Originally Posted by Balling

Mezzanine is 16 bit DCI-P3 (no BT.2020) with PQ transfer RGB. RGB does not have chroma. And does not have chroma siting.

Lucky duck, the highest I've ever got is a 12bit RGB, otherwise it's always YUV 4:2:2 or YUV 4:4:4 but 12bit or 10bit planar. I've never received a 16bit mezzanine...

(Not that I care much as I would have to dither it down to 10bit planar anyway as per our specs, but still, it would be nice to see a real world 16bit file).

[bokeron2020]

These last messages are quite informative to me, and I understand professionals/experts take this matters to a higher level, but I'm still wondering what to do. So... please let me just for a moment return to my particular situation, and repeat one last question to understand the importance of this for a home user.

What are the "real world" consequences of :

1. Keeping the left location [in order to prevent further degradation that comes from converting to topleft].
2. Not converting and just flagging it topleft

I mean, any differences for a home user now or in the future?

I'm finding a smorgasbord of BluRays and downloadable content with flags all over the place and I'm beggining to question if I'm not being a victim of diminishing returns, because correcting what others did wrong without knowing how wrong they really did it is getting on my nerves. I want to do it properly if I'm not gonna die trying... and right now it looks like I would...

[Balling]

HDR Blu-rays must already have top left chroma siting. So 2. "Not converting and just flagging it topleft". That is not done with -vf scale. That is just done with bsf filter.

https://ffmpeg.org/ffmpeg-bitstream-...c_005fmetadata

ffmpeg -i file.mp4 -c copy -bsf:v hevc_metadata=chroma_sample_loc_type=2 file1.mp4
Obviously 0, 1, 2, 3, 4, 5 are possible but also -1 can be applied to remove any chroma siting info from VUI in SPS.

"What are the "real world" consequences of"

You can read that in this post https://www.avsforum.com/threads/spe.../post-60439344

and this pattern https://www.avsforum.com/threads/spe.../post-60430955

[bokeron2020]

Originally Posted by Balling

HDR Blu-rays must already have top left chroma siting. So 2.

They must but apparently not all of them have it. The same can be said for downloadable content. Some are 0, some are 2.

Originally Posted by Balling

You can read [the real world consequences] in this post [...] and this pattern [...]

If I'm understanding it, you mean I'll get misplaced chroma with the effects described in that thread?
But when will that happen, when I keep out-of-specs videos or when I flag them without properly converting?


What I'm trying to decide for now is... should I convert to within-specs or could I ignore the specs and keep it like that?

• Downscaling and fixing out-of-specs chroma location has a cost in quality in all the PSNR-SSIM-VMAF tests I've made. It degrades the resulting video slightly more compared to just downscaling the video without fixing chroma.
  
• As far as I can tell, with the software/hardware I use now, media with out-of-specs chroma plays OK and I see no ill efects - which doesn't mean they aren't there, only that I don't see them if they are.

So, for keeping a home media server, does it pay in the long run to fix out-of-specs chroma despite the quality cost or will I seldom face problems related to out-of-specs chroma in the foreseeable future?

[FranceBB]

Originally Posted by bokeron2020

What are the "real world" consequences of :

1. Keeping the left location [in order to prevent further degradation that comes from converting to topleft].
2. Not converting and just flagging it topleft

I mean, any differences for a home user now or in the future?

Ok, so, from a home user perspective, let's start by saying that anything official should have the right metadata.

Taken that aside, there's a staggering amount of incorrect material out there in the web almost always 'cause it has been re-encoded by someone who had no clue whatsoever about what he was doing.

So, let's suppose we have the following scenario:

- UHD file with MPEG-2 chroma location flag and that is actually with the MPEG-2 chroma location

- UHD file with the MPEG-2 chroma location flag that is instead with the 4:2:0 Type 2 chroma location


The first one, although some might argue that it's not standard, doesn't have any problem whatsoever when you play it back on a compatible device 'cause the metadata and the location are both correct, so my advice would be to leave it as it is.

The second one is a problem 'cause the flag says 4:2:0 while the chroma location is 4:2:0 Type 2, so the decoder will use the wrong location when decoding and it will lead to artifacts. Some people like our Ben say that it's not so noticeable, especially at 4K resolution, which is actually true as people tend not to notice this that much, however it's technically not correct and will lead to degradation when decoding. In that case, I would suggest you to re-encode the file and fix the chroma location with ffmpeg.



A quick example: since in Avisynth I generally work with the old MPEG-2 alignment, whenever I have to output a 4:2:0 Type2, I use:

ffmpeg.exe -i "AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither

followed by other x265 options.
So far no one ever complained about this kind of conversion and I even sent those files to Samsung for the Samsung The Wall demonstration with Sky contents before the whole COVID-19 pandemic happened and no one ever said anything, which means that it was ok.

Originally Posted by bokeron2020

I'm finding a smorgasbord of BluRays and downloadable content with flags all over the place and I'm beggining to question if I'm not being a victim of diminishing returns, because correcting what others did wrong without knowing how wrong they really did it is getting on my nerves. I want to do it properly if I'm not gonna die trying... and right now it looks like I would...

Bluray? Official BDs? Really? O_O
Oh God... something went wrong down the production chain if the chroma location is wrong in an official bluray, like if it's 4:2:0 Type2 in metadata but it's actually MPEG-2 4:2:0 inside and what bothers me is that those kind of things shouldn't happen since there's generally a rigorous automatic and manual QC for those kind of things...
Well, now I've lost faith in humanity ehehehehehe

No, I mean, jokes aside, if you find things wrong on contents downloaded on the web, you should correct them 'cause it's almost always the case that whoever re-encoded that had no clue about what he was doing.
As technologies get more and more advanced, people get more prone to make mistakes, so we did see that coming, but professionally wise they should be ok.

[Balling]

"you to re-encode the file and fix the chroma location with ffmpeg."
You do not need to reencode the file. My command above will do it losslessly, changing just one value all over bitstream ffmpeg -i file.mp4 -c copy -bsf:v hevc_metadata=chroma_sample_loc_type=0 file1.mp4. As for your command of Samsung the wall, please show mediainfo. Did you actually tag the file? It should print YUV 4:2:0 (Type 2), where type 2 is the actual info in VUI. Mediainfo can read all 6 types of chroma siting.

I can also confirm that

 -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0

is correct for BT.2020-ncl Type 2 (top-left) chroma. So congrats.

[bokeron2020]

Thanks for your answer! It helps me, along with all the previous answers, to know where I stand, what to do ATM and directions to learn at least enough to reach a basic working knowledge for a home user.
The are two points I'd like to ask about again:

Originally Posted by FranceBB

UHD file with the MPEG-2 chroma location flag that is instead with the 4:2:0 Type 2 chroma location.
...
I would suggest you to re-encode the file and fix the chroma location with ffmpeg.

But... how do I find the real chroma location? Any tool or procedure to find it?

Originally Posted by FranceBB

ffmpeg.exe -i "AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither

I've seen most people here use scale instead of zscale, and pipe from ffmpeg to x265.exe.
Why not zscale? And why not to use integrated ffmpeg x265 codec?

Originally Posted by FranceBB

Bluray? Official BDs? Really? O_O
[...]
Well, now I've lost faith in humanity ehehehehehe

The bad blurays I've found are mostly local market editions dating back to the first bucnh of HDR releases and some documentaries, concerts, etc, from small publishers.
Some buy&keep downloadable movies have shown also this problem (ie, from chili.com), though it mostly predates non-released in Spain content that I have to find elsewhere, like "Space Adventure Cobra" which sometimes seem to be encoded by a blind monkey.


One last question about missing metadata. Any free tool out there to calculate MaxCLL,MaxFALL?