Transparent encodes of fake 4k blurays

[YaBoyShredderson]

My goals when encoding are to be indistinguishable from the source based on my viewing conditions, with a bit of quality buffer should those viewing conditions change (closer seating, larger tv etc).

I havent started encoding my 4k blurays just yet, only 1080p, as i want to get a ryzen 4000 processor before to speed things up a tad. But i wanted to know if i should downsample them to 1080p?

Obviously true 4k i should leave as is, but with fake 4k, stuff that was mastered at 2k and then upscaled to 4k, should i leave that in the full 4k? Or will it be the same quality at 1080p as thats what it mastered at? I would likely save a whole lot more space doing this.

[benwaggoner]

>1080p doesn't matter so much with SDR, but absolutely can with HDR so that sharp specular highlights can be preserved.

There isn't always a hard line between fake and real 4K, as titles may combine 4K elements with 2K VFX shots. How much grain and motion blur have a big impact on potential detail as well.

[scharfis_brain]

Also 2k Upscaled UHD Bluray benefit from better chroma resolution. (Near 444 related to 2k)

When downscaling an UHD Bluray with 2k content to 1080p, one will reduce the chroma to a quarter of its original resolution.

[YaBoyShredderson]

Ok, makes sense, leaving at 4k is the best for me then. Thanks.

[foxyshadis]

Before you make a choice that will hugely impact both encoding time and storage space, just watch a few downsampled to 1080p and then upscaled again to see if you can tell. Some can, many can't, and there's not much point if you can't.

[Boulder]

Before you make a choice that will hugely impact both encoding time and storage space, just watch a few downsampled to 1080p and then upscaled again to see if you can tell. Some can, many can't, and there's not much point if you can't.

This. I use a method that utilizes the clever tool called Zopti to determine optimal (well, at least better than using the same values for all sources) parameters b and c for BicubicResize to downsample. There's a huge difference compared to just using Spline36 or Lanczos. For native 4K, I go for 1440p and upscaled 2K goes down to 1080p.

[K.i.N.G]

scaling down to 1440p might be a good middle ground.

[benwaggoner]

And note that even "uprezzed" 2K to 4K titles don't use some fire-and-forget algorithim like bicubic. There is a lot of shot- and scene-based tweaking of parameters to get best results. For major titles from major studios, it's more like a lightweight remastering. And if it was a DCI-P3 master getting converted to HDR, it really IS a remaster. What you'd get scaling back down to 1080p isn't going to be the same as the original 2K master (which was 2048x, not 1920x; ~14% more source pixels).

Note a cinema "4K" projector is 4096x, not the home video/broadcast 3840x.

[FranceBB]

Also 2k Upscaled UHD Bluray benefit from better chroma resolution. (Near 444 related to 2k)

When downscaling an UHD Bluray with 2k content to 1080p, one will reduce the chroma to a quarter of its original resolution.

Not necessarily. If someone doesn't care about playback compatibility, when using a reverse upscale filter, the idea would be to reverse upscale the luma inverting the kernel (i.e Debilinear, Debicubic etc) to the original resolution but leaving the chroma as it is, which means that you're gonna end up with a FULL HD 4:4:4 10bit file from an upscaled UHD 4:2:0 10bit which is a win win.

[benwaggoner]

Not necessarily. If someone doesn't care about playback compatibility, when using a reverse upscale filter, the idea would be to reverse upscale the luma inverting the kernel (i.e Debilinear, Debicubic etc) to the original resolution but leaving the chroma as it is, which means that you're gonna end up with a FULL HD 4:4:4 10bit file from an upscaled UHD 4:2:0 10bit which is a win win.
It would be VERY unusual content for there to be a visible difference between downscaling to 4:4:4 or 4:2:0. Something like very sharp red text on a blue background or something. I don't believe I've seen actual TV/movie content where 4:2:0 at 1080p and above wasn't adequate.

Screen recordings, sure. Turn off ClearType before doing screen recordings, people! It messes up the chroma for people with different types of panels.

[scharfis_brain]

I don't believe I've seen actual TV/movie content where 4:2:0 at 1080p and above wasn't adequate.

Actually I see it all the time, when the content is crisp enough.
It bothers me.
That's why I try to use MadVRs chroma reconstruction/upsampling most of the time I use thr PC to playback movies.

Anyways YUV444 files won't be too compatible with SmartTV or Bluray players. So we are stuck here with 420 chroma.

[benwaggoner]

Actually I see it all the time, when the content is crisp enough.
It bothers me.
Can you give some examples where you see this?

That's why I try to use MadVRs chroma reconstruction/upsampling most of the time I use thr PC to playback movies.
Is the problem perhaps in the display pipeline, not the encode? There are obviously different algorithms that could be used for how to optimally go from the 4:2:0 to the display's RGB. Nearest Neighbor is going to be a lot worse even compared to a basic bilinear.

Anyways YUV444 files won't be too compatible with SmartTV or Bluray players. So we are stuck here with 420 chroma.
>4:2:0 chroma samples haven't ever been used in a broadly supported distribution format, disc or streaming. 4:4:4 doubles memory buffer and bandwidth requirements, and increases bitrate some, and demonstrated gains have never been worth the extra cost.

Put another way, 1440p 4:2:0 is fewer samples-per-second than 1080p 4:4:4, and will look better for 4K source content.

[Cary Knoop]

4:4:4 doubles memory buffer and bandwidth requirements, and increases bitrate some, and demonstrated gains have never been worth the extra cost.

I would challenge the assertion that not using chroma subsampling would increase the bitrate somewhat for the same perceptual quality.

It would be much more efficient for a CODEC to determine what perceptual compressions to make instead of feeding an already chroma-subsampled source.

There are five things, I believe need to change in video processing:

1. More to a float32 code value standard
2. Abandon the difference between full and limited range (no clipping while float 0.0 to 1.0 is always the visible range)
3. No more chroma subsampling (but allow chroma subsampling for legacy sources)
4. No more interlaced sources (but allow interlaced for legacy sources)
5. Allow for multiple framerates in the same source (each frame will have an absolute time duration marker)

This will make life a lot simpler!

[benwaggoner]

I would challenge the assertion that not using chroma subsampling would increase the bitrate somewhat for the same perceptual quality.

It would be much more efficient for a CODEC to determine what perceptual compressions to make instead of feeding an already chroma-subsampled source.
I agree. And thanks to Moore's Law, memory requirements become less expensive year-on-year.

There are five things, I believe need to change in video processing:

1. More to a float32 code value standard
2. Abandon the difference between full and limited range (no clipping while float 0.0 to 1.0 is always the visible range)
3. No more chroma subsampling (but allow chroma subsampling for legacy sources)
4. No more interlaced sources (but allow interlaced for legacy sources)
5. Allow for multiple framerates in the same source (each frame will have an absolute time duration marker)

This will make life a lot simpler!
1-3 would be a HUGE increase in memory and memory bandwidth requirements. Today 8-bit video is 12 bit/pixel. Going to fully sampled float32 is an 8x increase in memory requirements!

Doing 1080p30 in your proposed would have the same memory requirements as 4Kp60. Even 10-bit HDR would still be 6x more efficient using 4:2:0.

4. Is pretty much accomplished at this point. There is no interlaced happening with resolutiosn beyond 1080i or with HDR. And H.264 was the last codec that really engineered for interlaced efficiency (HEVC doesn't have MBAFF, for example). Interlaced is already a shrinking legacy technology, thank goodness.

5. We've had media formats that just give each frame a duration for ages and ages - supporting a fixed frame rate was itself an innovation that didn't get broadly implemented until about 15 years ago. The real challenge with variable frame rate video is that technologies like HDMI don't handle it gracefully. Over a 60p connection, switching between 24 and 60 has intrinsic judder. Having everything 120 Hz would be a lot easier for NTSC countries. PAL countries mixing 24/25/30/50/60. THe least common multiplier for all standard frame rates is 600. And that's not accouting for the NTSC 29.97/30 timing headache.

[Cary Knoop]

1-3 would be a HUGE increase in memory and memory bandwidth requirements. Today 8-bit video is 12 bit/pixel. Going to fully sampled float32 is an 8x increase in memory requirements!

Not really, the floats are converted to 8 or 10 bit using a byte stream depending on the destination bit-depth during (or right after) decoding. Practically speaking there are no commercial monitors that can display over 10-bit of accuracy, there is no need to actually render float values to the monitor.

I believe it should be possible when someone makes a documentary with mixed framerate footage not to worry about making the overall framerate unique but to allow multiple segments. There is absolutely no technical reason why modern monitors cannot switch framerates on the fly (or get a change signal a few frames ahead).

HDMI is an awful, protectionist, and very limited technology.

[huhn]

adaptive sync for gaming is not a special feature anymore more which HDMI can do but rarely used with HDMI displays. usually only a range of 48-60 is present which means a practical range of 50-58 in short not useful at all yet.

HDMI 1.3 was always able to do 1080p 60 hz 12 bit and TV where able to accept this type of signal too and it was used on PC with consumer grade hardware.

HDMI 2.0 is the first major used version that couldn't do the target refreshrate/resolution with 12 bit for 23p 12 bit is a default feature.

if i now take in to consideration that every TV except one i have tested in my life span performance much worse in term of banding when 10 bit or more bit was send into the display instead of 8 bit i don't even know what the point of it is. guess what my new X900h get's "destroyed" when 10 bit is send into it instead of 8 bit and i wasn't expecting anything else.

hasn't it been proven that downscaling is a terrible compression algorithm and that 4:4:4 which a higher chroma qp offset setting is "general" better for quality even for luma then 4:2:0 because more bandwidth is used on luma?

the chroma scaler in madVR weren't created for fun and they where discussed with real world examples. if i really have to i can hunt some... or we just take a esport computer game gaming is already bigger then hollywood so lot's and lot's of content where 4:4:4 matter a lot.

pretty much every new not low end LG TV can or is planned to do 120hz 4K 4:4:4.

just for fun nvidia can do 4.4:4 hardware "encoding" is limited but it can do it.

modern computer are very good at 16-256 bit so i don't see a huge issue here and aren't people already doing this with "avisynth" and vapoursynth? we are talking about processing right not encoding to 16-32 bit?

[benwaggoner]

Not really, the floats are converted to 8 or 10 bit using a byte stream depending on the destination bit-depth during (or right after) decoding. Practically speaking there are no commercial monitors that can display over 10-bit of accuracy, there is no need to actually render float values to the monitor.
It is still memory and bandwidth required in the decoder itself, even if the output is always normalized to 10-bit.

I believe it should be possible when someone makes a documentary with mixed framerate footage not to worry about making the overall framerate unique but to allow multiple segments. There is absolutely no technical reason why modern monitors cannot switch framerates on the fly (or get a change signal a few frames ahead).
Variable frame rate monitors for gaming have been around for a few years, and we're seeing those techs make it into some televisions. But that only works when all devices support it. I don't know of any media playback solution that's tried to leverage variable frame rate at all. For pretty much all consumer delivery, the masters have to get conformed to a specific frame rate for a master to be used broadly.

It's a fine idea, and I could see something like the PS5 or XBox Series X being able to support it eventually. It might "just work" under a few specific high-end Windows gaming setups. But that'd be <0.1% of the installed base at best, today.

Sounds like a potential SMPTE spec.

HDMI is an awful, protectionist, and very limited technology.
And startlingly undertested for performance. Most interoperability testing stops with "I saw and heard something" not "Did 4k HDR with Atmos work correctly without having to fiddle with menu settings?"

[huhn]

does 4:4:4 decoding really cost more silicon when a hardware decoder can do 8K 4:2:0 and 4:4:4 is limited to 4k in this case?

BTW. freesync and gsync screen work out of the box with pretty old hardware and even entry level screens have often support for it (mostly useless like TVs).
and player seem to support it too. a high end gaming PC is clearly not needed.
https://github.com/mpv-player/mpv/issues/6137

https://forums.blurbusters.com/viewtopic.php?t=3509

[benwaggoner]

does 4:4:4 decoding really cost more silicon when a hardware decoder can do 8K 4:2:0 and 4:4:4 is limited to 4k in this case?
I don't think it be a material difference. Although any potential visible benefit of >4:2:0 is going to be even smaller at 4K.

BTW. freesync and gsync screen work out of the box with pretty old hardware and even entry level screens have often support for it (mostly useless like TVs).
and player seem to support it too. a high end gaming PC is clearly not needed.
https://github.com/mpv-player/mpv/issues/6137

https://forums.blurbusters.com/viewtopic.php?t=3509
Well, that is cool. I will look more into it.

But the broader ecosystem challenge is that there are plenty of devices that can't play the content back, and not much content exists that could truly take advantage of a variable frame rate. Most titles that used sources with different frame rates had those assets conformed to the project's fps before editing. Getting end-to-end VFR working would require big overhauls to editing and content creation software upstream and then a new generation of content using those technologies.

Still, it'd make it possible to have the 48p versions of The Hobbit movies work on home video, which hasn't been possible to date as 48 fps support is far from universal, and isn't a broadcast or HDMI standard.

[huhn]

for 1080p is pretty easy to show "major" difference been chroma scaler.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.

[Cary Knoop]

for 1080p is pretty easy to show "major" difference been chroma scaler.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.
I am sure there are 8k and 12k raw samples online you can demosaic to 4:4:4.

[benwaggoner]

for 1080p is pretty easy to show "major" difference been chroma scaler.
Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.
Studio-provided IMF mezzanines often will be 12-bit 4:4:4. And most pro sources are at least 10-bit 4:2:2. Some of the blender.org projects are available as 16-bit RGB 4:4:4 frames as well.

Game footage is actually pretty good for finding chroma edge cases, since it gets rendered per-pixel and anti-aliasing tech is far from perfect. HUD details like red text in a compass or something can have quite sharp chroma details.

[Cary Knoop]

Studio-provided IMF mezzanines often will be 12-bit 4:4:4.
But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.

[huhn]

Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.
we do these test with real world samples from actual content not test pattern. we used test patterns to but there is s reason for quite a lot of chroma optimised scaler clearly not for test pattern.

for game footage i could just take rimworld make a screen it's extrem obvious because red text is used every where the esport titles should show this very easily to. these are not edge cases it's visible the whole time.

for real world camera footage you nearly always have to edge cases but not everything is made with a camera.

[foxyshadis]

does 4:4:4 decoding really cost more silicon when a hardware decoder can do 8K 4:2:0 and 4:4:4 is limited to 4k in this case?

BTW. freesync and gsync screen work out of the box with pretty old hardware and even entry level screens have often support for it (mostly useless like TVs).
and player seem to support it too. a high end gaming PC is clearly not needed.
https://github.com/mpv-player/mpv/issues/6137

https://forums.blurbusters.com/viewtopic.php?t=3509

There is a silicon cost, in doubling the available memory -- which admittedly isn't really all that expensive anymore, but every bit counts that deep. The bigger cost is the upfront engineering labor that they don't expect any ROI on, so why spend it in the first place?

This might change now that screensharing has suddenly become something everyone is doing, instead of just for the rare business meetings.

[foxyshadis]

But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.

That's beside the point; 4:4:4 is about keeping all available chroma information, instead of throwing away 75% of it. For naturally captured videos, losing 75% of the chroma is essentially meaningless. Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.

[Cary Knoop]

Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.
I disagree with that.

I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.

[huhn]

and what is not digitally created these days?

[Asmodian]

I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.

I do wonder how long it will take to discard all that quality damaging baggage from the video standards developed for the analog-converted-to-digital days. That all standard UHD still uses 4:2:0 video level encoding is pretty sad really.

Even YouTube and Netflix et al. are encoding in YUV 4:2:0 video levels.

Probably because the video decoding side has traditionally also been really poorly supported on computers. Until hardware decode became standard (very recently) no one payed attention to how to do things correctly. GPU drivers still cannot get basic standards correct all the time. This means there would have been terrible issues for a lot of people for a while after any change, so no one was ever willing to change.
Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard. :(

AV1 should take the opportunity to go pure full-range 4:4:4, but they won't. :p

[Cary Knoop]

Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard. :(

But I would venture you actually save bandwidth if you go 4:4:4 and let the encoder do its perceptual compression.

4:2:0 is simply a rather crude form of perceptual compression. By using 4:4:4 you give the encoder the full and much more flexible power on how to compress efficiently.

[huhn]

that's not what he means decoding 4:4:4 needs double the memory as 4:2:0 the image is now 2 times the size it is as it is.

and full range has other issues as long as YCbCr is used it can create out of range values this could be "fixed" with YCoCg or ICtCp. RGB is not an option.

the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma

[Cary Knoop]

the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma
Actually the original gamma OETF/EOTF SD video system was designed to provide an overall 1.2 resulting gamma not a net 1.0 gamma as the document indicated.

It's a jungle out there though: sRGB, Rec709, BT1886.
True gammas, gammas with a linear segment, camera gamma without linear segment.

[benwaggoner]

But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.
And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!

https://upload.wikimedia.org/wikipedia/commons/thumb/3/37/Bayer_pattern_on_sensor.svg/500px-Bayer_pattern_on_sensor.svg.png
The only TRUE 4:4:4 content is computer rendered, where it is actually created at native RGB 4:4:4.

Also, the human visual system is far more sensitive to edges in the luma domain than to chroma overall. While we have highly accurate color vision, we process it with much less spatial and temporal detail than basic black-and-white vision which is what most of our evolutionary ancestors only had. Luma is what keeps you from getting eaten by a tiger stalking you across a treeline. Chroma is what keeps you from eating unripe or poisonous fruit. But seeing precise color detail in something that's moving just isn't something we're wired for.

We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.

Fortunately the misbegotten YUV-9 (one chroma sample per 4x4 block of luma) used in 90's codecs like Sorenson Video and Indeo died. That was definitively TOO supersampled. Especially with 320x240 video, which would have only 80x60 chroma samples. Colored text was a nightmare.

Another classic problem was going from NTSC DV25, which used 4:1:1 subsampling (one chroma per four pixels horizontally) to DVD (4:2:0) which netted out 4:1:0 color, so one chroma per 4x2 block of pixels. Key was to do all motion graphics after DV25, rendering as 4:2:0. Natural images were okay-ish with 4:1:1, as long as all graphics were done with more precision.

[Cary Knoop]

And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!

Perhaps you should pay a bit more attention to what I actually write?

I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design

[Cary Knoop]

We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.

A clear argumentum ad antiquitatem "we have always done it like that" fallacy.

By leaving the question on how to do the best perceptual compression to the codec on a frame by frame basis you will get the best solution, not by some arbitrary constraint impacting every frame the same way before you even start the compression.

[benwaggoner]

That is why I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design

So, like using a 6K sensor for making 4K video? Alas, no one in Hollywood is actually doing post at >4K outside of tech experiments. Whatever the native camera format, it's always turned into a XYZ, RGB, or YUV 4K (or lower) before any real creative work is done. That said, if it's using ACE or ProRes 4444 XQ, it isn't subsampled. That's seen more in higher-budget films, with TV production being mostly done in 4:2:2.

Something thing people often miss is that a 4K sensor with only one color per "pixel" is actually LOWER detail than 4K with 4:2:0, since luma and chroma samples are colocated. going from a 4096x sensor to a 3840x file helps that a bit, even though they are both called "4K."

There is some very interesting alchemy that goes into making a good source out of camera's native formats!

[Cary Knoop]

So, like using a 6K sensor for making 4K video?
Blackmagic Design just came out with a 12k (non-Bayer) sensor.

[benwaggoner]

Blackmagic Design just came out with a 12k (non-Bayer) sensor.
What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).

[Cary Knoop]

What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).
It is an RGBW sensor using a 6x6 grid (6xG, 6xB, 6xR plus 18x white pixels).

Here the link to the patent app:
https://patents.google.com/patent/US20190306472A1/en?inventor=Carsten+Buettner&sort=new

Personally I think it is a step in the wrong direction, sure, it may be that this configuration turns out to be superior dynamic range wise, but the demosaicing is more complex (and perhaps even theoretically "wrong").

What we really need is to eventually get rid of Bayer or other sensors that require demosaicing.

Requiring demosaicing, just like interlaced video, chroma subsampling, fixed-bit code values, all those things are not features they are hacks.

[huhn]

i can "just" go and record a lossless 4:4:4 4K RGB game video.
or 1440p/1080p if my disk or CPU can't handle so much throughput and just do the old test of if subsampling actually improves image quality with the same bit rate?

i know this has been tested with x264 back in the days but i can't remember x265.

i mean we can talk a lot but test are the real thing.

[benwaggoner]

i can "just" go and record a lossless 4:4:4 4K RGB game video.
or 1440p/1080p if my disk or CPU can't handle so much throughput and just do the old test of if subsampling actually improves image quality with the same bit rate?
Some kind of lightweight lossless or perceptually lossless codec is ideal here. There's software that can do this, ideally leveraging the CPU's HW encoder so as not to burn a lot of CPU during that game. That said, a lot of those encoders are subsampled by default so some tweaking might be needed. I've not done screen recording for ages, so I'm not up on what today's state of the art is.

Alternatively you can capture HDMI out if you have that available. The cheap HDMI capture solutions are 4:2:0, though.

I don't know what can capture to >8-bit on-PC. An interesting comparison would be between 10-bit 4:2:0 and 8-bit 4:4:4, when those are availble. Some devices, like Xbox One X, can even render and output 12-bit over HDMI.

i mean we can talk a lot but test are the real thing.
Huzzah!

And heck, I see I have some 1080p60 RGBA recorded game clips on my RAID. I'll try to whip out some quick tests.

Any preferences between:

CSGO
DOTA 2
GTA V
Hearthstone
Minecraft
Rust
Starcraft 2
Witcher 3


There are captures from over four years ago, so they won't have all the latest bright and shiny effects.

[benwaggoner]

It looks like DOTA2 has the most extreme examples of single-pixel transitions between brightly saturated to no chroma, so I'll do some test runs like that.

On an initial test with Witcher 3, CRF 18 still left a lot of visible blocking and banding. The default aq settings aren't well tuned/effective adapting for that sort of material. DOTA2 is quite different, so I'm going to stick with CRF 18 to start.

Bitrates were pretty much identical between 420 and 444, probably because of:
x265 [warning]: halving the quality when psy-rd is enabled for 444 input. Setting cbQpOffset = 6 and crQpOffset = 6
Which should make apples-to-apples comparisons easier, as the same amount of bits should be allocated to both luma and chroma in either mode. Although any potential gain from 444 might require lower Qps for chroma to retain those edges.

Here's the DOTA2 command line I'm kicking off. These seem reasonable?

ffmpeg.exe -i DOTA2.mov -pix_fmt yuv420p -f yuv4mpegpipe - | x265.exe - --y4m --preset slower --pools "+,-" --keyint 240 --crf 18 --vbv-maxrate 5000 --vbv-bufsize 15000 -o DOTA2_CRF-18.hevc

ffmpeg.exe -i DOTA2.mov -pix_fmt yuv444p -f yuv4mpegpipe - | x265.exe - --y4m --profile main444-8 --preset slower --keyint 240 --crf 18 --vbv-maxrate 5000 --vbv-bufsize 15000 -o DOTA2_CRF-18-444.hevc

[huhn]

i have everything setup to record lossless on my gaming PC (x264rgb CRF=0 preset=ultrafast) i didn't try 4K but 1080p is no problem i may even be able to do 60-240 HZ.

only nvidia can record lossless 444 but not in RGB so it's already not great and not lossless.

DOTA 2 should be an OK test. i have no clue how old your sample is the game changed drastically i will capture a new one.

I don't know what can capture to >8-bit on-PC. An interesting comparison would be between 10-bit 4:2:0 and 8-bit 4:4:4, when those are available. Some devices, like Xbox One X, can even render and output 12-bit over HDMI.

rendering 10 bit is already kinda complicated and games rarely do that. outputting 12 bit RGB you can do that with windows 7 and a i don't know 10 year old consumer grade GPU and it was and is done with madVR.
12 bit doesn't exist only 16 bit and the driver has to dither this to 12 bit you can try this with MPDN. games don't use stuff like this there are always exception and microsoft doesn't stop adding new feature so this could be easily "dated" information.

i'm pretty sure ni no kuni 2 can do that when rendering HDR and 10 bit but that's not going to be fun to capture if it is even possible at all.

edit: i have a 1080p60 38 sec lossless RGB clip on my system now it's ~5GB so i need a place to upload it. i can't do UHD at the moment AMD has a strange bug forcing me to run the game at 240 hz making recording 60 FPS a nightmare. the system and the disk should be fast enough for this if i can get it to 60 HZ.

[benwaggoner]

edit: i have a 1080p60 38 sec lossless RGB clip on my system now it's ~5GB so i need a place to upload it. i can't do UHD at the moment AMD has a strange bug forcing me to run the game at 240 hz making recording 60 FPS a nightmare. the system and the disk should be fast enough for this if i can get it to 60 HZ.
Doing 7zip LZMA2 Ultra can shrink game capture stuff down amazingly. You can also use ffmpeg or whatever to decimate to 60 fps first.

[huhn]

i just need a place to upload.

the problem is i can't capture 240 hz UHD and i can't run the game at 240 hz so decimation is risky and frame times would be inaccurate so i have to limited the game to 60 manual which would be much more fun by just using 60 HZ and vsync.

edit: this clip will not show the benefts from 4:4:4 on a visual level the game benefits not enough from it.

D:\encode\x>D:\encode\x\x264_64_8.exe --input-range PC --colorprim BT709 --range PC --preset veryslow --crf=0 --output-csp RGB --output "d:\encode\x\test.mkv" "E:\2020-09-03 01-08-49.mkv"
[matroska,webm @ 00000000001e5c40] Stream #0: not enough frames to estimate rate; consider increasing probesize
[matroska,webm @ 00000000001e5c40] decoding for stream 0 failed
lavf [info]: 1920x1080p 0:1 @ 0/0 fps (vfr)
resize [warning]: converting from gbrp to rgb24
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x264 [info]: profile High 4:4:4 Predictive, level 5.1, 4:4:4, 8-bit

that should be irrelevant because that lossless?
i will upload the file to google drives.

edit2: https://drive.google.com/file/d/1JCWW0d-v93U4TQGC_QwtOqWgSzvdsd9B/view

[huhn]

i used your shared encode settings and here are the files:
https://drive.google.com/file/d/1Z20b4HkknY_pCzMIZfa3H7ly0Litvjjx/view?usp=sharing
https://drive.google.com/file/d/1Tfyr0eNnmpAs_RuNDYnJAaUndLdXnhTh/view?usp=sharing


on a first look 444 is doing fantastic better then i would have expected from the source.

the encode was done from the original ultrafast compressed lossless x264 source not the shared one the FFMPEG or x265 couldn't deal with it.

the 4:2:0 encode had --pool the 4:4:4 not i just copied the settings form you didn't notice before i even started encoding.

4:2:0: encoded 2322 frames in 620.47s (3.74 fps), 5152.00 kb/s, Avg QP:35.55
4:4:4: encoded 2322 frames in 837.02s (2.77 fps), 5152.05 kb/s, Avg QP:35.47

bilinear chroma 4:2:0: https://abload.de/img/420dota2k1ktm.png
4:4:4: https://abload.de/img/444dota2lkkf7.png
source: https://abload.de/img/lldota282j4c.png

[benwaggoner]

i used your shared encode settings and here are the files:
https://drive.google.com/file/d/1Z20b4HkknY_pCzMIZfa3H7ly0Litvjjx/view?usp=sharing
https://drive.google.com/file/d/1Tfyr0eNnmpAs_RuNDYnJAaUndLdXnhTh/view?usp=sharing


on a first look 444 is doing fantastic better then i would have expected from the source.

the encode was done from the original ultrafast compressed lossless x264 source not the shared one the FFMPEG or x265 couldn't deal with it.

the 4:2:0 encode had --pool the 4:4:4 not i just copied the settings form you didn't notice before i even started encoding.

4:2:0: encoded 2322 frames in 620.47s (3.74 fps), 5152.00 kb/s, Avg QP:35.55
4:4:4: encoded 2322 frames in 837.02s (2.77 fps), 5152.05 kb/s, Avg QP:35.47

bilinear chroma 4:2:0: https://abload.de/img/420dota2k1ktm.png
4:4:4: https://abload.de/img/444dota2lkkf7.png
source: https://abload.de/img/lldota282j4c.png
The differences between the 444 and 420 versions are a lot smaller than each has from the source.

I wonder if content that could benefit from 444 needs to have the chroma-qp deltas reduced to increase bits spent on chroma. The automatic +6 qp in 444 could be quite material.

Of course, then the comparison is about whether there are cases where it's worth it to reduce luma bitrate to increase chroma bitrate.

[huhn]

maybe we are not looking at the same parts but this bird for example is much better with 444. his icon is not even close.
the player name looks massively better with 444 too.

even through i'm using a really "bad" chroma scaler here. the difference could be smaller but bilinear is sadly a realistic chroma scaler.

another issue is luma is very different which i don't understand why shouldn't it similar with the same QP as the target? but they trade different artifacts the green orb firing little guy has chroma ringing on this right arm in 444 but has something even more wired thingy on his back in 420.

i can try and make a lossless recording of FTL it's 720p and pixel art with perfect use of chroma. another game would be starsector (native 1080p) it uses "1" pixel green lines with zero AA.

but there are other issues her there is clearly a color space mismatch my guess would be the ffmpeg conversation. the image shouldn't look this bad with CFR 18 too.

the results are far better then i could have hoped for. luma didn't obviously get worst and chroma is clearly better

[huhn]

does someone know how to use VT 709 for color conversation instead of bt 601 in ffmpeg?

these setting where used.
ffmpeg.exe -i DOTA2.mov -pix_fmt yuv420p -f yuv4mpegpipe - | x265.exe - --y4m --preset slower --pools "+,-" --keyint 240 --crf 18 --vbv-maxrate 5000 --vbv-bufsize 15000 -o DOTA2_CRF-18.hevc

ffmpeg.exe -i DOTA2.mov -pix_fmt yuv444p -f yuv4mpegpipe - | x265.exe - --y4m --profile main444-8 --preset slower --keyint 240 --crf 18 --vbv-maxrate 5000 --vbv-bufsize 15000 -o DOTA2_CRF-18-444.hevc

[takla]

For bit depth in video games, I recommend to check out this nifty tool (https://discourse.differentk.fyi/). It is developed by Kaldaien. He made it his mission to streamline HDR in PC games, which currently follows basically no standards and is mostly brocken (at least brocken when compared to what can be achieved with this tool properly set up). It can show you what native bit depth and color space a game uses and overwrite it, when supported, with scRGB HDR 16-bit.

And for 4:4:4 chroma talk, personally, I watch a lot of twitch tv streams which I pipe to MPV. It is very obvious to me to tell the difference of 540p to 2160p chroma on my 55" 4k Samsung TV. I use the excellent KrigBilateral glsl shader to get chroma to almost native res. Good examples I recommend to test for chroma are UI heavy games like World of Warcraft and Path of Exile.

And thankfully this samsung tv also supports freesync, which removes all judder in low fps videos once mpv is in exclusive fullscreen (default key is f) this is achieved by the gpu driver telling the display to double its refresh rate when outside freesync range (24>48fps with 48to60hz range)