x264 Settings for Video Games footage

[andiandi]

Hi,

I've got several (realistic) video games footages to encode (PS2/GC/Xbox Era), I will not keep the lossless source files and I would like to have some settings recommandations. I think video game footages should be in a very good quality (to better appreciate the technical showcase), but I don't want a too high bitrate.
I assume using more ReFrames would help, but for the rest I don't really know. Maybe aq and psy since image is generally clean and fidelity is important for these sources ?


Also, a side question : Would it be possible to eventually implement some x265 settings to x264 ? The kind of settings which only affects the encoding but not the decoding (speed, cpu load) nor compatibility - like subme and me.

Thanks

[benwaggoner]

Does it need to be AVC? HEVC has some really good tools (SAO, transform skip, lossless CU, much more flexibility in block/TU size) that can get a HEVC <<50% the bitrate of a AVC of a similar quality with this kind of game footage. That era of games had TONS of aliasing, and those sharp edges are very hard on frequency transforms, which HEVC can bypass.

--preset animation may be a good start with x264, and you could tweak from there. You shouldn't have any noise (unless the game has a film grain effect; turn off if possible!), and frame-to-frame matches can be really good, so animation is the closest analog in the presets.

How are you capturing? If via a HDMI capture system, you should set the console to output the native resolution in progressive scan and capture that, without any upsampling. GC was always 480, Xbox original had some 720p (Halo: CE, IIRC), and I never played on a PS2. Getting the native pixels will make encoding easier and faster.

[andiandi]

Does it need to be AVC? HEVC has some really good tools (SAO, transform skip, lossless CU, much more flexibility in block/TU size) that can get a HEVC <<50% the bitrate of a AVC of a similar quality with this kind of game footage. That era of games had TONS of aliasing, and those sharp edges are very hard on frequency transforms, which HEVC can bypass.

--preset animation may be a good start with x264, and you could tweak from there. You shouldn't have any noise (unless the game has a film grain effect; turn off if possible!), and frame-to-frame matches can be really good, so animation is the closest analog in the presets.

How are you capturing? If via a HDMI capture system, you should set the console to output the native resolution in progressive scan and capture that, without any upsampling. GC was always 480, Xbox original had some 720p (Halo: CE, IIRC), and I never played on a PS2. Getting the native pixels will make encoding easier and faster.

Yes AVC preferably, I think it does the job pretty well in general (if well tuned/tweaked) and I tend to prefer my videos to remain compatible with older hardwares (even if it implies more bitrate and/or limit resolution). Also HEVC is high CPU consuming.

OK, I precisely planned something quite close to animation tune, at least :

- More Refs
- aq-strength 0.6 (Edges tending to be sharp + UI and I think lowering aq-strengh can be fine for SD since edges have less pixels anyway)
- psy-rd 0.5 (Not quite sure there, but I believe psy-rd distortions and artifacts are more noticeable for SD in general and I assume it could be more bothering for video game footages)
- deblock -1,-1 instead of 1:1 or 0:0

I chose to capture with emulators since they're very very accurate for these consoles nowadays and also because progressive scan require component cables whose output is not necessarily very good quality while forcing 4:2:2 colorspace as well. Emulators, on an other hand allows RGB capture (so 4:2:0 conversion from RGB will be better than from 4:2:2).
There're other things that consoles don't always do very well like scaling to correct aspect-ratio (sometimes too blurry).

And yeah I will encode in SD (especially as hardware upscaling is improving further and further) and since internal resolutions of these consoles are often unusual (640x448, 512x448 etc) I wondered if it was better to change the display aspect-ratio or the pixel aspect-ratio (but not sure that Lanczos is the most appropriate in this case).

[benwaggoner]

Yes AVC preferably, I think it does the job pretty well in general (if well tuned/tweaked) and I tend to prefer my videos to remain compatible with older hardwares (even if it implies more bitrate and/or limit resolution). Also HEVC is high CPU consuming.
Makes sense for your goals. Although for this sort of sub-SD encoding, HEVC should still be really fast.

OK, I precisely planned something quite close to animation tune, at least :

- More Refs
- aq-strength 0.6 (Edges tending to be sharp + UI and I think lowering aq-strengh can be fine for SD since edges have less pixels anyway)
- psy-rd 0.5 (Not quite sure there, but I believe psy-rd distortions and artifacts are more noticeable for SD in general and I assume it could be more bothering for video game footages)
- deblock -1,-1 instead of 1:1 or 0:0
That generally makes sense. More B-frames could also really help. I'm not sure about deblock. Generally stronger values are better for synthetic content and lower for natural image content.

I chose to capture with emulators since they're very very accurate for these consoles nowadays and also because progressive scan require component cables whose output is not necessarily very good quality while forcing 4:2:2 colorspace as well. Emulators, on an other hand allows RGB capture (so 4:2:0 conversion from RGB will be better than from 4:2:2).
Getting lossless frame captures of the rendered pixels is ideal. For SD console stuff, native RGB encoding could make sense, although that would be a big HW decoder compatibility hit.

There're other things that consoles don't always do very well like scaling to correct aspect-ratio (sometimes too blurry).

And yeah I will encode in SD (especially as hardware upscaling is improving further and further) and since internal resolutions of these consoles are often unusual (640x448, 512x448 etc) I wondered if it was better to change the display aspect-ratio or the pixel aspect-ratio (but not sure that Lanczos is the most appropriate in this case).
I think you're best off encoding the exact rendered pixels, and setting SAR as appropriate to get the proper aspect ratio on playback. That'll optimize clarity, compression efficiency, and encoding speed.

[andiandi]

Makes sense for your goals. Although for this sort of sub-SD encoding, HEVC should still be really fast.

I also think that for HEVC, it might still be possible to limit to settings that doesn't use more CPU than AVC while having a better result than with AVC (unless it's inherently more cpu-intensive than AVC - I didn't put much interest to HEVC until now).
Regarding compatibility, does it necessarily requires a modern hardware or can we use it on older hardwares as long as they're not too weak, I ask because I remember that sometimes on older phones for example, AVC was decoded smoothly but with some visual flaws (Btw I always keep an old smartphone for benchmark purposes just in case).


That generally makes sense. More B-frames could also really help. I'm not sure about deblock. Generally stronger values are better for synthetic content and lower for natural image content.

OK, for deblock I'll compare then (I guess it can change depending on the game)


I think you're best off encoding the exact rendered pixels, and setting SAR as appropriate to get the proper aspect ratio on playback. That'll optimize clarity, compression efficiency, and encoding speed.


Alright, that's good.

[benwaggoner]

I also think that for HEVC, it might still be possible to limit to settings that doesn't use more CPU than AVC while having a better result than with AVC (unless it's inherently more cpu-intensive than AVC - I didn't put much interest to HEVC until now).
x265 --preset medium is faster than x264 --preset veryslow, so quality @ perf @ bitrate should certainly be better with HEVC for this kind of pixel-accurate synthetic content.

Either x264 and x265 should be faster than real time with these low resolutions on a CPU that supports AVX2.

Regarding compatibility, does it necessarily requires a modern hardware or can we use it on older hardwares as long as they're not too weak, I ask because I remember that sometimes on older phones for example, AVC was decoded smoothly but with some visual flaws (Btw I always keep an old smartphone for benchmark purposes just in case).
Pretty much every discreet or integrated GPU has supported HEVC HW decode for 8-bit standard def for many years, and pretty much all mobile devices do (thanks to rapid replacement cycles). Any 4K HDR TV supports HEVC, as do pretty much all Smart TVs 2017+. You can embed it a web page for Edge, Safari, and Chrome now (but not Firefox). And VLC and other media player apps include software decoders even if hardware isn't there.

OK, for deblock I'll compare then (I guess it can change depending on the game)
Yeah. And while I generally recommend using SAO, you should definitely test with it off, which would help maintain the crispness of individual pixels. SAO mainly reduces ringing at higher QP/CRF, but I expect you'd be able to do perceptually lossless at <200 Kbps with a well-tuned x265 encode. I've been able to do 1080p24 at <150 Kbps with HEVC with simple content and appropriate tuning.

If you can share a couple of minutes of your uncompressed captures, we can offer some more specific tuning advice. Encoding to lossless H.264 or HEVC would be a compact way to transport them, but with these low resolutions, even a .y4m in 7z LZMA2 Ultra should be quite compact.

[andiandi]

x265 --preset medium is faster than x264 --preset veryslow, so quality @ perf @ bitrate should certainly be better with HEVC for this kind of pixel-accurate synthetic content.

Either x264 and x265 should be faster than real time with these low resolutions on a CPU that supports AVX2.

Faster at encoding you mean ? Because to me, encoding speed isn't an issue at all - even if it takes hours.


Pretty much every discreet or integrated GPU has supported HEVC HW decode for 8-bit standard def for many years, and pretty much all mobile devices do (thanks to rapid replacement cycles). Any 4K HDR TV supports HEVC, as do pretty much all Smart TVs 2017+. You can embed it a web page for Edge, Safari, and Chrome now (but not Firefox). And VLC and other media player apps include software decoders even if hardware isn't there.


It's compatible with less recent hardwares than I thought, even though I prefer to stick to AVC as much as possible. But it's a good point for HEVC (even though I consider using it mostly for videos at 1080p and higher). Also hardwares are replaced quite quickly.


Yeah. And while I generally recommend using SAO, you should definitely test with it off, which would help maintain the crispness of individual pixels. SAO mainly reduces ringing at higher QP/CRF, but I expect you'd be able to do perceptually lossless at <200 Kbps with a well-tuned x265 encode. I've been able to do 1080p24 at <150 Kbps with HEVC with simple content and appropriate tuning.

If you can share a couple of minutes of your uncompressed captures, we can offer some more specific tuning advice. Encoding to lossless H.264 or HEVC would be a compact way to transport them, but with these low resolutions, even a .y4m in 7z LZMA2 Ultra should be quite compact.

OK, here are some captures :


https://mega.nz/file/9rADlRZC#0HhRBJML0JHiyp4ESSBhZwyNvH-UkrA_KRUbRwX2ReI

https://mega.nz/file/BzRwGBJD#VIulP9irawGyf2J_aNVSBk7kp_U4bz7anxyium_weus

https://mega.nz/file/IzRzABqA#p3zDBxpuLL03oyPRmkbSN1NLPhQSKE1LEmrz9xDaqgM

[benwaggoner]

Faster at encoding you mean ? Because to me, encoding speed isn't an issue at all - even if it takes hours.
At these low resolutions, x265 --preset placebo --tskip --cu-lossless should still be only a few multiples of realtime.

It's compatible with less recent hardwares than I thought, even though I prefer to stick to AVC as much as possible. But it's a good point for HEVC (even though I consider using it mostly for videos at 1080p and higher). Also hardwares are replaced quite quickly.
I think in 2023 it is safe to use HEVC by default unless you know you need compatibility with H.264-only clients, like Firefox or 7+ year old PCs where users can't install VLC.

OK, here are some captures :

https://mega.nz/file/9rADlRZC#0HhRBJML0JHiyp4ESSBhZwyNvH-UkrA_KRUbRwX2ReI

https://mega.nz/file/BzRwGBJD#VIulP9irawGyf2J_aNVSBk7kp_U4bz7anxyium_weus

https://mega.nz/file/IzRzABqA#p3zDBxpuLL03oyPRmkbSN1NLPhQSKE1LEmrz9xDaqgM
I will check them out!

[andiandi]

I think in 2023 it is safe to use HEVC by default unless you know you need compatibility with H.264-only clients, like Firefox or 7+ year old PCs where users can't install VLC.


That's a good thing, it'll probably depend of the best I can get with AVC without having an excessive bitrate, but now I'm much more interested by HEVC (quality is more important to me for this particular case).

I will check them out!

Thanks !

[benwaggoner]

That's a good thing, it'll probably depend of the best I can get with AVC without having an excessive bitrate, but now I'm much more interested by HEVC (quality is more important to me for this particular case).
Yeah, good to try both.

I ran a bunch of x265 tests over the weekend, and whew, those are challenging clips. A huge amount of big differences in adjoining pixels. I'd underestimated the difficulty badly.

The solution is presumably in combining recursion down to 4x4 TUs plus using assymetric motion partitions to get as small a transform as possible. Plus using rskip to skip frequency transform entirely as needed, and potentially enabling --cu-lossless if there are blocks where lossless encoding is actually more efficient.

Even with all that thrown at it, I'm still getting >4 Mbps at crf 23 for these sub-SD sources. This is Sol Levante level difficulty! A very chewy encoding challenge. Thanks for sharing!

I've found setting --aspect 4:3 in ffmpeg before piping into the encoder will calculate the correct SAR irrespective of the actual rendered pixels.

[andiandi]

Even with all that thrown at it, I'm still getting >4 Mbps at crf 23 for these sub-SD sources. This is Sol Levante level difficulty! A very chewy encoding challenge. Thanks for sharing!

Yeah, from what I tested, it was barely watchable even at really high bitrates. I also tried 720p upscaling thinking it could be a better tradeoff after all but in CRF 22 with AVC it gives about 8000 kbps (even though it's 60 fps, it's still a lot)

[Emulgator]

SC3 PS2: I would guess that emulator add-generated a scanline-look which might not have been there in the original game output, making it harder to encode.
The other clips have their fair amount of resize flicker too. Not that much to save bitrate-wise I guess too, concurring.

[andiandi]

SC3 PS2: I would guess that emulator add-generated a scanline-look which might not have been there in the original game output, making it harder to encode.
The other clips have their fair amount of resize flicker too. Not that much to save bitrate-wise I guess too, concurring.

For SC3 it's supposed to be dithering, IIRC it could be fixed with the emulator but I don't know if it's an issue or if it's in the game. Regardless, these are really problematic sources, maybe it would've been better to use emulator enhancements (better antialiasing/texture filtering/mip-mapping etc.). But it'd have been less representative of the real experience.

[andiandi]

Also, there's something I forgot for upscaling regarding YUV 4:2:0 colors converted from RGB : they're better preserved in HD than in SD. That's a good argument in favor of upscaling but given the complexity of the sources, it's not easy.

[benwaggoner]

Also, there's something I forgot for upscaling regarding YUV 4:2:0 colors converted from RGB : they're better preserved in HD than in SD. That's a good argument in favor of upscaling but given the complexity of the sources, it's not easy.
It'd love to have chroma subsampling become a problem, but the luma is so challenging to even notice any issues. A native full range 444 RGB encode could potentially do a lot better, although pretty much only SW decoders could display it.

H.264 wasn't any better.

I suspect a well-tuned AV1 encode could be a lot better. AV1 Main has a lot of screen encoding tools built-in that could really help. The HEVC screen encoding extensions would probably help a lot too, but I don't know of any practical encoders for it. All the old screen recording codecs were optimized before GUIs were full of transparencies and anti-aliasing, so wouldn't be much help.

Maybe something meant for video games like Bink could do well?

[andiandi]

It'd love to have chroma subsampling become a problem, but the luma is so challenging to even notice any issues. A native full range 444 RGB encode could potentially do a lot better, although pretty much only SW decoders could display it.

H.264 wasn't any better.

I suspect a well-tuned AV1 encode could be a lot better. AV1 Main has a lot of screen encoding tools built-in that could really help. The HEVC screen encoding extensions would probably help a lot too, but I don't know of any practical encoders for it. All the old screen recording codecs were optimized before GUIs were full of transparencies and anti-aliasing, so wouldn't be much help.

Maybe something meant for video games like Bink could do well?

And would 4:2:2 be a good compromise ? (what about compatibility ?), consoles used progressive scan in 4:2:2 with component cables so it'd remain close to the real experience.

Yes, I thought that other codecs could have more suitable tools for these kind of sources. For AV1 I assume we'll need to rely more on software decoders. Btw, I also wonder if software decoders are necessarily slower than hardware decoders, even if they are more recent? (especially on mobile devices) ?

In any case, I think the best thing to do is to encode with multiple codecs since I don't keep the lossless sources (fortunately, most of my captures are short), I would say:
In x265 in SD with the settings you recommended, then in x264 upscaled (to mitigate perceived quality loss) with a CRF of about 21-22 (8-10 Mbps, which is still less than Blu-ray, and I believe this kind of source have to be in very high quality by definition), and potentially in another codec to achieve the best possible compression before deleting the sources. Since x264 is the less efficient, I should try maxing it out and tweaking/tricking it as much as possible (maybe adjusting qpmin/qpmax? More rc-lookahead for mbtree ? Lowering psy-rd to prevent it allocating too much bits in aliased areas?).

[benwaggoner]

And would 4:2:2 be a good compromise ? (what about compatibility ?), consoles used progressive scan in 4:2:2 with component cables so it'd remain close to the real experience.
Maybe? Since they could fall back to composite output as well, generally they shouldn't have had too much chroma density to avoid issues with that. That's something to test and see. Note that 422 decode is less common, so there would be a compatibility hit.

Note VMAF entirely ignores chroma, so isn't of any use for that kind of testing,.

Yes, I thought that other codecs could have more suitable tools for these kind of sources. For AV1 I assume we'll need to rely more on software decoders. Btw, I also wonder if software decoders are necessarily slower than hardware decoders, even if they are more recent? (especially on mobile devices) ?
Yeah, a good share of new devices have AV1 HW decode, but the installed base is <<50%. Software decoding certainly can work if available, and at these low resolutions wouldn't require much CPU to decode, even on quite low-end devices.


In any case, I think the best thing to do is to encode with multiple codecs since I don't keep the lossless sources (fortunately, most of my captures are short), I would say:
In x265 in SD with the settings you recommended, then in x264 upscaled (to mitigate perceived quality loss) with a CRF of about 21-22 (8-10 Mbps, which is still less than Blu-ray, and I believe this kind of source have to be in very high quality by definition), and potentially in another codec to achieve the best possible compression before deleting the sources. Since x264 is the less efficient, I should try maxing it out and tweaking/tricking it as much as possible (maybe adjusting qpmin/qpmax? More rc-lookahead for mbtree ? Lowering psy-rd to prevent it allocating too much bits in aliased areas?).
Honestly, your lossless captures are't THAT much bigger than a transparent lossy encode might be. It might be cheaper to buy some 16 TB backup drives than paying for all the electricity to transcode.

A lossless HEVC RGB 444 might be somewhat smaller thanks to intraframe and interframe prediction and better entropy coding. I'd be hesitant to delete any sources without having nailed a transparent quality archival format.

[andiandi]

Maybe? Since they could fall back to composite output as well, generally they shouldn't have had too much chroma density to avoid issues with that. That's something to test and see. Note that 422 decode is less common, so there would be a compatibility hit

Note VMAF entirely ignores chroma, so isn't of any use for that kind of testing,.

OK


Honestly, your lossless captures are't THAT much bigger than a transparent lossy encode might be. It might be cheaper to buy some 16 TB backup drives than paying for all the electricity to transcode.

A lossless HEVC RGB 444 might be somewhat smaller thanks to intraframe and interframe prediction and better entropy coding. I'd be hesitant to delete any sources without having nailed a transparent quality archival format.


Yeah, I think I'm going to keep the Lossless sources after all. I did several other tests, and it seems almost impossible to do better with these sources (even with x265) and I guess there're no filters that can help a bit either. I think the real solution will probably come from newer codecs and tools (unless current and older codecs keep getting improved significantly, even through forks).

[benwaggoner]

Yeah, I think I'm going to keep the Lossless sources after all. I did several other tests, and it seems almost impossible to do better with these sources (even with x265) and I guess there're no filters that can help a bit either. I think the real solution will probably come from newer codecs and tools (unless current and older codecs keep getting improved significantly, even through forks).
I suspect that x265 doing RGB 8-bit lossless could be quite a bit smaller. Lossless mode skips frequency transform entirely, so should be better suited, and there is plenty of intraframe and interframe redundancy to compress with

[andiandi]

I suspect that x265 doing RGB 8-bit lossless could be quite a bit smaller. Lossless mode skips frequency transform entirely, so should be better suited, and there is plenty of intraframe and interframe redundancy to compress with

For x265 lossless (for archiving), I tested with VirtualDub2 and I didn't know that presets worked in lossless (I thought they were ignored and that only "lossless" settings applied). Thus, filesize could be significantly reduced in veryslow. However, Vdub detect the source as RGB32, but is limited to RGB24 for x265 lossless (there isn't any color loss in what I compared but I prefer to be sure).

As for lossy encodings (for watching) I think I'll just stick to x264 with a CRF around 16-18, as my sources are short overall (the longest ones are more or less 5 min) and with the most appropriate settings.

[Emulgator]

You can set VD2 to decode to RGB24.
Video -> Decode Format -> Decompression format RGB24

[andiandi]

You can set VD2 to decode to RGB24.
Video -> Decode Format -> Decompression format RGB24

I don't know if it's important to know whether the source is actually in RGB32 or RGB24? (I guess not, since x264 and x265 shouldn't take that into account). What's a bit confusing too, is that some Lossless codecs with Vdub2 only support RGB32 even when you set the decode format to RGB24. I don't think it's very important, but just in case.

[benwaggoner]

I don't know if it's important to know whether the source is actually in RGB32 or RGB24? (I guess not, since x264 and x265 shouldn't take that into account). What's a bit confusing too, is that some Lossless codecs with Vdub2 only support RGB32 even when you set the decode format to RGB24. I don't think it's very important, but just in case.
RGB32 is just 8-bit with an alpha channel. It is effectively interchangeable with RGB24 if alpha isn’t used. If you have a choice, do 24.

[nevcairiel]

fwiw, 24 is terrible for optimizations, because its not a power of 2. 3 byte alignment doesn't play well with CPUs, so you might see it being slower. Worth testing at least.

[andiandi]

RGB32 is just 8-bit with an alpha channel. It is effectively interchangeable with RGB24 if alpha isn’t used. If you have a choice, do 24.


OK, apparently the alpha channel can be present but not really used. I'm not quite sure how to check that. (In any case, the DC and GC were in 24-bit color, and the PS2 and Xbox were in 32-bit, if that can be an indication.)

fwiw, 24 is terrible for optimizations, because its not a power of 2. 3 byte alignment doesn't play well with CPUs, so you might see it being slower. Worth testing at least.

OK, to be tested then.

[benwaggoner]

OK, apparently the alpha channel can be present but not really used. I'm not quite sure how to check that. (In any case, the DC and GC were in 24-bit color, and the PS2 and Xbox were in 32-bit, if that can be an indication.)

OK, to be tested then.
24-bit should never be slower than 32-bit. The gap is more than it materially faster due to byte alignment.

Also, definitely try encoding as RGB 444 8-bit full range; exactly the same as your sources. I think some of the encoding challenge comes from color space conversion rounding away the palletized nature of the sources, reducing the efficiency of lossless and transform skip modes.

[andiandi]

24-bit should never be slower than 32-bit. The gap is more than it materially faster due to byte alignment.

Also, definitely try encoding as RGB 444 8-bit full range; exactly the same as your sources. I think some of the encoding challenge comes from color space conversion rounding away the palletized nature of the sources, reducing the efficiency of lossless and transform skip modes.

For the color range according to MediaInfo, it's limited range with x265 Lossless RGB 8-bit (with Vdub2), but it's strange because normally RGB should be full range (?).

Another thing I'm not sure about regarding 4:2:0 lossy encodings (x264 and x265) is whether it's better to feed the encoder with the RGB source or first convert the source to 4:2:0 (I see a slight difference, but nothing significant).

[benwaggoner]

For the color range according to MediaInfo, it's limited range with x265 Lossless RGB 8-bit (with Vdub2), but it's strange because normally RGB should be full range (?).
I'm on the road without my workstation the rest of the week, so I can't check the files. I concur limited range would be weird, but wouldn't be stunningly weird for devices of these devices predating HDMI by years.

Another thing I'm not sure about regarding 4:2:0 lossy encodings (x264 and x265) is whether it's better to feed the encoder with the RGB source or first convert the source to 4:2:0 (I see a slight difference, but nothing significant).
x264 and x264 don't natively include RGB->YUV conversion or subsampling, although many builds have added those features. As long as an implementation does a good job, it doesn't really matter where it happens. My go to method for "trusted quality and good speed" is zscale in ffmpeg. Other things can be as good or as fast, but I don't know anything actually better.

[andiandi]

I'm on the road without my workstation the rest of the week, so I can't check the files. I concur limited range would be weird, but wouldn't be stunningly weird for devices of these devices predating HDMI by years.


Alright. I looked at the various RGB options with Vdub2 (Compression -> Pixel Format -> Other), and indeed there's only Full Range available for RGB, so, I assume that MediaInfo is mistaken.

x264 and x264 don't natively include RGB->YUV conversion or subsampling, although many builds have added those features. As long as an implementation does a good job, it doesn't really matter where it happens. My go to method for "trusted quality and good speed" is zscale in ffmpeg. Other things can be as good or as fast, but I don't know anything actually better.

OK

[benwaggoner]

Alright. I looked at the various RGB options with Vdub2 (Compression -> Pixel Format -> Other), and indeed there's only Full Range available for RGB, so, I assume that MediaInfo is mistaken.
Take a look in the VDub2 histogram in the Levels filter to know for sure. Whether or not VDub2 supports it doesn't matter; full/limited is determined by the range output by your emulator.

[andiandi]

Take a look in the VDub2 histogram in the Levels filter to know for sure. Whether or not VDub2 supports it doesn't matter; full/limited is determined by the range output by your emulator.

OK. I always see differences, except when I set it to 0-255 for both input and output (even with YUV sources...), maybe I'm missing something ?

[benwaggoner]

OK. I always see differences, except when I set it to 0-255 for both input and output (even with YUV sources...), maybe I'm missing something ?
Do you see differences if you set both to 16-235?

The basic test is that output video black areas should be as black (RGB=0) in both cases. If 16-235 at darkest is dark gray instead of black, the source is likely full range.

If you encode limited range as full range, darks, whites, and saturation will be muted. If you encode full range as limited range, shadow detail gets turned into flat black, and bright areas get extra white.

Mathematically, a mismatch is essentially turning contrast too high or too low.

[andiandi]

Do you see differences if you set both to 16-235?

The basic test is that output video black areas should be as black (RGB=0) in both cases. If 16-235 at darkest is dark gray instead of black, the source is likely full range.

If you encode limited range as full range, darks, whites, and saturation will be muted. If you encode full range as limited range, shadow detail gets turned into flat black, and bright areas get extra white.

Mathematically, a mismatch is essentially turning contrast too high or too low.


Yeah, it's only with both 0-255 that there's no difference, for the others, the contrast changes except for 0-255 (input) -> 16-235 (output), which looks more like gamma correction, for example :

https://mega.nz/file/cihWDR7B#bGnwXtkaA347m1vBF3tJsfgDa52kbhHlUR2EPRqAsGI

Normally, I should conclude that the source is full range, but it's the same with all videos I've tested with (even with YUV 4:2:0 videos...).

[benwaggoner]

Yeah, it's only with both 0-255 that there's no difference, for the others, the contrast changes except for 0-255 (input) -> 16-235 (output), which looks more like gamma correction, for example :

https://mega.nz/file/cihWDR7B#bGnwXtkaA347m1vBF3tJsfgDa52kbhHlUR2EPRqAsGI

Normally, I should conclude that the source is full range, but it's the same with all videos I've tested with (even with YUV 4:2:0 videos...).
If MediaInfo says that an elementary stream in a .mp4 is limited range, that is generally quite accurate.

[andiandi]

If MediaInfo says that an elementary stream in a .mp4 is limited range, that is generally quite accurate.

Good, with samples converted to YUV 4:4:4 (since Vdub2 allows choosing Limited range and Full range for YUV), there aren't all these differences, and it seems to match MediaInfo.

[benwaggoner]

Good, with samples converted to YUV 4:4:4 (since Vdub2 allows choosing Limited range and Full range for YUV), there aren't all these differences, and it seems to match MediaInfo.
What exactly is matching?

[andiandi]

What exactly is matching?

The converted YUV 4.4.4 samples (limited range and full range) matches with the source (a Dreamcast source captured in x264 Lossless RGB by the emulator, Limited Range according to MediaInfo). There is no difference in Full Range either, but since there isn't any in Limited Range too, I assume it's limited range for this tested source (as in MediaInfo).