MPC-HC playback of VirtualDub-processed video encoded by libx264 appears washed out

[theultramage]

I've had a long-standing problem with my video toolchain. Whenever I use libx264 (ffmpeg or vfw codec) to encode a video processed by VirtualDub 1.10.4, and then play it back using MPC-HC, it appears washed out, like there's a filter over it. Using the mspaint dropper tool, I see that pure black (0,0,0) becomes (16,16,16).

This has plagued me for years, and my early workaround was to patch Vdub and tell x264vtw to encode as bgr24. Later I ditched vdub entirely and edited my videos straight through the ffmpeg commandline. Today I got hit by this again, and I really like the Vdub gui, so I thought I'd finally ask about it. Note that only MPC-HC seems to do this, and only on h264 yuv444p or yuv420p video that passed through vdub. Vdub and VLC show it fine, and a direct ffmpeg encode of the raw source shows fine as well. So I don't think the problem is in the actual pixels of the video.

There must be something in the video header or structure that's being mis-interpreted. It even happens if the vdub output is a raw DIB file... just changing the output's Color Depth from bgr24 to yv24_444p is enough to trigger this. MPC-HC's Mediainfo tab says in one case "Format: RGB, Codec ID: 0x00000000, Codec ID/Info: Basic Windows bitmap format...", and in the other case just "Format: YV24, Codec ID: YV24".

When looking at the libx264 encodes, the direct one says "Frame rate mode: Variable", while the vdub processed one says "Frame rate mode: Constant, Frame rate: 30.000 FPS, Bits/(Pixel*Frame): 0.064". No other noted differences.

I'm honestly at a loss as to what's going on here. I speculate that vdub's output is making libx264 generate yuv video files that make MPC-HC incorrectly apply a color range transform.

[Klaus1189]

full vs limited RGB?
0-255 vs 16-235?

[theultramage]

Yes, something like that. When I was investigating this long ago, I read something about low-quality videos using the compressed range while HD used the full one. Or something like that. I did not manage to find details on this, nor information on how such a thing is expressed in the files, if at all. Maybe it's based entirely on heuristics.

[clsid]

Movie/TV content is almost always in limited range (16-235 aka TV range). Renderers also usually assume this if the range isn't specified in the file.

If you make a recording of your desktop then it is in full range (aka PC range).

Add "--range pc" to the encoding parameters of x264 to signal that your content is full range.

[theultramage]

Oh hey, thanks for pointing me in what's hopefully the right direction. So I investigated a bit, and learned new things...
- There's "ffmpeg.exe -h full", normal listing doesn't show all the options.
- Vdub's YUV presets in the Color Depth menu are all Limited color range type, as revealed in the detailed listing when clicking on 'Other'.
- The preferred ffmpeg option seems to be -color_range pc (supposedly yuvj is deprecated?), but it doesn't work.

Now I could use some more help with explaining this behavior.
> ffmpeg -i raw-bgr24.avi -c:v libx264 -color_range pc -loglevel debug out.mp4

Input #0, avi - Stream #0:0: Video: rawvideo, bgr24, 1920x1080, 30 fps, 30 tbr, 30 tbn, 30 tbc
[format @ 000000000041abc0] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|...'
[auto_scaler_0 @ 000000000041ba40] picking yuv444p out of 11 ref:bgr24 alpha:0
Output #0, mp4 - Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p(pc), 1920x1080, q=-1--1, 30 fps, 15360 tbn, 30 tbcThis video still looks wrong. Its metadata does say "Color range: Full".

To actually get it to work, I need to specify "-pix_fmt yuvj444p" (j = JPEG). The "-color_range pc" parameter is not needed as it gets set automatically. The two files are indistinguishable in the MPC-HC UI (codec still says yuv444p and all metadata match), the only difference is in file size.

> ffmpeg -i raw-bgr24.avi -c:v libx264 -pix_fmt yuvj444p -loglevel debug out.mp4

Input #0, avi - Stream #0:0: Video: rawvideo, bgr24, 1920x1080, 30 fps, 30 tbr, 30 tbn, 30 tbc
[format @ 000000000037aa00] Setting 'pix_fmts' to value 'yuvj444p'
[swscaler @ 00000000003c8fc0] deprecated pixel format used, make sure you did set range correctly
Output #0, mp4 - Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p(pc), 1920x1080, q=-1--1, 30 fps, 15360 tbn, 30 tbc

Deprecated, huh. But it doesn't want to work any other way. Is MPC-HC parsing it wrong then? Would be kinda glad if it were only a player issue.
Back then it was such a problem that I started encoding my archive with x264 in rgb mode, just to avoid this.

EDIT: Phew, I got sidetracked. As I said earlier, the same file plays in VLC just fine. Also I get the issue with yuv444p encodes, but not with yuv420p encodes. I don't think it's an encoder issue, and even if I'm getting the parameters wrong, all that should cause is reduced quality, not a color space mismatch, yes? All of these parameters seem to affect the output; I'm not sure there's options to override the input format when it's not a raw stream, it seems to be autodetected.