FFDShow with H.264 10bit support

[patrick_]

I searched the web, but had a hard time finding a FFDShow build that supports H.264 10-bit. Finally I found one included in CCCP. I hate codec packs so I decided to replace the files of the default FFDShow Installer with the ones from CCCP. I've only tested it with a single 10-bit file using MPC-HC*, but it worked :D

Download FFDShow rev 3925 with H.264 10-bit support (BETA)
http://www.filesonic.com/file/1444561154/ffdshow_rev3925_20110711-10bit.exe

*if you use MPC-HC, don't forget to deactivate the internal filters.

[Hypernova]

Thanks. I was looking for that also. I don't mind CCCP in comparison to bigger codec pack like k-lite, but still this is nice.

[clsid]

Those builds are alpha quality and are not stable. They should be used with caution. For example 9-bit H.264 will just crash.

Btw, K-Lite has small variants as well. Its latest beta version even has Blu-ray playback capability.

[JanWillem32]

I know for a fact that quite a few people are working on this. I absolutely can't recommend usage of the 10-bit system yet with MPC-HC, until:
1.- A real ffdshow beta comes out that can output at least one of the recommended 10- or 16-bit formats for DirectShow to the player: http://msdn.microsoft.com/en-us/library/bb970578%28v=VS.85%29.aspx .
(Currently all output is rounded to 8-bit formats, so there's no gain by the 10-bit precision yet.)
2.- At least one of the internal renderers has been updated to become able to take in and mix the 10- or 16-bit formats.
3.- At least some quality and reliability testing has been done.

[Dark Shikari]

I know for a fact that quite a few people are working on this. I absolutely can't recommend usage of the 10-bit system yet with MPC-HC, until:
1.- A real ffdshow beta comes out that can output at least one of the recommended 10- or 16-bit formats for DirectShow to the player: http://msdn.microsoft.com/en-us/library/bb970578%28v=VS.85%29.aspx .
(Currently all output is rounded to 8-bit formats, so there's no gain by the 10-bit precision yet.)This is completely false. 10-bit display support isn't merely unnecessary; it's beyond useless.

[patrick_]

JanWillem32, using 10bits in compression doesn't have anything to do with 10bit output. If it worked that way, you would need a 10bit display as well.

clsid, I updated the installer primary to allow x264 users to test/compare 8bit and 10bit outputs.

[JanWillem32]

10-bit display support is something entirely different than 10-bit input to the mixer.
A 10-bit bt.601 or bt.709 encode has a limited ranges [64, 940] [64, 960], [64, 960] Y'CbCr encoding, usually with chroma sub-sampling.
A 10-bit display has a full range [0, 1023] RGB display matrix.

In the link I gave, there's a list with the recommended replacement formats for the regular 8-bit YV12, I420/IYUV, NV12 and AYUV formats that do support more than 8-bit to feed to the mixer.

There's a lot going on in between to get the input format mixed and rendered to the output display. Even if the output of the display is just 8-bit RGB, there's no doubt that quality during mixing and rendering suffers if the input Y'CbCr format is rounded from 10- to 8-bit before even the mixer and renderer can receive the image.

By the way, there's plenty of scientific basis to why the Digital Cinema Initiative set 12-bit as a minimum requirement for both the encoding format and display capability for licensing.
There's also plenty of reasons why the studio formats store XYZ color data in a 32-bit floating-point format.

[Dark Shikari]

10-bit display support is something entirely different than 10-bit input to the mixer.
A 10-bit bt.601 or bt.709 encode has a limited ranges [64, 940] [64, 960], [64, 960] Y'CbCr encoding, usually with chroma sub-sampling.
A 10-bit display has a full range [0, 1023] RGB display matrix.

In the link I gave, there's a list with the recommended replacement formats for the regular 8-bit YV12, I420/IYUV, NV12 and AYUV formats that do support more than 8-bit to feed to the mixer.

There's a lot going on in between to get the input format mixed and rendered to the output display. Even if the output of the display is just 8-bit RGB, there's no doubt that quality during mixing and rendering suffers if the input Y'CbCr format is rounded from 10- to 8-bit before even the mixer and renderer can receive the image.It isn't rounded, it's dithered. There's a slight, but critical difference.

The effect of your display on the effectiveness of 10-bit is negligible. A 6-bit $50 LCD benefits from 10-bit just as much as the world's most expensive IPS monitor because 10-bit is about internal codec precision, not output precision.

[JanWillem32]

How awful to use the dithering compromise right at the start of a rendering instance. I'm having enough trouble with performing convolutions and other filter passes on the inputs full of synthetic noise as it is.
I guess the dithering is less than a structured 128×128 dithering map lookup for every channel, too?
If it's about internal codec precision, please make the decoder 32-bit floating point or better on output, so I don't have to static cast every element of the mixer input to 32-bit floating-point anymore. I can handle pretty much any input quantization, only the double precision for the color management section can be a bit intense to process.
What comes out of the renderer takes at least 7 conversion passes from the decoder's output to the back buffer of the allocator-presenter. What goes on screen can hardly be called raw from what the decoder outputs. Depending on settings, things can look very bad. For instance, a low gamma setting of 2.0 to 2.2 is murder on darker scenes with most consumer-grade video.

[LoRd_MuldeR]

If it's about internal codec precision, please make the decoder 32-bit floating point or better on output, so I don't have to static cast every element of the mixer input to 32-bit floating-point anymore. I can handle pretty much any input quantization, only the double precision for the color management section can be a bit intense to process.

As far as I know, h.264 (and probably most video formats) internally use integer math only. So floating point output doesn't make much sense. You can as well do the conversion yourself, if you need FP math for your post-processing. Also: Even if the original input source only used 8-Bit precision (per color channel) and the final output is going to be 8-Bit again, using 10-bit (12-bit) internal codec precision improves compression efficiency. Whether the decoder outputs 8-Bit or 10-Bit (12-Bit) is up to the decoder's choice, i.e. we don't necessarily need "true" 10-bit (12-bit) output to benefit from "high bit-depth" h.264 video...

[Dark Shikari]

If it's about internal codec precision, please make the decoder 32-bit floating point or better on output, so I don't have to static cast every element of the mixer input to 32-bit floating-point anymore.
a) The H.264 spec only supports up to 14-bit.
b) Floating point math is incredibly slow relative to integer math. A decoder written in 32-bit float would probably not be able to decode 1080p on an overclocked 6-core Core i7.

[JanWillem32]

When looking at the specifications, I really don't see any reason to round any output to integer, and most certainly not 8-bit. The decoded structures are only guaranteed to be 8-bit on the 8-bit lossless profile. All lossy modes will generate in-between values, even on encoding integer input.
I've had absolutely no issues with floating-point performance, as long as you don't rely on the FPU to do the work. A nice example on how to use packed SSE (readable for non-programmers, too): http://software.intel.com/en-us/blogs/2010/12/20/visual-studio-2010-built-in-cpu-acceleration/ . It's also convenient to blend in GPU power when heavy floating-point operations are wanted, but the programming for that is very specific, and the GPU has only very little integer math power.

A bit of information why consumer formats are frustrating for studio workers:
You start off with a big, raw camera image format.
You take the camera color calibration scheme and project the input images to a nice studio format with the full XYZ color palette and plenty of quantization.
You edit in that studio format, towards the cinema format 2048×1080 or with less height on 1:2.40 movies. (The 4k profile of 4096×2160 is still very rare.)
You make the cinema screener: the XYZ color space remains intact in the encode, only when encoding the JPEG2000 video, colors are quantized to 12- or 14-bit and the gamma is set to 2.6. The encoding profile is lossless up to 250 Mbit/s. There's no visible change from the studio master to the encode at all. Features are generally some 300 GB in size.
On approval, the final version is encoded for distribution to cinemas with the same type of encoding.
When it's time to do the blu-ray and DVD release, things get nasty.
The image is clipped to 1920 wide for blu-ray, and usually the same amount for DVD, too.
The XYZ color space is converted to the HD, PAL and NTSC color spaces, losing about 2/3 of all possible colors in the space.
The input is limited on maximum lightness (a lowpass filter), because of limitations in the HDTV and SDTV standards.
The gamma is typically set to 2.4.
With DVD, the image is scaled down.
Chroma (2 color channels, relative to grayscale) is sub-sampled to half-resolution in height and width (4:2:0).
On the encoding step, the images are heavily dithered to mask the rounding to 8-bit+limited ranges.

There are plenty of "magic" filters in use by studios when doing the encodes, but you simply can't overcome the limitations set by the encoded format. Studio or cinema video and consumer video don't look nearly the same.
If there are no consumer products that can actually decode anything better than 8-bit, 4:2:0 Y'CbCr, studios hardly have a reason to use anything better. If the rare decoder found that can decode more than 8-bit, but rounds/dithers it again afterwards, studios don't have a reason either. They can take care of a much better set of filters for dithering to 8-bit themselves.
Considering the relative ages of the JPEG2000 and h.264 codecs, I do believe better results can be achieved with the 50 or 100 GB of space on a blu-ray.

On the other side: what I'm working on now. The mixer and renderer have to deal with images that have starvation in input precision, by the quantization level, by the image quality loss trough lossy encoding, by horizontal/vertical resolution and by the limitations in color space.
When writing filters for mixing and rendering stages, I've seen poor quality results with simple filters on even a lossless 8-bit RGB input from a BMP file. (The still image filter in Windows supports it.)
Quantization is one of the things that can be become better than what it is now. The recommended list of formats to transport 10- or 16-bit has been around for a while. Using those formats to for at least the 9-bit and better images is only logical.

[Dark Shikari]

When looking at the specifications, I really don't see any reason to round any output to integer, and most certainly not 8-bit. The decoded structures are only guaranteed to be 8-bit on the 8-bit lossless profile. All lossy modes will generate in-between values, even on encoding integer input.Incorrect. The specification requires bit-exact decoding or decoder/encoder desync will occur, resulting in artifacts.

I've had absolutely no issues with floating-point performance, as long as you don't rely on the FPU to do the work.SSE is part of the FPU. It shares the same execution units.

A nice example on how to use packed SSE (readable for non-programmers, too): http://software.intel.com/en-us/blogs/2010/12/20/visual-studio-2010-built-in-cpu-acceleration/ . It's also convenient to blend in GPU power when heavy floating-point operations are wanted, but the programming for that is very specific, and the GPU has only very little integer math power.I've written thousands of lines of assembly code. I think I know how SIMD works.

Floating point addition in SSE is typically 3/1 (latency/invthroughput). By comparison, integer addition is typically 1/0.5, and with 16-bit integers you get twice as many per register (and four times as many with 8-bit). In the end this means the typical throughput from integer math is 4-8 times higher than floating point.

This isn't even considering the fact that integer math allows all sorts of useful shortcuts, like shifting instead of multiplication, bitmasking, and other performance tricks which are often impossible with floating point math.

[JanWillem32]

Incorrect. The specification requires bit-exact decoding or decoder/encoder desync will occur, resulting in artifacts.Okay, so it's a "cleanup" cycle in decoding. Too bad, I had hoped for sine-wave-like transformations and convolution like with lossy audio that does benefit from floating-points.
SSE is part of the FPU. It shares the same execution units.I mostly meant relying on the classical 1-level approach, instead of 2 or more SIMD instructions at a time.This isn't even considering the fact that integer math allows all sorts of useful shortcuts, like shifting instead of multiplication, bitmasking, and other performance tricks which are often impossible with floating point math.Bitmasking and such is indeed tricky on floating-points. I've done some on GPU work, and I can say that the bit field in floating-point mode is too variable to pull it off correctly usually. Only sign and some exponent tricks work well bit-wise.
The fact that I need to rely more and more on doubles and some floats for various things, does decrease my usage of bitmasking and such. The SSE example shows how to get at least some decent performance on doubles and floats on a CPU. (And give at least some impression to what it is for the regular people here.)

[Dark Shikari]

Okay, so it's a "cleanup" cycle in decoding. Too bad, I had hoped for sine-wave-like transformations and convolution like with lossy audio that does benefit from floating-points.That's because audio codecs are, in terms of prediction, stuck in the 1980s: they still generally have no inter prediction. CELT and AAC Main (LTP) are the only audio codecs with inter prediction, and both do require nearly-bit-exact decoding as a result. IIRC, Long Term Prediction actually requires an implementation of 16-bit floats in order to work correctly.

[JanWillem32]

Interesting material. It seems that many lossless image formats also use a strict step rounding mechanism of the lossy internal structure, just before error correction to lossless.
I know the half float format very well. We currently use the D3DXFloat32To16Array function to write out to D3DFMT_A16B16G16R16F (can also carry non-ABGR data). It's not very fast, so I'd love to replace that one with something better. The structure it outputs is an unsigned __int16*, so the CPU can't really do anything useful with it, either. Once a D3DFMT_A16B16G16R16F texture is created, it goes straight into the GPU with a DMA transfer. Most GPU's nowadays don't have a half float calculation mode (anymore). It's just a supported format to save memory bandwidth. With every texture transfer, all vertices and subpixels are converted to 32-bit float for calculation. Other types are allowed in DirectX 10 for some calculation stages, DirectX 11 allows even doubles on the GPU. The performance is low with anything else but 32-bit float. That's one of the reasons why DXVA parts are separate on the silicon of a GPU.

Anyway, a bit more on topic. I very much hope that this advancement to leads a proper ffdshow beta soon. I'm more than willing to help writing and testing code for a video mixer part, so it's able to receive 10-bit or zero-padded 16-bit Y'CbCr formats. I'm looking at this from the perspective of the regular consumer and professional, too. Once a complete system has been proven to be able to maintain more measurable significant quality than what there was before, it becomes advertisable to the public and professionals in general.

[TheRyuu]

For example 9-bit H.264 will just crash.

That's not going to get fixed until tryouts gets unborked.

No one uses 9bit h264 so I don't really think it matters that much.

[clsid]

That's not going to get fixed until tryouts gets unborked.

No one uses 9bit h264 so I don't really think it matters that much.That part of the code is a huge mess and the people who wrote it are not around anymore. So any help is welcome. I have already done some major cleanup in Swscale, so that the difference with vanilla FFmpeg is now minimal.

As long as x264 supports 9-bit, such files will be made. If ffdshow can not be fixed to support it, then it should reject decoding such files. Crashing is unacceptable.

[aegisofrime]

I searched the web, but had a hard time finding a FFDShow build that supports H.264 10-bit. Finally I found one included in CCCP. I hate codec packs so I decided to replace the files of the default FFDShow Installer with the ones from CCCP. I've only tested it with a single 10-bit file using MPC-HC*, but it worked :D

Download FFDShow rev 3925 with H.264 10-bit support (BETA)
http://www.filesonic.com/file/1444561154/ffdshow_rev3925_20110711-10bit.exe

*if you use MPC-HC, don't forget to deactivate the internal filters.

Thanks so much for this, plays my 10-bit x264 encodes perfectly :)

BTW, is there a link or repository where I can keep up with future revisions?

[SamKook]

BTW, is there a link or repository where I can keep up with future revisions?

You can get beta for ffdshow anywhere, just do a search on google.

Or try here: http://www.afterdawn.com/software/audio_video/codecs/ffdshow.cfm#all_versions

Not sure if they all have 10-bit support enabled though, I don't have anything to test them with.

[ForceX]

Thanks so much for this, plays my 10-bit x264 encodes perfectly :)

BTW, is there a link or repository where I can keep up with future revisions?
Not for FFDshow separately, but you can get the latest CCCP packs from here http://www.cccp-project.net/beta/ which have the version with 10-bit support.

[aegisofrime]

Not for FFDshow separately, but you can get the latest CCCP packs from here http://www.cccp-project.net/beta/ which have the version with 10-bit support.

Hmmm I'm currently on K-Lite and would prefer to keep it that way. Nothing against CCCP, I have just been using K-Lite without a problem so I see no reason to uninstall it. I would prefer a FFDShow installer, that way it's more convenient than uninstalling and installing whole codec packs.

You can get beta for ffdshow anywhere, just do a search on google.

Or try here: http://www.afterdawn.com/software/audio_video/codecs/ffdshow.cfm#all_versions

Not sure if they all have 10-bit support enabled though, I don't have anything to test them with.

I tried a version of FFDShow from here:

http://www.codecs.com/download/FFDShow.htm

Rev. 3943. 10-bit didn't work with it. Seeing as 3943 is newer than the rev. 3925 that the topic creator provided seemed to indicate to me that 10-bit support might not be integrated into the usual builds.

[JEEB]

The 10bit playback support is not yet in mainline -tryouts. Lord is working hard with trying to get things to work on even some kind of a level (filters etc.), and we will be pushing the patches towards clsid when it reaches a level where not only playback works, but even other features could be usable to a level (basically the patches have been more or less alpha'ish level where Lord has been trying fixing various parts with various ways -- the way -tryouts works isn't always straightforward).

You can check overall looks of the patches from my github repo's blitzkrieg branch. I don't push every version of the patch that gets built in there, but if it looks like something that's going towards a more working setup, it usually gets pushed.

[patrick_]

There's a newer 10-bit build up on CCCP, but it's unstable (crashes MPC-HC on closing). I'll update the FFDShow installer when the next more-or-less stable (doesn't crash on the first video) version becomes available. There isn't much difference betweer r3925 and r3943 anyways.

[JEEB]

There's a newer 10-bit build up on CCCP, but it's unstable (crashes MPC-HC on closing). I'll update the FFDShow installer when the next more-or-less stable (doesn't crash on the first video) version becomes available. There isn't much difference betweer r3925 and r3943 anyways.
That's old already, and wasn't really related to 10bit at all. That was a -tryouts bug. I recommend the 2011-07-20_03 build, has filters mostly if not fully working.

edit: read: tested most filters quickly, except for dscaler and bitmap overlay with hi10p 4:2:0, only awarpsharp's output looked odd, but that might've as well been the settings :) Of course, since the amount of filters is humongous, this wasn't a partial rewrite of each and every filter, but an edit that basically made those filters get 8bit 4:2:0 :)

[TheElix]

I tried this file BBB444p10.mkv from: http://x264.nl/developers/Dark_Shikari/ in JanWillem32's build with 10-bit output option and Pixel Format on my PDP set to YCbCr 4:2:2. Does this stats mean that it's ouputting 10 bit? http://rghost.ru/15421901/image.png I wasn't able to get full 24 fps =\

[cyberbeing]

The BBB444p10.mkv is 10bit 4:4:4 as the filename implies. Why are you having FFDShow downsample the chroma to 8bit 4:2:0 NV12? Since the source is 4:4:4, you should output 8bit 4:4:4 RGB32 instead.

[clsid]

It does not output 10bit. The 10bit you see in the MPC stats related to the internal renderer, not the video input that MPC gets.

[TheElix]

The BBB444p10.mkv is 10bit 4:4:4 as the filename implies. Why are you having FFDShow downsample the chroma to 8bit 4:2:0 NV12? Since the source is 4:4:4, you should output 8bit 4:4:4 RGB32 instead.Hmm, good point. But I tried RGB32 and didn't find any difference with NV12 when comparing screenshots. And I looked really hard.

@clsid, I see. Thanks.

[JEEB]

Hmm, good point. But I tried RGB32 and didn't find any difference with NV12 when comparing screenshots. And I looked really hard.

@clsid, I see. Thanks.
Which build is that with (date-wise)? NV12|YV12 output should be quite different from RGB32.

[TheElix]

The link to that build is in the first post. Yeah, I thought so too, but I saw no difference. I made screenshots in D3DFS mode with grab screen software which works like Fraps. It usually works when you want to capture what you really see (which isn't the case with alt+i/F5)

[JEEB]

The link to that build is in the first post. Yeah, I thought so too, but I saw no difference. I made screenshots in D3DFS mode with grab screen software which works like Fraps. It usually works when you want to capture what you really see (which isn't the case with alt+i/F5)
Ugh, so a build from ~11 days ago >_>

Do try the 2011-07-22_01 installer from the CCCP's beta folder and check off everything else but the -tryouts build if you don't need anything else.

[TheElix]

Please, make it clear to me. Does these ffdshow tryout versions included in CCCP pack any different from tryout versions here (http://www.xvidvideo.ru/ffdshow-tryouts-project-x86-x64/)? I only downloaded the old version in the first post because someone said that regular tryouts don't have 10-bit support.

[cyberbeing]

Correct, the CCCP BETA includes additional FFDShow patches for high bit-depth h264 support which do not exist in FFDShow-Tryouts. Which is why you should uninstall FFDShow-Tryouts and install FFDShow from the latest CCCP BETA (http://www.cccp-project.net/beta/) when available, if you want up-to-date support for 10-bit playback. They will get committed to FFDShow-Tryouts when they are considered stable, finalized, and/or don't break things.

[ForceX]

Please, make it clear to me. Does these ffdshow tryout versions included in CCCP pack any different from tryout versions here (http://www.xvidvideo.ru/ffdshow-tryouts-project-x86-x64/)? I only downloaded the old version in the first post because someone said that regular tryouts don't have 10-bit support.

Yes, they are different. Only the ffdshow-tryout in CCCP supports 10-bit H.264. Although you can play 10-bit H.264 files with current 8-bit decoders (CoreAVC/DivX/non-CCCP ffdshow), the colors will be all messed up and there will be blocking/banding etc. You can get the latest version of CCCP with the special 10-bit decoding build from here: http://www.cccp-project.net/beta/

[madshi]

I absolutely can't recommend usage of the 10-bit system yet with MPC-HC, until:
1.- A real ffdshow beta comes out that can output at least one of the recommended 10- or 16-bit formats for DirectShow to the player: http://msdn.microsoft.com/en-us/library/bb970578%28v=VS.85%29.aspx .
(Currently all output is rounded to 8-bit formats, so there's no gain by the 10-bit precision yet.)
2.- At least one of the internal renderers has been updated to become able to take in and mix the 10- or 16-bit formats.
3.- At least some quality and reliability testing has been done.
FWIW, both the LAV Video and madVR internal decoders are already able to fully decode 9bit and 10bit h264 and pass it on in full quality to madVR. Works perfectly fine here with all the 9bit, 10bit and 4:4:4 h264 samples I have tried.

[JanWillem32]

Once you pass step 3, then it's fine, of course. I was mostly warning about the "still untested with this" part. I've seen too many jumping on this item too soon.
To point at my later posts; I don't know what's worse, dithering to 8-bit or rounding to 8-bit and something about getting getting studios finally interested in encoding better consumer-level video than 8-bit 4:2:0 stuff (as in: don't disappoint them).
In the mean time, progress is coming along fine on more than one side, I'm only a bit unlucky to be stuck with the EVR and VMR-9 mixers that apparently call the video card driver for support of every DirectShow format (which is pretty limited, for what I've seen).

[madshi]

I don't know what's worse, dithering to 8-bit or rounding to 8-bit
Rounding to 8bit is a lot worse, IMHO. If the original 10bit encoding actually contains smooth 10bit content (which is probably very rare, though), dithering it down to 8bit will preserve the smoothness and avoid banding. Rounding down to 8bit will introduce banding artifacts. Of course, it the original source was 8bit and was just encoded as 10bit, the difference between rounding and dithering will be much lower. I'd still prefer dithering, though. In any case, the combination of LAV Video + madVR, or madVR alone, won't either dither down nor round down the decoded 10bit data. Instead it's passed in full bitdepth to the processing chain. Dithering is then only done at the very last step of the madVR rendering chain.

I'm only a bit unlucky to be stuck with the EVR and VMR-9 mixers that apparently call the video card driver for support of every DirectShow format (which is pretty limited, for what I've seen).
Yeah, I know, trying to make this all work with EVR and VMR9 is probably a major pain in the ***. There's a reason I decided to stay away from anything EVR/VMR9 when starting to write madVR. Of course you could try writing your own mixers, but if you do that, which part of EVR/VMR9 are you still using at all? Not much, I guess...

[nevcairiel]

In any case, the combination of LAV Video + madVR, or madVR alone, won't either dither down nor round down the decoded 10bit data. Instead it's passed in full bitdepth to the processing chain. Dithering is then only done at the very last step of the madVR rendering chain.

And if using LAV Video with EVR or any other renderer that only supports 8-bit input, dithering is applied, its not rounded. :)

Speaking of the whole topic, is there a way to indicate 9-bit in a media type, like using P010 and somehow indicating that only 9 bit are valid? Every value will be left-shifted to fit 10-bit anyway, but would it be useful to indicate that only 9-bit are actual data?

[madshi]

Speaking of the whole topic, is there a way to indicate 9-bit in a media type, like using P010 and somehow indicating that only 9 bit are valid? Every value will be left-shifted to fit 10-bit anyway, but would it be useful to indicate that only 9-bit are actual data?
Yes, it might be useful. Not right now, but maybe in the future. With VIDEOINFOHEADER2 I believe you can't transport this information. But we added this information capability to the "Open Media Format":

http://forum.doom9.org/showthread.php?t=156051

[JanWillem32]

Yeah, I know, trying to make this all work with EVR and VMR9 is probably a major pain in the ***. There's a reason I decided to stay away from anything EVR/VMR9 when starting to write madVR. Of course you could try writing your own mixers, but if you do that, which part of EVR/VMR9 are you still using at all? Not much, I guess...The function-specific library's loaded for the 3 main custom renderers in MPC-HC are: mfplat.dll (Media Foundation enumeration), Quartz.dll (DirectShow parts), DXVA2.dll (to interface the virtual DXVA2 driver), avrt.dll (thread scheduler) and dwmapi.dll (desktop composition).
EVR Sync may use some more functions for its special sync clock, but otherwise, the 3 main custom renderers are normal DirectX 9 renderers (although the 3D part of it is pre-multiplied to become mostly 2D).
I can indeed live without evr.dll and the VMR-9 parts of Quartz.dll. Having to rely so heavily on semi-hardcoded mixing by the video card's drivers is annoying to say the least. I've written plenty of filters for the mixing parts already (deinterlacing, chroma resizing, color controls, color matrixing and such). It's just that I'm still very new at this. I've started programming for DirectX 10 less than 2 years ago (with a very controlled renderer environment and plenty of templates to use). The DirectX 9 downgrade isn't a big problem, but the DirectShow and Media Foundation parts still confuse me a lot. The fact that the code for handling the mixer parts is pretty much the same as the reference Microsoft code samples doesn't help much either. Next, I'm also scheduled to merge the 3 renderers into 2 (a light one and a heavy one)...
I must say, you're in a pretty lucky position in these aspects.

[madshi]

I can indeed live without evr.dll and the VMR-9 parts of Quartz.dll. Having to rely so heavily on semi-hardcoded mixing by the video card's drivers is annoying to say the least. I've written plenty of filters for the mixing parts already (deinterlacing, chroma resizing, color controls, color matrixing and such). It's just that I'm still very new at this. I've started programming for DirectX 10 less than 2 years ago (with a very controlled renderer environment and plenty of templates to use). The DirectX 9 downgrade isn't a big problem, but the DirectShow and Media Foundation parts still confuse me a lot. The fact that the code for handling the mixer parts is pretty much the same as the reference Microsoft code samples doesn't help much either. Next, I'm also scheduled to merge the 3 renderers into 2 (a light one and a heavy one)...
I must say, you're in a pretty lucky position in these aspects.
Yeah, not having to deal with any VMR/EVR stuff makes rendering related things easier for me. However, the DirectShow framework complexity is quite problematic for me, too.

[Fadeout]

Is there a version of these codecs that can work on MPC-HC 64bit?

[ForceX]

No. If you need a 64 bit decoder for H.264 Hi10P, you can use LAVVideo decoder.

[JEEB]

Is there a version of these codecs that can work on MPC-HC 64bit?
Should be buildable in all theory for win64 as well, but I will not be compiling this. Pushed the sources for the 2011-07-30 CCCP release FFDShow-tryouts to the github (https://github.com/jeeb/ffdshow-tryouts/commits/blitzkrieg). The widening of the colorspace list from 32bit to 64bit was deemed too big of a change by Lord for not to have possible problems in it, so that was reverted for this patchwork.

9bit H.264 and related shouldn't crash -tryouts with this patch, and most (if not all) filters should work.

[ForceX]

10bit H.264 support has been included in the main ffdshow-tryouts batch, so x64 builds are now available.

[madshi]

Does ffdshow-tryouts output it as 10bit, or is it forcefully dithered down to 8bit output?

[ForceX]

Dithered. It's just the patch from CCCP's version.

[cyberbeing]

The only difference should be swscale scaling. FFDShow-Tryouts uses Bilinear and CCCP FFDShow uses Bicubic.

FFDShow's swscale dithering is a bit unique though, and can result in a different color hue than MPlayer2 swscale, ffmpegsource2 ordered(?) dithering, and madVR dithering (all three which are very close).

[clsid]

The custom stuff in ffdshow's swscale is for threaded resizing. The colorspace handling stuff is pretty much vanilla FFmpeg code.

[cyberbeing]

I'll re-post my 10-bit dithering comparison here for anybody interested:
Raw YCbCr output (via madVR special switch) converted with VDub TV filter and enhanced, no dithering (http://img143.imageshack.us/img143/1010/0000ci.png)
MPlayer2, 10-bit output to OpenGL -vo gl:yuv=3:colorspace=2:lscale=0:cscale=0, no dithering (http://img153.imageshack.us/img153/9766/mplayer2gl.png)

MPlayer2 w/ Direct3D -vo direct3d output (10bit to 8bit dithered by swscale) (http://img593.imageshack.us/img593/6295/mplayer2d3d.png)
MPlayer2 w OpenGL -vo gl:yuv=3:colorspace=2:lscale=0:cscale=0 -vf format=yv12 output (10bit to 8bit dithered by swscale) (http://img684.imageshack.us/img684/5572/mplayer2glyv12.png)
madVR decoder /w madVR (10bit to 8bit dithered by madVR) (http://img220.imageshack.us/img220/7114/madvr.png)
FFMpegSource2 /w AvsPMod (10bit to 8bit with crappy ffmpeg/libav dithering) (http://img684.imageshack.us/img684/3894/ffms2.png)
FFDShow decoder w/ madVR (10bit to 8bit dithered by custom ffdshow swscale + madVR dithering) (http://img15.imageshack.us/img15/4485/ffdshows.png)

madVR, MPlayer2 D3D/OGL output, and FFMpegSource2 all seem relatively close in color tone. FFDShow is a bit of an odd-ball with its green tint and choice of dithering. You can kind of see how it got there with the non-dithered vo gl screenshot, but since it's unique that means it's probably a bug with how FFDShow interacts with swscale.

[sneaker_ger]

Are you sure it's not an issue of BT.601 vs BT.709 ?

[cyberbeing]

Yes, I'm sure. It was YV12 out (exception being madVR decoder) with MPlayer2, madVR, and AvsPMod set to BT.709, hence why they match for the most part.

FFDShow has a different color tint, appearing to just smooth out the brighter blocking in non-dithered MPlayer2 OpenGL output, unlike the others which appear to take into account the darker background behind the blocking. Very non-technical description, but something different is going on in FFDShow swscale dithering (or colorspace handling).

[clsid]

Possibly fixed in 3260.

[cyberbeing]

Unfortunately, FFDShow r3260 shows no change. :(

[Reimar]

I'll re-post my 10-bit dithering comparison here for anybody interested:
MPlayer2, 10-bit output to OpenGL -vo gl:yuv=3:colorspace=2:lscale=0:cscale=0, no dithering

Note that for MPlayer (and I assume MPlayer2 will be no different) whether or not that command-line results in dithering or not depends entirely on the graphics hardware/driver.
I have read some people claiming ATI cards will do dithering, but I have no idea (honestly I just can't see the differents on any displays I have), and it might only be true for professional cards with 10-bit framebuffer support.
As far as the OpenGL standard is concerned, GL_DITHER is enabled by default, so when supported by the hardware it should be used.

[cyberbeing]

I figured that was the case, this is an old 7800GTX 512 though, and I can see it results in no dither with 10bit OpenGL input (it has the same pattern as the RAW YCbCr screenshot from madVR). mplayer and mplayer2 output are identical.

As an additional oddity, with the mplayer2 w/ runtime CPU detection, I actually get better performance with swscale dithering than raw 10bit output. With mplayer athlon and p4 builds, it's the opposite, with 10bit output being slightly faster.

Either way, I'm somewhat curious why http://oss.netfarm.it/mplayer-win32.php says "Please use athlon builds for AMD cpu, even for AMD64 (amd is faster with 3dnow)." when the P4 build is up to 10% faster on my AMD X2 4800+ (939)? The mplayer2 runtime detection build is amazingly fast with swscale for some reason. Is there anyplace which builds win32 mplayer svn with runtime detection for comparison?

Short benchmark 446 frames:
mplayer2-20110729
CPUflags: MMX: 1 MMX2: 1 3DNow: 1 3DNowExt: 1 SSE: 1 SSE2: 1 SSSE3: 0

[swscaler @ 013caeb4]using unscaled yuv420p10le -> yuv420p special converter
VO: [gl] 1920x1038 => 1920x1038 Planar YV12
V: 18.4 0/ 0 28% 38% 0.0% 0 0
BENCHMARKs: VC: 5.169s VO: 7.041s A: 0.000s Sys: 0.287s = 12.497s
BENCHMARK%: VC: 41.3619% VO: 56.3415% A: 0.0000% Sys: 2.2966% = 100.0000%

VO: [gl] 1920x1038 => 1920x1038 Planar 420P 10-bit little-endian
V: 18.4 0/ 0 40% 29% 0.0% 0 0
BENCHMARKs: VC: 7.490s VO: 5.228s A: 0.000s Sys: 0.458s = 13.176s
BENCHMARK%: VC: 56.8458% VO: 39.6782% A: 0.0000% Sys: 3.4760% = 100.0000%

MPlayer-athlon-svn-33883
CPUflags: MMX: 1 MMX2: 1 3DNow: 1 3DNowExt: 1 SSE: 0 SSE2: 0 SSSE3: 0

[swscaler @ 01261df4]using unscaled yuv420p10
VO: [gl] 1920x1038 => 1920x1038 Planar YV12
BENCHMARKs: VC: 5.429s VO: 8.760s A: 0.000s Sys: 0.186s = 14.375s
BENCHMARK%: VC: 37.7670% VO: 60.9391% A: 0.0000% Sys: 1.2939% = 100.0000%

VO: [gl] 1920x1038 => 1920x1038 Planar 420P 10-bit little-endian
BENCHMARKs: VC: 8.405s VO: 5.228s A: 0.000s Sys: 0.416s = 14.049s
BENCHMARK%: VC: 59.8263% VO: 37.2126% A: 0.0000% Sys: 2.9611% = 100.0000%

MPlayer-p4-svn-33883
CPUflags: MMX: 1 MMX2: 1 3DNow: 0 3DNowExt: 0 SSE: 1 SSE2: 1 SSSE3: 0

[swscaler @ 01240154]using unscaled yuv420p10le -> yuv420p special converter
VO: [gl] 1920x1038 => 1920x1038 Planar YV12
BENCHMARKs: VC: 5.141s VO: 7.641s A: 0.000s Sys: 0.273s = 13.055s
BENCHMARK%: VC: 39.3795% VO: 58.5293% A: 0.0000% Sys: 2.0912% = 100.0000%

VO: [gl] 1920x1038 => 1920x1038 Planar 420P 10-bit little-endian
BENCHMARKs: VC: 7.687s VO: 4.817s A: 0.000s Sys: 0.460s = 12.964s
BENCHMARK%: VC: 59.2950% VO: 37.1567% A: 0.0000% Sys: 3.5483% = 100.0000%

[JEEB]

Just noting that after I tested LAV Video's and FFDShow-tryouts' output with DSS2 (both output YV12 to the filter), manual BT.709 ConvertToRGB32 conversion and picture output the md5 hashes of both files matched :)

So FFDShow-tryouts, even though having a modified libswscale and friends, seems to do exactly the same thing as an unpatched ffmpeg.

Will try to see if I can get a similar setup with LAV Video + libav, to see if their output differs dithering-wise.

[cyberbeing]

So now the question is which way is correct?

FFDShow-Tryouts | LAV-Video
OR
MPlayer | FFmpegSource2 | MadVR


http://mailman.videolan.org/pipermail/x264-devel/2011-July/008685.html

Hey,

I've just been sent a test pattern which was encoded with x264 to 10bit. The
original 8bit test pattern has these black and white YCbCr values:

8 bit black, Y: 16, Cb: 128, Cr: 128
8 bit white: Y: 235, Cb: 128, Cr: 128

I've now checked the raw decoding output of the 10bit video file encoded
with x264 (ffmpeg/libav decoder). I'm getting these values:

10 bit black, Y: 64, Cb: 514, Cr: 514
10 bit white: Y: 943, Cb: 514, Cr: 514

The 8bit -> 10bit conversion is clearly incorrect. I guess that the
conversion is done by using "/ 255 * 1023". However, the correct conversion
would be a simple shift << 2.

As a reference, the BT.709 specification says:

Black level, 8bit: 16, 10bit: 64
Achromatic, 8bit: 128, 10bit: 512
Nominal peak, 8bit: 235, 10bit: 940

The incorrect conversion means that peak white is encoded with 943/514/514
instead of the correct 940/512/512. Furthermore, using "/ 255 * 1023" means
that a bit of banding is introduced. Both problems would not occur with a
simple << 2 shift.

Best regards, madshi.

Actually now that I think about it, what we are likely seeing is once again the x264 8bit to 10bit levels bug which madshi confirmed with a test chart sample I created a few days ago. Which once again means that MadVR, MPlayer, and FFmpegSource2 are decoding the incorrect levels in the encode as-is (which is correct decoding), and reproducing the slight color shift which x264 created.

FFDShow-Tryouts & LAV-Video are either using the same broken swscale conversion as x264 or clipping input levels and is actually partially reversing the problem skewed levels in the encode creates (which is incorrect decoding). Which likely means both FFDShow-Tryouts and LAV-Video will need to keep their eye out for x264 fixing their swscale levels bug, so they can apply the same fix.

What this really comes down to is don't use x264's built-in 8bit to 10bit conversion or your encode won't have proper levels and will decode/render with a color tint until the bug is fixed. Someone should really check if the same thing happens with 16bit x264 input to 10bit.

[Dark Shikari]

I highly doubt that being off by 0.5 units of pixel value is responsible for a "chroma shift". I doubt such a thing would even be perceptible if the images were side by side.

If you have a problem with x264's conversion, submit a patch.

[madshi]

If you have a problem with x264's conversion, submit a patch.
The x264 development page (http://www.videolan.org/developers/x264.html) says:

"Please report any bugs to the mailing list."

Which I've done... :p

[madshi]

Edit: Patch was incorrect, I think.

[cyberbeing]

I highly doubt that being off by 0.5 units of pixel value is responsible for a "chroma shift". I doubt such a thing would even be perceptible if the images were side by side.

MPlayer, FFmpegSource2, MadVR end up with around all values 8-bit values Red +1, Green +0, Blue +1.

FFDShow-Tryouts & LAV-Video end up with around all 8bit values Red +0, Green +1, Blue +0.

It's easily perceivable with a good monitor when comparing the two, especially on anime content.
You probably wouldn't even need to compare against anything if you knew something was supposed to be a neutral gray tone (R=G=B).

@Dark Shikari, how about this patch?
Come-on madshi, don't take the easy way out with a 4 line patch. You know you want to write a multi-platform open-source replacement for swscale to take care of all these nasty colorspace, bit-depth, dithering conversions, and resizing. You could even port it from madVR and call it madScale. ;)

[madshi]

I think my patch was wrong. The offending function seems to be "scale_image" in depth.c. There is reads:

/* this function mimics how swscale does upconversion. 8-bit is converted
* to 16-bit through left shifting the orginal value with 8 and then adding
* the original value to that. This effectively keeps the full color range
* while also being fast. for n-bit we basically do the same thing, but we
* discard the lower 16-n bits. */
I don't know who invented this upconversion method, but it's definitely bad. Keeping the full color range is not what BT.709 calls for. Also Microsoft is pretty clear on how to convert video bitdepths:

http://msdn.microsoft.com/en-us/library/bb970578%28VS.85%29.aspx#_420formats

These formats use a fixed-point representation for both the luma channel and the chroma (C'b and C'r) channels. Sample values are scaled 8-bit values, using a scaling factor of 2^(n − 8), where n is either 10 or 16, as per sections 7.7-7.8 and 7.11-7.12 of SMPTE 274M. Precision conversions can be performed using simple bit shifts. For example, if the white point of an 8-bit format is 235, the corresponding 10-bit format has a white point at 940 (235 × 4).

[Reimar]

I don't know who invented this upconversion method, but it's definitely bad.

Well, it's the correct and standard method for RGB and full-range YUV, and I think that 10-bit RGB was first in libswscale (not sure though).
Guess it did "spill over" to places where it's not appropriate.

[Dark Shikari]

Eesh, so the upscaling method isn't the same for full-range vs non-full-range? That's horrific.

[madshi]

Well, it's the correct and standard method for RGB and full-range YUV
I'm sorry, but I disagree. The upconversion is bad even for full-range data. Why? For two reasons:

(1) Full-range YUV has luma and chroma components. For chroma the x264 upscaling is especially bad, full-range or not, because it converts a neutral 8bit 128 value to 10bit 514, which is not neutral, anymore. Full-range 10bit neutral chroma is identical to limited-range neutral: 512.

(2) Let's look at what x264 does:

16bit = (8bit << 8) + 8bit;
10bit = 16bit >> 6;

We have two steps here. One increases the bitdepth, which is fine. The second step decreases the bitdepth. A reduction in bitdepth requires dithering to be used, which is not done by x264. As a result we will get quantization noise. Let's double check that with a smooth 8bit gradient:

8bit full-range RGB data:
60, 61, 62, 63, 64, 65, 66, 67

10bit full-range RGB data, upscaled with x264:
240, 244, 248, 252, 257, 261, 265, 269

Do you see the problem? The gradient isn't smooth, anymore, because the 16bit -> 10bit downconversion by x264 was not done with the required dithering.

The proper solution to increase bitdepth in video processing is to use a simple shift:

10bit = 8bit << 2;

Smooth gradients stay smooth, without needing dithering. That applies to full range content, too, IMHO. Ok, I understand that a full range 8bit 255 peak white value is supposed to be 1023 in 10bit, but I think it's still preferable to use a simple shift, for the reasons explained above. The very slight loss in dynamic range is not really a problem, IMHO. At least it's a much smaller problem compared to the 2 problems mentioned above.

when converting eight-bit data to ten-bit data the least two significant bits of the ten-bit word shall be set to 0.

[Dark Shikari]

It sounds like we need to change swscale too then.

[madshi]

I think so. This is all from my point of view/understanding, though. I might be missing some cases where the current upscaling method is preferable. Still, I think if it's used like that, dithering should at least be optionally possible, whenever the output bitdepth is smaller than 16bit.

[Reimar]

I'm sorry, but I disagree. The upconversion is bad even for full-range data. Why? For two reasons:

(1) Full-range YUV has luma and chroma components. For chroma the x264 upscaling is especially bad, full-range or not, because it converts a neutral 8bit 128 value to 10bit 514, which is not neutral, anymore. Full-range 10bit neutral chroma is identical to limited-range neutral: 512.

Chroma is really signed, using unsigned upscaling is just broken. See e.g. UNORM vs. ZNORM in Direct3D which also specifies one set of "official" conversion methods (but I admit they are targeted towards RGB, it might make sense to go for something else with YUV - but the mismatch between expanding before or after RGB conversion is not nice either, but can be solved by defining full-range 10-bit YUV to only got up to 255*4 for example - I do not know how this is handled).
My conclusion would be that probably both the upscaling and the YUV->RGB conversion factors for > 8 bit need to be changed.

[madshi]

Chroma is really signed, using unsigned upscaling is just broken.
But unsigned upscaling is what swscale and x264 are doing right now for chroma, isn't it? FWIW, using a simple shift << 2 works just fine for both signed and unsigned chroma representations. So implementing the simple shift would be a rather easy fix for swscale and x264, because conversion math wouldn't have to be changed to signed for chroma.

See e.g. UNORM vs. ZNORM in Direct3D which also specifies one set of "official" conversion methods
Direct3D works in floating point and is totally optimized for full range game rendering. So of course it makes sense for Direct3D to set the range to exactly 0.0 to 1.0. I wouldn't use the Direct3D conversion logic as the base for what video processing should ideally do, though. Two pairs of shoes, really. For video processing we should look at what BT.709, SMPTE 274M and similar specifications say. Also, Microsoft themselves advertizes using a simple shift for video processing, unlike what Direct3D does.

And as I said before, *if* we keep using the current x264/swscale logic for full range luma (and RGB) content, then a dithering step needs to be added. My main worry is limited range content, though, which is probably 99% of the content out there. For that it's pretty clear that a simple shift must be used. I don't care much what swscale/x264 do with full range content.

[mandarinka]

A question: is 16-bit -> 10-bit conversion in x264cli also wrong like this?

(For example if one uses functions from the Dither package & avs2yuv in 16-bit raw mode to feed x264.)

[Reimar]

And as I said before, *if* we keep using the current x264/swscale logic for full range luma (and RGB) content

Just to verify, do you see an issue with always using a simple shift for YUV (also full-range) while keeping the current method for RGB?
It would make the conversion matrix depend on the bit-depth I guess which at least is ugly, but I have a feeling doing RGB conversion with only shifts would cause more issues.

[madshi]

Just to verify, do you see an issue with always using a simple shift for YUV (also full-range) while keeping the current method for RGB?
It would make the conversion matrix depend on the bit-depth I guess which at least is ugly, but I have a feeling doing RGB conversion with only shifts would cause more issues.
I see no issue at all if you go from 8bit to 16bit and stay there. But if you go from 8bit to 16bit, then back down to 10bit, dithering is needed, even for full-range RGB content. The processing is similar to changing volume in audio processing: Basically you multiply the integer audio data with a floating point value, which results in floating point data. If you convert the floating point audio data back to a lower (integer) bitdepth, you have to use dithering. The same principle generally also applies to video processing. Decreasing bitdepth from 16bit to 10bit in video processing means that you need to use dithering if you want to preserve full gradient smoothness. You can see that in the "60, 61, 62, 63, 64, 65, 66, 67" number row in one of my earlier posts. If you convert that to 10bit with the current x264/swscale method, the gradient isn't perfectly smooth, anymore.

Let me sum up what I would suggest swscale should do:

(1) For limited-range content (both RGB and YCbCr) I would definitely use simple shifting. That's not only simpler and faster, but also more correct. No disadvantages here, as far as I can see.

(2) For full-range content (both RGB and YCbCr) I would suggest to use either simple shifting or to add dithering to the current conversion algorithm. The latter solution would produce the best quality for full-range content, but it would lower lossless/lossy compression efficiency, due to the random nature of dithering. So maybe it would make sense to always use simple shifting. Or maybe to use the current algorithm + dither only if the final output is uncompressed. If the current algorithm stays (either with or without dithering), chroma scaling needs to be changed to signed math.

My opinion is that the easiest and quickest fix would be to replace the current solution with simple shifting. For full-range content it wouldn't be perfect, but IMHO it would still be better than the current solution.

Maybe we should discuss this on the libav mailing list?

[TheElix]

What about VC1 and AVC1 10-bit support for ffdshow?

[ForceX]

Huh? AVC1 and H.264 are the same. VC-1 doesn't even support 10 bit encoding...

[TheElix]

Okay, thanks for answering stupid questions. Also, how to get ffdshow to decode 10-bit H.264? I have the required version, but I get 4:2:0 8-bit as an output.

[sneaker_ger]

ffdshow doesn't support 10 bit output atm, 10 bit input is always dithered down to 8 bit output.

[TheElix]

Another misconception of mine. Thanks for getting this straight.

[patrick_]

There's a bug in 10bit FFDShow :( When converting to RGB, it uses BT.601 for SD video (my sample is 768x480), even if the stream specifies BT.709.

[madshi]

I don't think ffdshow looks at the video bitstream to find out which decoding matrix to use. madVR does that, though, if you let it do the YCbCr -> RGB conversion. So you should be able to use ffdshow -> madVR, and then you should automatically get BT.709. You need to configure ffdshow to output YCbCr, though, for that to work.

[clsid]

ffdshow actually does use some info from the bitstream for its RGB32 conversion.

[cyberbeing]

There's a bug in 10bit FFDShow :( When converting to RGB, it uses BT.601 for SD video (my sample is 768x480), even if the stream specifies BT.709.

For 10bit RGB conversion, FFDShow (swscale) will always use BT.601 YUV TV-range->RGB PC-range. It's not really a FFDShow bug, just a known limitation of the swscale defaults. Someone would need to hack at swscale in order to make it obey the FFDShow RGB output settings.

[nevcairiel]

You can tell swscale which matrix to use, it has options for that.

[madshi]

ffdshow actually does use some info from the bitstream for its RGB32 conversion.
Didn't know that, thanks for the correction.

[cyberbeing]

You can tell swscale which matrix to use, it has options for that.
Lord was aware that it allowed specifying a matrix for a single direction, but hooking it into FFDShow-Tryouts settings and logic for RGB output wasn't something simple enough to implement for the initial 10bit patch. The current 10bit implementation bypasses FFDShow-Tryouts stuff almost completely and just lets swscale do its thing with near-default settings. If you are familiar enough with what needs to be done to swscale to integrate it into FFDShow-Tryout's RGB settings, maybe you could submit a patch.

[patrick_]

I don't think ffdshow looks at the video bitstream to find out which decoding matrix to use. madVR does that, though, if you let it do the YCbCr -> RGB conversion. So you should be able to use ffdshow -> madVR, and then you should automatically get BT.709. You need to configure ffdshow to output YCbCr, though, for that to work.
Thanks! I didn't know that madVR uses the stream's matrix info.

[madshi]

Thanks! I didn't know that madVR uses the stream's matrix info.
It uses the stream's matrix, primaries & full range info (only for h264, VC-1 and MPEG2, though). That feature was added just recently.

[06_taro]

Fix bit depth conversion and dithering.diff (http://pastebin.com/iHqenZtb)

for x264
use different upscaling/dithering method for full-range/non-full-range according to this thread.
full-range upscaling method is the same with x264's original one.
non-full-range upscaling method is shifting bits only, e.g. <<2 for 8->10 conversion.

[madshi]

Fix bit depth conversion and dithering.diff (http://pastebin.com/iHqenZtb)

for x264
use different upscaling/dithering method for full-range/non-full-range according to this thread.
full-range upscaling method is the same with x264's original one.
non-full-range upscaling method is shifting bits only, e.g. <<2 for 8->10 conversion.
Thank you!! :)

Two small problems:

(1) When using the simple << 2 shift, please do *not* apply dithering. It is not mathematically needed and will just decrease compression efficiency.

(2) For RGB your patch should be fine. But the full range conversion doesn't work properly for YCbCr chroma channels. For full range chroma, you need to use signed math. Alternatively you could use the dithered full range conversion for the luma channel, only, while using simple << 2 shifting for the chroma channels. Basically you need to make sure that 8bit 128 (which is neutral / gray chroma) stays neutral after upscaling to 10bit. The final result in 10bit must be 512. With your current dithering full range upscaling the result will be (on average) 514.

[06_taro]

Agree. Dithering after 8 bit to 10 bit conversion is useless.

I didn't touch the full range conversion because I haven't find any standard documents referring to full range upscaling. So I kept x264's original method. But level offset like 128 turning to be 514 is really a problem. I think the upscaling and dithering method for full range conversion should be completely re-writed.

[madshi]

Can you change the full range conversion so that for YCbCr content (but not for RGB) the luma channel is upscaled with the current full range upscaling method, and that the chroma channels are upscaled by using a simple << 2 shift? I think that would be the best compromise for now. Dithering should be done for the current full range upscaling method, but not for the simple << 2 shift method.

Thanks.

[06_taro]

So should dithering be applied on full range YCbCr content's chroma channels any more?

[madshi]

If you change the full range YCbCr chroma channel math to use a simple << 2 shift (as I suggested) then no dithering should be applied.

As a general rule, dithering is only needed if you cut of bits which might be non-zero, by using a >> shift or a "/" division. The dithering reduces the negative effects of cutting off data bits. So if you use only a << shift, where no data bits are cut off, dithering has no use.

[thewebchat]

If we all know what the problem is, can we finally get this taken care of at the root (i.e. x264 and libswscale) so that people can actually start using 10-bit encoding? It seems that we know exactly what's wrong, and exactly how to fix it, and yet I see absolutely 0 developer interest in resolving this. In fact, there is absolutely no mention of this on either libav or ffmpeg mailing lists.

[mandarinka]

madshi, is the patch as linked in post #89 correct now, when it comes to the limited-range 8->10 conversion?
(I don't speak any programming languages, so I can't tell if dithering is used there or not.)

Maybe somebody could provide a testing build of x264 with that applied in the meantime?

[madshi]

The patch in post #89 applies dithering for limited range which is not good.

[mandarinka]

I see. Oh well, so much for that idea.

[cyberbeing]

Oskar Arvidsson (aka irock, author of 9-bit/10-bit support in x264/libav) seems to have taken interest in fixing the problem:
http://thread.gmane.org/gmane.comp.video.libav.devel/9285

But there is still a question of what actually needs to be changed. With Uoti Urpala (MPlayer2 dev), Ronald S. Bultje (aka BBB, x264/libav/Google dev, currently in the process of re-writing parts of swscale?), Michael Niedermayer (ffmpeg dev & original swscale author?), and madshi (madVR dev) all discussing it, I hope they have luck coming to an agreement without too much drama. All that's missing is Reimar (MPlayer dev) and Dark Shikari, if either have an opinion on the matter.

[JEEB]

Oskar Arvidsson @ libav seems to have taken interest in fixing the problem:
http://thread.gmane.org/gmane.comp.video.libav.devel/9285

But there is still a question of what actually needs to be changed. With Uoti Urpala (MPlayer2 dev), Ronald S. Bultje (aka BBB, x264/libav/Google dev, currently in the process of re-writing parts of swscale?), Michael Niedermayer (ffmpeg dev & original swscale author?), and madshi (madVR dev) all discussing it, I hope they have luck coming to an agreement without too much drama. All that's missing is Reimar (MPlayer dev) and Dark Shikari, if either have an opinion on the matter.
Nice, so irock (Oskar Arvidsson, the developer of the hi10p encoder parts in x264, as well as the decoder in libavcodec in the end) finally got the time to discuss the algorithms. I and BugMaster poked him a few weeks ago on the issue, and he said he'd talk with people about it after he got some time.

[nevcairiel]

Michael applied a change to ffmpegs swscale now to always use a simple shift for the chroma planes and limited-range luma, and only use the byte-duplication for full-range luma.
Its not a final solution, but it fixes the common use-case for limited range content.

However, this is ffmpeg, not x264, so .. someone get the changed in there as well. :p

I do hope the development of the changes goes into the right direction. The latest talk is about simply changing the 10->8 downscaler to correct for the upscaling error, which sounds terrible.

[JEEB]

Yeah, saw that on the IRC channel when it got in.
<CIA-107> ffmpeg: Michael Niedermayer master * rabe0b8e9f3 ffmpeg/ (9 files in 3 dirs):
<CIA-107> ffmpeg: sws: use shift for chroma sample up convertion
<CIA-107> ffmpeg: sws: use shifts for non full range luma sample upconvertion
<CIA-107> ffmpeg: Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
<kierank> irock: ^
<irock> oh
<irock> well, it was the right thing to do - but I assumed we were going to discuss this further first

Anyways, the good thing is that now something goes by some specification (BT.709 in this case IIRC), full range content can be discussed further, as well as the 10->8 dithering algorithm + possible other specifications.

Also, in related things, I wish I could understand filters/video/depth.c better >_>

[JEEB]

Posted a test build of x264 with a corrected limited range 8bit -> output bit depth algorithm here (http://forum.doom9.org/showthread.php?p=1522281#post1522281). Would be interested to see results.

[Cman21]

So now the question is which way is correct?

FFDShow-Tryouts | LAV-Video
OR
MPlayer | FFmpegSource2 | MadVR



Actually now that I think about it, what we are likely seeing is once again the x264 8bit to 10bit levels bug which madshi confirmed with a test chart sample I created a few days ago. Which once again means that MadVR, MPlayer, and FFmpegSource2 are decoding the incorrect levels in the encode as-is (which is correct decoding), and reproducing the slight color shift which x264 created.

FFDShow-Tryouts & LAV-Video are either using the same broken swscale conversion as x264 or clipping input levels and is actually partially reversing the problem skewed levels in the encode creates (which is incorrect decoding). Which likely means both FFDShow-Tryouts and LAV-Video will need to keep their eye out for x264 fixing their swscale levels bug, so they can apply the same fix.

What this really comes down to is don't use x264's built-in 8bit to 10bit conversion or your encode won't have proper levels and will decode/render with a color tint until the bug is fixed. Someone should really check if the same thing happens with 16bit x264 input to 10bit.

so does the latest x264 revision2074 on 2011-08-25 have this issue fixed or is it still pending?

[cyberbeing]

Still pending. The patches were pulled from the x264-devel tree since they are still in the process of being re-worked, and won't be included in the upcoming 4:2:2 encoding support push. Which means it will probably be a couple more weeks, or whenever the push after 4:2:2 happens, for this to finally be fixed once and for all.

[sneaker_ger]

You can get builds with an intermediate patch for limited range conversions from JEEB's blog (http://x264.fushizen.eu/).

[Cman21]

OH! thanks for pointing that out to me i didnt notice it the first time >_<

anyway i went and did a quick test encoding and it worked great! here are some comparisons if anyone cares:

http://screenshotcomparison.com/comparison/80048
http://screenshotcomparison.com/comparison/80050

[chapas]

Cman21, those look great! So 10bit solves the color blotches attributed to the encoder being too aggressive?

Can I get a sample of a 10bit file?

Thanks.

[Cman21]

umm sure i can upload a sample, sry for the delay but my C:\ failed on me so i have been a little preoccupied trying to get my computer back to an operable state... just send me a PM and i'll shoot you a link to the samples i used to make that comparison.

also i see that we are now on revision 2085 but on the change log i dont see anything about correcting this shift. but im going to ask anyway as half that stuff went over my head but is it corrected in r2085 or do i still need to use the non official one?

EDIT: Crap... i completely forgot i saved those videos on my desktop which i lost when i lost my C:\ so i cant upload them at the moment. i could always run it again but i dont know when i'll have the time to do so. but i do have more screenshots that i didnt post before cuz i didnt think they where completely relevant at the time so i'll just post those now.

the gradient comparisons show the main strong point in 10bit vs 8bit with the first screenshot is obvious why 10bit is better and the second is more subtle. on the second image on the gradient comparison look around the snow flakes near the top, you should see the 10bit has a much smoother transition between the white and blue. now after looking at quite a few detail oriented screenshots i think it is a wash between which is better, 10bit or 8bit. this is because 10bit's main advantage is in gradients and most detail oriented content has few gradients. but then you must also keep in mind the ~5-10% file size reduction that 10bit has, when encoding with the same settings, which is why 10bit still better.

the only other thing to note on these screenshots is that they where made using the official x264 r2074 which does not contain the correction that i compared in the previous post, which is why you see the shifting on the first gradient comparison. other than that single image i can barley, or not at all, see any shifting of colors.

Gradient Comparisons:
http://screenshotcomparison.com/comparison/80034

Detail Comparisons:
http://screenshotcomparison.com/comparison/80038
http://screenshotcomparison.com/comparison/80039
http://screenshotcomparison.com/comparison/80040
http://screenshotcomparison.com/comparison/80041
http://screenshotcomparison.com/comparison/80042