Log in

View Full Version : NV12 and yuv420p10 in MPV hwdec


FranceBB
15th April 2021, 01:15
Hi there,
I'm the not-so-proud owner of a crappy GPU which doesn't have H.265 4:2:0 10bit support but does however have H.265 4:2:0 8bit support, in fact I was able to play H.265 yv12 60p videos just fine without dropping a single frame.
Now, the thing is that not being able to decode 10bit files annoys me A LOT 'cause each and every UHD-BD is indeed in 10bit, which makes both my GPU (NVIDIA GTX 950M and Intel HD Graphics 530) useless...
I've been digging a bit into what MPV does and it seems that it converts the video to NV12 which is pretty much like yv12, so 8bit and then feeds it to the GPU for decoding, in fact whenever I click "I" on the player it says that the pixel_format is indeed NV12. So... I'm wondering why my decoding is wrong. I mean, if I FORCE GPU decoding of 10bit H.265 contents, what I see on my screen is awful and full of artifacts 'cause the GPU is clearly trying to decode a 10bit video as if it was 8bit, hence producing glitches and any kind of issue.

Here is a simple screenshot:

Broken Hardware Acceleration ON (VAAPI) (https://i.imgur.com/vhjhh8g.jpg)

Software Decoding (everything is fine) (https://i.imgur.com/7Dneg1o.jpg)


What makes me wonder, though, is: if MPV is actually converting everything to NV12 as it's the only available color space for the GPU and NV12 is 8bit only, shouldn't I be able to see it correctly? Is there something I'm missing? At which stage is it converted to NV12? After getting it from the GPU? What I think it's happening here is that the GPU decodes the yuv420p10 and delivers a bad NV12. Is my hypothesis correct? Is there anything I can do to make software decoding less resource intensive?

This is my mpv.conf file:


#
#
# Warning:
#
# The commented example options usually do _not_ set the default values. Call
# mpv with --list-options to see the default values for most options. There is
# no builtin or example mpv.conf with all the defaults.
#
#
# Configuration files are read system-wide from /etc/mpv/mpv.conf
# and per-user from ~/.config/mpv/mpv.conf, where per-user settings override
# system-wide settings, all of which are overridden by the command line.
#
# Configuration file settings and the command line options use the same
# underlying mechanisms. Most options can be put into the configuration file
# by dropping the preceding '--'. See the man page for a complete list of
# options.
#
# Lines starting with '#' are comments and are ignored.
#
# See the CONFIGURATION FILES section in the man page
# for a detailed description of the syntax.
#
# Profiles should be placed at the bottom of the configuration file to ensure
# that settings wanted as defaults are not restricted to specific profiles.

##################
# video settings #
##################

# Start in fullscreen mode by default.
#fs=yes

# force starting with centered window
#geometry=50%:50%

# don't allow a new window to have a size larger than 90% of the screen size
#autofit-larger=90%x90%

# Do not close the window on exit.
#keep-open=yes

# Do not wait with showing the video window until it has loaded. (This will
# resize the window once video is loaded. Also always shows a window with
# audio.)
#force-window=immediate

# Disable the On Screen Controller (OSC).
#osc=no

# Keep the player window on top of all other windows.
#ontop=yes

# Specify high quality video rendering preset (for --vo=gpu only)
# Can cause performance problems with some drivers and GPUs.
#profile=gpu-hq

# Force video to lock on the display's refresh rate, and change video and audio
# speed to some degree to ensure synchronous playback - can cause problems
# with some drivers and desktop environments.
#video-sync=display-resample

# Enable hardware decoding if available. Often, this does not work with all
# video outputs, but should work well with default settings on most systems.
# If performance or energy usage is an issue, forcing the vdpau or vaapi VOs
# may or may not help.

#Here we enable hardware decoding for everything
hwdec=auto
hwdec-codecs=all

#This breaks 10bit H.265 on my Intel i7 6700HQ CPU (Intel® HD Graphics 530 GPU) :(
vd-lavc-check-hw-profile=no

#Here we fallback to software if there's no hardware decoding
vd-lavc-software-fallback=yes

#Here we set the Linear Interpolation + Blending
#interpolation
#interpolation-threshold=-1
#tscale=bicubic

#Here we set the video sync (useful with GNOME)
video-sync=display-resample-desync

#Here we set the Display Server to avoid dropping frames when hwdec is disabled (Linux Only)
gpu-context=x11vk

#Dithering down to 8bit with Fruit (no floyd steinberg 'cause too much power)
#dither-depth=8



##################
# audio settings #
##################

# Specify default audio device. You can list devices with: --audio-device=help
# The option takes the device string (the stuff between the '...').

#Here we use ALSA instead of pulse (Linux Fedora, please change if on Windows)
audio-device=alsa/default
ao=alsa


# Do not filter audio to keep pitch when changing playback speed.
#audio-pitch-correction=no

# Output 5.1 audio natively, and upmix/downmix audio with a different format.
#audio-channels=5.1
# Disable any automatic remix, _if_ the audio output accepts the audio format.
# of the currently played file. See caveats mentioned in the manpage.
# (The default is "auto-safe", see manpage.)
#audio-channels=auto

##################
# other settings #
##################

# Pretend to be a web browser. Might fix playback with some streaming sites,
# but also will break with shoutcast streams.
#user-agent="Mozilla/5.0"

# cache settings
#
# Use a large seekable RAM cache even for local input.
#cache=yes
#
# Use extra large RAM cache (needs cache=yes to make it useful).
#demuxer-max-bytes=500M
#demuxer-max-back-bytes=100M
#
# Disable the behavior that the player will pause if the cache goes below a
# certain fill size.
#cache-pause=no
#
# Store cache payload on the hard disk instead of in RAM. (This may negatively
# impact performance unless used for slow input such as network.)
#cache-dir=~/.cache/
#cache-on-disk=yes

# Display English subtitles if available.
#slang=en

# Play Finnish audio if available, fall back to English otherwise.
#alang=fi,en

# Change subtitle encoding. For Arabic subtitles use 'cp1256'.
# If the file seems to be valid UTF-8, prefer UTF-8.
# (You can add '+' in front of the codepage to force it.)
#sub-codepage=cp1256

# You can also include other configuration files.
#include=/path/to/the/file/you/want/to/include

############
# Profiles #
############

# The options declared as part of profiles override global default settings,
# but only take effect when the profile is active.

# The following profile can be enabled on the command line with: --profile=eye-cancer

#[eye-cancer]
#sharpen=5

huhn
15th April 2021, 01:34
this is a maxwell GPU so it has 10 bit HEVC decoder.
even your intel IGPU can do main10.

to verify it you should try a different API. if you have a windows install just try MPC-HC play the same file and see if it works. i highly doubt the hardware here is the issue. and i only know about one card type that had a HEVC decoder limited to 8 bit and i can't even remember the name of that joke of a AMD card.

videoh
15th April 2021, 01:41
At which stage is it converted to NV12? After getting it from the GPU? What I think it's happening here is that the GPU decodes the yuv420p10 and delivers a bad NV12. Is my hypothesis correct? First note huhn's salient response. Could add that the nVidia GPU directly delivers NV12 (via CUVID/NVDec) in either 8-bit or 16-bit. So 10-bit content is padded to 16-bit. The NV12 must be de-interleaved to planar for Avisynth/Vapoursynth. Don't know what MPV is doing.

FranceBB
18th April 2021, 12:33
this is a maxwell GPU so it has 10 bit HEVC decoder.
even your intel IGPU can do main10.


Crikey! So I do have Main10 support... that's interesting and an unexpected turnaround...


nVidia GPU directly delivers NV12 (via CUVID/NVDec) in either 8-bit or 16-bit. So 10-bit content is padded to 16-bit.

Ahh!! I see!!
Ok I think what's going on here then: everything is working fine and the GPU delivers NV12 as 10bit padded to 16bit which is shown incorrectly as it assumes it's a real 16bit and bad things happen when it's finally converted to RGB and displayed to the monitor... This makes sense! And the fact that I force 8bit doesn't solve anything 'cause it's still incorrectly converting to 8bit after the GPU delivered a 10bit disguised as 16bit stream!

Well, now I know what's going on and what's wrong, but I wonder how I can fix it... I guess I'm gonna open a ticket to the MPV guys as MPV just uses the same libraries FFMpeg uses so it's just a matter of getting the right settings to interpret the disguised 10bit and telling it that it's not a real 16bit.




to verify it you should try a different API. if you have a windows install just try MPC-HC play the same file and see if it works. i highly doubt the hardware here is the issue. and i only know about one card type that had a HEVC decoder limited to 8 bit and i can't even remember the name of that joke of a AMD card.

I can't test it on Windows 'cause all my Windows installations on this Laptop (Win98Se, WinXP Pro and Win10 Enterprise) are Virtual Machines, I only use Windows with Avisynth on bare metal at work with my NVIDIA Quadro P4000, but what you wrote makes a lot of sense, so I guess it's just MPV assuming 16bit as a real 16bit instead of a 10bit disguised as 16bit and wrongly converts it to 8bit RGB.

Thank you both, huhn and Donald, you're always an inestimable source of knowledge! :)

Asmodian
20th April 2021, 03:26
Ok I think what's going on here then: everything is working fine and the GPU delivers NV12 as 10bit padded to 16bit which is shown incorrectly as it assumes it's a real 16bit and bad things happen when it's finally converted to RGB and displayed to the monitor...

Why would there be a difference in RGB values when converting from 10 bit or 10 bit padded to 16 bit? It should convert to exactly the same RGB values either way.

There is no difference between 'real' 16bit and 10bit padded to 16bit, except lots of 0s in the least significant bits.

FranceBB
29th April 2021, 15:30
Uhm... right... so technically it should just truncate the zeroes and nothing bad should happen anyway, right?
Then I have no idea what's going on...

huhn
29th April 2021, 22:14
well except for tons of banding.

the artefacts look like the same type when 10 bit h264 was a thing and "decoder" couldn't handle it correctly.
not sure if it ever was a decoder issue but instead a video format conversation issue.

clsid
29th April 2021, 23:19
This is clearly a user error. You have disabled the hw profile check. The result is that it is now decoding Main10 video with the 8bit Main profile decoder. That obviously result in messed up output.

videoh
30th April 2021, 01:37
I agree with clsid. Also, truncating the zeros is not going to cause banding. If there is banding at 10-bits that would be a source issue if clsid's mechanism does not apply.

huhn
30th April 2021, 06:39
it show NV12 which is 8 bit that's where my banding comment is based on.
p016 and p010 is not the issue here.

nevcairiel
1st May 2021, 02:06
The pixel format for 10-bit or 16-bit should be called p010 or p016 as huhn pointed out. nv12 is solely 8-bit. And if the decoder is running in 8-bit mode on a 10-bit stream, there is clearly something wrong. Maybe disabling the profile check also makes it fail to select the proper decoding settings for 10-bit, I woul definitely turn those checks on.

Balling
14th April 2022, 06:27
NV12 is not 8 bit, since that would mean 24 bit per pixel. Yet NV12 is 2 times less than that, 12 bpp, so 8 bit is used for Y and Cb, Cr both are only 4 bits, that is why 444 is not an option!

Do you think this is enough? Yep.

See: (4+2+0)/(4+4+4) is 1/2 so instead of 3*8 you get 1/2 *3*8 and that is 12 bit.

You could have read ffmpeg code to see that: https://github.com/FFmpeg/FFmpeg/blob/e85958ece84a8d3a6390f50ac2690addf28a082e/libavutil/pixfmt.h#L89

For HDR 444 you need NV30 actually. Or PX16, where x is no 0.

Now all this BS with dithering 10 bit content to 8 bit 420 is long gone: removed here https://github.com/FFmpeg/FFmpeg/commit/81147b5596ea19f7c5c153f4a534e9314d291fd3

As for vaapi this is a known bug.

huhn
14th April 2022, 07:24
NV12 is 8 bit per component and it is always 4:2:0 because that's what it is made for.
and it has 12 bit per pixel.

so yes it's 8 bit obviously just ignore the 12 in the name i'm not even sure if it is based on the 12 bit per pixel and it doesn't matter because it doesn't change anything.

this thread is a year old come on...

Balling
15th April 2022, 23:06
>NV12 is 8 bit per component

It is 8 bit per 1st component, 2 bits per second and 2 bits per 3rd, or it is 4 bits per component, if 3 components.

nevcairiel
16th April 2022, 00:02
Absolutely nothing in that signal is either 2 bit nor 4 bit, every actual value you ever process is 8 bit. Averages don't make up pixels, actual pixel make up pixel. Please stop spreading your confusing half-truths. There should really be a rule against that around here.

huhn
16th April 2022, 04:17
the problem is that bpp is used and and user don't understand that it is a very bad way to to describe the format.

everything in nv12 is 8 bit and it is 4:2:0 that's what matters.
it is always 3 components or call it channel like Microsoft because it is not mono color.

if you don't like bit per component(it means something else on apple system) then use bit per channel.

bit per pixel is also used for reason beyond me:
https://chromium.googlesource.com/libyuv/libyuv/+/HEAD/docs/formats.md

here have fun: https://docs.microsoft.com/en-us/windows/win32/medfound/10-bit-and-16-bit-yuv-video-formats

they use bit per channel.
you can even find the correct pixel format for the 4:4:4 "HDR"(what ever HDR has to do with a pixel format...) you randomly brought up it's v410 or v416.

Balling
16th April 2022, 07:09
Absolutely nothing in that signal is either 2 bit nor 4 bit, every actual value you ever process is 8 bit. Averages don't make up pixels, actual pixel make up pixel. Please stop spreading your confusing half-truths. There should really be a rule against that around here.
It can be also subsamled 4:2:0 RGB, yes, JPEG allows that too. And yes, averages do not make pixels, because left chroma siting is not average. Far from it.

By HDR 444 I meant yuv444p10, sorry. Yes, of course that can be SDR too, whatever. That requires 30 bits, 10 bit per 3 component, NV30 is not implemented yet in ffmpeg, this is next step. yuv420p10 only requires 15 bits (10 bits per Y and half of that for Cb, Cr). There is also yuv420p9, did not look up how that works.

You cannot use bit per channel for subsampled formats because not all logical pixels have all samples, see hex view in YUView.

Finaly, there you can passthrough NV12 directly to HDMI. 420 is supported.

https://user-images.githubusercontent.com/2803622/127140346-fa3aa3c7-1b55-4b1f-8e40-856456af4cd6.png

huhn
16th April 2022, 07:57
if you don't want to read the links just leave please.

you still don't get that Y, Cb and Cr is a component the resolution doesn't matter when you define the bit deep so does chroma subsampling not matter because it is 8 bpc for nv12, YV12, YUY2, YV24, AYUV, RGB24 and even RGB32 by haven 4 components.

why would you even even bring subsampling of rgb up it has nothing todo with this at all.

a pixel format doesn't care about chroma subsample position that's just meta data that's important for 4:2:0 to 4:4:4 and only for the algorithm this has nothing to do with the pixel format to make that absolutely clear.

There is also yuv420p9, did not look up how that works.

like you even looked up how nv12 works if you have we wouldn't "talk" about this.

there are countless pixel formats out there 4:2:1 subsample and such luckily we don't have to care what so ever.

a pixel format doesn't care about what you call logical pixel.
a 1920x1080p image stored in nv12 has a Y component with the resolution of 1920x1080 a Cb component of 960x540 and Cr component of 960x540 which each of these component using uint8_t or unsinged char if you like that word more.

because this format has 8 bit per component.

because there are A LOT of pixel formats i had to google NV30 and no it doesn't seem to exists and it doesn't matter because V410 is made for 10 bit 4:4:4 this format store 3 components of 10 bit each in 32 bit and has a 2 bit alpha channel.

You cannot use bit per channel for subsampled formats because not all logical pixels have all samples, see hex view in YUView.

why would i look at that if i want to know how nv12 or YV12 (both store the same type of data differently) when i instead can just look up how it is stored digitally.

huhn
16th April 2022, 08:02
Finaly, there you can passthrough NV12 directly to HDMI. 420 is supported.

no you can not do that like that and again it doesn't matter has nothing todo with the topic.

NV12 is not a presentation surface and even with a 4:2:0 surface you still have to fix the chroma position first making it very questionable to use it in the first place for the simple reason that you have to use the chroma subsample position defined by the HDMI spec and what ever that maybe.
and don't worry BD and UHD BD don't have the same subsample position so end of story.

Balling
17th April 2022, 14:37
NV30 does exist, you CANNOT USE GOOGLE apparently. https://lkml.org/lkml/2020/7/6/977

https://github.com/maskedeken/CoreELEC-RR/blob/836187f6190a8d0976756676263256eb68c5b384/projects/Rockchip/patches/linux/default/linux-0021-drm-from-list.patch#L6

AGAIN, you can just use 30 bit pixel format like some of PX16...


>have to use the chroma subsample position defined by the HDMI spec and what ever that maybe

Sure, that is what I meant. Though it is quite buggy.

>you still don't get that Y, Cb and Cr is a component the resolution doesn't matter when you define the bit deep so does chroma subsampling not matter because it is 8 bpc for nv12, YV12, YUY2, YV24, AYUV, RGB24 and even RGB32 by haven 4 components.

This whole idea is wrong. 4:2:0 is two times less than 4:4:4 why would people use all 4:4:4 for 4:2:0? GPU bandwidth is not inifinite. Because of people like you we cannot have good things.

P.S. libyuv is hacky BS library from Google. It is much worse than ffmpeg. But even it mentions: "The 12 in NV12 refers to 12 bits per pixel. NV12 has a half width and half height chroma channel, and therefore is a 420 subsampling."

P.S.2 "why would you even even bring subsampling of rgb up it has nothing todo with this at all." It has everything to do with this. Wow, this is just dumb.

v0lt
17th April 2022, 18:13
this is a maxwell GPU so it has 10 bit HEVC decoder.
No, it doesn't. Only the desktop version of the GTX 950 has a hardware decoder for HEVC 10-bit. The mobile version is made on a different chip.
GTX 950M - GM107 - VDPAU Feature Set E
GTX 950 - GM206 - VDPAU Feature Set F

even your intel IGPU can do main10.
HD Graphics 530 (Skylake) has poor HEVC 10-bit support. Normal support for HEVC 10-bit appeared in Kaby Lake.

huhn
17th April 2022, 18:37
noted another special case for maxwell. i guess i have to look up here one when it comes to maxwell.

how poor is skylake?
spec says it can. is it useless like hybrid or more like AMD bare minimum?

v0lt
17th April 2022, 19:33
NVDEC -> Supported Format Details (https://developer.nvidia.com/nvidia-video-codec-sdk)
Maxwell (GM206) is the only Maxwell that has a full HEVC 4:2:0 10-bit hardware decoder.
Video cards based on the GM206 chip: GTX 960, GTX 950 and GTX 750 SE. Wikipedia (https://en.wikipedia.org/wiki/Nvidia_PureVideo#Table_of_GPUs_containing_a_PureVideo_SIP_block).

how poor is skylake?
From Wikipedia (https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video#Development):
The Skylake microarchitecture adds a full fixed-function H.265/HEVC main/8-bit encoding and decoding acceleration, hybrid and partial HEVC main10/10-bit decoding acceleration
I saw a couple reviews of Skylake graphics a few years ago. The playback stability score of 4k HEVC 10-bit was poor.

huhn
17th April 2022, 19:38
NV30 does exist, you CANNOT USE GOOGLE apparently. https://lkml.org/lkml/2020/7/6/977

yes i will not find the spec of this format.
all i find after longer looking (and again it doesn't matter) is some NVidia stuff.
if they want to add more of these countless formats i have no problem with that.

this is the problem...
NV12 is not 8 bit, since that would mean 24 bit per pixel. Yet NV12 is 2 times less than that, 12 bpp, so 8 bit is used for Y and Cb, Cr both are only 4 bits, that is why 444 is not an option!
it is 4.2:0. 4:4:4 is not an option because it is 4.2:0 end of story. and each component is 8 bit not 4. because it is 8 bit and the AVG is 12 pixel which is a bad name because people come up with none sense like yours not understanding the meaning of 12 bit per pixel.
>have to use the chroma subsample position defined by the HDMI spec and what ever that maybe

Sure, that is what I meant. Though it is quite buggy.
and again it has nothing todo with a pixel format.

>you still don't get that Y, Cb and Cr is a component the resolution doesn't matter when you define the bit deep so does chroma subsampling not matter because it is 8 bpc for nv12, YV12, YUY2, YV24, AYUV, RGB24 and even RGB32 by haven 4 components.

This whole idea is wrong. 4:2:0 is two times less than 4:4:4 why would people use all 4:4:4 for 4:2:0? GPU bandwidth is not inifinite. Because of people like you we cannot have good things.

nv12 4:2:0 8 bit... it's not 4.4:4 i wonder how many times you need to hear this before you read spec of it...
using YUV as a name here
it uses an array of Y followed by an half sized array where U and V are alternated(packed) what so hard to understand about it.
because presentation surfaces on windows are RGB that's why you need 4:4:4 and RGB.
btw... nv12 works just like nv15 if the github page is correct it just uses 5 bytes "packed" instead of 1 byte component.
but don't read tomuch into this because i don't have the spec
P.S.2 "why would you even even bring subsampling of rgb up it has nothing todo with this at all." It has everything to do with this. Wow, this is just dumb.
yeah it has everything todo with something totally random.
let's subsample RGB for videos. YCbCr is created so you can subsample without destroying an image let's bring RGB up.

and about GPU bandwidth... just no. PCIe bandwidth maybe but this topic is about hardware decoding and NV12 doesn't waste a single bit but what ever...

Balling
27th April 2022, 14:43
Here is 4:4:4 10 bit from RGB 64 bit (16 bit per component, 4 components, RGBA). https://drive.google.com/drive/u/0/m...v6VjbSK1hAXwJH

Requires Y410, at least 30 bit per pixel.

mpv.com -v --hwdec=nvdec -gpu-api=vulkan --vo=gpu "01. test444_23.976.mp4"

Will cause HW decode on 20 series Nvidia and later. No HW decode with --hwdec=d3d11, alas. IT IS POSSIBLE with those patches https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=527 at least on Intel® UHD Graphics 770 (12th gen BigCore).

Madvr does not HW decode.
Davinci is HW decoded too, since it does not use d3d11, it uses directly nvdec.

Balling
3rd June 2022, 07:36
422 for 8 bit will be added in https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220601203337.1116134-1-matthieu.bouron@gmail.com/

Indeed 422 requires (4+2+2)/(4+4+4)=8/12=2/3 of data so 24 bit for 8 bit bit 444 is only 24/3*2= 16 pixls

Balling
19th July 2022, 20:42
422 for 8 bit will be added in https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220601203337.1116134-1-matthieu.bouron@gmail.com/

Indeed 422 requires (4+2+2)/(4+4+4)=8/12=2/3 of data so 24 bit for 8 bit bit 444 is only 24/3*2= 16 pixls

Applied in https://github.com/FFmpeg/FFmpeg/commit/0a6bb7da55709d82b1cbb12d420db2d1fbb867d7

Now we need NV61.

Balling
14th August 2022, 12:38
Y410, Y416, Y216 will be added here: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220813211821.14916-2-philipl@overt.org/

Balling
5th September 2022, 08:46
Added. Only NV61 left.

Balling
20th December 2022, 07:47
So when NV61? please end it finally. Last format.

Balling
26th September 2023, 16:03
So when NV61? please end it finally. Last format.

Still not applied.