madVR - high quality video renderer (GPU assisted)

[nijiko]

@huhn , does Lenovo Thinkpad use a 6bit displayer?

[James Freeman]

Originally Posted by Ver Greeneyes

You need at least a 10-bit monitor to take advantage of 10-bit output. Using it on your 8-bit monitor will give you worse quality since it will basically bypass madVR's dithering to fall back on the dithering of your display or GPU.

Are you sure?

I think switching to 10bit in madVR does not disable dithering, but it does make the dithering less strong.
If madVR would have 16bit option with dithering enabled, on an 8bit monitor it would look completely undithered.
As an opposite to that statement, we already can see that if one selects a lower bit depth, the dithering is a lot stronger.

If I understand correctly, madVR does not dither the exact integer steps, but only the ones that do not match.
As a result if one selects 10bit in madVR, madVR will not dither the steps that should fall exactly on a 10bit display.
In other words, banding will be generated the higher the bit depth mismatch between the Display and the GPU output.

[The 8472]

There's a new superresolution upscaler for still images: https://github.com/nagadomi/waifu2x/ based on http://arxiv.org/abs/1501.00092

There also is a webservice running that implementation which has been trained with anime-style images and provides very good results on that type of material, especially on fine lines that are barely a pixel wide and fairly aliased. It also seems to introduce fewer fake details on flat surfaces than nnedi.
If you run it on real life images you'll get oil paintings.
Specialization a specific kind of content seems to have a pretty big impact.

Source image
new upscaler w/ noise reduction
new upscaler w/o noise reduction
NNEDI3 32 w/ SuperRes
NNEDI3 64 w/o SuperRes


I figure it's too expensive to run this in realtime, thus my question is can we get madvr any closer to those results? E.g. by having separate live action / animation modes for nnedi3?

Edit: added needi w/o superres

[Ver Greeneyes]

Originally Posted by James Freeman

Are you sure?

I think switching to 10bit in madVR does not disable dithering, but it does make the dithering less strong.
If madVR would have 16bit option with dithering enabled, on an 8bit monitor it would look completely undithered.
As an opposite to that statement, we already can see that if one selects a lower bit depth, the dithering is a lot stronger.

If I understand correctly, madVR does not dither the exact integer steps, but only the ones that do not match.
As a result if one selects 10bit in madVR, madVR will not dither the steps that should fall exactly on a 10bit display.
In other words, banding will be generated the higher the bit depth mismatch between the Display and the GPU output.

Right, that's what I meant. madVR will only dither the least significant bit, so if you send a dithered 10-bit image to an 8-bit display, that will at most affect the display's (or GPU's) dithering decisions in representing this 10-bit image as an 8-bit one. madVR's dithering will likely do very little.

[ryrynz]

Originally Posted by The 8472

I figure it's too expensive to run this in realtime, thus my question is can we get madvr any closer to those results? E.g. by having separate live action / animation modes for nnedi3?

I just did my own test with one of their test images and NNEDI3 is superior when used at higher neuron counts. What you're seeing as a difference there with your own shots is the SuperRes sharpening.. drop that and it'll look a lot closer. Plus that particular resizer has a smoothing function (hence the flatness) when I did the test I clicked no noise reduction for a fair test (madVR doesn't do that with NNEDI) and NNEDI3 delivered a nicer picture without the excessive sharpening I got from waifui2 which made the JPEG artifacts of their test image even more "noisy"

TBH it looked quite terrible in comparison, it's not until you use their denoising filter (which wouldn't be something you'd use generally with video presentation these days (anyone still watch blocky MPEG video?)) that it looks any "good".

So NNEDI3 blows waifu2 away in the most important area (line construction) check out left side hair clip (nearest neighbour upscale? eww) and the hair outlines.
Sure waifu2 used with a blur is cleaner and sharper but it's totally overdone. madVR at some point may add an actual denoiser or deblocker but you could achieve better results than waifu easily using avisynth/ffdshow and combine that with NNEDI3. The only area I see where waifu2 does better is determining line accuracy in the deep corners (near right eye and right arm) If NNEDI3 could be modified to get the same results there that would be a nice improvement.

My results with their test image.

[huhn]

Originally Posted by Ver Greeneyes

You need at least a 10-bit monitor to take advantage of 10-bit output. Using it on your 8-bit monitor will give you worse quality since it will basically bypass madVR's dithering to fall back on the dithering of your display or GPU.

10-bit output is basically for people with displays advertising 'deep color' and things like that. With the advent of Blu-Ray there's quite a few of those these days. Also considering madVR is all about getting the best display quality 'at any cost' (so long as it can be done in realtime on high end hardware), this thread is probably not representative of the average adoption rate for 10-bit displays

displays alter the picture quite a lot. if it is chroma sub sampling color correction or other stuff. so sending 10 bit to a native 8 bit panel can help a lot. and displays can dither even my TV can do this.

Originally Posted by nijiko

@huhn , does Lenovo Thinkpad use a 6bit displayer?

i don't know. but these are laptops there displays are usually cheap.

[Ver Greeneyes]

Originally Posted by huhn

displays alter the picture quite a lot. if it is chroma sub sampling color correction or other stuff. so sending 10 bit to a native 8 bit panel can help a lot. and displays can dither even my TV can do this.

Sure, but the dithering quality isn't necessarily going to be very good. If it only accepts 4:2:0 input I could see 10-bit helping a lot though, yeah, assuming it uses the 10-bit information internally.

[The 8472]

Originally Posted by ryrynz

I just did my own test with one of their test images and NNEDI3 is superior when used at higher neuron counts. What you're seeing as a difference there is the SuperRes sharpening.. drop that and it'll look a lot closer. Plus that particular resizer has a smoothing function (hence the flatness) when I did the test I clicked no noise reduction for a fair test

Good points, I'll add some more variants based on that to my previous post

Originally Posted by ryrynz

NNEDI3 blows waifu2 away, no contest in the most important area (line construction) sure waifu2 used with a blur is cleaner and sharper but it's overdone.

I have difficulty parsing your statement there. Are you saying that NNEDI3 or that waifu2 performs better at line construction? In my eyes it's the latter. Especially regarding the scarf in my sample images.

[huhn]

Originally Posted by Ver Greeneyes

Sure, but the dithering quality isn't necessarily going to be very good. If it only accepts 4:2:0 input I could see 10-bit helping a lot though, sure, assuming it uses the 10-bit information internally.

madVR used for years random dithering. and why should the dithering be bad? even with 8 bit RGB input a screen will dither/process it's not like the whole color correction and other stuff isn't applied anymore.

even my cheap old display accept 10/12 bit and is dithering it.

[Nevilne]

The 8472, your NNEDI3 images are 130kb jpegs.

[ryrynz]

Originally Posted by The 8472

I have difficulty parsing your statement there. Are you saying that NNEDI3 or that waifu2 performs better at line construction? In my eyes it's the latter. Especially regarding the scarf in my sample images.

I've edited my post, take a look. NNEDI3 is superior, don't confuse line sharpness with line construction

[The 8472]

Originally Posted by Nevilne

The 8472, your NNEDI3 images are 130kb jpegs.

Ugh, imgur is compressing them. The originals are PNG. One minute, I'll replace them.

Edit: fixed

[leeperry]

Originally Posted by ryrynz

If NNEDI3 could be modified to get the same results there that would be a nice improvement.

NNEDI3 would appear to be a black box whose secrets are only known by tritical himself and he hasn't released stuff in a while AFAIK so I wouldn't set my hopes too high.

[ryrynz]

Originally Posted by leeperry

NNEDI3 would appear to be a black box whose secrets are only known by tritical himself and he hasn't released stuff in a while AFAIK so I wouldn't set my hopes too high.

Apparently he was working on a NNEDI4? He's been contacted recently, so it might not be terribly hard to improve it.
Looks like the waifu code is available to use for anyone that wants it judging from the license.. I wonder if shiandow could adapt NNEDI3 with waifu2x's improved neural guesswork.

[The 8472]

Originally Posted by ryrynz

I've edited my post, take a look. NNEDI3 is superior, don't confuse line sharpness with line construction

Hum, ok, maybe my jargon is off.

Compare any of the nnedi3 images vs. the waifu2 I've posted and look at the scarf. scarf, stairs, nnedi

What waifu2 seems to do better is avoiding the stairs effects on fine lines.

How would you call that? It's obviously more than just dumb smoothing/sharpening.

The stairs that nnedi "preserves" are what sticks out like a sore thumb to me, because that's an artifact of rasterization of finer lineart.

[huhn]

waifux2 is clear better in term of details at the cost of a lot of aliasing.

but the source has a ton of aliasing too so hard to judge

[ryrynz]

Originally Posted by The 8472

What waifu2 seems to do better is avoiding the stairs effects on fine lines.

Probably shouldn't continue much further with this discussion here but those words link to the same file.

Originally Posted by huhn

waifux2 is clear better in term of details at the cost of a lot of aliasing.
but the source has a ton of aliasing too so hard to judge

Fine details judging from what 8472 says.. I guess we could open a new thread for more comparisons.

I was right about 8bit + FRC NOT being the same as true 10bit. nVidia just added an option to select color depth in its latest 352.86 drivers and it will not let me select anything other than 8bit on my 8bit + FRC monitor (Eizo Foris FG2421 - using DisplayPort). True 10bit monitor would be allowed a setting of 10bit in nVidia CP. Using 10bit setting in madVR is a bad idea for 8bit + FRC monitors. I will either not work or make rendering quality worse than it would be if you selected 8bit. Bit depth selection and HDMI Full/Limit color range can be selected under "Change resolution" in nVidia CP in 352.86 drivers. Just scroll down and you will see the new options.

I have this odd problem. I can use Direct3D 11 with Sync sub-option underneath it on my 120Hz FG2421 monitor without ANY frame drops or problems. However, when I use my HDTV at 23Hz, I get constant frame drops with Direct3D 11 with or without the Sync sub-option underneath it. Why is that? Is it because ReClock works harder @ 23Hz or what? I thought 120Hz would be the one with problems because it requires faster rendering.

[huhn]

Originally Posted by XMonarchY

I was right about 8bit + FRC NOT being the same as true 10bit. nVidia just added an option to select color depth in its latest 352.86 drivers and it will not let me select anything other than 8bit on my 8bit + FRC monitor (Eizo Foris FG2421). True 10bit monitor would be allowed a setting of 10bit in nVidia CP. Using 10bit setting in madVR is a bad idea for 8bit + FRC monitors. I will either not work or make rendering quality worse than it would be if you selected 8bit.

P.S. Bit depth and HDMI Full/Limit color range can be selected under "Change resolution" in nVidia CP. Just scroll down and you will see the new options.

what connector are you using?
and if your device doesn't accept 10 bit input it's kind of worthless to send 10 bit to the driver.

but not all 8bit+ FRC doesn't accept 10 or 12 bit input.

Originally Posted by huhn

what connector are you using?
and if your device doesn't accept 10 bit input it's kind of worthless to send 10 bit to the driver.

but not all 8bit+ FRC doesn't accept 10 or 12 bit input.

I use DisplayPort. I guess its just my monitor that does not accept 10bit. I wonder if there is a way to force it... I mean there is DEFINITELY dithering going on on my monitor. I can see it easily.