madVR - high quality video renderer (GPU assisted)

[huhn]

not sure about this downscaler. is sharper than bicubic but terrible aliased at the same time.

just look at the hair.

the comparison algorithm are pretty weak to say it friendly.

[ashlar42]

I concur, if I have to limit myself at the images displayed on that page, the algorithm seems pretty terrible, to be honest. I like a sharp image, but judging from these images one is supposed to watch a Minecraft game, not a movie. :)

Edit: pictures here are better, though https://graphics.ethz.ch/~cengizo/Files/Sig15HighResNaturalImages.html

Aliasing is still there, but at least I can see a reason for taking it into consideration. They really ought to change the pics in the homepage.

Edit 2: these https://graphics.ethz.ch/~cengizo/Files/Sig15UserStudyImages.html are even better. Interesting...

[KoD]

Bug report: with the provided file (http://wikisend.com/download/199646/qs-adaptive.mkv) (35Mb), madVR breaks in many different ways when using QuickSync as the decoder in MPC-HC. EVR (custom presenter) has no problems, with the same decoder. It's a straight mux to mkv of the first minute of the first episode of Star Trek Voyager, from a PAL DVD disc (25i material).


I am listing the steps to follow for what I describe below, because based on what you do, the issues you get are different:
- connect your display (TV in my case) directly to your motherboard HDMI port; if you connect your display to another graphic card, and just use QS decoding in the background, the results below will be different (like, no issues in some cases and much worse issues in others);
- switch your display to a 50Hz refresh rate (again, if you have a different refresh rate, the issues are much worse)
- reset the madVR settings, using the "restore default settings.bat" in the madVR folder
- in MPC-HC Options: Playback\Output\Auto-zoom = enabled, and set to AutoFit (Larger only)
- in MPC-HC Options: Playback\Fullscreen\Launch files in Fullscreen = disabled


Now, in MPC-HC, let's first configure to use EVR as the video renderer, select xySubfilter as the subtitle renderer, and QuickSync as the hardware decoder, and play the file. During playback, make sure to try and skip back and forth in the file (using the keyboard arrows, for instance), and to switch between windowed mode and full-screen mode, just to make sure that everything works fine. Notice that subtitles are located nicely at the bottom of the image, too.
- with "Enable Adaptive HW deinterlacing = disabled" -> no issues
- with "Enable Adaptive HW deinterlacing = enabled" -> no issues

Now select madVR as renderer, and make sure xySubfilter is also selected as the subtitle renderer. Start playback, but pause it immediately to configure madVR a bit more:
- devices->display modes->switch to matching display mode = disabled
- processing->zoom control->move subtitles = disabled
- rendering->general settings->full-screen exclusive mode = disabled.

Playback results:
- with "Enable Adaptive HW deinterlacing = disabled", madVR using DX9 video path, while the file plays in a MPC-HC window of the same size as the video material, everything seems to look fine; the opening credits are scrolling, and then the action in the video begins, with some talking heads; it's at this point that you see subtitles in the video, too; but try to switch to fullscreen mode -> the subtitles are no longer at the proper location in the video ! They're now in the middle-left part of the screen.

- same as above, but with "Enable Adaptive HW deinterlacing = enabled": again, everything seems to be fine; let's try and switch to full-screen mode-> what do you know, this time the subtitles are in the proper location (at the bottom of the screen)!

Don't forget to switch between full-screen mode and windowed mode, and to skip in the file back and forth, to make sure that everything else is fine!

Ok, time to switch madVR to use the DX11 path:
- rendering->general settings->use Direct3D 11 for presentation = enabled
- rendering->general settings->present a frame for every VSync = enabled


Now, playback results:
- with "Enable Adaptive HW deinterlacing = disabled", madVR using DX11 video path, while the file plays in a MPC-HC window of the same size as the video material, everything seems to look fine, at least at the beginning; when the talking heads appear, you might notice some judder, but you might also think it's a fluke. So, after these talking heads appear, let's switch to fullscreen mode -> very likely you'll see an extreme case of frames jumping back and forth, making the chaotic video even more chaotic. Or, if per chance this does not happen, just skip back a bit, I guarantee the extreme jumping will start. And, did you notice the subtitles are once again located in the middle-left part of the image?
- with "Enable Adaptive HW deinterlacing = enabled", madVR using DX11 video path, switching to full-screen does not have the same dramatic effect as above, you notice some jumps but not that bad. But it's enough to skip back or forth in the file, and the extreme frame jumping starts.


I'll make one more comment, about having the display connected to another graphics adapter, instead of your motherboard video port. In this case, having "Enable Adaptive HW deinterlacing = disabled" might give a working picture, but turn this setting on, and this is by far the worst playback scenario - you might get a frame every 1 minute or so, if you have the patience to wait for it; we can't really say there is any playback in this case.


Finally, switch back to EVR (custom presenter) as the video renderer, to have proof that everything can work fine when using the same decoder and the same file.


These kind of issues don't happen with every file. But when it happens, it's really bad.

[madshi]

Thanks!
Sorry, but the problem is not fixed here :(
The log:
https://drive.google.com/file/d/0B7a6LffuxvKUSS1UZTRyeGx6Ylk/view?usp=sharing
Argh. Ok, can you create another log with the next madVR build? I've added some more logging output, maybe it helps me finding out what's going on there.

OK great, would there be room for further improvement there in your opinion?
I'm not sure what you mean.

It's useful at a glance.. Could you at least state the profiles that are running?
You do realize that there can be 17 different profile groups? Kodi/DSPlayer actually has a mode which creates the max number of profile groups. I don't really want to add (up to) 17 new information fields to the OSD.

Some more thoughts on SuperRes radius:
I think one should simply use a lower SR strength for very high scaling factors, e.g. not above 2.
With it, I think aliasing is not an issue with real content in the wild and with high scaling factor, the effect of SR subjectively looks stronger to me in general.
So imho a radius of 0.66 is fine in general and if one wants less aliasing but still a nice sharpness boost, SR strength of 2 should be combined with the new sharpen options (at least once they are more affordable).
I would find it really bad if the radius was higher, as I really like the naturally sharp impression when doing 1080p -> WQHD. I suppose a radius of 0.66 also looks fine with 1080p -> 4k, as the aliasing seems to depend on a very low pixel count of thin structures.
Ok.

The media player reacts. When I press stop button then it is pressed and the sound of the actual file stops playing. The video remains frozen until I close mpc-hc.

I would also add information that the system have two displays. One is the primary PC monitor. The second one is TV where the movies are played fullscreen. I operate mpc-hc from the main PC monitor taskbar (for this kind of test).
Please activate debug mode, then reproduce the problem, let the media player stay frozen for 20 seconds, then please terminate it with the task manager. Then please zip and upload the log somewhere (not here) for me to look it. Please also create 2-3 freeze reports (Ctrl+Alt+Shift+Pause/Break). Thanks.

I'm not so sure about that, but let's wait and see.

There's another isssue with those interlaced DVDs since the introduction of the automatic switch to DXVA scaling:
When playback starts the clock deviation is heavily off, maybe 10^6 % or so, and the DVD essentially hangs for 10 to 20 sec until this value goes down to an acceptable level and plays fluidly thereafter.
Sounds strange. I don't think I have that problem here. Do you have another PC to double check this with?

anyways, I wanted to draw some attention to a new developed downscaling algorithm by ETH Zurich which is called "Perceptually Downscaling" which gets pretty amazing results:
https://graphics.ethz.ch/~cengizo/imageDownscaling.htm
not sure about this downscaler. is sharper than bicubic but terrible aliased at the same time.

just look at the hair.

the comparison algorithm are pretty weak to say it friendly.
I concur, if I have to limit myself at the images displayed on that page, the algorithm seems pretty terrible, to be honest. I like a sharp image, but judging from these images one is supposed to watch a Minecraft game, not a movie. :)

Edit: pictures here are better, though https://graphics.ethz.ch/~cengizo/Files/Sig15HighResNaturalImages.html

Aliasing is still there, but at least I can see a reason for taking it into consideration. They really ought to change the pics in the homepage.

Edit 2: these https://graphics.ethz.ch/~cengizo/Files/Sig15UserStudyImages.html are even better. Interesting...
Could be worth a try. It does look like it tends to produce heavy aliasing in some situations, but maybe it could be improved by running some sort of smoothing filter. I don't have time for this atm, though.

Bug report: with the provided file (http://wikisend.com/download/199646/qs-adaptive.mkv) (35Mb), madVR breaks in many different ways when using QuickSync as the decoder in MPC-HC. EVR (custom presenter) has no problems, with the same decoder. It's a straight mux to mkv of the first minute of the first episode of Star Trek Voyager, from a PAL DVD disc (25i material).
Which exact GPU are you using when getting the DX11 frame jumping? And which OS are you using? And which discrete GPU are you using when *not* connecting your display to the mainboard HDMI output?

[madshi]

madVR v0.89.17 released

http://madshi.net/madVR.zip

* significant speedups for "sharpen edges" and "thin edges" algos
* small quality change (hopefully improvement?) for "thin edges"
* fixed: repeated frames were reported although smooth motion FRC was on
Some benchmarks on my development PC:

AMD HD7750 1080p24:
AdaptiveSharpen: 6.34ms
sharpen edges v0.89.16: 17.15ms
sharpen edges v0.89.17: 9.87ms
thin edges v0.89.16: 25.62ms
thin edges v0.89.17: 5.41ms

Intel HD4000 720p24:
AdaptiveSharpen: 9.92ms
sharpen edges v0.89.16: 36.2ms
sharpen edges v0.89.17: 15.9ms

NVidia GTX650 1080p24
AdaptiveSharpen: 15.51ms
sharpen edges v0.89.16: 27.59ms
sharpen edges v0.89.17: 11.69ms
And yes, the results are weird on the NVidia compared to AMD and Intel. But I've double checked, the results are correct. So "sharpen edges" is now about 50% slower than AdaptiveSharpen on my AMD & Intel GPUs, but 25% faster than AdaptiveSharpen on my NVidia GPU.

[aufkrawall]

Thanks for the performance optimizations.
With GTX 980, now Adaptive Sharpen and sharpen edges roughly seem to be equally fast. Adaptive Sharpen is still slightly faster, I get a little lower GPU usage and rendering time with it. So there seems to be a difference to Kepler.

Regarding SuperRes again:
I read here that SR would produce an overly sharp picture to some when doing 1080p -> WQHD. However, I can hardly believe this. I made a sharpness comparison with a game screenshot rendered in WQHD, and despite of SR with strength of 3 on top of Jinc AR scaling, it's still noticeably softer than the original (and much closer to ground truth than just Jinc AR without SR):

Original in WQHD:
http://abload.de/thumb/wqhdoriguhryd.png (http://abload.de/image.php?img=wqhdoriguhryd.png)

Downscaled to 1080p with C-R GL:
http://abload.de/thumb/1080pouqwf.png (http://abload.de/image.php?img=1080pouqwf.png)

Upscaled to WQHD Jinc AR without SR:
http://abload.de/thumb/nosrlhpe4.png (http://abload.de/image.php?img=nosrlhpe4.png)

SR strength 3:
http://abload.de/thumb/srj8q6r.png (http://abload.de/image.php?img=srj8q6r.png)

[leeperry]

I'm not sure what you mean.
Well, 2X SR improves edges clarity so how about another extra pass on top of it for instance? Most users here don't care about increased GPU load if PQ improves.

Thanks a lot for .17 but apparently 2X is still disabled for SR only? Based on my screenshots comparison, you seemed to agree that 2X SR improves PQ so why not? I got no use for 1X only SR :(

[rodrigo081089]

Is it possible to use madvr with h265 10 bits videos? Because if i play it i cant get hardware decoding. The only way to make it works it´s with EVR

[edigee]

Is it possible to use madvr with h265 10 bits videos? Because if i play it i cant get hardware decoding. The only way to make it works it´s with EVR
LAV filters ,DXVA copyback should work with no problems. Of course ,it depends of the HEVC decoding capabilities of the GPU.
AFAIK the only cards that can do that are GTX 960/950. Maybe there are other platforms( integrated GPU/CPU) ,laptops that can do hevc hardware decoding.

[KoD]

Which exact GPU are you using when getting the DX11 frame jumping? And which OS are you using? And which discrete GPU are you using when *not* connecting your display to the mainboard HDMI output?

The CPU is a i7 4790K (HD4600 iGPU), and the discrete GPU is a GTX 980. OS is Win 8.1 64 bit.

What I looked for was to have QS doing the decoding and deinterlacing, and having the GTX 980 doing the resizing. But decoding + deinterlacing on QS was not working properly when using the dGPU for output (as soon as I skipped in the file, playback started grinding), so I tried to debug what was going on, and connected the TV directly to the motherboard HDMI port. That's how I encountered all of the above.

madshi, is it certain that the bat file really resets all the settings? And that it does not have to be run as administrator? Because I did not have the issue with subtitle positioning on this same file when I used 0.89.15 with the ar fixed dlls. But the issue started happening after switching to 0.89.16 (and doing a reset settings to defaults). Then I switched back to 0.89.15 + the ar fixed dlls, did a reset to defaults, and still had the subtitle issue. Finally, I started modifying all the settings in madVR, just enabling stuff, pressing Apply, and then disabling them back, and after some such random operations the subtitle issue was gone again with 0.89.15. But I also started doing the same in the Intel Graphics CP (like disabling the 3D Application Optimal Mode setting - when this is enabled the madVR OSD flickers badly on that file), so I'm not sure what fixed the positioning. The frame jumping back and forth is still there though, as are the issues when using the GTX 980 for output and QS for decoding and deinterlacing.

[aufkrawall]

Well, 2X SR improves edges clarity so how about another extra pass on top of it for instance? Most users here don't care about increased GPU load if PQ improves.

Are you sure you wouldn't achieve a 99% identical result with sharpen edges on top of current SR?

[ace960]

When next file in a directory is played then the audio is played only. The video freezes at the previous file.

Does the media player still react to mouse and keyboard events? E.g. can you press the stop button and it reacts?

The media player reacts. When I press stop button then it is pressed and the sound of the actual file stops playing. The video remains frozen until I close mpc-hc.


Please activate debug mode, then reproduce the problem, let the media player stay frozen for 20 seconds, then please terminate it with the task manager. Then please zip and upload the log somewhere (not here) for me to look it. Please also create 2-3 freeze reports (Ctrl+Alt+Shift+Pause/Break). Thanks.

I uploaded zipped log and the freeze reports to http://ulozto.net/xPtByEpB/madvr-zip.
note: I generated the files with mpc-hc 1.7.10.5 and madvr 0.89.17.

[rodrigo081089]

LAV filters ,DXVA copyback should work with no problems. Of course ,it depends of the HEVC decoding capabilities of the GPU.
AFAIK the only cards that can do that are GTX 960/950. Maybe there are other platforms( integrated GPU/CPU) ,laptops that can do hevc hardware decoding.

Thanks, that fixed my problem. Now it works awesome

[Sp00kyFox]

Could be worth a try. It does look like it tends to produce heavy aliasing in some situations, but maybe it could be improved by running some sort of smoothing filter. I don't have time for this atm, though.
fair enough. I also created a thread in the avisynth development area (http://forum.doom9.org/showthread.php?p=1746651) for it, if anyone feels up to the task. I'm not an avisynth dev so I can't do it myself but at least I shared the word. maybe it's possible to implement it as an avisynth script though. like I said, pseudocde does look rather easy.

[leeperry]

Are you sure you wouldn't achieve a 99% identical result with sharpen edges on top of current SR?
Well, there currently isn't any version that would allow me to A/B 2X SR Vs 1X SR + edges sharpening but anyway it's all subjective, 2X does improve PQ as proven by my screenshots comparison and at the end of the day I like what SR does to motion-blur. All this said, I didn't see any regular sharpening filter competing with a second pass of SR in .16.

And I'd love to try 2X SR + "thin edges" :devil:

[hu1kamania]

Amazing results Madshi, thank you! Looking forward to testing the optimized filters tonight.

[Magik Mark]

madVR v0.89.17 released

http://madshi.net/madVR.zip

* significant speedups for "sharpen edges" and "thin edges" algos
* small quality change (hopefully improvement?) for "thin edges"
* fixed: repeated frames were reported although smooth motion FRC was on
Some benchmarks on my development PC:

AMD HD7750 1080p24:
AdaptiveSharpen: 6.34ms
sharpen edges v0.89.16: 17.15ms
sharpen edges v0.89.17: 9.87ms
thin edges v0.89.16: 25.62ms
thin edges v0.89.17: 5.41ms

Intel HD4000 720p24:
AdaptiveSharpen: 9.92ms
sharpen edges v0.89.16: 36.2ms
sharpen edges v0.89.17: 15.9ms

NVidia GTX650 1080p24
AdaptiveSharpen: 15.51ms
sharpen edges v0.89.16: 27.59ms
sharpen edges v0.89.17: 11.69ms
And yes, the results are weird on the NVidia compared to AMD and Intel. But I've double checked, the results are correct. So "sharpen edges" is now about 50% slower than AdaptiveSharpen on my AMD & Intel GPUs, but 25% faster than AdaptiveSharpen on my NVidia GPU.

It definitely made rendering faster by at least 10ms. All new enhancements running

[Sp00kyFox]

played around with the new version and some settings. I don't really get the negative feedback for the biliteral chroma upscaling. I tried it with some real world movies and TV shows and it does a great job. using the luma information as a guidance seems to me is the best way to do it, so yeah.. it's a good chroma scaler in my eyes.

another thing. can it be that the nnedi3 implementation in madvr doesn't correct the center shift that the algorithm introduces? if I pause a frame and activate luma doubling with nnedi3 (neuron number doesn't matter) I can cleary see that the image content slightly moves.

[Shiandow]

anyways, I wanted to draw some attention to a new developed downscaling algorithm by ETH Zurich which is called "Perceptually Downscaling" which gets pretty amazing results:
https://graphics.ethz.ch/~cengizo/imageDownscaling.htm

I think that could be a gread addition to madvr. according to the paper the performance of is pretty good (they implemented it with matlab) and it should be possible to execute it in realtime. it seems that they didn't publish the code though but the algorithm is explained in the paper and presented in pseudo code, it only consists of convolutions and sums.

That looks interesting, bit worried about the 'patent pending' though.

not sure about this downscaler. is sharper than bicubic but terrible aliased at the same time.

just look at the hair.

the comparison algorithm are pretty weak to say it friendly.

I concur, if I have to limit myself at the images displayed on that page, the algorithm seems pretty terrible, to be honest. I like a sharp image, but judging from these images one is supposed to watch a Minecraft game, not a movie. :)

Edit: pictures here are better, though https://graphics.ethz.ch/~cengizo/Files/Sig15HighResNaturalImages.html

Aliasing is still there, but at least I can see a reason for taking it into consideration. They really ought to change the pics in the homepage.

Edit 2: these https://graphics.ethz.ch/~cengizo/Files/Sig15UserStudyImages.html are even better. Interesting...


Could be worth a try. It does look like it tends to produce heavy aliasing in some situations, but maybe it could be improved by running some sort of smoothing filter. I don't have time for this atm, though.

The aliasing you're seeing seems to be because they've upscaled the images using point sampling. Presumably to make it easier to see differences (otherwise you'd have to look at a really small image). The other algorithms are simply too blurry to make the aliasing noticeable.

[har3inger]

played around with the new version and some settings. I don't really get the negative feedback for the biliteral chroma upscaling. I tried it with some real world movies and TV shows and it does a great job. using the luma information as a guidance seems to me is the best way to do it, so yeah.. it's a good chroma scaler in my eyes.


Bilateral scaling can be really really good, but also runs into some exceptions that make it look like trash. Any time you have solid red shapes on solid black, Bilateral has zero luma information to draw on and gets ugly at all the edges (happens on logo/studio sequences before films, rare scenes from certain movies, anime) Furthermore, it sometimes "fills to edges" inappropriately, giving a dramatically different image (chroma bleed) from what ground truth is.

It's just not consistent enough to be left on for all video content. 80-90% of the time it's great, the other 10% it looks so bad I think something's corrupted or broken.

[ryrynz]

madVR v0.89.17 released
* small quality change (hopefully improvement?) for "thin edges"


Looks a lot more accurate to the source now, I'm seeing less haloing and overall better picture quality. The lines are thicker now at the same settings so people may wish to revisit them.

FWIW, applying thinning after upscaling at a setting of 2.0 seems to look basically identical to doing it @ 1.0 every 2x upscale, you save some rendering time doing it this way also.
Crispen edges almost does nothing (certainly nothing good) when refining after upscaling though.

Madshi, do you think crispen edges goes a little too far? > 2.0 starts to look quite bad. > 3.0 makes me wonder why it's even there.

[Anima123]

The aliasing you're seeing seems to be because they've upscaled the images using point sampling. Presumably to make it easier to see differences (otherwise you'd have to look at a really small image). The other algorithms are simply too blurry to make the aliasing noticeable.

Shiandow, it make me curious if it helps with the overall quality once a good downscaler was adopted into SuperRes.

[madshi]

Thanks for the performance optimizations.
With GTX 980, now Adaptive Sharpen and sharpen edges roughly seem to be equally fast. Adaptive Sharpen is still slightly faster, I get a little lower GPU usage and rendering time with it. So there seems to be a difference to Kepler.
Yep, but roughly equally fast is still different to Intel & AMD, where sharpen edges is still 50% slower than AdaptiveSharpen.

Regarding SuperRes again:
I read here that SR would produce an overly sharp picture to some when doing 1080p -> WQHD. However, I can hardly believe this. I made a sharpness comparison with a game screenshot rendered in WQHD, and despite of SR with strength of 3 on top of Jinc AR scaling, it's still noticeably softer than the original (and much closer to ground truth than just Jinc AR without SR)
Yes, I agree. The only thing that sometimes bothers me is that SuperRes makes the edges sharper but leaves textures untouched. That can make the image look comic like sometimes. But I hope adding a bit of "enhance detail" in the latest madVR build might help there. (Should probably only be used on good sources, though.)

Well, 2X SR improves edges clarity so how about another extra pass on top of it for instance? Most users here don't care about increased GPU load if PQ improves.
You can't run SuperRes without upscaling first. So this would have to be done by using SuperSampling. I'm not sure if doing that would be worth the extra performance hit. I'd rather try the 4 new sharpening algos instead.

Thanks a lot for .17 but apparently 2X is still disabled for SR only? Based on my screenshots comparison, you seemed to agree that 2X SR improves PQ so why not? I got no use for 1X only SR :(
Do you have the power to make my days have 30 hours instead of 24? If not, then please try to have a bit of patience.

The CPU is a i7 4790K (HD4600 iGPU), and the discrete GPU is a GTX 980. OS is Win 8.1 64 bit.

What I looked for was to have QS doing the decoding and deinterlacing, and having the GTX 980 doing the resizing. But decoding + deinterlacing on QS was not working properly when using the dGPU for output (as soon as I skipped in the file, playback started grinding), so I tried to debug what was going on, and connected the TV directly to the motherboard HDMI port. That's how I encountered all of the above.
Ok. The combination of Intel GPU with newer OSs and DX11 is known to sometimes cause this frame jumping in madVR. Unfortunately I can't reproduce it on my development PC with my HD4000. I hope this will change once I upgrade to win10, but I don't know for sure yet.

I'll try to reproduce the QS specific problems later, when I find some time.

is it certain that the bat file really resets all the settings? And that it does not have to be run as administrator?
It works for me, but there's no guarantee. You can double check yourself: The bat file should delete the "settings.bin" file in the madVR folder (if there is one), and it should delete the settings in "HKCU\Software\madshi\madVR". So if you want to be certain, double check that the bat file does all that correctly.

The frame jumping back and forth is still there though, as are the issues when using the GTX 980 for output and QS for decoding and deinterlacing.
A quick guess on my part: I once had a user reporting problems and after some digging we found out that uploading to the GPU was extremely slow for him, when uploading to RGB textures, but was much faster when uploading to NV12 surfaces. In order to test this, please activate DXVA scaling and zoom into the image a bit, so that DXVA scaling actually gets active. This forces madVR to upload the frames as NV12. When doing that, do the QS problems go away? The subtitle positions are probably a completely different topic.

I uploaded zipped log and the freeze reports to http://ulozto.net/xPtByEpB/madvr-zip.
note: I generated the files with mpc-hc 1.7.10.5 and madvr 0.89.17.
Thanks, will have a look at this later...

can it be that the nnedi3 implementation in madvr doesn't correct the center shift that the algorithm introduces? if I pause a frame and activate luma doubling with nnedi3 (neuron number doesn't matter) I can cleary see that the image content slightly moves.
Fixing the center shift costs performance and quality, and you would never notice the shift when you don't compare to a different algorithm, so that's why madVR does *not* fix the center shift. The situation changes when you use SuperRes because SuperRes requires the center to be unshifted. So in that case madVR does correct the center shift. Also, when using quadrupling, madVR uses NNEDI3 in such a way that quadrupling shifts the center into the opposite direction, so the overall center shift after quadrupling is still there, but smaller than when just using doubling.

played around with the new version and some settings. I don't really get the negative feedback for the biliteral chroma upscaling. I tried it with some real world movies and TV shows and it does a great job. using the luma information as a guidance seems to me is the best way to do it, so yeah.. it's a good chroma scaler in my eyes.
Bilateral scaling can be really really good, but also runs into some exceptions that make it look like trash. Any time you have solid red shapes on solid black, Bilateral has zero luma information to draw on and gets ugly at all the edges (happens on logo/studio sequences before films, rare scenes from certain movies, anime) Furthermore, it sometimes "fills to edges" inappropriately, giving a dramatically different image (chroma bleed) from what ground truth is.

It's just not consistent enough to be left on for all video content. 80-90% of the time it's great, the other 10% it looks so bad I think something's corrupted or broken.
If it were just the problems with a flat luma channel, I might invest the work to detect that situation and switch to a different algo in that case. But the inappropriate "edge filling" which sometimes occurs is a problem that I don't know how to solve. Which is why I haven't tried to improve Bilateral chroma upscaling yet. Maybe Shiandow has some ideas there? I'm not sure if anything can be done there, though. Maybe it's not possible to solve the inappropriate "edge filling" without removing all the benefits Bilateral has in other situations.

That looks interesting, bit worried about the 'patent pending' though.
Yep, noticed that, too. Was planning to wait until they have code available (website says "soon") and then to email them to ask about the license/patent situation.

The aliasing you're seeing seems to be because they've upscaled the images using point sampling. Presumably to make it easier to see differences (otherwise you'd have to look at a really small image). The other algorithms are simply too blurry to make the aliasing noticeable.
Oh, that could be an explanation. In any case, it's worth a try, if the license situation allows it.

Looks a lot more accurate to the source now, I'm seeing less haloing and overall better picture quality.
Glad to hear that!

So how does it compare to warpsharp now?

FWIW, applying thinning after upscaling at a setting of 2.0 seems to look basically identical to doing it @ 1.0 every 2x upscale, you save some rendering time doing is this way also.
Yes, I just wish the other algos had the same properties, but they don't. Which makes it hard for me to streamline all the options into something less complex.

Madshi, do you think crispen edges goes a little too far? > 2.0 starts to look quite bad. > 3.0 makes me wonder why it's even there.
I haven't really done full comparisons myself yet. crispen edges has a somewhat different look to it which I generally like, but with high settings it does start to fall apart. Maybe I can improve it in the future, I don't know...

[ryrynz]

Glad to hear that!
So how does it compare to warpsharp now?


It's looking like a pretty good alternative, there's some line warping properties from the last version that look closer to the original source so I think it could still use a little tweaking but this was a step in the right direction. I'll send you some screenshots etc.

[hubblec4]

Hi madshi

I have posted in the LAV thread (http://forum.doom9.org/showthread.php?p=1746830#post1746830) an issue.

With the new madVR0.89.17 play the DVD Aliens(test dvd without the big-vobs (http://forum.videohelp.com/attachments/34047-1444852648/Aliens.7z)) with 4:3 format (menu and video), but it should be 16:9. With EVR it plays fine with 16:9 format.

[leeperry]

The only thing that sometimes bothers me is that SuperRes makes the edges sharper but leaves textures untouched. That can make the image look comic like sometimes. But I hope adding a bit of "enhance detail" in the latest madVR build might help there.
That's especially what I like about it I think as edges are sharper but not the picture itself, that does wonderful things to fast motion but yes you're right as it gives a "comic-like" look indeed but it's sooo smooth and it's not like NNEDI3 didn't look artificial on its own so at some point you have to find an acceptable compromise between sharpness and naturalness I think and as much as the thin look of the latter kinda bothers me SR appears right up my alley :)

You can't run SuperRes without upscaling first. So this would have to be done by using SuperSampling. I'm not sure if doing that would be worth the extra performance hit.
Ouh, wet dreaming of SS SR now and I need an excuse to buy a new GPU anyway ^^

Do you have the power to make my days have 30 hours instead of 24? If not, then please try to have a bit of patience.
Yeah so sorry I figured afterwards that a checkbox on the end-user side could mean a lot of changes in the back with those four new options and all :o

Yep, noticed that, too. Was planning to wait until they have code available (website says "soon") and then to email them to ask about the license/patent situation.
Ouh, this would be most promising in mVR if any possible.

[Thunderbolt8]

the perceptual downscaling image comparisons look interesting!

[Shiandow]

Shiandow, it make me curious if it helps with the overall quality once a good downscaler was adopted into SuperRes.

It's kind of difficult to predict what effect a different downscaler will have, especially a non-linear one. It might also be the wrong way to go about it, I think there's a better way to use their ideas in SuperRes.

Basically, the basis of their algorithm is similar to SuperRes; they generates a resized image which is in some way 'similar' to the input. In their case they generate a smaller image, instead of a bigger image (which means that there's a unique solution). But they also use a different way to measure similarity. It would make more sense to see if SuperRes could use their method to measure similarity, which would actually remove the need to use a downscaling algorithm at all.

Coincidentally, if you use SuperRes's method for measuring similarity to downscale an image you'd get the 'box' downscaling algorithm, which isn't that bad, but Mitchell-Nertavalli is easily better.

[XMonarchY]

Will SuperRes for Chroma Upcaling be updated any time soon? Last I heard SuperRes was updated only for Upscaling Refinement, not Chroma Upscaling.

[avinab]

@madashi in the latest build nnedi3 is not working.even if nnedi3 is selected in chroma upscaling or image doubling,it is showing JINC3.
But super-xbr is working.please look into it.

[huhn]

what is your OS, GPU and driver version?

[KoD]

the frame jumping back and forth is still there though, as are the issues when using the gtx 980 for output and qs for decoding and deinterlacing.
a quick guess on my part: I once had a user reporting problems and after some digging we found out that uploading to the gpu was extremely slow for him, when uploading to rgb textures, but was much faster when uploading to nv12 surfaces. In order to test this, please activate dxva scaling and zoom into the image a bit, so that dxva scaling actually gets active. This forces madvr to upload the frames as nv12. When doing that, do the qs problems go away? The subtitle positions are probably a completely different topic.
Hi madshi, you were right, the settings are reset, both the file and the registry entries get deleted, so that's not the reason why the subtitle positioning got fixed. In fact, after doing what you asked me, I can now say with certainty that the subtitle positioning issue happens only when DXVA is used for the scaling of the luma and of the chroma. If DXVA is not used for scaling, then the subtitles are at their proper location in the image. It's as if DXVA is used for scaling, then the subtitles are kept at their original position in the frame in absolute values, which is now somewhere in the middle-left part of the image. And yes, using software decoding + DXVA scaling also shows this issue; it's not related to QuickSync.

When "Enable Adaptive HW deinterlacing = disabled" for QuickSync, then the frames were sent interlaced to the renderer and since "use DXVA chroma upscaling when doing DXVA deinterlacing = enabled" after a madVR reset settings, madVR was using DXVA to do the scaling and that's why the subtitle positioning issue was always happening in this configuration.

These are my results when using QuickSync for decoding, but the resizing and video output are performed by the GTX 980:

1. With QuickSync "Enable Adaptive HW deinterlacing = disabled", DXVA is always used for deinterlacing, and these are the playback results:
- DX9 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma Bicubic + chroma Bicubic -> luma Bicubic + chroma Bicubic are used, and there are no issues with switching fullscreen and back, nor with skipping in the file, nor with subtitle positioning
- DX9 + "use DXVA chroma upscaling when doing DXVA deinterlacing = enabled" + luma Bicubic + chroma Bicubic -> luma DXVA + chroma DXVA are used, and there are issues only with subtitle positioning; I find it strange that luma DXVA is used in this configuration - is this normal?

- DX11 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma Bicubic + chroma Bicubic -> luma Bicubic + chroma Bicubic are used, and just like for DX9 there are no issues whatsoever
- DX11 + "use DXVA chroma upscaling when doing DXVA deinterlacing = enabled" + luma Bicubic + chroma Bicubic -> just like for DX9, luma DXVA + chroma DXVA are used, and there are issues only with subtitle positioning

2. With QuickSync "Enable Adaptive HW deinterlacing = enabled", QuickSync is doing the deinterlacing, and these are the playback results:
- DX9 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma Bicubic + chroma Bicubic -> luma Bicubic + chroma Bicubic are used, and if you just wait after opening the file, the sound plays but no image, the madVR buffers stay at 1; if you get impatient and start skipping in the file, past the scrolling credits part towards the talking heads segment, you might get lucky and get unstuck and playback is smooth as if there's nothing wrong; skipping again might prove lucky, or you might get stuck again on some frame freeze, and you might even get in a state where the sound gets moving but the image is like refreshing one frame every few seconds or so, or not recovering at all, but you can close the playback in the player without issues (so the player does not freeze); no issues with subtitle positioning
- DX9 + "use DXVA chroma upscaling when doing DXVA deinterlacing = enabled" + luma Bicubic + chroma Bicubic -> same as above

- DX11 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma Bicubic + chroma Bicubic -> same as above
- DX11 + "use DXVA chroma upscaling when doing DXVA deinterlacing = enabled" + luma Bicubic + chroma Bicubic -> same as above

- DX9 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma DXVA + chroma Bicubic -> luma DXVA + chroma DXVA are used (again, is this normal?), and there are all the issues above + the issue with subtitle positioning
- DX11 + "use DXVA chroma upscaling when doing DXVA deinterlacing = disabled" + luma DXVA + chroma Bicubic -> same as for DX9


I think it's fair to also say what the results are with EVR CP:
- with QuickSync "Enable Adaptive HW deinterlacing = disabled" -> no issues whatsoever
- with QuickSync "Enable Adaptive HW deinterlacing = enabled" -> I get the same issues as with madVR (in this configuration with the GTX 980 for output; there were no issues when the iGPU was used for output!)

Hi madshi

I have posted in the LAV thread an issue.

With the new madVR0.89.17 play the DVD Aliens(test dvd without the big-vobs) with 4:3 format (menu and video), but it should be 16:9. With EVR it plays fine with 16:9 format.
I confirm there is an AR issue for DVD playback myself, with a DVD of my own. But this is not new, it's here since at least 0.89.15 (did not test further back).

The copyright notice is 4:3 and is displayed properly, the studio intro is also 4:3 and no issues, and then the menu appears, which is a static image, and should be displayed as 16:9, but madVR displays it as 4:3. In EVR it's 16:9. No sample from me, but I think there needs to be the two videos included as well to reproduce this, and I don't know how to cut the vobs with the intro videos in.

[Knight77]

Warner306 in relation to the new scalers which are the settings you suggest now for 720p to 1080p and 1080p to 1080p?

[Warner306]

Warner306 in relation to the new scalers which are the settings you suggest now for 720p to 1080p and 1080p to 1080p?

I gave a response to this in the Kodi forums. These sharpeners are new to me, so I would ask others instead of relying on settings posted months ago. I will continue to change my preference into the future and find it is harder to recommend specific settings with these shaders.

So I don't know what is best.

I used to use SuperRes alone, but might try adding enhance detail in the future.

[Knight77]

I gave a response to this in the Kodi forums. These sharpeners are new to me, so I would ask others instead of relying on settings posted months ago. I will continue to change my preference into the future and find it is harder to recommend specific settings with these shaders.

So I don't know what is best.

I used to use SuperRes alone, but might try adding enhance detail in the future.

Thanks Warner306 and please don't get my question wrong or feel bothered about it: thanks you in particular I tried and get focused on some very good settings so to me you are like a trustable source. Obviously I'll check other options too, but by now looks like sharpen edges (0.2) and enhance detail (0.5) are a good starting point.

[digitech]

@Madshi can you pls give us a proper MadVR profile for 1.- 1080p, 2.- 720p and 3.- below 720p (SD) profile for all kinds of video files.., i have been trying lots of combinations but always thing get mixed up, sd files playing in hd and viceversa

[JarrettH]

I just updated to the latest from 88.21

Zoom control...a little help?...

Do I need to use this if I'm just playing video files on a computer and watching via computer monitor? I want to maintain the aspect throughout the movie.

Using touch window from inside in MPC

:thanks:

[Ver Greeneyes]

If that new SSIM-based downscaling algorithm can be used, would be interesting to see how it affects SuperRes (which relies on downscaling, right?).

[Warner306]

Thanks Warner306 and please don't get my question wrong or feel bothered about it: thanks you in particular I tried and get focused on some very good settings so to me you are like a trustable source. Obviously I'll check other options too, but by now looks like sharpen edges (0.2) and enhance detail (0.5) are a good starting point.

Low values with image enhancements are important. Higher values should only be used when upscaling by using sharpening applied from upscaling refinement.

[Warner306]

I just updated to the latest from 88.21

Zoom control...a little help?...

Do I need to use this if I'm just playing video files on a computer and watching via computer monitor? I want to maintain the aspect throughout the movie.

Using touch window from inside in MPC

:thanks:

These settings are primarily aimed at users with Constant Image Height (CIH) projection set-ups, so it would be unlikely you would have to change anything if you didn't use these settings before.

[huhn]

these settings are very useful for 4/3 sources with 16/9 content on a 16/9 screen. just as an example.

screen config is for CIH projectors

[JarrettH]

This must be a bug...

https://c1.staticflickr.com/1/759/22458122143_6e8a969572_o.jpg

I don't have Lanczos3 AR enabled, I have Jinc3 AR ticked. To check, I tried every upscaler, they work, but when Jinc3 AR is chosen it uses Lanczos3 AR instead.

Also, can someone explain what's happening in this chain?

720x576 video --> 1680x1080

chroma -> super-xbr
image y -> super-xbr
image x -> super-xbr <- catmull-rom

The height was doubled and downsized - what happened to the width? It looks like it was doubled, but what upscaler brought it to 1680?

:thanks: for the zoom control feedback

[huhn]

jinc3 AR can't downscale so lanczos is used. working as intended.

even through the images shows upscaling the image is still downscaled at the same time AR correction.

BTW. this file need deintelracing and is flag wrong or totally wrong encoded.

[JarrettH]

Yeah, it was encoded in the video. It looks terrible :D

Thanks

[chros]

Zoom control...a little help?...

Do I need to use this if I'm just playing video files on a computer and watching via computer monitor? I want to maintain the aspect throughout the movie.

Using touch window from inside in MPC
The "crop black bars" feature is useful, since it's applied before scaling, so scalers have less job with contents that have black bars.
And the top checkbox on this tab (I don't remember what it's called), that allows not to upscale contents if the cut lines are just couple of lines, is really useful with cropping and with some strange sources (like web download files that are sometimes not perfect full hd).

[avinab]

what is your OS, GPU and driver version?

Os is windows 10 build 1511 ,nvidia gtx 980 and latest nvidia driver version 358.91 .NNEDI3 was working on madvr 0.89.16. but in 0.89.17 NNEDI3 is not enabled.It goes to JINC

[Shiandow]

If that new SSIM-based downscaling algorithm can be used, would be interesting to see how it affects SuperRes (which relies on downscaling, right?).

While I agree it would be interesting, it might not necessarily be the best way to implement this SSIM downscaling algorithm. For more information see my previous comment (http://forum.doom9.org/showpost.php?p=1746841&postcount=34288).

If it were just the problems with a flat luma channel, I might invest the work to detect that situation and switch to a different algo in that case. But the inappropriate "edge filling" which sometimes occurs is a problem that I don't know how to solve. Which is why I haven't tried to improve Bilateral chroma upscaling yet. Maybe Shiandow has some ideas there? I'm not sure if anything can be done there, though. Maybe it's not possible to solve the inappropriate "edge filling" without removing all the benefits Bilateral has in other situations.

I think preventing this 'edge filling' entirely will require a different algorithm, you'll need to detect edges in a somewhat more intelligent manner. Although, if you don't mind doing an extra processing pass, I did manage to get some nice results if I combine a somewhat modified Bilateral scaler it with a softened SuperRes pass, that's still highly experimental though.

[Ver Greeneyes]

While I agree it would be interesting, it might not necessarily be the best way to implement this SSIM downscaling algorithm. For more information see my previous comment (http://forum.doom9.org/showpost.php?p=1746841&postcount=34288).
Thanks, I missed that :)

[iSunrise]

...If it were just the problems with a flat luma channel, I might invest the work to detect that situation and switch to a different algo in that case. But the inappropriate "edge filling" which sometimes occurs is a problem that I don't know how to solve. Which is why I haven't tried to improve Bilateral chroma upscaling yet. Maybe Shiandow has some ideas there? I'm not sure if anything can be done there, though. Maybe it's not possible to solve the inappropriate "edge filling" without removing all the benefits Bilateral has in other situations....
Do you happen to have some screenshots lying around that show some of the mentioned problems? Personally I absolutely love the bilateral chroma upscaling from madVR in the screenshot comparisons I did some months ago, especially when you take into account the very low performance cost. I took some examples from recorded Amiga games (lots of pixelation), real world stuff (movies, TV shows and very old video camera recordings) and also artificially created graphics from the PC demo scene.

I found some research on the web about an adaptive bilateral upscaling filter implementation, maybe you can use or at the very least borrow some ideas (if you don't know the paper already):
http://www.interdigital.com/research_papers/2012_01_07_adaptive_bilateral_filter_for_video_and_image_upsampling (direct download (http://www.interdigital.com/download/54313edae262284525000224))

Apart from NNEDI3 even the simple bilateral upscaler is definitely my favourite chroma upscaler.

[fredgido]

Love the thin edges enhancement!:thanks: On anime it makes the outlines look so sharp (I set it to maxium)! Shapen and crispen looks slightly better on very moderate amounts. On anime, 1080p BD encodes it improves more consistently a lot more than on 480p dvd rips from what I tested. Looks like it is more hard to detect edges on low resolution blurry anime. Enhance detail only makes encoding artifacts more visible.

I will use the enchantments and try to tweak when watching anime and try different quality and art animes for some time then post my experience.