madVR - high quality video renderer (GPU assisted)

[vivan]

So, where does it run right now? What HW (CPU/GPU), what part of HW (if not GPU shaders).Shaders. He is talking about "custom shaders" interface. It allows to execute only single shader (but you can chain them) with input = previous output and without ability to change resolution. Those limitations is the problem, not HW.
Shiandow's deband runs in several passes: first n passes downscale image by a factor of 2 each, last one uses all of downscaled images as input (or last n passes, each using n-th image, dunno).

[Hyllian]

Why don't you use your super-xbr twice, like 2*2x = 4x? That's what I'm doing with NNEDI3/NEDI, too.

Ok, for your wishes --> http://i.imgur.com/C1xv6Pr.png

That's using the shaders two times. The anti-ringing is ON two times!



Ouch, looks very ugly without anti-ringing!

I like it, besides the rings, of course. It's so much more organic! I think the anti-ringing destroys some textures beyond the rings, kkk. You should use jinc as a second pass.



Which license does your super-xbr algo (and the shaders) come with? I might be interested trying it for madVR, but it depends on the license. Thanks!



For now it's GPL, though I don't mind to change it for any use. I chose that only because Retroarch was using it.

[ryrynz]

Ok, for your wishes --> http://i.imgur.com/C1xv6Pr.png

That's using the shaders two times. The anti-ringing in on two times!


That does look pretty good to me, I'd be interested in comparing results.

[Anima123]

That does look pretty good to me, I'd be interested in comparing results.

Beside of that, the power consumption does matter, at least in the sense it can be spared for other image enhancement computing.

[blindbox]

I'm sorry to bring up the topic on FreeSync again, but can't madVR force the output FPS (that is, the 3d/DirectX FPS) to be an integer multiple of the video, and let Freesync or G-Sync handle the rest?

i.e. a 24 fps video is converted to 48 fps by frame doubling/repeating, then let the output FPS be equal to the output video FPS (in this case, 48 dx9 fps) and since it falls into the Free/G-Sync working range, Free/G-sync gets activated.

From what I've read with regards to G-Sync and FreeSync, the game developers don't have to do anything to implement them. It's all driver side. The only thing it requires is that the FPS must be in the working range. And the game must be in full screen.

I haven't seen this suggested so I decided to point this out.

[MistahBonzai]

I see lines in both cases on your screenshot, on a PC monitor with guaranteed 4:4:4 reproduction. Maybe it interacts badly with your chroma sampling test?
Same here on HDTV with verified 4:4:4 chroma capability.

[Anima123]

I did some testing of SuperRes for different resolution video files and these settings found to be quite good to my eyes.

For 1024x576 -> 1920x1080, the following settings are used
image doubling: off
Chroma upscaling: Bicubic 75 + AR + SuperRes
image upscaling: Bicubic75 + AR
image downscaling: Catmull-Rom + AR + LL
upscaling refinement: passes 2, medium, strength 0.65, sharpness 0.25, softness 0.25, anti-aliasing 0.15, anti-ringing 0.14, refine the image only once after upscaling is complete


While with 720x406 -> 1920x1080, difference with the former settings as following:
image doubling: NEDI
upscaling refinement: choose refine the image after every ~2x upscaling step produces better result, less blocky (the original compressed video has block artefacts).

Since when 1024x576 -> 1920x1080, if the image doubling was used, it ends up with a DOWNSCALING to reach the screen resolution, while 720x406 ends up with an UPSCALING, I guess similar effect might stands to videos with other resolutions.

With 1024x576 -> 1920x1080 files, the non-double defaults does not produce good quality at least for Bicubic 75 + AR upscaling. Actually I have tried SuperRes with other upscaling algorithms, including bilinear, it seems SuperRes works well with Bicubic and Bilinear, not with other algorithms, to my eyes of course.

One question, if using image doubling with 1024x576 -> 1920x1080 and 'choose refine the image after every ~2x upscaling step' is chosen with upscaling refinement settings, would the refinement been applied twice, or just once after doubling; how exactly it's working with 'refine the image only once after upscaling is complete' set to yes?

[madshi]

Any idea why 1 prepresented frame is being dropped when using D3D11 though? With frames in advance set to 16, 15-15/15 present queue with D3D11, 15-16/16 present queue with D3D9.
That's a limitations of DX11.

Ok, for your wishes --> http://i.imgur.com/C1xv6Pr.png

That's using the shaders two times. The anti-ringing is ON two times!
Thanks! Looks quite promising to me eyes!

I like it, besides the rings, of course. It's so much more organic! I think the anti-ringing destroys some textures beyond the rings, kkk. You should use jinc as a second pass.
In some image areas ringing is beneficial, in others not. I think we would get better results by using your algo once, then sharpening the result in a clever way, then run your algo again, then sharpen it again. The sharpening should have a similar effect to running jinc. Of course the sharpening would need to have some anti-ringing incorporated, too. A combination of LimitedSharpenFaster (AviSynth script) and FineSharp should work well.

For now it's GPL, though I don't mind to change it for any use. I chose that only because Retroarch was using it.
If you could change that to (or dual license it as) MIT or BSD license, that would be most awesome! Of course I would give you proper credit in my readme file (and later in some credits window, if I ever add one, not sure yet). If you find MIT/BSD too generous, LGPL would also work, but if I could choose, I'd *strongly* prefer MIT/BSD.

I'm sorry to bring up the topic on FreeSync again
Please use the advanced search for this thread, search for my user name and "FreeSync", and you should get all the answers you need.

I did some testing of SuperRes for different resolution video files and these settings found to be quite good to my eyes.

For 1024x576 -> 1920x1080, the following settings are used
image doubling: off
Chroma upscaling: Bicubic 75 + AR + SuperRes
image upscaling: Bicubic75 + AR
image downscaling: Catmull-Rom + AR + LL
upscaling refinement: passes 2, medium, strength 0.65, sharpness 0.25, softness 0.25, anti-aliasing 0.15, anti-ringing 0.14, refine the image only once after upscaling is complete


While with 720x406 -> 1920x1080, difference with the former settings as following:
image doubling: NEDI
upscaling refinement: choose refine the image after every ~2x upscaling step produces better result, less blocky (the original compressed video has block artefacts).

Since when 1024x576 -> 1920x1080, if the image doubling was used, it ends up with a DOWNSCALING to reach the screen resolution, while 720x406 ends up with an UPSCALING, I guess similar effect might stands to videos with other resolutions.

With 1024x576 -> 1920x1080 files, the non-double defaults does not produce good quality at least for Bicubic 75 + AR upscaling. Actually I have tried SuperRes with other upscaling algorithms, including bilinear, it seems SuperRes works well with Bicubic and Bilinear, not with other algorithms, to my eyes of course.

One question, if using image doubling with 1024x576 -> 1920x1080 and 'choose refine the image after every ~2x upscaling step' is chosen with upscaling refinement settings, would the refinement been applied twice, or just once after doubling; how exactly it's working with 'refine the image only once after upscaling is complete' set to yes?
I've just double checked my source code to be extra sure what I'm saying is correct:

If you activate "refine after every ~2x upscaling step", madVR splits the upscaling into as many upscaling steps as needed to reach the target resolution. It usually does exact 2x upscaling steps, except for the very last upscaling step, which is done with the exact factor needed to reach the target resolution. For 2x steps it doesn't matter if you use image doubling or not, the behaviour is the same. E.g. if you have image doubling activated, madVR uses NNEDI3/NEDI to double resolution. If you don't have image doubling activated, madVR doubles resolution with your selected upscaling algorithm (e.g. Jinc). Some examples:

scaling factor 1.9x -> upscaling 1.9x + superres
scaling factor 2.2x -> upscaling 2.2x + superres
scaling factor 2.3x -> upscaling 2x + superres + upscaling 1.3x + superres
scaling factor 4.1x -> upscaling 2x + superres + upscaling 2.1x + superres

Please note that the upscaling factors 1.3x and 2.1x mentioned above are not fully correct, but you will probably know what I mean. Also please note that e.g. upscaling by 1.9x might include image doubling + downscaling, if you've activated NNEDI3/NEDI doubling.

If you deactivate "refine after every ~2x upscaling step", madVR first scales in the same way as v0.87.21 would do, and then runs superres only once directly after upscaling (or doubling + up/downscaling) is complete. Superres is skipped in this situation if the overall upscaling factor is smaller than 1.125x.

1024x576 -> 1920x1080 is less than 2x in both directions, so superres will only be applied once, regardless of your settings. So you could leave "refine after every ~2x upscaling step" activated all the time, it would not change anything.

[Anima123]

My observations still stand then, it seems that I don't like image doubling when the scale factor less than 2.0 when the downscaling C-R AR LL involved, which is consistent with my pm to you.

Madshi, maybe a detailed status report for the process would help us better understand it. Right now ctrl+J didn't show things correctly according to you explain in the post above.

[madshi]

Maybe you simply don't like the look of CatmullRom AR LL for downscaling? It does have higher aliasing levels than just CatmullRom AR. Just turn LL off if you don't like it!

[toniash]

@madshi

" Superres is skipped in this situation if the overall upscaling factor is smaller than 1.125x"

:confused: Why?
SuperRes can be very useful as a sharpener even on not upscaled images.

Another question: How processing changes when SuperRes is "applied first"?

[nlnl]

I see lines in both cases on your screenshot, on a PC monitor with guaranteed 4:4:4 reproduction. Maybe it interacts badly with your chroma sampling test?
Yes, you are right. Pioneer 5090 TV (video mode) does have problem with 4:4:4 reproduction.
http://i70.fastpic.ru/thumb/2015/0610/5b/e2b2d2fd62375a972e87cbb0a1c0e45b.jpeg (http://fastpic.ru/view/70/2015/0610/e2b2d2fd62375a972e87cbb0a1c0e45b.jpg.html)

But does Fine Sharp ON, Linear Ligt ON (image enhancement) options magically fix the problem? :)
http://i70.fastpic.ru/thumb/2015/0610/bb/_b5a8fafbf11b2a52f394e0e1c6721cbb.jpeg (http://fastpic.ru/view/70/2015/0610/_b5a8fafbf11b2a52f394e0e1c6721cbb.png.html)

[huhn]

nope you shouldn't see 4:2:2. just a some lines

[James Freeman]

But does Fine Sharp ON, Linear Ligt ON (image enhancement) options magically fix the problem? :)


Heh, No.
How can pre-processing fix a hardware trait? ... It can't.

Going to test the sharpening shaders as soon as I'll have some time.

[nevcairiel]

Yes, you are right. Pioneer 5090 TV (video mode) does have problem with 4:4:4 reproduction.
http://i70.fastpic.ru/thumb/2015/0610/5b/e2b2d2fd62375a972e87cbb0a1c0e45b.jpeg (http://fastpic.ru/view/70/2015/0610/e2b2d2fd62375a972e87cbb0a1c0e45b.jpg.html)

But does Fine Sharp ON, Linear Ligt ON (image enhancement) options magically fix the problem? :)
http://i70.fastpic.ru/thumb/2015/0610/bb/_b5a8fafbf11b2a52f394e0e1c6721cbb.jpeg (http://fastpic.ru/view/70/2015/0610/_b5a8fafbf11b2a52f394e0e1c6721cbb.png.html)

These patterns work by exploiting how subsampling works in the majority of PCs. If a post-processing algorithm changes the pattern, they stop working.

[blindbox]

Please use the advanced search for this thread, search for my user name and "FreeSync", and you should get all the answers you need.


I did. Many times. I've been following freesync on this thread for quite a while. And my question stands. Why not just set the output fps to 48 fps for a 24 fps video, and let the freesync drivers handle the rest? Maybe I'm just unable to comprehend that there's possibly a big difference between video frame delivery and game engine's frame delivery.

Unfortunately, I don't have a freesync monitor to try this myself, possibly by spoofing my refresh rate towards madvr.

[nlnl]

These patterns work by exploiting how subsampling works in the majority of PCs. If a post-processing algorithm changes the pattern, they stop working.
Yes, get it, but post-processing (FS+LL) hides the problem very well.
4:2:2 side of the pattern masked very well, practically only 4:4:4 is visible.:):)
http://i72.fastpic.ru/thumb/2015/0610/55/_865534f7c1f0cc02e5a3bd4c7395b555.jpeg (http://fastpic.ru/view/72/2015/0610/_865534f7c1f0cc02e5a3bd4c7395b555.png.html)

[Barnahadnagy]

And my question stands.

Basically, it would require MadVR to deliver frames when they are needed to be displayed. This works for games (frame rendered then displayed instantly), but MadVR has a timing for the frame. However, MadVR cannot guarantee to send the frame when this time comes (No CPU time, and/or simply not precise enough timers).

This is why we have queues in FSE (apart from more stability), to let the GPU handle the presenting times of frames by using a HW circuit.

TL;DR: You need to present frames exactly when they need to be presented, with no queue. Current systems can't do this.

[vivan]

Yes, get it, but post-processing (FS+LL) hides the problem very well.It doesn't hide the problem, because this image doesn't demonstrate a problem.

Again, this pattern was created for easy subsampling detection. If you change this pattern - then it stops working.

Read what color subsampling is: http://en.wikipedia.org/wiki/Chroma_subsampling

[kalston]

I did. Many times. I've been following freesync on this thread for quite a while. And my question stands. Why not just set the output fps to 48 fps for a 24 fps video, and let the freesync drivers handle the rest? Maybe I'm just unable to comprehend that there's possibly a big difference between video frame delivery and game engine's frame delivery.

Unfortunately, I don't have a freesync monitor to try this myself, possibly by spoofing my refresh rate towards madvr.

madshi has clearly said he is not interested in working on FreeSync/G-Sync support for the time being (and since all the VRR monitors are purely gaming monitors, I think we should respect that and drop the subject for the time being). It can be made to work and it will certainly happen in the future (I read something about FreeSync and WMP in Windows 10) but right now it would most likely be a waste of time for madshi to bother with this.

VRR monitors like my Rog Swift have no issue running at a fixed refresh rate compatible with virtually any content anyway (100, 120, 144, 59, 60, 85...) and I could probably even make a custom resolution if I needed to.

[webs0r]

Barnahadnagy, blindbox already answered this. I'll just explain in different words in case it helps.

Game = don't know when next frame will come (e.g. could be 5ms later, or 60ms)
Video = know exactly the frame spacing (e.g. always 41.6ms for 24fps)

So for video, just match the monitor refresh to the video frame rate (or multiple). Gsync is useless in this case.

A case where it might be useful is if that monitor doesn't support a particular refresh rate. But then you run into that other problem, where if we drove it via software, the software is unlikely to be able to have it hit e.g. 48 Hz exactly. Games just present the next frame when its ready, it doesn't need to worry about "hang on wait until exactly 41.6ms from the last frame, then present". Computers aren't set up to time this precisely that well.

We'd really need those monitors to be able to switch to any arbitrary refresh rate - that would be more useful instead.

[madshi]

" Superres is skipped in this situation if the overall upscaling factor is smaller than 1.125x"

:confused: Why?
SuperRes can be very useful as a sharpener even on not upscaled images.
The original purpose of SuperRes was to enhance/improve the image quality when doing upscaling. One integral part of the algorithm compares the upscaled image to the original image. Doing this doesn't make too much sense if the upscaling factor is very small.

But yes, SuperRes has some algorithms in it which "do something" even if you don't upscale at all. So yes, it could be an option to run SuperRes even if no scaling is performed at all, or even when downscaling. But this is really outside of what SuperRes was originally written for. I do wonder if running dedicated sharpening algorithms wouldn't produce better results when not scaling. Anyway, for now it is as it is. Once we've dumbed all the settings down to low/medium/high, we can discuss whether it might make sense to allow SuperRes also when not upscaling, but let's first reduce all those options, to make image quality comparisons easier.

Another question: How processing changes when SuperRes is "applied first"?
It doesn't change much. This option only defines in which order FineSharp, LumaSharpen and SuperRes are applied, if you have enabled more than one of those 3 options in "upscaling refinement". That's all.

[Hyllian]

If you could change that to (or dual license it as) MIT or BSD license, that would be most awesome! Of course I would give you proper credit in my readme file (and later in some credits window, if I ever add one, not sure yet). If you find MIT/BSD too generous, LGPL would also work, but if I could choose, I'd *strongly* prefer MIT/BSD.

There you have it --> https://github.com/libretro/common-shaders/tree/master/xbr/super-xbr

MIT license from now on! I was already willing to do it for some time.

[sandro08]

Hello everyone

I am new to the forum and I am the last few years
I ask your help for madVR passage 10 bit
I made some screenshots for you to see my parametrage
my graphics card is a nvidia 970gtx and my diffuser is a w1070 benq I think to be consistent after the explanations given here
My problem is the following

I can not seem to have the 10-bit option in the control panel nvidia, I can only access 8 bits
I plug in hdmi

http://img11.hostingpics.net/pics/7171532015060512h2043.png (http://www.hostingpics.net/viewer.php?id=7171532015060512h2043.png)
http://img11.hostingpics.net/pics/2820772015060512h3009.png (http://www.hostingpics.net/viewer.php?id=2820772015060512h3009.png)
http://img11.hostingpics.net/pics/9163732015060512h3032.png (http://www.hostingpics.net/viewer.php?id=9163732015060512h3032.png)
http://img11.hostingpics.net/pics/5422412015060513h1300.png (http://www.hostingpics.net/viewer.php?id=5422412015060513h1300.png)
http://img11.hostingpics.net/pics/61481820150605121501.jpg (http://www.hostingpics.net/viewer.php?id=61481820150605121501.jpg)

excuse me if the question was asked already

friendly

[huhn]

your screen doesn't support 10 bit.

maybe a limitation of your HDMI cable but very unlikely.

[sandro08]

your screen doesn't support 10 bit.

maybe a limitation of your HDMI cable but very unlikely.

ok though when I did the test ca looks ok
cacle my hdmi is 1.3

thank you

[yok833]

ok though when I did the test ca looks ok
cacle my hdmi is 1.3

thank you

I have the same issue with my Panasonic plasma St50 which is quite recent too... It can only support 12bit with 23hz but no option available with 60hz (I tried with an hdmi cable highspeed 2.0)... So I guess my TV cannot support it... :(

[Della]

your screen doesn't support 10 bit.

maybe a limitation of your HDMI cable but very unlikely.

Just curious, how can you tell that from the information he provided?
Simply because the CP doesn't present the option?
Thanks

[Shiandow]

In some image areas ringing is beneficial, in others not. I think we would get better results by using your algo once, then sharpening the result in a clever way, then run your algo again, then sharpen it again. The sharpening should have a similar effect to running jinc. Of course the sharpening would need to have some anti-ringing incorporated, too. A combination of LimitedSharpenFaster (AviSynth script) and FineSharp should work well.

Might be interesting to combine super-xBR with SuperRes (or one of it's variations), I suspect that those two will combine especially well.

[Hyllian]

Might be interesting to combine super-xBR with SuperRes (or one of it's variations), I suspect that those two will combine especially well.

super-xbr has a internal param that let you control the level of edge detection. In those images it's set to 1.0. If I had set it to 0.0, it would try to find edge even in grass and other textures. I think it has the same objective of SuperRes, right?

[Shiandow]

super-xbr has a internal param that let you control the level of edge detection. In those images it's set to 1.0. If I had set it to 0.0, it would try to find edge even in grass and other textures. I think it has the same objective of SuperRes, right?

Not exactly, the objective of SuperRes is to upscale an image such that downscaling it will give the original image. It then imposes some constraints on the image to make this process well defined and numerically stable.

So far one of the biggest problems has been that this process doesn't necessarily remove aliasing. If done well it doesn't add any, but removing it still seems a bit difficult. However it is pretty good at recovering detail. So the idea is that if you combine it with an algorithm which is good at removing aliasing, you get an image with low aliasing and good sharpness.

[Hyllian]

Not exactly, the objective of SuperRes is to upscale an image such that downscaling it will give the original image. It then imposes some constraints on the image to make this process well defined and numerically stable.

Interesting. Though I think it's a strange objective, because I don't know if it can guarantee you'll get a good result. Depending on the downscale method, a nearest neightbor upscaled image would be a perfect result for SuperRes.

[madshi]

Interesting. Though I think it's a strange objective, because I don't know if it can guarantee you'll get a good result. Depending on the downscale method, a nearest neightbor upscaled image would be a perfect result for SuperRes.
It does sound a bit strange, but as you say it depends a lot on the algorithm used for downscaling, and on the algorithm used to upscale the difference/error comparison result back to the upscaled image size. If you use linear sampling for downscaling and nearest neighbor for upscaling the difference/error, then probably the perfect SuperRes result would be nearest neighbor. But if you use better downscaling/error upscaling algorithms, that is no longer true. E.g. imagine you downscale the NNEDI3 (or your) image back to the original image size, using a decent downscaling algorithm. The difference to the original image will be very *very* small. So you take that very small difference, scale it back up to the upscaled image and apply it to the upscaled image. It does not really result in nearest neighbor at all, if you upscale the error/difference with a good algorithm. E.g. using Jinc upscaling, the upscaled difference should be pretty much free of aliasing. The only situation where the difference gets bigger is if the original upscaling algorithm made some really bad decisions somewhere. E.g. this whole concept works quite well to remove some of the directional artifacts of the NEDI algorithm (not NNEDI3, but NEDI).

That said, I'm not totally convinced yet that SuperRes is better than "simple" good sharpening, when starting with a good artifact-free upscaling algorithm. Still need to do some more investigation into the matter. SuperRes is a relatively "young" algorithm.

@Shiandow, what I've found is that the SuperRes downscaling/difference/upscaling sequence has somewhat similar effects to an unsharp mask. The downscaling/upscaling is somewhat comparable to the gaussian blur in the unsharp mask. Which means that the upscaled image is sharpened by SuperRes, even if you set sharpness to 0.0, and also gets some ringing. It might make sense to include some anti-ringing surpression right when applying the difference to the upscaled image, instead of relying on the extra anti-ringing pass. It's usually better for image quality to avoid adding ringing instead of trying to remove it afterwards. Just a thought, though...

[madshi]

P.S: @Hyllian, thank you very much for relicensing your algorithm & shaders under MIT, that's very generous! :) I'll try to implement your shaders in madVR during the weekend.

I suppose your images in this thread have been created with passes 0-1, is that correct? Or have you used passes 0-2 for those?

[Hyllian]

P.S: @Hyllian, thank you very much for relicensing your algorithm & shaders under MIT, that's very generous! :) I'll try to implement your shaders in madVR during the weekend.

I suppose your images in this thread have been created with passes 0-1, is that correct? Or have you used passes 0-2 for those?

They were created with the three passes, though the third pass is optional.

I think the third pass improves a bit the IQ, in exchange of power, of course. I don't know if it's worth. Anyway, it's optional and can be skipped if power is a problem.

As you'll see, my implementation has much room for optimization. I calc the luminance inside all shaders and it could be done in a first pass (see NEDI shaders, for example) only.

The first pass must work at 2x scale factor (the same way NEDI shaders), and the second and third passes work at 1x scale factor. Use nearest sampling on all texture lookups, as it doesn't need precise texture sample locations.

You'll see too that my blendings aren't done in linear light. I didn't test this algorithm in linear light.

[ryrynz]

P.S: @Hyllian, thank you very much for relicensing your algorithm & shaders under MIT, that's very generous! :) I'll try to implement your shaders in madVR during the weekend.


Awesome Hyllian, looking forward to the test build.

[MysteryX]

Ok, for your wishes --> http://i.imgur.com/C1xv6Pr.png

That's using the shaders two times. The anti-ringing is ON two times!
Personally I find the image unnatural. Too "cartoonish" for my taste. But then that probably works for Mario Bros :D

That said, I'm not totally convinced yet that SuperRes is better than "simple" good sharpening, when starting with a good artifact-free upscaling algorithm. Still need to do some more investigation into the matter. SuperRes is a relatively "young" algorithm.
Simple good sharpening doesn't work with material that has noise and artifacts in it. SuperRes works GREAT to neutralize noise and artifacts while bringing out the real details. Even when using low-quality material with bilinear resizing, with SuperRes it still looks good. That can't be denied.

To the point that I really need to find a way to use SuperRes in AviSynth. madVR does a GREAT job but is limited by what my computer can handle in real-time. I developed a software that allows easy re-encoding of videos through AviSynth by applying EEDI3, NNEDI3, denoise, sharpening, SVP and other things as needed to greatly enhance the quality of the video; and then re-encode at higher quality. But... madVR has gone beyond what is available in AviSynth!

[MysteryX]

Hey I have an idea to simplify SuperRes. Instead of having all these settings and asking whether to apply once or every 2x scaling, why not simply ask how many passes he wants to run? If he runs 2 passes with a 3.5x upscaling factor, it will do 1 pass mid-way and 1 pass after. The user generally selects the number of passes based on what his graphic card can handle, but he really doesn't need to know when each pass actually gets applied. You can determine the best distribution of passes yourself.

[MysteryX]

One more thing. This is a bit off-topic but the tool I designed to re-encode videos is very useful to convert 288p or 360p into 720p, and then play it with madVR to fit your display. For really low-quality videos, it looked MUCH better than by simply running madVR. BUT, with SuperRes, it's almost even between my version and madVR's live version. My video is clearer with less noise, while the original file with madVR looks sharper because of SuperRes. I'd be happy to release that tool for any of you guys if interested (although the software itself was designed for something else, but has the encoder as part of it).

There are two issues I'm having problems with, however.

1. Playing .AVS (avisynth) script files in Windows Media Player ActiveX control, to preview the changes before encoding

2. Getting SuperRes to work in AviSynth (best bet would probably be editing AviShader which uses a shader in AviSynth)

You can contact me if you believe you can help with either of these issues.

[Hyllian]

Personally I find the image unnatural. Too "cartoonish" for my taste. But then that probably works for Mario Bros :D


You may be right, in the sense that I, in fact, don't have a good knowledge of how to adjust a image of a video/movie (I'm completely newbie in this realm).

The algorithm is adjustable and I think you all video experts can extract much more from this than me in that task.

When you develop an edge detector algorithm, you tend to like edges a bit too much. :D

[ryrynz]

The algorithm is adjustable and I think you all video experts can extract much more from this than me in that task.


Exactly. I have no doubt madshi will find some acceptable settings and or allow us to fine tune it based on feedback.

[madshi]

They were created with the three passes, though the third pass is optional.

I think the third pass improves a bit the IQ, in exchange of power, of course. I don't know if it's worth. Anyway, it's optional and can be skipped if power is a problem.

As you'll see, my implementation has much room for optimization. I calc the luminance inside all shaders and it could be done in a first pass (see NEDI shaders, for example) only.

The first pass must work at 2x scale factor (the same way NEDI shaders), and the second and third passes work at 1x scale factor. Use nearest sampling on all texture lookups, as it doesn't need precise texture sample locations.

You'll see too that my blendings aren't done in linear light. I didn't test this algorithm in linear light.
Ok, thanks. I'll try linear light and with/without the 3rd pass.

Personally I find the image unnatural. Too "cartoonish" for my taste.
If you upscale that much (400%), the image must become either very very soft, or a little bit cartoonish, that's just the way it is. NNEDI3 has a similar look.

Simple good sharpening doesn't work with material that has noise and artifacts in it. SuperRes works GREAT to neutralize noise and artifacts while bringing out the real details. Even when using low-quality material with bilinear resizing, with SuperRes it still looks good. That can't be denied.
You can filter out noise and artifacts before sharpening. Yes, madVR can't do that yet. But that doesn't mean that SuperRes is superior in concept. Well, it *may* be, but I don't take that as a given yet.

Hey I have an idea to simplify SuperRes. Instead of having all these settings and asking whether to apply once or every 2x scaling, why not simply ask how many passes he wants to run? If he runs 2 passes with a 3.5x upscaling factor, it will do 1 pass mid-way and 1 pass after. The user generally selects the number of passes based on what his graphic card can handle, but he really doesn't need to know when each pass actually gets applied. You can determine the best distribution of passes yourself.
If you scale an image 200%, you need much less sharpening than when you scale an image 400%. Your suggestion does not take that into account at all.

[Sunset1982]

Feature request:


@madshi:

are you planning the use of directx 12 for madvr in near future?


The use of internal GPU would be a nice fauture to think about:

http://winfuture.de/videos/Spiele/DirectX-12-interne-GPUs-arbeiten-jetzt-mit-Grafikkarten-zusammen-14444.html

just a thing to think about. Keep on your great work! you give a great time watching movies on our htpcs! :)

[nevcairiel]

People really need to stop assuming that improvements for game rendering will automagically make video rendering better.... :)
Personally, I don't think DX12 will offer anything for Video. Its not a replacement for DX11 afterall, its an alternative for developers that want absolute low-level control.

For multi-adapter, thats probably not viable for video rendering, as you can't off-load parts of the processing chain to another GPU. Its either the entire image, or nothing. And copying the entire image around is a slow process.

[dansrfe]

Is there even a significant performance advantage of DX11 over DX9 assuming an identical madVR setup? (no additional features)

[huhn]

AFAIK DX 11 is slower and dx9 old path is the fastest. but dx 12 is made for performance unlike 10/11. I'm pretty sure he will add DX 12 if it is gives better performance and is needed for a great new feature so just wait.

[aufkrawall]

It would give him quite a task for memory management etc. Not sure if this is worth it for a video renderer.

[pirlouy]

Ahah, Nevcairiel = dream breaker.

Everywhere they say Dx12 is a major improvement, threading , low level, etc. And you break everything. :D

[MysteryX]

You can filter out noise and artifacts before sharpening. Yes, madVR can't do that yet. But that doesn't mean that SuperRes is superior in concept. Well, it *may* be, but I don't take that as a given yet.
I tried optimizing videos with denoising the image before upscaling to the best I could. It gives a good result, but SuperRes alone gives a result almost as good as all the processing I've done. If I could use somehow use SuperRes in AviSynth, then I could know for sure and compare. Denoising the image first and applying SuperRes at the end most likely would give an excellent result even on very bad material.

If you scale an image 200%, you need much less sharpening than when you scale an image 400%. Your suggestion does not take that into account at all.
We need to configure profiles for each types of videos, so that's taken into account there.

[madshi]

http://www.anandtech.com/show/7889/microsoft-announces-directx-12-low-level-graphics-programming-comes-to-directx/2

"As with Mantle, the primary goal for Direct3D 12 is to greatly reduce the CPU overhead [...]"

No benefit for madVR. There are many differences between what a game typically does and what madVR does. madVR just draws 2 triangles per frame. It's a whole different world. I don't see anything in DX12 that would be interesting to madVR. All the improvements I can use are already in DX11.