madVR 4K GPU performance test

[chros]

The purpose of this thread to test the speed of different GPUs using latest stable v0.92.17 madVR's NGU algorithms:
- it's supposed to be reflect real life scenario with today's/tomorrow's standard
- testing/reporting takes around an hour

Constraints:
Since the result can depend on so many details, we use the following constraints:
- use a 4k screen (if you can't you can still test it but make a note about it), set your display device in madVR to:
-- 10 bit output
-- this display is already calibrated: DCI-P3 + gamma 2.2/2.4
-- hdr: passthrough HDR to display + send HDR metadata to the display (we don't test hdr2sdr conversion until a final build)
- use d3d11 native in LAV Video (see LAV settings in "settings.zip")
-- hardware decoder: D3D11
-- hardware device to use: Automatic (Native)
- use d3d11 for representation to achieve 10 bit output
-- on Windows 8.1: use FSE
-- on Windows 10: use SFW ("settings.bin" is already set up this way)
-- on Windows 7: I don't know
- match refresh rate of the display with the content if it's possible (if not possible then use SmoothMotion with 60p and make a note about it)
-- e.g.: 2160p23, 2160p24, 2160p25, 2160p29, 2160p30, 2160p50, 2160p59, 2160p60

Prerequisities:
- make a backup of your "settings.bin" in madvr directory first!
- reset your madVR settings and download the test video files (~2.6 GB) from mega.nz
-- there are 12 files all together: 3 files (23p/29p/60p) in 4 categories (2160p/1080p/720p/576p)
- profiles for different video content is already set up in "settings.bin":
-- they are all set up to use the NGU highest settings (2160p/1080p profiles use NGU Sharp, 720p/576p profiles use NGU Standard; NGU Anti-Alias is faster than the rest of NGU algo hence it's not used)


Testing and reporting:
- play back every file and try to lower NGU chroma *and* image upscaling settings to a point when there's no dropped frames
-- use MPC-HC / BE for simplicity
-- monitor playback with the help of OSD (ctrl+j) and a GPU monitoring util (GPU-Z, nvidiainspector, etc.)
-- if your card can pass a preset as it is then state: 2160p@24p: passed
--- in this case you enable doubling of 2160p contents and take a note about those
--- and take a note of the GPU usage/frequency and the actual/max rendering time
-- if it can't then take a note of your passed settings, e.g.: 1080p@29p: chroma up: NGU Sharp High ; image up: NGU Sharp High + High
- also report: driver and OS version and try to link your card to a comparison table (e.g. on Wikipedia)

Happy testing!

[chros]

Results:

- MSI GeForce GTX 1060 Gaming X 6GB

[chros]

MSI GeForce GTX 1060 Gaming X 6GB (GP106-400-A1 , 2016.07.19, max freq: ~1900 MHz) + v385.28 driver + Win10 LTSB 1607:
- 2160p@24p:
-- chroma up: NGU Sharp High
- 2160p@29p:
-- chroma up: NGU Sharp High
- 2160p@60p:
-- chroma up: NGU Sharp Medium
- 1080p@23p:
-- chroma up: NGU Sharp High
-- image up: NGU Sharp High + NGU Sharp Low
- 1080p@29p:
-- chroma up: NGU Sharp High
-- image up: NGU Sharp High + NGU Sharp Low
- 1080p@59p:
-- chroma up: NGU Sharp Low
-- image up: NGU Sharp Medium + Bicubic60 AR (let madvr decide)
- 720p@23p:
-- chroma up: NGU Standard Low
-- image up: NGU Standard High > NGU Standard High < SSim1D100AR + Bicubic60 AR > Bicubic60 AR (normal)
- 720p@29p:
-- chroma up: NGU Standard Low
-- image up: NGU Standard Medium > NGU Standard Medium < SSim1D100AR + Bicubic60 AR > Bicubic60 AR (let madvr decide)
- 720p@59p:
-- chroma up: Bicubic60 AR
-- image up: NGU Anti-Alias Low > NGU Anti-Alias Low < Bicubic60 AR + Bicubic60 AR (let madvr decide)
-- up refinements: none selected
- 576p@23p:
-- chroma up: NGU Standard Low
-- image up: NGU Standard Very High > NGU Standard High < SSim1D100AR + NGU Standard Low > Bicubic60 AR
- 576p@29p:
-- chroma up: NGU Standard Low
-- image up: NGU Standard High > NGU Standard High < SSim1D100AR + Bicubic60 AR > Bicubic60 AR
- 576p@59p:
-- chroma up: Bicubic60 AR
-- image up: NGU Standard Low > NGU Standard Low < Bicubic60 AR + Bicubic60 AR > Bicubic60 AR

Note: since the factory overclocked 1060 6GB Gaming X can't even handle NGU Sharp Very High settings with 2160p/1080p 23p contents, I use it underclocked (-350MHz) with slightly worse settings.

[huhn]

how do you even want to sort this?

there is no d3d11 chain madVR is d3d9.
why add an overhead with SM why add an overhead with calibration
why don't you just use d3d9 for presentation to remove as many variables as possible
what the point of 12 test files...
576p is not a real world resolution dvds are interlaced.

[chros]

Originally Posted by huhn

there is no d3d11 chain madVR is d3d9.
why don't you just use d3d9 for presentation to remove as many variables as possible
why add an overhead with SM why add an overhead with calibration

Thanks, corrected: "d3d11 for representation"
Something has to be set, and I want to reflect real life usage, mainly 10 bit output.
About SM: it can be a valid real life scenario as well and I didn't want to discard users from the test just because they can't match refresh rate.

Originally Posted by huhn

what the point of 12 test files...
576p is not a real world resolution dvds are interlaced.

To provide more versatile result set.
720p content is the most demanding (if we don't use 2160p doubling) on a 4K screen, different refresh rates help to understand what a card can do at a given resolution.
560p is the worst case scenario of the lower SD resolutions hence I included it. It won't take much more time to test them.

Originally Posted by huhn

how do you even want to sort this?

It can be done and I will unless an existing GPU can pass all the preconfigured presets (maybe a 1080TI, or 2080TI). If it can then just enable doubling of 2160p contents and take a note about those settings.

Now, let's get back to testing!

[huhn]

i'm not going to do this test it will not even get an image if i follow this...

720p is not the hardest content just don't supersample.
you don't have to match refresh rates it doesn't have an real effect on performance but SM has.

using path through heck HDR is pointless it's no real difference for the GPU to do pass through or sdr you just add more variables.

[chros]

Originally Posted by huhn

i'm not going to do this test

No worries, hopefully others will

[ng4ever]

Originally Posted by chros

No worries, hopefully others will

I will. Just because.

My only problem is how do I do it ?

Yes I read but I don't have a settings.bin file anywhere. Sorry.

[chros]

Nice to hear it, which GPU do you have?
You don't have settings.bin in madvr directory because probably it's installed in the system directory where your regarular user doesn't have write permission. Modify fhe write permission of that dir or move the dir to e. g. D drive somwhwere where you will have and register the filter from there. Next time you start the player madvr should create the settings.bin file.

[mclingo]

Hi, been thinking, MADVR processing and gaming abilities have a direct correlation why dont we use a gaming benchmark evaluate cards, are there any about which just evaluate GPU performance ignoring the system in general?

[chros]

Originally Posted by mclingo

MADVR processing and gaming abilities have a direct correlation why dont we use a gaming benchmark evaluate cards

Which one(s)? Most GPU behave differently in different type of benchmarks: d3d9, d3d11, vulkan, etc. let alone the actual games.
If there was at least 1 then we won't need a thread like this.

Seriously, (not just you) please stop brainstorming/questioning/arguing/etc ...
The task is really simple, even a 10 years old could do it ...
All you need to do is testing ... (I already did.)

[clsid]

Ultimately we just want to know what a GPU is capable of in terms of madVR settings. Based on that someone can compare GPUs and decide which one best matches his/her requirements.

1) Make a list of input/output combinations. For example "720p23 -> 1080p" and "1080p23 -> 4k" and so on.
2) Make a discussion topic for each combo. Starting with the most commons ones. Here users can post their test results. There is a central topic with instructions on how to perform the test and how to format the results.
3) For each combo there is a list of standardized "doom9 approved" scaling settings/presets. Initially just simple ones that only involve scaling. Later maybe a few more complex ones for super high end cards.
4) Users test these scaling presets and post their average render times. The test report contains following details: GPU, driver version, OS, refreshrate, SM=on/off, 3dlut=yes/no.
Because we have dedicated topics for specific inputs, it becomes easier to evaluate and compare the results that are posted, so anomalies can be spotted quickly.
5) Results are collected and input in a spreadsheet. Two pages for each input/output combo. One with 3dlut and one without. The performance impact of SM is relatively small. So in that case we put the worst test result in the table.
Rows for GPUs, columns for the scaling presets.
Colored cells with render times. Green = ok, yellow = barely ok, orange = barely too slow, red = too slow

This data could eventually also be used by a little tool to automatically create settings for a specific GPU.

[Warner306]

I like the idea of separating these tests. Does anyone care about anything besides 1080p -> 4K NGU image doubling and HDR to SDR tone mapping?

Providing a pre-populated settings.bin would make the test more reliable and easier to carry out.

[chros]

Originally Posted by Warner306

Providing a pre-populated settings.bin would make the test more reliable and easier to carry out.

? There *is* a provided settings.bin as well.

[Warner306]

Alright. But what about separating the tests? Yes. No.

[mclingo]

sorry to have to say, where I personally really appreciate the work that has already gone into this I dont think you're not going to get many takers here.

Its needs to be stripped right down to one one single test to run. Note, you dont need to test everything, all we really need is a basic processing grunt hierarchy and we should be able to do that with just one test. We'd need two downloads, one for SDR users, one for HDR users.

[chros]

Originally Posted by Warner306

Alright. But what about separating the tests? Yes. No.

Originally Posted by mclingo

Its needs to be stripped right down to one one single test to run. Note, you dont need to test everything, all we really need is a basic processing grunt hierarchy and we should be able to do that with just one test.

They are separated, aren't they? You can do just 1 or 2 or 6 or 12. It's all up to you

Originally Posted by mclingo

We'd need two downloads, one for SDR users, one for HDR users.

Can be, but that one must be done by someone else (preferably in a separate thread)

[Warner306]

Well, you're not getting many customers. I thought clsid's input was an improvement. It would keep discussion more focused.

[chros]

Originally Posted by warner306

well, you're not getting many customers.

I'm pretty sure they are on Holiday

[mclingo]

I'm still hoping for MADSHI to come out from under his HDR tone mapping rock and come up with something for us, in the meantime i'll just stick with gaming benchmarks are I think there is a reasonable enough correlation between the two, I'll always just be comparing AMD cards anyway as i'm still using 3D.