Objective comparison of renderers possible?

tetsuo55 · 18th September 2009, 09:46

I think many people want to know which renderer to use, there are so many different ones, and some applications have custom renderers too.

I suggest we make a list of different renderers.
Decide which things are important and can be objectively measured.

Some ideas are:
-Chroma sampling
-Scaling
-Cpu/Gpu cycles required
-Features supported

Obviously this requires some form of benchmark.

The results from these tests can be used to improve custom renderers like VMR9-Renderless and EVR-CP in MPC-HC.

Please help with building this test so we can end the video renderer nightmare

18th September 2009, 19:36

I'm not sure this is the right time for this test. We have a moving target: except VMR9, none of the other "common" renderers are mature yet, i.e. EVR-CP, madVR, Haali all have bugs, issues and quirks that make them perform worse (in terms of PQ, speed, etc) than desired by their own goals.

In other words, the nightmare *might* only end when one of the renderers reaches a level of maturity. With the absence of madshi and haali I don't see this happening any time soon.

leeperry · 18th September 2009, 19:45

don't you think EVR is more usable than VMR9? but well, I hate its built-in sharpening

tetsuo55 · 18th September 2009, 20:00

Quote:

Originally Posted by honai

I'm not sure this is the right time for this test. We have a moving target: except VMR9, none of the other "common" renderers are mature yet, i.e. EVR-CP, madVR, Haali all have bugs, issues and quirks that make them perform worse (in terms of PQ, speed, etc) than desired by their own goals.

In other words, the nightmare *might* only end when one of the renderers reaches a level of maturity. With the absence of madshi and haali I don't see this happening any time soon.

In MPC-HC we are only working on VMR9 renderless and EVR-CP.
The other renderers are not supported anymore, haali and madVR are supported by their makers.

A lot of people keep naming reasons why renderer X is better than renderer Y
We would like to end this discussion by pointing out all the downsides and benefits of each renderer, and then work on improving VMR9 renderless and EVR-CP to the point where all the downsides and benefits of each renderer is included.

Astrophizz · 18th September 2009, 21:06

Aren't there certain inherent limitations as to what VMR9 and EVR-CP can do that would prevent fixing some of their downsides?

tetsuo55 · 18th September 2009, 21:16

Quote:

Originally Posted by Astrophizz

Aren't there certain inherent limitations as to what VMR9 and EVR-CP can do that would prevent fixing some of their downsides?

that's exactly what i want to find out with this comparison

CruNcher · 18th September 2009, 22:34

Linux + GPU

though especially for Nvidia Cards people should look in the new Cuda SDK 2.3 for Windows (XP) and make use of OpenGL instead of Direct3d (good bye VMR/EVR and all the DXVA problems and limitations, though performance wise it's better to stay with VMR/EVR at least on Vista/7 as OpenGL gets wrapped into Direct3D since Vista)

Jong · 19th September 2009, 08:16

I cannot see how VMR9 or EVR could be adapted to acheive the PQ improvements of madVR, especially chroma upsampling and the dithering to RGB output bitdepth. This would take a new generation of GPU I think.

That said, although these are REALLY clear in test patterns such as a smooth luma ramp (and some well chosen red on black, titles) I have looked for noticeable improvements in actual Blu-ray mvoie footage and have never been able to spot it. Maybe it is just my eyesight! MadVR fans may be able to comment!

I admire the work going on to make madVR the absolute best for PQ but, for me at least, the Nvidia/ATI solutions make acceptable (if not theoretically ideal!) tradeoffs.

What are needed are vsync improvements, a la Gothsync.

and if MPC-HC is going to be "stand-alone" I think the Gothsync enhancements would need to incorporate the speedup/speeddown capabilities of Reclock. But that is a lot of work and personally I think Reclock does it's job well, so I would settle for a renderer (maybe triple buffered VMR9/EVR) that allows Reclock to do all it's stuff unhindered. I know ar-jar is interested/considering this, because we have discussed it, but I do understand that Reclock is a pig to understand and configure and for those with well matched frame rate/refresh rates built-in "Gothsync" synchronisation will be MUCH easier.

tetsuo55 · 19th September 2009, 08:39

Do you happen to have these samples available?

Obviously image stability is one of the comparison factors.

EVR-CP already does 10bit RGB conversion on supporting hardware.
All custom MPC-HC renderers support correct chroma sampling

Mark_A_W · 19th September 2009, 09:14

Smoothness of playback is the most important to me.

Don't forget that (although I'm not sure how you will evaluate it in a standised way).

Jong · 19th September 2009, 10:05

Quote:

Originally Posted by tetsuo55

Do you happen to have these samples available?

Obviously image stability is one of the comparison factors.

EVR-CP already does 10bit RGB conversion on supporting hardware.
All custom MPC-HC renderers support correct chroma sampling

Have a look in the MadVR thread for the chroma upsampling tests, I think they are there.

Any HD luma ramp will do for the dithering test. Such as in AVS-HD 709 (http://www.avsforum.com/avs-vb/showthread.php?t=948496).

I am confident MadVR does better on these tests than any GPU (the luma ramp test is quite amazing!), but as I say, not sure I can see the difference when watching something "real!".

Oh.... while I remember. We need Reclock's ability to change refresh rates to suit the media to. This is much better done by the renderer as it can pause the graph and restart it after the change. I think ar-jar has been considering that too. Reclock's scripts mostly work,but they are a "crude hack".

foxyshadis · 21st September 2009, 06:22

Quote:

Originally Posted by Jong

That said, although these are REALLY clear in test patterns such as a smooth luma ramp (and some well chosen red on black, titles) I have looked for noticeable improvements in actual Blu-ray mvoie footage and have never been able to spot it. Maybe it is just my eyesight! MadVR fans may be able to comment!

Aside from the black/grey issue, I don't care what renderer I use for HD, I doubt you'd ever see much difference. For that, only speed and judder matters to me. It makes a huge difference on regular DVD/TV footage however, most of what I watch is still SD.

leeperry · 21st September 2009, 14:01

Quote:

Originally Posted by Mark_A_W

Smoothness of playback is the most important to me.

both mVR and EVR(w/ a proper CP) can do 29.97fps@89.91Hz....HR and VMR cannot, they constantly hiccup.

Beliyaal gave me a technical explanation to that, but I can't seem to find his post...it was in the MPC HC thread somewhere around April 2009. IIRC it had to do w/ the mandatory A/V sync that EVR and mVR don't care about..

18th September 2009, 09:46	#1 \| Link
tetsuo55 MPC-HC Project Manager Join Date: Mar 2007 Posts: 2,317	Objective comparison of renderers possible? I think many people want to know which renderer to use, there are so many different ones, and some applications have custom renderers too. I suggest we make a list of different renderers. Decide which things are important and can be objectively measured. Some ideas are: -Chroma sampling -Scaling -Cpu/Gpu cycles required -Features supported Obviously this requires some form of benchmark. The results from these tests can be used to improve custom renderers like VMR9-Renderless and EVR-CP in MPC-HC. Please help with building this test so we can end the video renderer nightmare __________________ MPC-HC, an open source project everyone can improve. Want to help? Test Nightly Builds, submit patches or bugs and chat on IRC

18th September 2009, 22:34	#7 \| Link
CruNcher Registered User Join Date: Apr 2002 Location: Germany Posts: 4,926	Linux + GPU though especially for Nvidia Cards people should look in the new Cuda SDK 2.3 for Windows (XP) and make use of OpenGL instead of Direct3d (good bye VMR/EVR and all the DXVA problems and limitations, though performance wise it's better to stay with VMR/EVR at least on Vista/7 as OpenGL gets wrapped into Direct3D since Vista) __________________ all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 18th September 2009 at 23:00.

19th September 2009, 08:39	#9 \| Link
tetsuo55 MPC-HC Project Manager Join Date: Mar 2007 Posts: 2,317	Do you happen to have these samples available? Obviously image stability is one of the comparison factors. EVR-CP already does 10bit RGB conversion on supporting hardware. All custom MPC-HC renderers support correct chroma sampling __________________ MPC-HC, an open source project everyone can improve. Want to help? Test Nightly Builds, submit patches or bugs and chat on IRC

18th September 2009, 19:36	#2 \| Link
honai Guest Posts: n/a	I'm not sure this is the right time for this test. We have a moving target: except VMR9, none of the other "common" renderers are mature yet, i.e. EVR-CP, madVR, Haali all have bugs, issues and quirks that make them perform worse (in terms of PQ, speed, etc) than desired by their own goals. In other words, the nightmare might only end when one of the renderers reaches a level of maturity. With the absence of madshi and haali I don't see this happening any time soon.

18th September 2009, 19:45	#3 \| Link
leeperry Kid for Today Join Date: Aug 2004 Posts: 3,477	don't you think EVR is more usable than VMR9? but well, I hate its built-in sharpening

18th September 2009, 21:06	#5 \| Link
Astrophizz Registered User Join Date: Jul 2008 Posts: 184	Aren't there certain inherent limitations as to what VMR9 and EVR-CP can do that would prevent fixing some of their downsides?

19th September 2009, 08:16	#8 \| Link
Jong Registered User Join Date: Mar 2007 Location: London, UK Posts: 576	I cannot see how VMR9 or EVR could be adapted to acheive the PQ improvements of madVR, especially chroma upsampling and the dithering to RGB output bitdepth. This would take a new generation of GPU I think. That said, although these are REALLY clear in test patterns such as a smooth luma ramp (and some well chosen red on black, titles) I have looked for noticeable improvements in actual Blu-ray mvoie footage and have never been able to spot it. Maybe it is just my eyesight! MadVR fans may be able to comment! I admire the work going on to make madVR the absolute best for PQ but, for me at least, the Nvidia/ATI solutions make acceptable (if not theoretically ideal!) tradeoffs. What are needed are vsync improvements, a la Gothsync. and if MPC-HC is going to be "stand-alone" I think the Gothsync enhancements would need to incorporate the speedup/speeddown capabilities of Reclock. But that is a lot of work and personally I think Reclock does it's job well, so I would settle for a renderer (maybe triple buffered VMR9/EVR) that allows Reclock to do all it's stuff unhindered. I know ar-jar is interested/considering this, because we have discussed it, but I do understand that Reclock is a pig to understand and configure and for those with well matched frame rate/refresh rates built-in "Gothsync" synchronisation will be MUCH easier.

19th September 2009, 09:14	#10 \| Link
Mark_A_W 3 eyed CRT supporter Join Date: Jan 2008 Location: Or-strayl-ya Posts: 563	Smoothness of playback is the most important to me. Don't forget that (although I'm not sure how you will evaluate it in a standised way).