Objective comparison of renderers possible?

[tetsuo55]

I think many people want to know which renderer to use, there are so many different ones, and some applications have custom renderers too.

I suggest we make a list of different renderers.
Decide which things are important and can be objectively measured.

Some ideas are:
-Chroma sampling
-Scaling
-Cpu/Gpu cycles required
-Features supported

Obviously this requires some form of benchmark.

The results from these tests can be used to improve custom renderers like VMR9-Renderless and EVR-CP in MPC-HC.

Please help with building this test so we can end the video renderer nightmare

[honai]

I'm not sure this is the right time for this test. We have a moving target: except VMR9, none of the other "common" renderers are mature yet, i.e. EVR-CP, madVR, Haali all have bugs, issues and quirks that make them perform worse (in terms of PQ, speed, etc) than desired by their own goals.

In other words, the nightmare *might* only end when one of the renderers reaches a level of maturity. With the absence of madshi and haali I don't see this happening any time soon.

[leeperry]

don't you think EVR is more usable than VMR9? but well, I hate its built-in sharpening :o

[tetsuo55]

I'm not sure this is the right time for this test. We have a moving target: except VMR9, none of the other "common" renderers are mature yet, i.e. EVR-CP, madVR, Haali all have bugs, issues and quirks that make them perform worse (in terms of PQ, speed, etc) than desired by their own goals.

In other words, the nightmare *might* only end when one of the renderers reaches a level of maturity. With the absence of madshi and haali I don't see this happening any time soon.In MPC-HC we are only working on VMR9 renderless and EVR-CP.
The other renderers are not supported anymore, haali and madVR are supported by their makers.

A lot of people keep naming reasons why renderer X is better than renderer Y
We would like to end this discussion by pointing out all the downsides and benefits of each renderer, and then work on improving VMR9 renderless and EVR-CP to the point where all the downsides and benefits of each renderer is included.

[Astrophizz]

Aren't there certain inherent limitations as to what VMR9 and EVR-CP can do that would prevent fixing some of their downsides?

[tetsuo55]

Aren't there certain inherent limitations as to what VMR9 and EVR-CP can do that would prevent fixing some of their downsides?that's exactly what i want to find out with this comparison

[CruNcher]

Linux + GPU ;)
though especially for Nvidia Cards people should look in the new Cuda SDK 2.3 for Windows (XP) and make use of OpenGL instead of Direct3d (good bye VMR/EVR and all the DXVA problems and limitations, though performance wise it's better to stay with VMR/EVR at least on Vista/7 as OpenGL gets wrapped into Direct3D since Vista) :)

[Jong]

I cannot see how VMR9 or EVR could be adapted to acheive the PQ improvements of madVR, especially chroma upsampling and the dithering to RGB output bitdepth. This would take a new generation of GPU I think.

That said, although these are REALLY clear in test patterns such as a smooth luma ramp (and some well chosen red on black, titles) I have looked for noticeable improvements in actual Blu-ray mvoie footage and have never been able to spot it. Maybe it is just my eyesight! MadVR fans may be able to comment!

I admire the work going on to make madVR the absolute best for PQ but, for me at least, the Nvidia/ATI solutions make acceptable (if not theoretically ideal!) tradeoffs.

What are needed are vsync improvements, a la Gothsync.

and if MPC-HC is going to be "stand-alone" I think the Gothsync enhancements would need to incorporate the speedup/speeddown capabilities of Reclock. But that is a lot of work and personally I think Reclock does it's job well, so I would settle for a renderer (maybe triple buffered VMR9/EVR) that allows Reclock to do all it's stuff unhindered. I know ar-jar is interested/considering this, because we have discussed it, but I do understand that Reclock is a pig to understand and configure and for those with well matched frame rate/refresh rates built-in "Gothsync" synchronisation will be MUCH easier.

[tetsuo55]

Do you happen to have these samples available?

Obviously image stability is one of the comparison factors.

EVR-CP already does 10bit RGB conversion on supporting hardware.
All custom MPC-HC renderers support correct chroma sampling

[Mark_A_W]

Smoothness of playback is the most important to me.

Don't forget that (although I'm not sure how you will evaluate it in a standised way).

[Jong]

Do you happen to have these samples available?

Obviously image stability is one of the comparison factors.

EVR-CP already does 10bit RGB conversion on supporting hardware.
All custom MPC-HC renderers support correct chroma samplingHave a look in the MadVR thread for the chroma upsampling tests, I think they are there.

Any HD luma ramp will do for the dithering test. Such as in AVS-HD 709 (http://www.avsforum.com/avs-vb/showthread.php?t=948496).

I am confident MadVR does better on these tests than any GPU (the luma ramp test is quite amazing!), but as I say, not sure I can see the difference when watching something "real!".

Oh.... while I remember. We need Reclock's ability to change refresh rates to suit the media to. This is much better done by the renderer as it can pause the graph and restart it after the change. I think ar-jar has been considering that too. Reclock's scripts mostly work,but they are a "crude hack".

[foxyshadis]

That said, although these are REALLY clear in test patterns such as a smooth luma ramp (and some well chosen red on black, titles) I have looked for noticeable improvements in actual Blu-ray mvoie footage and have never been able to spot it. Maybe it is just my eyesight! MadVR fans may be able to comment!

Aside from the black/grey issue, I don't care what renderer I use for HD, I doubt you'd ever see much difference. For that, only speed and judder matters to me. It makes a huge difference on regular DVD/TV footage however, most of what I watch is still SD.

[leeperry]

Smoothness of playback is the most important to me.
both mVR and EVR(w/ a proper CP) can do 29.97fps@89.91Hz....HR and VMR cannot, they constantly hiccup.

Beliyaal gave me a technical explanation to that, but I can't seem to find his post...it was in the MPC HC thread somewhere around April 2009. IIRC it had to do w/ the mandatory A/V sync that EVR and mVR don't care about..