madVR - high quality video renderer (GPU assisted)

[racerxnet]

cyberbeing
Quote:
Originally Posted by madshi View Post
Ok. Is the NVidia decoder freeware and does it work on ATI cards, too?
It's not freeware, but there is a 30 day trial here: http://www.nvidia.com/object/dvd_dec...223-trial.html


It seems I remember hearing of people using it on ATI cards before, but not owning an ATI card myself currently, I can't confirm. Since you would be using it in software mode with madVR, I really don't see why it wouldn't work.

It works fine with ATI cards. I have been using this decoder since it first came out. Yes it will decode in software mode for YV12 output to MadVR. It will also provide DXVA for HD content with the Arcsoft filters. This has been shown in the AVS forum for HTPC use.

MAK

[TinTime]

Originally Posted by cyberbeing

When outputting YUY2 from the NVIDIA decoder to FFDshow which is outputting YV12 to madVR, that misaligned luma and chroma problem still happens. This only happens with madVR. Other renderers are fine. This suggests it doesn't have anything to do with the colorspace being output by the NVIDIA decoder.

Have you tried YV12 output from the Nvidia decoder into ffdshow? Does this work with other renderers for HD material?

[cyberbeing]

Originally Posted by TinTime

Have you tried YV12 output from the Nvidia decoder into ffdshow? Does this work with other renderers for HD material?

Yes it does work for other renderers. madVR is the only renderer with this problem so it must not be handling something correctly.

[Hypernova]

Originally Posted by madshi

From what I can see, both EVR and Haali are slightly sharper than your madVR screenshot. That might make a difference in banding, too? Which madVR upscaling filter are you using? You may want to try a sharper one (e.g. Lanczos) to see whether the increases banding or not.

My answer would be no, but I don't really trust my eyes outside of choosing things for myself, so here's the shot.



On other note, I use Bilinear with madVR (on both) right now because that's the only way I can get smooth playback fullscreen. If I had a choice, I probably go with lanczos4 or one of spline. Mainly because I mainly watch anime, so I prefer it sharp (but not with sharpening). madVR 0.9 certainly improve to the point that I can use Softcubic for SD contents (~35-38ms render time). So not all my hope is lost yet, maybe

[madshi]

Originally Posted by Hypernova

My answer would be no, but I don't really trust my eyes outside of choosing things for myself, so here's the shot.

Thanks! This madVR screenshot is now sharper than the EVR and Haali screenshots, but still madVR shows less banding compared to Haali and much less banding compared to EVR. So your screenshots are a really good example of other renderers showing more banding than madVR does.

Interestingly the colors in your madVR 0.9 screenshot are now similar to EVR. So probably your EVR settings were correct all the time...

[6233638]

Originally Posted by madshi

That goes to show that luma scaling algorithms are really a matter of taste. Most of the algorithms have some advantages and some disadvantages. Personally, I like Lanczos4 for its sharpness, but (obviously) I dislike the ringing. I like Mitchell, but actually I like SoftCubic50 even more, which I find very similar to Mitchell, but with less aliasing. So sometimes I'm using Lanczos4 and sometimes SoftCubic50.

I found that SoftCubic50, while initially looking better, actually introduced more artefacts into the picture. There was "blocking" in some scenes. I'll try to get some comparison shots done later.

Originally Posted by madshi

For chroma I'm using SoftCubic100. Interesting that both of you guys prefer SoftCubic50 for chroma.

I find that SoftCubic 100 smooths chroma out too much, which can make fine coloured details look a bit desaturated around the edges. (you'll see a similar effect if you turn up sharpness too high on your display)

Originally Posted by madshi

Lanczos and Spline produce very similar results, I think. However, you should compare with the same number of taps. Lanczos3 and Spline36 use 3 taps. Lanczos4 and Spline64 use 4 taps. Lanczos8 uses 8 taps. Personally, I can see some differences between 3 and 4 taps. But I don't really see much of a difference between 4 and 8 taps. Except that 8 taps rings quite a bit more...

What I thought was artefacts being added by Lanczos8 last night, looks like it might actually be double-ringing. So the initial ringing is lessened but it rings twice which makes it look worse.

Here's a few examples taken from the same images. (at 10x)

4, then 8:

[mark0077]

Just butting in to say keep up the good work. Great stuff being done in here. madshi did you manage to get a dvd sample yet, my bb has been slowed due to going over my download limits so might be 3-4 days until I can upload a dvd (requested by testuo) for you to test / fix the macrovision errors people get playing dvd's from hdd.

EDIT: With all of the discussion about different algorithms, when madVR is being improved / finalized, if different algorithms are generally agreed to be best (produce closes to original) for upscaling and downscaling, will madVR have a smart setting or some sort of Auto, which can select the best method based on what is being done.

I am always concerned with new work like this, that an auto setting is put in there for those of us who want the benefits, without having to do lots of research on the best settings. Would a type of auto setting be a good idea out of the bag? Cheers.

[nijiko]

Originally Posted by madshi

Ok. Is the NVidia decoder freeware and does it work on ATI cards, too?

It works well on ATI but probably no DXVA.
And it's a commerce software. You can learn from here:
http://www.nvidia.com/object/dvd_decoder.html

[yesgrey]

Originally Posted by madshi

@yesgrey, your opinion?

Maybe it's a good idea...
From the formulas:
Y' = Kr*R' + Kg*G' + Kb*B'
Pb = B' - Y'
Pr = R' - Y'
Cb = 0.5 + Pb
Cr = 0.5 + Pr
then
Cb = 0.5 + B' - Y'
Cr = 0.5 + R' - Y'

You could try this, considering 2x2 pixels in YV12:
1)Using the Y' values from all 4 pixels, calculate the equivalent Y'eq value using the same method that is used for the chroma downsampling (this could be a problem, because there are more than one method...).
2)Calculate B' = Cb + Y'eq - 0.5 and R' = Cr + Y'eq - 0.5.
3)Upsample B' and R' values instead of Cb and Cr like you are doing now.
4) Calculate the Cb and Cr values using the upsampled B' and R' and the Y' values of each pixel.

This is just a quick thought, so maybe it's just a very dumb idea...

[Casshern]

Originally Posted by madshi

You say that as if it was a fact. Have you seen this done? And have you seen that it's actually "much better"?


Doing it that way would mean making brighter pixels more saturated and darker pixels less saturated, right? I don't really see how that would be "more accurate". But I'm not an expert in this area. Is there any "scientific" reason for making brighter pixels more saturated than darker pixels?

As far as I can see, luma and chroma are independent and the luma value does not have any direct influence on chroma. So I don't really see how luma can help upsampling chroma better. But then we're talking about gamma corrected Y'CbCr and not linear light YCbCr. And IIRC I've been told that there is a bit of luminance in Cb and Cr, too. Argh, this is complicated.

@yesgrey, your opinion?


There is an article which handles border cases where chroma is spread to neighbor pixels, if luma is too dark or too bright to hold the upsampled chroma. I've yet to look into implementing a similar algorithm. But the article only handles such corner cases and does not *generally* reshuffle the chroma. Actually the author of that article told me that a friend of his suggested to use luma to form chroma better, but he was not convinced of his friend's efforts...

I am not talking about border cases, nor should luma affect chroma other than in upscaling it - so the colors themselves are not altered. The absolut simplest would be to use a 4 tap horizontal window on luma deltas (L) and 4 tap horizontal actual chroma window (of the simply doubled chroma info - so its a point upsampled chroma channel to the same resolution as luma) (C). Then one could do something like this:
Destination_chroma_for_pixel_p=Sum(Ci*(Sum_up_to_i(L)/Sum_all(L)))

p is in the middle of the tap window.

This ensures that the weights are monoton increasing and normalized in total to 1. Then the chroma info is resampled using these weights.

Naturally this should be extended to the vertikal res as well, and it one could refine it by subsampling on a 0.5 pixel raster as the luma deltas are shifted by 0.5 pixels. Also the linear interpolation can easily be extended to bicubic, lanczos etc. The last thing to think about is how to actually weight the chroma channels: do in rgb light, or UV or normal RGB. But i hope you get the idea...


NOTE: It just occured to me that the designers of YUV were pretty clever in coding chroma as differentials. When upscaling these with anything better than point sampling to 4:4:4 one could get the same results as above much easier. So an alternative pipeline would be to upscale YUV 4:2:2 to YUV 4:4:4 and only then convert/upscale to the target RGB resolution. Or directly upscale the chroma differentials to the target res. This might be easier then the above. So i might just have reinvented the wheel. Any thoughts on that? Other ideas?

[madshi]

Originally Posted by 6233638

I found that SoftCubic50, while initially looking better, actually introduced more artefacts into the picture. There was "blocking" in some scenes. I'll try to get some comparison shots done later.

I find that SoftCubic 100 smooths chroma out too much, which can make fine coloured details look a bit desaturated around the edges. (you'll see a similar effect if you turn up sharpness too high on your display)

Comparison shots for both problems would be welcome.

Originally Posted by mark0077

madshi did you manage to get a dvd sample yet

No.

Originally Posted by mark0077

With all of the discussion about different algorithms, when madVR is being improved / finalized, if different algorithms are generally agreed to be best (produce closes to original) for upscaling and downscaling, will madVR have a smart setting or some sort of Auto, which can select the best method based on what is being done.

If you read through the few previous posts you'll notice that everybody has his own favorites. There is no "best method". It's a matter of taste.

Originally Posted by yesgrey3

Maybe it's a good idea...
From the formulas:
Y' = Kr*R' + Kg*G' + Kb*B'
Pb = B' - Y'
Pr = R' - Y'
Cb = 0.5 + Pb
Cr = 0.5 + Pr
then
Cb = 0.5 + B' - Y'
Cr = 0.5 + R' - Y'

You could try this, considering 2x2 pixels in YV12:
1)Using the Y' values from all 4 pixels, calculate the equivalent Y'eq value using the same method that is used for the chroma downsampling (this could be a problem, because there are more than one method...).
2)Calculate B' = Cb + Y'eq + 0.5 and R' = Cr + Y'eq + 0.5.
3)Upsample B' and R' values instead of Cb and Cr like you are doing now.
4) Calculate the Cb and Cr values using the upsampled B' and R' and the Y' values of each pixel.

This is just a quick thought, so maybe it's just a very dumb idea...

That raises a number of questions for me:

(1) I understand Cb and Cr in your math above are still from the gamma corrected Y'CbCr, right? Maybe we should designate them Cb' and Cr', although that's wrong. Or maybe we should find a new name for them?
(2) What is "Pb" and "Pr" and "Kr/g/b"?
(3) You seem to be calculating B' and R' (gamma corrected blue/red), but without using any of the Rec601/709 coefficients. Wouldn't that result in incorrect results?
(4) You totally ignore G'. How can you recalculate Cb and Cr without using G'?

I just don't fully understand what your formulas are doing and why. Could you please explain? Thanks!

Originally Posted by Casshern

I am not talking about border cases, nor should luma affect chroma other than in upscaling it - so the colors themselves are not altered. The absolut simplest would be to use a 4 tap horizontal window on luma deltas (L) and 4 tap horizontal actual chroma window (of the simply doubled chroma info - so its a point upsampled chroma channel to the same resolution as luma) (C). Then one could do something like this:
Destination_chroma_for_pixel_p=Sum(Ci*(Sum_up_to_i(L)/Sum_all(L)))

p is in the middle of the tap window.

This ensures that the weights are monoton increasing and normalized in total to 1. Then the chroma info is resampled using these weights.

Naturally this should be extended to the vertikal res as well, and it one could refine it by subsampling on a 0.5 pixel raster as the luma deltas are shifted by 0.5 pixels. Also the linear interpolation can easily be extended to bicubic, lanczos etc. The last thing to think about is how to actually weight the chroma channels: do in rgb light, or UV or normal RGB. But i hope you get the idea...

I don't fully understand your formula. Maybe I'm being stupid?

But more importantly: I still don't see the physical/scientific reasoning for doing what you suggest. Now I don't have free time to spare. So I can't really afford to spend hours and hours on an algorithm without even understanding *why* it should be better in theory. So could you please spend a few lines on explaining why your formula should produce better results?

Originally Posted by Casshern

NOTE: It just occured to me that the designers of YUV were pretty clever in coding chroma as differentials. When upscaling these with anything better than point sampling to 4:4:4 one could get the same results as above much easier.

Well, that is exactly what all the better chroma upsampling algorithms out there (including madVR) are already doing!

[nlnl]

Originally Posted by yesgrey3

Maybe it's a good idea...
From the formulas:
Y' = Kr*R' + Kg*G' + Kb*B'
Pb = B' - Y'
Pr = R' - Y'
Cb = 0.5 + Pb
Cr = 0.5 + Pr
then
Cb = 0.5 + B' - Y'
Cr = 0.5 + R' - Y'

You could try this, considering 2x2 pixels in YV12:
1)Using the Y' values from all 4 pixels, calculate the equivalent Y'eq value using the same method that is used for the chroma downsampling (this could be a problem, because there are more than one method...).
2)Calculate B' = Cb + Y'eq + 0.5 and R' = Cr + Y'eq + 0.5.
3)Upsample B' and R' values instead of Cb and Cr like you are doing now.
4) Calculate the Cb and Cr values using the upsampled B' and R' and the Y' values of each pixel.

This is just a quick thought, so maybe it's just a very dumb idea...

yesgrey3
And if we did like this:
1) See above (downsample Y' --> Y'eq)
2) Transform between Y'eqCbCr --> R'G'B' (1/4 of source pixels)
3) Upsample B', R' and G' values R'G'B' --> R"G"B" (back to initial number of pixels, we could call it upscaling in gamma corrected RGB space)
4) Correct R"G"B" luminance using exact Y' values from the source Rc"=R"+d, Gc"=G"+d, Bc"=B"+d:
Kr*(R"+d) + Kg*(G"+d) + Kb*(B"+d)=Y', d=(Y'-(Kr*R"+Kg*G"+Kb*B"))/(Kr+Kg+Kb)
5) Degamma Rc"Gc"Bc" --> RGB
6) Scale
7) Gamma it RGB-->R'G'B'

[mark0077]

madshi, I would have to disagree (albeit with my limited knowledge) that there is no "best" method out of a group of scaling techniques for a certain operation. How can this be. Isn't there something wrong with this statement.

If we are trying to create an image thats as close to the original as possible, then (maybe with the use of a camera or some other method) to compare the original image with the various scaled ones and do some sort of comparison on those to see which is closest.

I can't think of a good way of doing this, maybe using a CRT, but to say there isn't a best method I think is wrong. One technique, at least for one specific image would be closer to the original (percieved by our eyes) than other methods.

[madshi]

@Mark, all the resampling filters are a compromise between aliasing, sharpness and ringing. There is no filter which is best in every category. E.g. Lanczos is the sharpest filter (good), but has the most ringng (bad). Look here:

http://forum.doom9.org/showthread.ph...90#post1272990

If ringing artifacts don't bother you, Lanczos is "best". If you hate ringing artifacts, Lanczos is "worst". So obviously there is no overall "best" filter. You may also want to reread the forum rules, especially rule 12.

[mark0077]

Well thanks for the reply and link but mathematically one image WILL be closer to the original than another right? I am not breaking rule 12, I am just discussing something....... anyways, one image will be closer to the original than another I think, whether it would have one of the many negative effects you talk about. If part of the aim / point of madVR is to get us the original image in all of its glory, and as accurately as possible, then the whole idea of a users preference goes out the window.

OK fair enough when it comes to something that can't be done perfectly / exactly like resizing users will always have a preference, but I still think strongly that user preference should have as little impact / effect as possible if there is a way to measure how close to the original a scaled image is perceptually (and I'm sure your away of some technique for doing such a thing?). I hope you see what I am saying here.

[madshi]

Lanczos is nearer to the original, I believe. But it adds artifacts to the image which other filters don't (or do less). So regardless of whether you like it or not, it still is a matter of taste.

This is my last post about this topic for now.

[yesgrey]

Originally Posted by madshi

(1) I understand Cb and Cr in your math above are still from the gamma corrected Y'CbCr, right? Maybe we should designate them Cb' and Cr', although that's wrong. Or maybe we should find a new name for them?

Yes, I agree with the Cb' Cr' designations. Soon we should start talking about YCbCr and we need to distinguish between them.

Originally Posted by madshi

(2) What is "Pb" and "Pr" and "Kr/g/b"?

From my understanding:
Kr, Kg and Kb are the contributions or R', G' and B', when all they are equal to 1.0, that gives the white point as Y' = 1.0. (This in [0.0 1.0] range).
Pb' and Pr' (they are not primed, but I will also use the prime symbol to avoid confusion with Pb, Pr when converting from linear RGB) are analog values. Cb' and Cr' are the designations when using them in the digital format.
Pb' and Pr' are defined in the range [-0.5 0.5], so we need to add 0.5 to them to be defined in the range [0.0 1.0] and allow them to be used in the digital format Cb' Cr'.

Originally Posted by madshi

(3) You seem to be calculating B' and R' (gamma corrected blue/red), but without using any of the Rec601/709 coefficients. Wouldn't that result in incorrect results?

No, because B' and R' are calculated using the coefficients, just not in the way you are used to see them.
The Rec601/709 only define the Kr and Kb coefficients. Kg is calculated from Kr+Kg+Kb = 1.
The coefficients that we are used to, are calculated based on the formulas I gave above. Those three formulas can be translated into a matrix of coefficients.
If you want more details on how to get the coefficients, see here:

Originally Posted by madshi

(4) You totally ignore G'. How can you recalculate Cb and Cr without using G'?

The only part of G' that is needed is contained in Y'.

Originally Posted by nlnl

yesgrey3
And if we did like this:
...
4) Correct R"G"B" luminance using exact Y' values from the source Rc"=R"+d, Gc"=G"+d, Bc"=B"+d:
Kr*(R"+d) + Kg*(G"+d) + Kb*(B"+d)=Y', d=(Y'-(Kr*R"+Kg*G"+Kb*B"))/(Kr+Kg+Kb)

The only problem I see, besides the extra steps, is you considering that R", G" and B" would get the same correction from Y' values. R', G' and B' have different contributions to Y', so R", G" and B" should also be corrected by different values, which could not be computed with the suggestion you made.
I think that my method should be pretty simple to try, because does not require significative changes by madshi. Then, if it gives any improvement, maybe we can think on more complex methods.

[madshi]

Originally Posted by yesgrey3

No, because B' and R' are calculated using the coefficients, just not in the way you are used to see them.

But your formulas don't list any coefficients! You list Kr, Kg and Kb in the first line and then you just drop them and never use them again. Also your line "Pb = B' - Y'" doesn't match what I read in Wikipedia. There the "Pb" formular is much more complicated. You seem to have simply dropped the whole Kb coefficient! I'm confused...

[Casshern]

Originally Posted by madshi

Well, that is exactly what all the better chroma upsampling algorithms out there (including madVR) are already doing!

Good! I only proposed this because you wrote that chroma and luma upsampling is completely unrelated. If you do it on the chroma differentials (heres the "hidden" connection) everything is fine - and the result should be the same. The only room to improve would be to use a psychovisually more "accurate" definition of Y and the chroma components (maybe through RGB light - then something like i proposed might be necessary again), but i doubt it would make perceptible differences. So everything is good enough for me- keep up the good work!

[yesgrey]

Originally Posted by madshi

You list Kr, Kg and Kb in the first line and then you just drop them and never use them again.

Y' = Kr*R' + Kg*G' + Kb*B'
Pb' = B' - Y'
Pr' = R' - Y'

If you substitute Y' in Pb' and Pr' you'll get:
Pb' = B' - (Kr*R' + Kg*G' + Kb*B')
Pr' = R' - (Kr*R' + Kg*G' + Kb*B')

But since Y' is already known, you can use it directly. The coefficients are used to calculate Pb' and Pr' from R', G'and B'; from Y' it would be a lot simpler.
Also, Rec BT709 defines Kr = 0.2126 and Kb = 0.0722, and Rec BT601 defines Kr = 0.2990 and Kb = 0.1140.

If you want to give it a try, just trust me for now, the formulas are correct. If you prefer to understand it first, I will continue to explain it to you. If you feel this is too off-topic let me know and I will PM you instead...
I only think that the method I explained above should be pretty simple to try, because you already have all the math written; is just using it with different values, nothing more.