madVR - high quality video renderer (GPU assisted)

[James Freeman]

OK guys, I'm letting go of this one (people starting to get angry). :)

AND, I agree that MadVR is not about "Good enough", its about the best of the best !

Sorry for polluting the thread with huge pictures. :D

[The 8472]

I might be willing to look into re-introducing a little bit of randomness, but I'll probably do that in such a way that it's the same for every frame, so that with static content the pattern stays the same. This might get rid of the worm artifacts without increasing the subjective noise level. However, it's not going to be free, performance wise. None of the computing platforms (OpenCL, DirectCompute) has a random function built in, so I'd have to create a random texture and read from it, which costs memory bandwidth. That said, Error Diffusion is supposed to be the high-quality alternative, so I guess a small increase in rendering times to get rid of the worm artifacts might be worth it, even if most people are not going to be able to see the difference without boosting the contrast.

Since this should only affect the least significant bits in the output colors maybe simply using a different rounding mode when converting high precision RGB to 6/7/8bit RGB would be sufficient. Depending on what you're using right now round to even or stochastic rounding might be sufficient.

[kgp700]

madVR v0.87.4

* workaround added: NNEDI3 upscaling failed/froze with newer NVidia GPUs
* fixed: NNEDI3 chroma upscaling produced wrong colors with 10bit sources
* got rid of some unnecessary texture sharing

I'm using GTX660 with 334.67 driver (windows 8.1)
and using potplayer, mpc-hc + madVR v0.87.4

but still have driver crash when using scaling algorithms - image doubling - NNEDI3 options
also have wrong colors issue (all green) when using scaling algorithms - chroma upscaling -NNEDI3

tested with 720p x264 video (not 10bit, it just commonness x264 video)

please solve this issues

when "enable automatic fullscreen exclusive mode" , driver don't crash and no color issue but too much frame drop and tearing

Have many advantages when use NNEDI3 upscaling?
for example 720p little more similar as 1080p?


p.s. tooooooo difficult random questions when write on this forum ... that's so subjective ...
please change random questions

[bacondither]

zhoufang (http://www.cb.uu.se/~cris/blog/matlabimages/dither_zhoufang.png) from cris' blog looks much better than ed+random fix. It's apparently just as fast too.
I agree, if the performance hit is not to much.

@madshi
Maby this will help?
The MWC64X Random Number Generator (http://cas.ee.ic.ac.uk/people/dt10/research/rngs-gpu-mwc64x.html)

[turbojet]

zhoufang without random noise might be interesting. It's supposed to workarond difficult greys.

[NicolasRobidoux]

Mathias: Now I think that I remember better:

Deblurring Jinc5 with a different recipe than the one used for "your" Jinc3 gives a scheme which actually is very different: It really "de-aliases" "pixel art" diagonal lines in a way that no reasonable Jinc3 can. (I do not remember if Jinc4 at about the same deblur gave close results to Jinc5. I do remember, making a note to myself that EWA Lanczoses seem to work better with odd numbers of lobes.)

Would you like me to dig this out/experiment or should I leave you alone in your NNEDI bliss?

[6233638]

@madshi
The error diffusion algorithm creates a static pattern with no temporal dithering and can create strange "worm" patterns. Your code seems to hade static thresholds and could be improved as suggested at Cris's Image Analysis Blog (http://www.cb.uu.se/~cris/blog/index.php/archives/355)I noticed this quite a bit when I was creating a new 3DLUT for my display. Certain levels near black (around the 0-15% range) introduced noticeable patterns on my display.
You also have to consider that it's not just about what dithering algorithm is used by madVR, but how that interacts with the display's own processing/pixel structure.

One level even resulted in obvious horizontal bands across the picture, though it was not that clear if you simply looked at the magnified dither pattern alone.

If you can see that with you naked eye... please contact the UN straight away because you may be trained as a pilot.8-bit is not enough where one step blends into the next. You need at least 10-bit for that, ideally 12-bit.

To quote some of the testing that was done when creating the DCI spec:
Results:
•Many of subjects could distinguish 2 counts in 12 bits (=11 bits)
•Almost all subjects could distinguish 4 counts in 12 bits (=10 bits)
•Conclusion –12 bits per color per pixel should be “safe” into the future

No, for full blu-rays (aka. Remux), or straight from disc.
Because the content is naturally dithered.I have an unfortunately large number of discs where that is not the case. It seems to be more common with things shot digitally rather than film. It seems to be a real problem with anything shot on RED. (it's immediately obvious to me which films have been)

Again, very little of madshi´s algorithms (like chroma upscaling differences between Jinc and NNEDI3) can be seen with your human eyes when talking about watching actual movies. However, when zoomed in or enhanced, you will actually see that there are differences. And these differences don´t just magically disappear. They will be there even if you don´t see them. I don´t know why you keep arguing like that. What I find is that you might see no difference in chroma algorithms (assuming they are sufficiently sharp) but every so often a scene appears where the difference is clearly visible. Most of the time, Bicubic 75 AR is going to look almost the same as Jinc 3 AR or NNEDI3. But every so often you're going to find an image where there is obviously aliasing caused by using Bicubic rather than Jinc or NNEDI3.

I want the image to look its best all the time, not just most of the time.
Deblurring Jinc5 with a different recipe than the one used for "your" Jinc3 gives a scheme which actually is very different: It really "de-aliases" "pixel art" diagonal lines in a way that no reasonable Jinc3 can. (I do not remember if Jinc4 at about the same deblur gave close results to Jinc5. I do remember, making a note to myself that EWA Lanczoses seem to work better with odd numbers of lobes.)It would be interesting to see, if nothing else. There were definitely some benefits to Jinc 4, but I did not feel that the increase in ringing was a worthwhile trade-off.

[cyberbeing]

Do you guys use debanding for your ripped Blu Rays or is it usefull only for low quality/SD content.

I use a customized lower than low deband preset for high quality content.

avgDIF 0.3, maxDIF 1.3, midDIF 0.6, angleBoost 2.0, maxAngle 0.08 + analyze gradient angles

Just barely strong enough to smooth out minor imperfections in detected gradients, while not harming detail. That said, this was tuned specially for the minimum visible improvement on my calibrated GDM-F520 CRT along with NNEDI3 luma doubling and chroma upsampling. If you have a display which cannot render perfectly smooth gradients, cannot show 1-step differences in any of the RGB channels distinctly, or are not using NNEDI3, you may need to go a bit stronger then this to have a positive effect. Just keep in mind that the madVR's default Low preset is already very borderline in terms of detail preservation, so that should be your upper limit for high quality content.

[NicolasRobidoux]

...
It would be interesting to see, if nothing else. There were definitely some benefits to Jinc 4, but I did not feel that the increase in ringing was a worthwhile trade-off.
AR changes the game.

[XMonarchY]

IMO, its not worth it.

Nobody super enhances the contrast in actual movie watching.
Nobody can actually see the worm pattern with their naked eye.

Let me remind you that the worm pattern is visible ONLY in flat, static, zoomed, and super contrast enhanced test patterns.

I don't want to add fuel to fire as I am not knowledgeable about this subject, but some things that you cannot see with a naked eye can still affect overall perception of the image and motion. Many can't tell pinpoint any dithering by looking at the picture without zooming, but they can tell that the image is different with and without it. Colorimeter measures also pick up on the dithering AFAIK. I couldn't tell what in the world Error Diffusion was even doing, but I could tell that image rendering was better than with regular random dithering. The same could be said about this case and I trust madshi's decision.

Another possible example (please don't kill me if its bogus) - I can definitely see the difference between default WMP rendering and madVR rendering, and then I can see a difference between madVR chroma & image upscalers, and other settings. Those settings target 4:4:4 chroma subscampling, while my TV only uses 4:2:2 @ 24Hz. Technically, I am not supposed to see a difference, but I do... and yet I can't pinpoint at what exactly is different either!

MadVR is all about the best quality rendering - its known to be GPU/CPU-heavy, so there is no reason to hold it back. I am glad that video rendering demands have finally caught up with 3D rendering as far as requirements go. It should push nVidia and AMD to improve their drivers and future videocard architecture to accommodate such needs.

[iSunrise]

What I find is that you might see no difference in chroma algorithms (assuming they are sufficiently sharp) but every so often a scene appears where the difference is clearly visible. Most of the time, Bicubic 75 AR is going to look almost the same as Jinc 3 AR or NNEDI3. But every so often you're going to find an image where there is obviously aliasing caused by using Bicubic rather than Jinc or NNEDI3.

I want the image to look its best all the time, not just most of the time.
Indeed, I also choose the settings that are giving me the most consistent results. That´s why in the past I went with Bicubic75, too, because it´s just amazingly cheap for what you get out of it. However, like you say, that comes with it´s own problems, like potentially too much aliasing or ringing, which may already be there in the source and which will get even more amplified by it´s relatively high sharpness. With the additional AR introduced, it completely changed the game, though, the negatives suddenly rarely mattered. Because without AR, I found it specifically distracting on content that needs deinterlacing or was deinterlaced wrong. SoftCubic worked much better in these (rare) cases.

With NNEDI3 though, this consistent behaviour has gotten a lot easier to achieve, because it actually is also perfectly applicable to game recordings for instance, not just for movies or other clips. And if the HQ dithering (whatever or however it may be implemented in the end) doesn´t leave behind strange artifacts, everyone should immediately benefit, because there´s a lot of different setups and content out there that will always show flaws more than others. It doesn´t even have to get into the completely insane territory, but a balanced and broad approach that improves upon what we already have, without much additional performance loss or other limiting side effects.

@madshi:
I got word back from Blaire about the reported OpenCL issue(s) >327.23, NV has actually reproduced it now, but they didn´t get into specifics of when a fix is going to happen. Not sure if that still matters though or what you´ve decided once you also have NNEDI3 running well in DirectCompute. But at least they know about the bug now.

[XMonarchY]

madVR v0.87.4

* workaround added: NNEDI3 upscaling failed/froze with newer NVidia GPUs
* fixed: NNEDI3 chroma upscaling produced wrong colors with 10bit sources
* got rid of some unnecessary texture sharing

I'm using GTX660 with 334.67 driver (windows 8.1)
and using potplayer, mpc-hc + madVR v0.87.4

but still have driver crash when using scaling algorithms - image doubling - NNEDI3 options
also have wrong colors issue (all green) when using scaling algorithms - chroma upscaling -NNEDI3

tested with 720p x264 video (not 10bit, it just commonness x264 video)

please solve this issues

when "enable automatic fullscreen exclusive mode" , driver don't crash and no color issue but too much frame drop and tearing

Have many advantages when use NNEDI3 upscaling?
for example 720p little more similar as 1080p?


p.s. tooooooo difficult random questions when write on this forum ... that's so subjective ...
please change random questions


The latest drivers have broken OpenCL. The last driver set that has functioning OpenCL is 327.23. nVidia most likely reproduced the error and will include a fix in future drivers.

[Shiandow]

This is a contrast enhanced and zoomed image of the error diffusion algorithm:

[...]

You want to avoid the ugly worm pattern in 18, 20 ,24.
Every bar should look like number 21 via introducing some randomness in the thresholds as suggested by Cris Luengo (http://www.cb.uu.se/~cris/blog/index.php/archives/355).

And random dithering (http://s23.postimg.org/kg2kjoyob/RD_0s.png) if someone want to compare.

Could someone do a similar test but with some random coloured pixels thrown in? This might cause coloured patterns to show up instead of gray ones, I'm interested to see if this would be more noticeable.

I also don't quite understand why these lines appear, I've tested the standard Flloyd-Steinberg error diffusion which didn't cause these patterns, from what I understand Madshi uses a method which should be superior not worse.

Edit: Apparently it can also happen using the Flloyd-Steinberg algorithm, but it might be possible to prevent it from happening.

[Ver Greeneyes]

Tell me if you see any worm pattern.Not to beat a dead horse, but.. Yes I can, on level 18. It's too small to see on my 24" monitor unless I get about twice as close to it as I usually do, and I probably wouldn't notice it if the image wasn't static, but I can imagine it being visible on a big-ass TV with a high contrast ratio where the pixels are much bigger :)

[6233638]

AR changes the game.Well, it removes the obvious effects of ringing, but not all of them.

Using an old example, comparing Lanczos 3 AR with Lanczos 8 AR:
http://www.abload.de/img/u7wqvvosvv.jpg

You can see that the right of the red circle is still affected, even though the obvious signs of ringing disappear.

[madshi]

zhoufang (http://www.cb.uu.se/~cris/blog/matlabimages/dither_zhoufang.png) from cris' blog looks much better than ed+random fix. It's apparently just as fast too.
No, it's not as fast.

Since this should only affect the least significant bits in the output colors maybe simply using a different rounding mode when converting high precision RGB to 6/7/8bit RGB would be sufficient. Depending on what you're using right now round to even or stochastic rounding might be sufficient.
I'm not familiar with "round to even" or "stochastic rounding". Can you clarify what that means?

madVR v0.87.4

* workaround added: NNEDI3 upscaling failed/froze with newer NVidia GPUs
* fixed: NNEDI3 chroma upscaling produced wrong colors with 10bit sources
* got rid of some unnecessary texture sharing

I'm using GTX660 with 334.67 driver (windows 8.1)
and using potplayer, mpc-hc + madVR v0.87.4

but still have driver crash when using scaling algorithms - image doubling - NNEDI3 options
also have wrong colors issue (all green) when using scaling algorithms - chroma upscaling -NNEDI3
It's nice that you read the v0.87.4 announcement post. It would have been even nicer if you had read the *whole post* and not just a part of it... ;)

Maby this will help?
The MWC64X Random Number Generator (http://cas.ee.ic.ac.uk/people/dt10/research/rngs-gpu-mwc64x.html)
Thanks, might help, will have to test.

zhoufang without random noise might be interesting. It's supposed to workarond difficult greys.
Please note that all the discussion on Cris' blog is about 8bit -> 1bit error diffusion. Some of the optimizations are especially targetted at exactly this operation. madVR is doing 16bit+ -> 8bit error diffusion, which in concept is similar, but not all of the "tricks" suggested on Cris' blog work for what madVR does.

I got word back from Blaire about the reported OpenCL issue(s) >327.23, NV has actually reproduced it now, but they didn´t get into specifics of when a fix is going to happen. Not sure if that still matters though or what you´ve decided once you also have NNEDI3 running well in DirectCompute. But at least they know about the bug now.
Great - thanks! I think I'll switch completely to DirectCompute, so the bug fix is probably not important for me, anymore. But it's not decided yet.

Mathias: Now I think that I remember better:

Deblurring Jinc5 with a different recipe than the one used for "your" Jinc3 gives a scheme which actually is very different: It really "de-aliases" "pixel art" diagonal lines in a way that no reasonable Jinc3 can. (I do not remember if Jinc4 at about the same deblur gave close results to Jinc5. I do remember, making a note to myself that EWA Lanczoses seem to work better with odd numbers of lobes.)

Would you like me to dig this out/experiment or should I leave you alone in your NNEDI bliss?
I would like to see what your modified Jinc5 can do. This test image might be a good candidate for pixel art:

http://madshi.net/SNES.png

Here are comparisons between NNEDI3 vs. Jinc3AR, and between Jinc3AR vs. Jinc3:

http://screenshotcomparison.com/comparison/59698
http://screenshotcomparison.com/comparison/59702

[*Touche*]

I use a customized lower than low deband preset for high quality content.

avgDIF 0.3, maxDIF 1.3, midDIF 0.6, angleBoost 2.0, maxAngle 0.08 + analyze gradient angles

Just barely strong enough to smooth out minor imperfections in detected gradients, while not harming detail. That said, this was tuned specially for the minimum visible improvement on my calibrated GDM-F520 CRT along with NNEDI3 luma doubling and chroma upsampling. If you have a display which cannot render perfectly smooth gradients, cannot show 1-step differences in any of the RGB channels distinctly, or are not using NNEDI3, you may need to go a bit stronger then this to have a positive effect. Just keep in mind that the madVR's default Low preset is already very borderline in terms of detail preservation, so that should be your upper limit for high quality content.

I'm using a 55" Panasonic ST60. I think it would benefit from a touch stronger settings, like default low, if I remember plasmas issues correctly.

[toniash]

Here are comparisons between NNEDI3 vs. Jinc3AR, and between Jinc3AR vs. Jinc3:

http://screenshotcomparison.com/comparison/59698
http://screenshotcomparison.com/comparison/59702

impressive!

[DragonQ]

That NNEDI3 looks pretty tasty for emulators. :D

[pie1394]

I have mentioned debanding_with_angle_detection + NNEDI3 chroma 2x even improves the Sony XCA7 super-resolution engine's performance for 1920x1080 4:2:0 contents. Comparing with Jinc3AR, it increases the perceptual resolution and stability on patterned object shapes under moving scenes -- in either interlaced or progressive format. :D

[The 8472]

I'm not familiar with "round to even" or "stochastic rounding". Can you clarify what that means?

round to even means on 0.5 it chooses to even integer value. 0.5 -> 0, 1.5 -> 2, 2.5 -> 2. 3.5 -> 4, 4.5 -> 4, etc.
stochastic rounding picks at random whether to round up or down on 0.5.

The differences are relevant to error propagation, always rounding up on 0.5 introduces some bias.

But rounding may not be the right thing to do at all if you're using integer RGB -> RGB since only the range [0.0,0.5] would result in black while (0.5,1.5) would result in 1, i.e. full black and full white would get squeezed into a smaller interval. It's probably more appropriate to use if you come from a larger color space and have to clamp anyway.

I only brought it up because I was thinking that if you want to introduce randomness in the dither then it should be no more significant than the quantization error, so if it could be incorporated in the quantization step it would probably be the best.

Another issue with grey-grey gradients is that if we add noise separately to the RGB channels it will end up propagating to different pixels, basically introducing colored noise when there shouldn't be any.

Of course it's very unlikely that real content will have perfectly monochrome gradients with zero deviation between the RGB channels.

[6233638]

Your Jinc3 (http://i1.someimage.com/c5pKJ03.png) images are bugged. This (http://i1.someimage.com/a5cCz3t.png) is what Jinc3AR actually looks like...

This is actually a bit concerning. I thought it was just a fluke, but I actually experienced this exact same "madVR blurry bug" yesterday when I was testing various settings on a paused video and taking screenshots.You're confusing Jinc with NNEDI. The images are correct - Jinc 3 has a tendency to blur together images like that. (which is actually beneficial for games, as that's simply dithered because they had a limited palette to work with)

For example, Pitfall via an RGB connection:
http://abload.de/img/qxshdo5e.png

Pitfall via a Composite connection:
http://abload.de/img/pxstore0.png

Source: http://www.neogaf.com/forum/showpost.php?p=72083811&postcount=77

[huhn]

pixel art can be a real problem for needi3:

image 1 source (http://abload.de/img/sourcez0k53.png)

jinc3ar (http://abload.de/img/schweifjinc3lkkkj.png)
spline3ar (http://abload.de/img/schweifspline3dpk0i.png)
neeid3x4spline3ar (http://abload.de/img/schweifx4spline3t9j2a.png)

huge artifacts in the bottom left and a lot of small ones with needi3, jinc is known to have problems with this as you can see

image 2 source (http://abload.de/img/bsource8ajv2.png)

jinc3ar (http://abload.de/img/bschweifjinc3gpjoc.png)
spline3ar (http://abload.de/img/bschweifspline3qljgk.png)
needi3x4spline3ar (http://abload.de/img/bschweifx4spline3s8kij.png)

with needi3 the face in the bottom left looks really good but there a artifacts in the fonts again.

64 neurons are used for all screens
the mice position can be of by 1 frame in this lossless rgb video i get a black screen when i use neeidi3 with my hd4000 for 1 frame i'm sorry

[6233638]

I think the goals for scaling up pixel art are very different from scaling up video, or even animation.
Pixel art was very useful for showing issues with the anti-ringing filter, but I don't know that it's useful for judging the quality of scaling algorithms.

[cyberbeing]

You're confusing Jinc with NNEDI.
Oops, I missed a setting (too many profiles...). Deleted that post.

I did experience a blurry bug yesterday though... Probably was just a fluke with PrtScn, so I won't worry about it.

[Ver Greeneyes]

round to even means on 0.5 it chooses to even integer value. 0.5 -> 0, 1.5 -> 2, 2.5 -> 2. 3.5 -> 4, 4.5 -> 4, etc.I think this is usually called "round to nearest, ties to even" or "round to nearest, half to even". It's the default rounding method for IEEE 754 floating point.

[kasper93]

Are we using madVR to play games or watch movies? C'mon guys... For such content I personally prefer Nearest Neighbour, I don't mind having pixels on screen and blurring them is not for me. :)

[The 8472]

Are we using madVR to play games or watch movies? C'mon guys... For such content I personally prefer Nearest Neighbour, I don't mind having pixels on screen and blurring them is not for me. :)

There are specialized pixel art upscaling algorithms, NN is not the best choice there. But it certainly isn't madVRs focus, so NNEDI does a decent job there for something it isn't optimized for.

[Shiandow]

Another issue with grey-grey gradients is that if we add noise separately to the RGB channels it will end up propagating to different pixels, basically introducing colored noise when there shouldn't be any.

Of course it's very unlikely that real content will have perfectly monochrome gradients with zero deviation between the RGB channels.

The issue with coloured noise probably already occurs when the image contains some coloured pixels before the gray-gray gradient. In that case the error-diffusion algorithm will no longer output exactly the same values for the different RGB channels.

The reason this doesn't occur on grayscale images is that the algorithm is deterministic and in a grayscale image all RGB channels are exactly the same, therefore it will simply give the same output for all channels. But this effect is more of an coincidence than a feature of the algorithm and is extremely unstable; if anywhere in the picture there is a rounding error which causes the RGB channel error to be different then you will get coloured patterns instead of gray ones.

You can prevent this from happening by performing the calculations in YCbCr space, but this is rather expensive. If you want to see some (exaggerated) examples then see my previous post (http://forum.doom9.org/showthread.php?p=1665961#post1665961).

By the way I've also been able to figure out why the line like patterns happen. This happens whenever a value is 1/3 of the way between the two possible ways to round this value. For example in the 1bit black-white case using error diffusion on a background which is 1/3 white will result in white lines on a black background, which if you think about it is actually 'optimal' since any 3x3 block contains 3 white pixels and 6 black pixels which averages to 1/3 white. I'm not entirely sure if I even agree with the people who claim that this looks worse, I've compared this to a randomized versions of error diffusion and I prefer the normal error diffusion.

Here are the results of the versions I tried:

normal error diffusion:
http://i.imgur.com/jpICuzK.png

Version where the error is diffused in 3 different ways, picked randomly:
http://i.imgur.com/ZpNJ9f0.png

Version with small amount of noise, error diffused in YCbCr:
http://i.imgur.com/sSIZZWb.png

Version with small amount of noise, error diffused in RGB:
http://i.imgur.com/sr64jkH.png

Note that using error diffusion with noise in RGB adds coloured noise.

[iSunrise]

I would like to see what your modified Jinc5 can do. This test image might be a good candidate for pixel art:

http://madshi.net/SNES.png

Here are comparisons between NNEDI3 vs. Jinc3AR, and between Jinc3AR vs. Jinc3:

http://screenshotcomparison.com/comparison/59698
http://screenshotcomparison.com/comparison/59702
This is a great example to test pixel art indeed.

While NNEDI3 4x paired together with Spline or some other Bicubic resizer already does a good job, I can easily see some additional ringing artifacts introduced.

Now I wonder, would it be possible to improve that even more with NNEDI 4x + your AR algorithm, because the AR applied with the additional luma resizer is already too late in the chain to correct it.

I´ve marked some spots in question with red circles and arrows.

http://abload.de/thumb/nnedi3_4x_spline36_alqejri.png (http://abload.de/image.php?img=nnedi3_4x_spline36_alqejri.png)

[The 8472]

Version with small amount of noise, error diffused in YCbCr:
[http://i.imgur.com/sSIZZWb.png]

Version with small amount of noise, error diffused in RGB:
[http://i.imgur.com/sr64jkH.png]

Note that using error diffusion with noise in RGB adds coloured noise.

Interesting. The YCbCr version seems to be self-stabilizing, even the few pixels of color-noise don't result in a cascade of more colored pixels.

How do you mix in the noise?

Of course those are still very artificial examples. Maybe we should compare various dithering methods with color A - color B gradients at some no-π angle instead. To see how the dithering fares in a more realistic scenario. Those do occur in animated content at least. Although it's certainly nice if we can also cover edge cases like grey gradients.

[Shiandow]

I mixed in the noise in the 'rounding' step of error diffusion, so instead of rounding 0.3 to 0 I added a small amount of noise so it has a small chance of rounding to 1. This is basically the same as dithering but I used a smaller amount of noise. By the way if you use a small enough amount of noise it is actually possible to remove the coloured pixels entirely from the YCbCr version, but it also means that there are larger regions with just lines.

[bacondither]

I have played around some in matlab and the results:

Regular Floyd Steinberg (http://s30.postimg.org/m6nuakxht/floyd.png)

Floyd Steinberg with "new = 255*(old >= 128+(rand)*90);" (http://s29.postimg.org/fblkels3r/floyd_rd90.png)

[Ver Greeneyes]

Here's another scaler for pixel art: Reverse Antialiasing Shader (http://board.byuu.org/viewtopic.php?f=10&t=3211) (not the original source, but it has more information and links back to the original article).
Floyd Steinberg with "new = 255*(old >= 128+(rand)*90);" (http://s29.postimg.org/fblkels3r/floyd_rd90.png)
The result looks nice, but 90 and 100 which was mentioned earlier both seem pretty arbitrary. How does something like 64 (in other words, 128 / 2) look? Other fractions that seem less arbitrary (but without justification): 2 * 128 / 3 = 85 and 3 * 128 / 4 = 96.

[iSunrise]

madshi, I have a prores file (MOV container) that LAV reports as yuv422p10le with PCM audio. When I activate deinterlacing and check "if in doubt, activate deinterlacing", madVR for some reason thinks that the file needs IVTC (it says "deinterlacing on: settings" in the OSD) and playback stutters like crazy. The weird thing is that when I check "if in doubt, deactivate deinterlacing" or force film mode, while deinterlacing is still active, madVR doesn´t do IVTC or deinterlace it and the file plays perfectly smooth. Also, it reports "deinterlacing off: says upstream". Does madVR for some reason ignore the upstream, when I leave it at "if in doubt, activate deinterlacing". Is that how it´s supposed to work? I´m a little confused by this.

Unfortunately I cannot make you a sample, because the file apparently has the header at the end of the file and it´s 6GB total. If you need a sample, anyway, I can remux it to MP4 though, if that helps and add it to the bug tracker if need be.

[bacondither]

The result looks nice, but 90 and 100 which was mentioned earlier both seem pretty arbitrary. How does something like 64 (in other words, 128 / 2) look? Other fractions that seem less arbitrary (but without justification): 2 * 128 / 3 = 85 and 3 * 128 / 4 = 96.

Here is a value of 64 (http://s22.postimg.org/x4pihfqxt/floyd_64.pnghttp://)
It does not hide all the error diffusion artifacts. It seems a value of >80-85 is minimum.

Using the random code from Zhou-Fang is maybe better...

new = 255*(old >= 128+(rand*96)*strg(img(ii,jj)+1)); Rand*96 instead of 128

Image (http://s23.postimg.org/5drazaoi3/floyd_96.png)

[The 8472]

I mixed in the noise in the 'rounding' step of error diffusion, so instead of rounding 0.3 to 0 I added a small amount of noise so it has a small chance of rounding to 1. This is basically the same as dithering but I used a smaller amount of noise. By the way if you use a small enough amount of noise it is actually possible to remove the coloured pixels entirely from the YCbCr version, but it also means that there are larger regions with just lines.

So given a value from in = [0.0, 1.0] and then quantizing it to an integer out = [0,1] your method of adding noise can flip out for any given value of in? Wouldn't it be better to add noise to the input bits that are less significant than the least significant bits of the output value? That way it would only add randomness if it's close to a threshold and otherwise behave deterministically.

Basically, are you adding noise before or after quantizing?

You have to remember that madVR is already operating with 8-bit output depth, so we don't want to add randomness in most cases, we only want to add a tiny little amount of randomness to break degenerate edge-cases where the propagating errors keep flip-flopping between some specific values. So we only really want some noise in the less-than-significant internal state to act as tie-breaker and leave the output state unperturbed when possible.

[Shiandow]

I added the noise before quantizeing and only used values between -1/9 and 1/9, so something like [0.0,1.0] would always round to [0,1]. If you combine this with the error from the other pixels then it is enough to change the output occasionally which prevents regular patterns from emerging. But personally I prefer the method where the error is diffused randomly to other pixels, this does not introduce any colours which did not exist in the original image.

It might also be a good idea to lower the error slightly before distributing since otherwise an error introduced on one side of the image can still have an effect on the other side of the image which leads to unexpected results. For example in the image from this post (http://forum.doom9.org/showthread.php?p=1666263#post1666263), you can clearly see that under the 22 and 24 the algorithm behaves differently, which it shouldn't.

Edit: Oh never mind that last part, the random dithering version of that image has the same problem so it seems to be a problem caused by the image, not the algorithm.

[The 8472]

But personally I prefer the method where the error is diffused randomly to other pixels, this does not introduce any colours which did not exist in the original image. Agreed, this seems to be preferable, especially in RGB.

It might also be a good idea to lower the error slightly before distributing since otherwise an error introduced on one side of the image can still have an effect on the other side of the image which leads to unexpected results. Would that preserve uniformity of the error distribution? On the one hand it would take more from one pixel than it gives to the neighboring ones, on the other hand errors go in both directions so it should even out. But that's assuming it's perceptually uniform.

Anyway, flat, perfectly-grey areas at specific threshold values are an edge case. I think gradients and especially color-color ones would serve as more realistic test cases that should easily break any pattering in the diffusion.

[Shiandow]

Would that preserve uniformity of the error distribution? On the one hand it would take more from one pixel than it gives to the neighboring ones, on the other hand errors go in both directions so it should even out. But that's assuming it's perceptually uniform.

It shouldn't change the uniformity much, it will only cause the algorithm to 'forget' an error gradually so the effect of 1 pixel shouldn't affect a pixel 100's of pixels away. It might help in the RGB case to 'realign' the different channels.


Anyway, flat, perfectly-grey areas at specific threshold values are an edge case. I think gradients and especially color-color ones would serve as more realistic test cases that should easily break any pattering in the diffusion.

I agree that these are edge cases and using an algorithm which is technically inferior to prevent patterns which are below the threshold of human sight, might be a bit silly. Although it would be nice if we could prevent coloured patterns from occurring on gray areas.

Edit: I tried it, gradually 'forgetting' the error does not help much to realign the different channels.

[SamuriHL]

Great - thanks! I think I'll switch completely to DirectCompute, so the bug fix is probably not important for me, anymore. But it's not decided yet.


That being said, having options is better than not. :D Meaning if they can fix it, that'd give you another avenue to explore should the need arise and you hit a wall with DirectCompute or decide that OpenCL can do something that DirectCompute can't down the road.

[Ver Greeneyes]

Here is a value of 64 (http://s22.postimg.org/x4pihfqxt/floyd_64.pnghttp://)
It does not hide all the error diffusion artifacts. It seems a value of >80-85 is minimum.

Using the random code from Zhou-Fang is maybe better...

new = 255*(old >= 128+(rand*96)*strg(img(ii,jj)+1)); Rand*96 instead of 128

Image (http://s23.postimg.org/5drazaoi3/floyd_96.png)
Thanks for testing, the second image does look nice :)

[iSunrise]

Here is a value of 64 (http://s22.postimg.org/x4pihfqxt/floyd_64.pnghttp://)
It does not hide all the error diffusion artifacts. It seems a value of >80-85 is minimum.

Using the random code from Zhou-Fang is maybe better...

new = 255*(old >= 128+(rand*96)*strg(img(ii,jj)+1)); Rand*96 instead of 128

Image (http://s23.postimg.org/5drazaoi3/floyd_96.png)
Wow, your second image looks almost too clean to be true. Unfortunately, madshi said it´s "not as fast", so I´m not sure if he´s willing to do that. But that just looks marvelous and I would be all for it if there´s not a huge performance hit. But that´s for madshi to decide.

If you compare it with the original image (http://www.cb.uu.se/~cris/blog/matlabimages/dither_zhoufang.png) from zhoufang, yours looks a lot more refined in the details of her face and her clothing. And of course, the white single dots in the black are complete gone.

The more I look at it, the more amazed I am. Would love to see how that translates to actual colored pictures or videos.

[Thunderbolt8]

welp since my card cant do that much special tricks anyway atm, Id say implement the best there is, because if I get a new card sometime, then it will be fine :D

[G_M_C]

Hmm, I've finaly found time to see if the newest developments in madVR work for me.

I've got a C2Q 9650XE with an AMD HD7850 @ 950 core / 1250 mem. I use win7-64. I installed latest stable AMD drivers today (from 13.1 to 13.12, using overwrite method -> these days AMD's recommended method). Also updated MPC-HT to latest stable (32bit x86) . Used the option 'reset settings' during install. Removed the older version of LAV filters beforehand.

Then installed madVR 87.4 with DirectCompute V3 build /.ax.

Seems i cannot use nnedi3 without getting dropped frames, not for chroma upscaling nor for image doubling. Tested with an 720p24 anime wich has to be upscaled to a 1080p60 display. Playing with settings (neurons, smooth motion on/off, error difusion on/off and so forth) did not help much.

Alas, but using 'old settings' as with 'older madVR', but with error diffusion still improved quality.

But something tells me my HD7850 should do better. Anyone ideas what could be wrong here ?

[ryrynz]

http://screenshotcomparison.com/comparison/59702

There's that thinning.. The White 0 top middle beside the blue rectangle. The right hand side of it, Jinc clearly draws a straight line, but with NNEDI and Spline there's that dimple.
Then there's all those nasty artifacts around the x's and the crazy job it does on the shading of the pillars, there's much to like but also much to not like.

[The 8472]

Tested with an 720p24 anime wich has to be upscaled to a 1080p60 display.
Using nnedi means it gets upscaled to 1440p and then downscaled to 1080p. So you have to also pick a downscaling algorithm that's not too expensive. It'll also have to do the chroma doubling to 4:4:4 and upscale that separately.
Error diffusion on top of that.

That can be quite some load. So if you want to sort of start with a minimal-load profile you could pick bilinear for everything, SM off, ED off. and then watch the GPU load and queue lengths.

Doing some DXVA stuff can also add a performance penalty due to bus transfers afaik.

In some cases making the queues a bit longer can help.

[DragonQ]

With all these great upscaling algorithms, how long will it be until we can enhance images by maybe 100x like in CSI? :D

[The 8472]

With all these great upscaling algorithms, how long will it be until we can enhance images by maybe 100x like in CSI? :D

Offtopic, but see http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0083325

[Dogway]

I lost image after version 0.86.9 on Techsmith codec videos. I use Intel HD 4600 on XP, on 7 it works fine. I'm waiting for a new dedicated card but thought you should know this.