Gamma-aware, dithered colorspace conversion and resizing

[madshi]

FWIW, most of these dithering algorithms are optimized for halftoning (8bit -> 1bit). I believe that for our use (high bitdepth -> 8bit) we have different needs. I would suggest to use the method proposed in the paper "Optimized Error Diffusion for Image Display" (PDF download available via google). This dithering method whitens the quantization error which should produce best results for our needs, IMHO.

But of course the optimal solution would be if AviSynth would be able to handle a full high bitdepth processing chain (16bit int or even 32bit float). This way dithering could be avoided altogether, or at least delayed to the very last step. E.g. dithering could be done to 10bit, with the final result fed to the new x264 10bit encoder. The current limitation of AviSynth to 8bit means that basically every processing step that internally produces higher bitdepth (resampling, color conversion, and many more) must do an internal dithering step for optimal results. If this is not done, rounding errors will add up. But even if it's done, dithering noise adds up. Having higher bitdepth support would be better in any case.

[PhrostByte]

Originally Posted by madshi

FWIW, most of these dithering algorithms are optimized for halftoning (8bit -> 1bit). I believe that for our use (high bitdepth -> 8bit) we have different needs. I would suggest to use the method proposed in the paper "Optimized Error Diffusion for Image Display" (PDF download available via google). This dithering method whitens the quantization error which should produce best results for our needs, IMHO.

But of course the optimal solution would be if AviSynth would be able to handle a full high bitdepth processing chain (16bit int or even 32bit float). This way dithering could be avoided altogether, or at least delayed to the very last step. E.g. dithering could be done to 10bit, with the final result fed to the new x264 10bit encoder. The current limitation of AviSynth to 8bit means that basically every processing step that internally produces higher bitdepth (resampling, color conversion, and many more) must do an internal dithering step for optimal results. If this is not done, rounding errors will add up. But even if it's done, dithering noise adds up. Having higher bitdepth support would be better in any case.

I've also wondered if the classic error diffusion algorithms were optimal for high bit depths. From my tests it seems to perform acceptably, but if there's something better I definitely want to know. Interesting paper, I'll read it. Thanks.

I've been considering making a backwards-compatible extension to IClip -- it would have GetFrame16 and GetFrame32F member functions, and transparently down-convert if GetFrame was called by a legacy filter. All anyone would need to do is call dynamic_cast<IClipEx*>(clip) to test if a clip offers the extended formats. It'd be a fair amount of work, but I think rewarding in the end. Would people use this?

[jmac698]

Yes, I would use it. For now I'm using a script-compatible system where the extra low bits are in a separate clip. There's other filters that use deep color processing as well. Could you make it compatible to my tools for now? It would be the easiest approach.
http://forum.doom9.org/showthread.ph...07#post1467907

[PhrostByte]

v3 is out. sub-pixel cropping, selectable resize kernels, and some small optimizations. big overhaul to support SSE touched a lot of code, so be on the lookout for bugs.

Download for 32-bit and 64-bit:
http://sourceforge.net/projects/int6...3.zip/download

Full documentation:
http://svn.int64.org/viewvc/int64/re...l?revision=255

[PhrostByte]

Originally Posted by jmac698

Yes, I would use it. For now I'm using a script-compatible system where the extra low bits are in a separate clip. There's other filters that use deep color processing as well. Could you make it compatible to my tools for now? It would be the easiest approach.
http://forum.doom9.org/showthread.ph...07#post1467907

Unfortunately transparent compatibility is basically impossible -- all the filters you're depending on would have to explicitly support it. Until they did, the video they see would just appear like any other clip (low bit depth and non-linear colorspace), so you wouldn't be able to get much out of it.

Given the amount of filters we have that haven't been updated in ages and have had their authors disappear, I don't really know how big of an impact it would be able to make. But I guess we have to start somewhere?

[Dogway]

Perceived bugs:
-Omitting or setting dither=false, causes a crash.
-looks like dither only works when changing colorspace from yv12 to rgb (not on resizing etc)
-Strange black artifacts around contrasty edges when DSTCOLORSPACE is RGB (independent of input)
-Overall there's strong ringing, black outline plus bright ringing.

Tell me if you want me to upload my tests.

[PhrostByte]

Originally Posted by Dogway

Perceived bugs:
-Omitting or setting dither=false, causes a crash.
-looks like dither only works when changing colorspace from yv12 to rgb (not on resizing etc)
-Strange black artifacts around contrasty edges when DSTCOLORSPACE is RGB (independent of input)
-Overall there's strong ringing, black outline plus bright ringing.

Tell me if you want me to upload my tests.

Please do, I've tested all the possible colorspace conversions and can't replicate any of that.

[Dogway]

Dither=false crashes no matter what. I even emptied the plugin folder.
Here are my tests:
http://www.mediafire.com/?l6t0pf2mfto32xg

[PhrostByte]

Originally Posted by Dogway

Dither=false crashes no matter what. I even emptied the plugin folder.
Here are my tests:
http://www.mediafire.com/?l6t0pf2mfto32xg

Thanks! v4 is up with everything fixed.

The crashes were a bug from my SSE preparations, and the dithering artifacts were due to resizing creating out-of-range values (which is normal, I just didn't clamp them properly).

I believe the extra ringing is merely a result of linear blending. Regular Spline36 has it too, just not as intense -- just like with other images, the sharp contrast is brought out more with linear blending.

Download for 32-bit and 64-bit:
http://sourceforge.net/projects/int6...4.zip/download

Full documentation:
http://svn.int64.org/viewvc/int64/re...l?revision=257

[tritical]

Originally Posted by PhrostByte

I believe the extra ringing is merely a result of linear blending. Regular Spline36 has it too, just not as intense -- just like with other images, the sharp contrast is brought out more with linear blending.

If you are enlarging an image, working with linear values will absolutely create more ringing around edges simply because you are enlarging the difference between dark/bright pixels... so the overshoot of resampling functions with negative coefficients will be greater than if you work with gamma corrected values. Really, for enlarging there is no reason to believe that working with the linear values will give a better result (with standard fir interpolation functions).. because the resampling functions do not approximate the true image better when fit to the linear values than when fit to the gamma corrected values. At least I have never seen any evidence of this, and my tests show that it isn't true.

Shrinking or downsizing an image is different because you have to low-pass filter the image before decimation (or interpolation) in order to avoid aliasing. It is the low-pass filtering step that requires working in linear space if you want to approximate an optical blur. Avisynth's resizing functions don't separate those two steps. The low-pass filtering is accomplished by increasing the support width of the resampling function by a factor of old_width/target_width. Obviously, the larger the downsizing ratio the stronger the low-pass filtering, and therefore the larger the difference between working with gamma-corrected versus linear values.

Random thought: I don't know how flexible Avisynth's resampling code is, but it would be really nice if you could manually control the support width - maybe by supplying a user specified scaling factor >= 1.0 - instead of having it always scaled by a factor of max(old_width/target_width,1.0).

[*.mp4 guy]

Increasing support width will blur things to all hell. The algorithm avisynth uses should be able to do this easily, iirc, if the code itself is this capable, I have no idea.

[cretindesalpes]

Nice! It would be great if you could also add support for 16-bit data as output and/or input (for example, with the MSB part stacked onto the LSB), so the dithering could be done later.

[Dogway]

@PhrostByte: Yes, everything looks right now : D
Just a question, still images should be treated as... Rec.601?

Also I made a funny experiment, ported resampleHQ to automttap3:

[PhrostByte]

Originally Posted by Dogway

Just a question, still images should be treated as... Rec.601?

The matrix is only used for YV12/YUY2 (it defines the mix of R,G,B that goes into Y, and the integer range for Y'CbCr). For everything else it is ignored. I'd assume still images would be RGB.

[Dogway]

ay! yes, what I had been thinking...

[tritical]

Originally Posted by *.mp4 guy

Increasing support width will blur things to all hell. The algorithm avisynth uses should be able to do this easily, iirc, if the code itself is this capable, I have no idea.

What I'm saying is that Avisynth's current method of scaling the support by max(old_width/target_width,1.0) is just a rule of thumb. There is no reason to believe that that value is optimal for every image, and it would be interesting if you could supply the scaling factor yourself (after looking at the code I think this would be very easy to do). However, what I'm really thinking about is if you were to perform low-pass filtering yourself before calling one of the built in resizers. In that case, you wouldn't want the filter support increased because you don't need low-pass filtering just interpolation. Anyways, getting off topic now.

[*.mp4 guy]

Originally Posted by tritical

What I'm saying is that Avisynth's current method of scaling the support by max(old_width/target_width,1.0) is just a rule of thumb.

Avisynths current method is naively theoretically correct for purposes of keeping the frequency response of the output as close to the input as can be done using the current resizing kernel. The only way to do better is to break out the non-linearity. Though, as you mentioned, there are other reasons to make it adjustable. I just don't want people to think that the avisynth interpolators are arbitrary.

[PhrostByte]

Originally Posted by tritical

What I'm saying is that Avisynth's current method of scaling the support by max(old_width/target_width,1.0) is just a rule of thumb. There is no reason to believe that that value is optimal for every image, and it would be interesting if you could supply the scaling factor yourself (after looking at the code I think this would be very easy to do). However, what I'm really thinking about is if you were to perform low-pass filtering yourself before calling one of the built in resizers. In that case, you wouldn't want the filter support increased because you don't need low-pass filtering just interpolation. Anyways, getting off topic now.

I guess you mean min(target_width/old_width, 1.0). I'll add an option to let you override it.

[tritical]

Originally Posted by *.mp4 guy

Avisynths current method is naively theoretically correct for purposes of keeping the frequency response of the output as close to the input as can be done using the current resizing kernel.

Under what [arbitrary] definition of close? Certainly it does not completely capture the only thing that really matters for image processing - the perceived quality of the person doing the resizing.

[SilaSurfer]

PhrostByte nice work! Just tried it.

@Dogway

How did you port resampleHQ to automttap3, and was that image resampled with it? Looks awasome.