Gamma-aware, dithered colorspace conversion and resizing

[PhrostByte]

Hello all, I've made a new filter that performs gamma-corrected colorspace conversion and Spline36 resizing in a single function.

Download version 8 for 32-bit:
http://sourceforge.net/projects/int6...8.zip/download

Full Documentation:
http://svn.int64.org/viewvc/int64/re...l?revision=341

Kernel visualization:
http://svn.int64.org/viewvc/int64/re...c/kernels.html

Usage:

ResampleHQ(clip
           [, int width]
           [, int height]
           [, string dstcolorspace]
           [, string srcmatrix]
           [, string dstmatrix]
           [, bool dither]
           [, float src_left]
           [, float src_top]
           [, float src_width]
           [, float src_height]
           [, string kernel]
           [, float karg1]
           [, float karg2]
           [, float kblur_x]
           [, float kblur_y]
           [, string chroma_kernel]
           [, float chroma_karg1]
           [, float chroma_karg2])

Too many settings to list here, see documentation above for what they all do.

Note: Dithering YV12 and YUY2 output is not currently supported and will come at a later version. Interlaced formats are also not supported.

Here's an example from Avisynth's gamma-ignorant Spline36Resize:



And now here's the gamma-aware ResampleHQ version:



Note how the ResampleHQ image is much brighter overall, and shows faint light in completely new places around Canada, Alaska, western United States, and along coastlines.

This image was cherry-picked to show how off Avisynth's resizers can be with sharp contrast. Most video will show far less improvement.

[Dogway]

music to my ears! will test it soon.

[jmac698]

http://avisynth.org/mediawiki/External_filters#Resizers
Updated.

[Dogway]

Just tested.
I can see dither still is not optimal.
Also I wanted to ask if the chroma around some edges is normal, notice the reddish outline:
http://i212.photobucket.com/albums/c...y/Misc/raw.png
http://i212.photobucket.com/albums/c...spline64-1.png
http://i212.photobucket.com/albums/c...ResampleHQ.png

[PhrostByte]

Thanks for trying it out. I see the red edges you're talking about, I'll check it out.

Dithering isn't even there yet. The code is finished but not yet integrated. Next build will have it.

[Archimedes]

What a lucky break. In the last view days I was thinking about the same issue (making a plugin for gamma corrected resizing). And now I see, somebody has already done the job.

Regarding the picture above, I can't reproduce red edges with ResampleHQ.

Spline36Resize >>
ResampleHQ >>

[Dogway]

I imagine it has to do with yuv content, and chroma subsampling. But maybe that is the expected behaviour, I was just wondering from my ignorance.
Talking about kernels any options for automttap3? Its a function so maybe it implies kernels from bilinear and lanczosresize.

[PhrostByte]

The red edges were due to a chroma conversion bug with Y'CbCr formats. I've fixed it already, am finishing up dithering before next release though. Some time today probably.

[pbristow]

This is good to see!

Once all the bugs are out of the colourspace/gamma correction & dither code, will you then be adding other options for the resize algorithm (e.g. Lanczos, or even good old Bilinear & Bicubic? )?

Is it modular enough that you could just add an option such as ' "resizer" = string ', and use it to call any existing AVIsynth resizer in between the input and output stages? That would open to door to things such as gamma-aware EDI, etc.

[Archimedes]

There is also a bug with the alpha channel (RGB32 input).

[PhrostByte]

Originally Posted by pbristow

This is good to see!

Once all the bugs are out of the colourspace/gamma correction & dither code, will you then be adding other options for the resize algorithm (e.g. Lanczos, or even good old Bilinear & Bicubic? )?

Is it modular enough that you could just add an option such as ' "resizer" = string ', and use it to call any existing AVIsynth resizer in between the input and output stages? That would open to door to things such as gamma-aware EDI, etc.

Other sinc kernels (Lanczos, Spline, etc.) will be added soon. Non-sinc algorithms will come later.

Unfortunately it's not possible to reuse other filters.

Originally Posted by Archimedes

There is also a bug with the alpha channel (RGB32 input).

Actually didn't realize the alpha channel was used by anything. Easy to fix, at least. I'll add it to the list.

[PhrostByte]

v2 is up, download here.

I'll be busy the next couple days so I decided to put out what I've finished. All the bugs identified are fixed (please test to make sure!), and dithering works for RGB output. I unfortunately wasn't able to finish YV12 and YUY2 dithering in time, so that'll come later.

[Dogway]

Thanks for all the effort!
I will wait until you release the dither version for a current project.
And hopefully make this one my default resizer.

[leeperry]

Originally Posted by Dogway

music to my ears!

to mine as well

I'm currently forced to use this for upscaled SD through madVR:

ColorMatrix(mode="Rec.601->Rec.709",clamp=0,opt=0)

I already use SmoothLevels() for a proper TV>PC conversion, it does add dithering...and so does madVR, so I'm not sure that I've got any headroom for more dithering.

How is your plugin supposed to be different/better than the aforementioned command? I strictly want 601>709, and especially not something that messes w/ saturations

[PhrostByte]

Originally Posted by leeperry

How is your plugin supposed to be different/better than the aforementioned command? I strictly want 601>709, and especially not something that messes w/ saturations

TV.601->PC.709 doesn't require any linear blending, so my filter won't be able to do much for you. Two possible improvements which would be negligible:

1. If you're using both ColorMatrix and SmoothLevels together, then ResampleHQ will give you less rounding error.
2. Proper dithering does require linear blending, so ResampleHQ will give more correct results than anything that does non-linear dithering. I'd be (pleasantly) surprised if whatever dithering SmoothLevels does is performed linearly.

The brightness difference you see in those pictures is actually what correct resizing is supposed to look like. ResampleHQ doesn't mess with brightness, saturation, or anything like that at all -- everything else just does it wrong

[IanB]

Neat filter.

For those interested in the source code, it appears to be here.

The dither seems to be error diffusion with a twist. r1 and l1 flip sign on each line. This is fine for l1 because it applies to the rerr2[] pointers of the next line, but r1 applies to the rerr1[] pointers of the current line, so the accumulating error is added to an already processed pixel

[PhrostByte]

Originally Posted by IanB

the accumulating error is added to an already processed pixel

if you scan left->right, then l1 is an offset to the left pixel, and r1 is an offset to the right pixel.

it alternates between scanning left->right and right->left (see the calculation of x). I flip the sign on l1/r1 each line too, so r1 is always an offset to the pixel next in line.

it's called "serpentine scanning" and helps avoid some rare bad looking patterns.

[Mug Funky]

a lil' feature request?

could you possibly implement cropping routines similar to avisynth's resizers? these allow precise (subpixel) cropping of the image which is necessary in all manner of video resizing applications (ie field alignment, correct 702 width scaling from and to HD, etc).

thanks!

btw, very nice work

[IanB]

Originally Posted by PhrostByte

if you scan left->right, then l1 is an offset to the left pixel, and r1 is an offset to the right pixel.

it alternates between scanning left->right and right->left (see the calculation of x). I flip the sign on l1/r1 each line too, so r1 is always an offset to the pixel next in line.

it's called "serpentine scanning" and helps avoid some rare bad looking patterns.

Ah, yes I missed this code

   color::dimension_type x = (y & 1) ? (width - xx - 1) : xx;

You have obviously thought about this a lot. So on the Even lines you diffuse error with this kernel

 1 [ - # 7 ]
16 [ 4 5 0 ]

and on the Odd line with this kernel

 1 [ 7 # - ]
16 [ 0 5 4 ]

What other advantages have you found instead of using the default Floyd and Steinberg kernel?

 1 [ - # 7 ]
16 [ 3 5 1 ]

[PhrostByte]

Originally Posted by Mug Funky

a lil' feature request?

could you possibly implement cropping routines similar to avisynth's resizers? these allow precise (subpixel) cropping of the image which is necessary in all manner of video resizing applications (ie field alignment, correct 702 width scaling from and to HD, etc).

Definitely planned.

Originally Posted by IanB

What other advantages have you found instead of using the default Floyd and Steinberg kernel?

 1 [ - # 7 ]
16 [ 3 5 1 ]

A while ago I read a study that did an automated comparison of various dithering techniques, including all common error diffusion patterns and even randomized ones. It concluded that while the Floyd-Steinberg pattern was near perfect for regular scanning, it was actually sub-optimal for serpentine scanning. The pattern I'm using tested best in its class, outmatched only by a slow and complicated edge-directed binary search algorithm.

I tried it out myself on a few handfuls of images and found that it did help avoid certain bad patterns, so I stuck with it.