Log in

View Full Version : HiAA: Anti-aliasing script with 16-bit support / LUtils / eedi3_resize


zero9999
12th December 2013, 21:03
HiAA v0.13
Download (https://raw.github.com/AviSynth/avs-scripts/master/HiAA.avsi)

To celebrate the release of Avisynth+ and quite a number of updated plugins, I've thrown together an anti-aliasing script for the new era.

Key Features:

Supports all 8-bit AviSynth+ YUV and RGB formats, as well as the 16-bit Dither formats
Anti-Aliasing methods: eedi3, Sangnom2, eedi3+Sangnom2, nnedi3
Supersampling methods: nnedi3, all AviSynth and Dither resizers
optional daa-like pre-pass antialiasing
Built-in mask presets
Supports multiple sharpeners: CSmod, LSFmod, aWarpSharp, MSharpen


While there are certain performance "optimizations" (use of Y8 where possible, processing only the required planes, using the mask as a prescreener for eedi3) the focus of this script is providing high quality anti-aliasing. Center shifts, chroma locations and colorspace conversions are (hopefully) properly taken care of and high-bitdepth processing is used (where possible) when either the input or output is 16-bit.

The script is still a work in progress, so watch out for bugs and sudden parameter changes!

Requirements:

AviSynth+ r1555
Dither v1.24.0
eedi3 v0.9.2 (Firesledge mod)
eedi3_resize v0.11
nnedi3 v0.9.4
nnedi3_resize16 v3.1
Contra-Sharpen mod v3.4
SmoothAdjust v2.90
Masktools v2.0b1
LUtils v0.11
Resize8 v1.1
RgTools v0.91
tmaskcleaner v0.9
Sangnom2 v0.35
Soothe
aWarpSharp2
LSFmod 1.9 (and prerequisites)
MSharpen 0.9
Variableblur 0.5

Input Formats:

Y8/YV12/YV16/YV24 (8-bit or 16-bit stacked)
RGB24/RGB32
RGB48Y/RGB48YV12


Parameters:

+ [string] aa (sangnom2)
* Antiaaliasing method.
* Currently supported:
* eedi3
* eedi3+sangnom2
* nnedi3
* sangnom2
+ [string] ss (nnedi3)
* Supersampling method. Currently supported: all the AVS+/Dither resizing kernels, nnedi3
+ [float] ssf (1.0 for eedi3/eedi3+sangnom2; 2.0 for sangnom2)
* Supersampling factor. Increase this to combat artifacts or excessive blurring.
+ [val] preaa (false)
* Enables daa type pre-pass antialiasing, which uses nnedi3 to interpolate from the 2 source fields
and averages the 2 resulting frames. Causes significant blurring, so only use it on particularly
bad sources where lines won't connect otherwise (e.g. due to bad Deinterlacing/IVTC)
* Will be run in both directions if set to true. Set it to "hor" or "ver" to only preaa in one direction
+ [bool] lsb_in (false), lsb_out (lsb_in)
* Tells the script whether the input clips are 16-bit stacked or 8 bit
* Setting either of these parameters to true will enable lsb processing (which results in a speed hit)
+ [int] y (3), u (2), v (u)
* Processing options for y/u/v planes:
* -x..0 : All the pixels of the plane will be set to -x (disables processing)
* 1 : Disables processing of the plane (plane will contain garbage)
* 2 : Copies plane from the input clip (disables processing)
* 3 : Plane will antialiased and masked
* 4 : Plane will be antialiased and output as is (ignoring the mask)
+ [string] fmt_in (autodetect) / fmt_out (fmt_in)
* Allows you to specify the input and output colorspaces
* Specifying fmt_in is required for rgb48y and rgb48yv12
+ [int] dither (6)
* Dither mode for conversion from 16-bit to 8-bit (refer to dither docs for more information)
+ [float] e_fac (2.0)
* Scaling factor for eedi3/nnedi3 interpolation (antialasing stage).
* Using anything other than multiples of two will incur an additional performance hit
+ [various] ee_alpha, ee_beta, ee_gamma, ee_nrad, ee_mdis, ee_hp
* Eedi3 parameters (refer to eedi3 docs for more information)
+ [bool/string] sharp (false)
* Sharpening applied to the supersampled clip after antialiasing
* Can be True/False or one of the following methods:
* "csmod" : contra sharpening (Default if set to true)
* "lsfmod" : LSFmod with configurable defaults and strength
* "awarpsharp2" : aWarpSharp2 applied at supersampling resolution (ssf*e_fac)
* "awarp4" : aWarpSharp2 applied at 4x source resolution
* "msharpen" : MSharpen
+ [int] cs_strength (75)
* Contra sharpening strength
+ [val] cs_limit (true)
* Limits contra sharpening to either:
* "ss" : the supersampled (see ss) and preprocessed (see preaa) source clip
* "src" : the source clip, seperately supersample with the Spline36 kernel
* Enable this if you want the output to be no sharper than the source
* If set to true, "src" will be used if preaa is enabled, otherwise "ss"
* You might want to override this on terribly aliased sources in case
the contra sharpening reintroduces some of the aliasing
+ [int] aw_thresh, aw_blur, aw_depth (4 for awarpsharp2; 2 for awarp4)
* Parameters passed to aWarpSharp2 (refer to awarpsharp2 docs for more information)
+ [various] lsf_strength (60), lsf_defaults (fast)
* Parameters passed to LSFmod (refer to LSFmod docs for more information)
+ [int] ms_strength (50), ms_threshold
* Parameters passed to MSharpen (refer to MSharpen docs for more information)
+ [string] kernel_d (Spline36)
* Kernel to use for downscaling back to the source resolution
+ [bool] noring (false)
* If enabled, a non-ringing kernel will be used for supersampling
* Doesn't apply to nnedi3 supersampling
+ [string] matrix (709 if input height >= 600, otherwise 601)
* Color matrix to be used for conversions between YUV and RGB
+ [bool] tv_range (true)
* Specifies the range of the pixel values of the input and output clips
+ [string] cplace (MPEG2)
* Chroma siting of the clip to be processed (refer to the Dither docs for more information)
+ [bool/string] mask (simple if no mclip is specified, otherwise false)
* Specifies whether or not the script should build an edge mask for masked antialiasing
* Can be either True/False or one of the following mask presets:
* simple: simple and fast mask, one for each plane
* simple-cmb: like simple, but uses a combined mask for all planes
* precise-cmb: cleaned mask that tries to skip small details
* lines: use this preset for a rather strict "line" mask that skips small details as well as larger detailed areas
* The mask is also used as a prescreener for eedi3 so using one will yield a considerable speedup
+ [clip] mclip (Undefined)
* Allows you to bring your own mask instead of relying on HiAA's internal masking
* The mask must be 8-bit YUV and of the same resolution as the source clip
* If you supply an Y8 mask, HiAA will automatically used its Y plane for masking all processed source planes
* For other color spaces the chroma subsampling must match that of the input clip (use YV24 for RGB input)
and HiAA will mask all planes separatly
+ [int] mthr (30)
* Binarization threshold for the internal masking.
* Decrease if the masking misses aliased edges, increase if the mask includes pristine details and flat areas
+ [bool] show (false)
* If enabled, visualizes the mask instead of running the antialiasing
* Useful for tweaking mask and mthr
+ [bool/int] sa_aa (true for aa methods that include sangnom, otherwise false) sa_aac (false)
* Enables/Disables Sangnom2 based antialiasing on the luma and chroma channels
* Enabled if the supplied is an integer, which will be passed on as aa/aac parameters to Sangnom2
+ [int] threads
* Number of threads to be used by eedi3/nnedi3/Sangnom2


Planned features: sclip support for eedi3+nnedi3, nnedi3gpu support, pre-aa downscaling, presets (suggestions welcome), mask preview for chroma channels, use tcolormask for protecting credits

eedi3_resize v0.11
Download (https://github.com/AviSynth/avs-scripts/blob/master/eedi3_resize.avsi)

eedi3 based resizing script that allows to resize to arbitrary resolutions while maintaining the correct image center and chroma location. Shamelessly copy-pasted from Based on mawen1250s nnedi3_resize16 v3.0. Can work both in 8-bit and 16-bit.

Requirements:

AviSynth+
Dither v1.24.0
eedi3 v0.9.2 (Firesledge mod)
Contra-Sharpen mod v3.4
SmoothAdjust v2.90
Masktools 2.0b1
LUtils v0.1


Input Formats: Y8/YV12/YV16/YV24 (8-bit or 16-bit stacked)

Parameters:

+ [int] target_width, target_height (source width/height)
* Resolution to resize to. Arbitrary resolutions are realized with Dither resizers.
+ [float] src_left (0), src_top (0), src_width(source width), src_height (source height)
* Parameters for cropping before resizing
+ [string] kernel_d (Spline36)
* Dither resizing kernel used for downscaling
* This kernel will also be used when upscaling to arbitrary resolutions that are not exactly multiples of 2
+ [string] kernel_u (Spline64)
* Dither resizing kernel used for upscaling
+ [bool] noring (false)
* Use non-ringing algorithms for Dither resizers
* Enabling this is not recommended, because eedi3_resize will use eedi3 for upscaling edges, anyway
* Downscaling with non-ringing resizers may result in blurring and aliasing artifacts
+ [various] alpha (0.4), beta (0.2), gamma (15.0), nrad (3), mdis (35), hp (false), threads (0)
* Parameters passed to eedi3. Refer to the eedi3 docs for more information
+ [float] ratiothr (1.125)
* When the scaling ratio is below this threshold, eedi3_resize will only use Dither resizers instead of
the eedi3+Dither_resize16 combo
+ [float] sharp (0)
* Strength of contra-sharpening applied to the upscaled clip
* A value of 0 disabled sharpening, sensible values are around 50-100
+ [bool] mixed (true)
* By default, eedi3_resize uses eedi3 (8-bit) and Dither resizers (16-bit) for edge areas
* If this parameter is enabled, the script will only use 16-bit Dither resizers for scaling flat areas to increase precision
* Disable this if you want flat areas to be resized using eedi3+Dither_resize16
+ [float] thr (1.0), elast (1.5)
* Threshold and elasticity for merging eedi3 output into the clip upscaled with 16-bit Dither resizers
* Increase thr to take less pixels from the eedi3 upscaled clip
* Increase elast to increase blending between pixels from the eedi3 and Dither upscaled sclips for smoother transitions
+ [string] output (input colorspace)
* Output format/color space
* May be either one of the AviSynth internal colorspace or one of the Dither formats (RGB48Y, RGB48YV12)
+ [string] curve (linear)
* Setting this parameter to any of the following values enables resizing in linear light:
* "709", "601", "170", "240", "sRGB", "2020"
* refer to the Dither documentation (Dither_y_gamma_to_linear) for more information
+ [float] gcor (1.0)
* Gamma correction to be applied when converting to linear light
* Requires curve to be something else than "linear"
+ [string] matrix (resolution-based)
* Matrix used for YUV->RGB conversion
* Defaults to "709" for HD material, "601" for SD material
* Refer to the Dither docs (Dither_convert_yuv_to_rgb) for more information
+ [bool] tv_range (true)
* Indicates whether the input clip is in TV or PC (full) range
+ [bool] cplace (MPEG2)
* Placement of the chroma samples on 4:2:0 YUV formats
* Supported values are MPEG2 and MPEG1
+ [int] y (3), u (2), v (u)
* Processing options for y/u/v planes:
* -x..0 : All the pixels of the plane will be set to -x (disables processing)
* 1 : Disables processing of the plane (plane will contain garbage)
* 2 : Copies plane from the input clip (disables processing)
* 3 : Plane will processed
+ [bool] lsb_in (false), lsb_out (lsb_in)
* Tells the script whether the input/output clips are 16-bit stacked or 8 bit
+ [bool] lsb
* When enabled, processing will be done in high bitdepth for increased precision and smooth gradients (but at the cost of speed)
* When disabled, eedi3_resize will use 8-bit resizers instead of 16-bit resizers
* Setting either lsb_in or lsb_out to true will enable high-bitdepth processing by default
* High-bitdepth processing is always enabled for RGB output, regardless of what the lsb parameter is set to
+ [int] dither (6)
* Dither mode for 16->8-bit coversion
* Refer to the Dither docs (DitherPost) for more information on the various dithering modes
+ [clip] mclip (Undefined)
* mask clip to be used as prescreener for eedi3. Depending on the content, it usually results in a significant speed gain


LUtils v0.11
Download (https://raw.github.com/AviSynth/avs-scripts/master/LUtils.avsi)

A collection of helper and wrapper functions meant to help script authors in handling common operations (especially 8/16-bit processing) and shortcomings of the avs scripting language without having to write the same boilerplate over and over again. Most of these functions require AviSynth+

Function list:

Lu16To8
Lu8To16
LuBlankClip
LuConvCSP
LuEdi
LUIsDefined
LuIsEq
LuIsFunction
LuIsHD
LuIsSameRes
LuLimitDif
LuLut
LuMatchCSP
LuMerge
LuMergePlanes
LUResize
LuRGB48YV12ToRGB48Y
LuSeparateColumns
LuStackedNto16
LuSubstrAtIdx
LuSubstrCnt
LuPlanarToStacked

vkusnyashka
15th November 2021, 17:19
It is very strange that for so many years there have been no discussions on this topic. Also, I did not find any new versions of HiAA.
I recently found that HiAA stopped working in AviSynth+ since version 3.6. "Not a clip" error. There are no problems in version 3.5.1 and below.
Maybe someone can make a fix or suggest another filter with similar HiAA functionality / result?

Blankmedia
22nd December 2021, 06:43
It is very strange that for so many years there have been no discussions on this topic. Also, I did not find any new versions of HiAA.

I recently found that HiAA stopped working in AviSynth+ since version 3.6. "Not a clip" error. There are no problems in version 3.5.1 and below.

Maybe someone can make a fix or suggest another filter with similar HiAA functionality / result?Was it a good filter? More than Santiag or others?

Envoyé de mon Pixel 6 en utilisant Tapatalk

vkusnyashka
22nd December 2021, 15:48
Was it a good filter? More than Santiag or others?
Wow! I was inattentive and thought that santiag is some kind of simple aa filter :eek:
I compared it with HiAA and it shows better results in "nnedi3" and "eedi3" modes - https://slow.pics/c/jXjbW7iH
In "sangnom2" mode, HiAA has a better result, and santiag v1.604 is worse than santiag v1.6 - https://slow.pics/c/rLd5zedl
I think maa2 mod by A.SONY shows the most similar result for "sangnom2" mode.
Blankmedia, :thanks:
The only thing is, santiag doesn't support 16 bits. HiAA at least has stack16 support.
upd: 16 bits are supported in "nnedi3" and "sangnom2" modes.

kedautinh12
22nd December 2021, 16:02
I think real.finder was added support high bit depth. Just use convertbits(16) in front of santiag
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/Santiag.avsi

New sangnom supported 8-32bit
https://github.com/Asd-g/AviSynth-SangNom2

Blankmedia
22nd December 2021, 16:19
Blankmedia, :thanks:
.

haha no trouble, but I genuinely didn't know.

vkusnyashka
23rd December 2021, 05:55
kedautinh12,
Oh, I was inattentive again and checked 16 bit support when type="eedi3" was set :D That's right, 16 bits are supported in "nnedi3" and "sangnom2" modes.

real.finder
15th January 2022, 20:42
It is very strange that for so many years there have been no discussions on this topic. Also, I did not find any new versions of HiAA.
I recently found that HiAA stopped working in AviSynth+ since version 3.6. "Not a clip" error. There are no problems in version 3.5.1 and below.
Maybe someone can make a fix or suggest another filter with similar HiAA functionality / result?

simple HiAA() work fine for me in AviSynth+ 3.7.1

and santiag v1.604 is worse than santiag v1.6 - https://slow.pics/c/rLd5zedl

can you test with this update https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/Santiag.avsi ?

vkusnyashka
20th February 2022, 18:01
Hello, real.finder! Sorry for the late reply.
simple HiAA() work fine for me in AviSynth+ 3.7.1
I updated the version to 3.7.1 but nothing has changed:
https://i6.imageban.ru/out/2022/02/20/3d19cb4073b7ce86db43ff7cb64bf1fa.png
Could this be my local problem? :confused: As I already wrote, it works on version 3.5.1 and below.
can you test with this update https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/Santiag.avsi ?
I checked with the new version but didn't notice almost any difference from the previous version - https://slow.pics/c/nUzvyNYZ
A small clarification about the screenshots in the comparisons. The filter was applied at a resolution of 1280x720:
blackmanresize(1280,720)
santiag(type="sangnom")
blackmanresize(1920,1080)

real.finder
20th February 2022, 19:13
using https://i.slow.pics/8HtBzuBC.png

ImageSource("8HtBzuBC.png")
converttoyv12
blackmanresize(1280,720)
santiag(type="sangnom")
blackmanresize(1920,1080)


original santiag https://forum.doom9.org/showthread.php?p=1393006#post1393006 give me same output as v1.605 in your screen shot, maybe you got some modded santiag from somewhere, or you got some local problems

vkusnyashka
21st February 2022, 07:38
using https://i.slow.pics/8HtBzuBC.png
Yes, after changing the resolution several times there is no difference. Please check with this frame in original resolution - https://i3.imageban.ru/out/2022/02/21/a2646bb8c4026ee8bc9f377381c2c8d0.png

real.finder
21st February 2022, 14:59
Yes, after changing the resolution several times there is no difference. Please check with this frame in original resolution - https://i3.imageban.ru/out/2022/02/21/a2646bb8c4026ee8bc9f377381c2c8d0.png

this is what I got https://slow.pics/c/TeDfFb9F

vkusnyashka
21st February 2022, 19:39
this is what I got https://slow.pics/c/TeDfFb9F
I think I've solved this riddle. I do not understand how this can be, but there are no differences if the source is an image. If the source is a video, then everything is as in my previous comparisons. Up to this point, I thought that the output of antialiasing filters on an image and video is the same.
Here is a comparison of v1.6 with the image and with the video as source - https://slow.pics/c/ySBQmnmn
If you are interested to verify this, please check it out on a sample video that I will send you in private message.
In any case, HiAA is better than Santiag in "sangnom2" mode.

real.finder
21st February 2022, 21:42
I note it now

it's because there was bug in 1.6 that I fixed, try https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/Santiag.avsi

kedautinh12
22nd February 2022, 00:21
I think if it was bug, sangnom_bug must be set default value to false

vkusnyashka
22nd February 2022, 06:55
real.finder
Yes, now the result is the same as in version 1.6 :thanks:
Can you explain why it was only noticeable when the source is a video? :confused:
PS: Also I would like to thank you for all the work you do with avisynth filters! :thanks:

kedautinh12
In any case, I think it's worth making a more detailed comment about this bug in the script.

real.finder
22nd February 2022, 12:48
Can you explain why it was only noticeable when the source is a video?

because when you use image you deal with rgb not yuv, every time you convert to or from rgb you loss some details https://forum.doom9.org/showthread.php?t=164737