Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

 
 
Thread Tools Search this Thread Display Modes
Prev Previous Post   Next Post Next
Old 19th May 2021, 18:24   #1  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,364
Dogway's Filters Packs

Dogway's Filters Packs


GitHub repo.

If you are new to AviSynth, to get things up and running refer to this post.

TIP: For AviSynth+ front end GUIs such as MeGui/RipBot, follow the next suggestions to update to latest AVS+ version; for MeGui and RipBot.

----

As others before I thought on creating one single thread to list and explain my updated filter packs and avs+ modernization efforts.

The main goal of the updates is to reduce redundant, outdated or slow functions to modern alternatives often with more features, like HBD support (32-bit float inc.), frame properties and improved performance among others.
Basic building block functions like those in masktools2, RgTools and smoothadjust have been replaced with internal Expr() wrappers, this allows liquid and easily editable code for others to inspect, debug or branch.
Additionally higher level filters have also been created or ported like those in SharpenersPack or GradePack (read below).
For performance reasons many expressions have also seen major refactors so due to this and modern AVS+ syntax updates probably any version earlier than v3.7.3 won't work properly, but this is a necessary evil to move things forward and make HBD filtering of HD|UHD sources something that is counted in hours and not days. Special thanks to pinterf for his continuous work on AVS+.


ORIGINAL

ExTools: Wrapper library for Expr() expressions that covers (and expands) most masktools2 and removegrain functions, including lutspa and convolutions. Also adds Array helper functions that expands those included internally and function approximations like 'atan', 'expr'... faster than internal. Syntax and arguments are kept so it's easy to update old scripts to the new counterparts. This pack will be required for all the following scripts.
Post about STTWM() (and initial release), Adaptive Threshold, ex_bilateral() (vs Dither_bilateral16() ), ex_shape() GIF. Morphological mask filtering.

Transforms Pack: Divided in 3: Main, Models, Transfers. Modern color and tone response technical transforms functions for color managing AviSynth+. Goal is usability, functionality and accuracy, works over any bitdepth, supports any luma range, extra color spaces and color models among them real RGB based HSV, reversible YUV and YCoCg, IPT, OkLab, ICtCp, IPTPQc2 and more. Includes also a SoftLimiter() and building block matrix functions.
Example converting an ACEScg exr to Rec709-1886 with gamut compression, tonemapping and filmic contrast.
Example converting a Dolby Vision IPTPQc2 (DVp5 or DVp8) clip to Rec709-1886 with gamut compression and tonemapping.

Grade Pack: Look transforms. Includes ex_levels() with native HBD support (same usage than native Levels() ), ex_autolevels(), ex_contrast(), ex_blend(), ex_glow(), ex_posterize(), greyscale_rgb(), FindTemp(), WhitePoint(), Vignette(), Skin_Qualifier(), GamutWarning(), PseudoColor(), GreyWorld(), HSVxHSV() and ex_vibrance(), a saturation and vibrance function.

SMDegrain: Simple MDegrain Mod. Easy to use, fool proof degraining wrapper of MDegrain and company. Initially a small few lines wrapper by Caroliano that I took over and implemented YUY2 support, interlaced support, 16-bit dither support, contrasharpening, prefiltering, debug view, documentation, globals, and good practice code. Later on real.finder took over and adapted it to modern code, added bugfixes, HBD support and so on, so forth. Now I ported it to ExTools, sanitized the code, removed old avisynth support, Dither support, and YUY2 support and included some new features like alternative degrainers, recursion, low frequency restoration (here too), DCT flicker, optimized UHD performance, ex_DGDenoise() and ex_BM3D() prefilters, and multi-scale retinex.
Here some explanations on iterative temporal filtering. In this post a draft for a SAD sampler. Recommendations for heavy grain prefiltering. 16mm film restoration (+here). Original 2011 SMDegrain thread.

Resizers Pack: Pack of functions involving resizing operations, like deep_resize() (and here) a refactored port of nnedi3_resize16() six times the speed and at higher quality, nnedi3resize(), a nnedi3 based arbitrary size scaler, RatioResize() which can resize by a single factor; percentage, adjust to width/height, to PAR, DAR and so on. PadResize() to crop or pad given input dimensions, PadBorders() like an advanced AddBorders()+Crop() with option to mirror, dilate or fill borders, MatteCrop() to automatically fix (crop+resize) movies with random bordered shots, and some utilities like mmod() to crop/pad/auto to mod, and nmod() to mod values with extra features like min value or bankers' rounding.

Masks Pack: Mask and limiter filters. BoxMask(), FlatMask(), LumaMask(), CornerMask() (cheap alternative here) and MotionMask() for masks and ex_limitchange(), ex_limitdif() and Soothe() for limiters.

Scenes Pack: SceneStats() opens the doors to scene based workflows. It writes current (frame) scene range bounds into '_SceneRange' frame properties and current scene change into '_SceneChangePrev', also scene motion into '_SceneMotion', scene details to '_SceneDetails' (a complexity index for average of edges), scene exposure index to '_SceneExposure' and pixel stats into '_SceneStats', on the fly or by offloading it to a file. ReadStats() can load an optionally exported SceneStats() stats file for faster processing at encoding stages. ClipStats() will otherwise load them and convert them to clip global stats, to help you decide better clip-wide constant settings in your filters.
Example for SMDegrain.
Example for FilmGrainPlus.

FilmGrain+: Made from the ground up, an accurate and performant synthetic film grain filter with presets for the most common negative films.

Logo: Easily add static logos or watermarks, with blur, fade in/out, opacity, and blending controls. Eventually also for video based logos.

Stabilization Tools Pack (legacy): Initially a simple mod of Stab() which grew bigger and currently includes various strategies for edge filling. Also includes FilmGateFix() mainly aimed at anime sources.


EX/MIX MODS

Normally there are 2 flavors of each mod:
EX mods are future proof with ExTools wrappers and minimal dependencies, this can also come handy when running on Linux/macOS which very few plugins support.
MIX mods use carefully chosen masktools2 and removegrain functions to maximize speed but come with these and probably other dependencies as well.

QTGMC+: Reference deinterlacer. Ported to ExTools from v3.382 (~40% faster in HBD). Includes ex_vinverse() (now legacy for int bitdepths), ex_bob() and ex_reduceflicker() functions.

LSFplus: Based on LSFmod, also one of the best sharpeners out there. Optimized (+74% with no SS), ported to ExTools and added more features.

GradFun3plus: Port of cretindesalpes' excellent GradFun3 debanding filter to internal AVS+ calls (+66% gain in smode=0)

Sharpeners Pack: Collection of high quality sharpeners optimized and ported to ExTools for HBD support and performance. In total 29 sharpeners, among them; Adaptive Sharpen, ex_unsharp, CASP, NVSharpen, ex_ContraSharpening, SeeSaw, FineSharpPlus, NonlinUSM, ReCon, blah and Plum.

Deblock Pack: Pack containing different deblocking functions from famous Deblock_QED() (29% speed gain), to CCD(), SmoothD2c(), SmoothDeblock() (WIP) or feisty2's Oyster (Oyster includes also deringing and else)

Similarity Metrics: Pack containing all the similarity/distance metrics ported by Asd-g to AVS+ from WolframRhodium VapourSynth repo. I collected, sanitized and updated the code for x4 speed gain on GMSD(), x2 on MDSI(), x3 on vsSSIM() and added+refactored BSSIM() from zorr. Also created SVM(), a metric for image sharpness. For more metrics check the cost functions in ex_makediff() in ExTools.

yugefunc: Collection of VapourSynth filters ported to AVS+ and optimized on the way with ExTools and other expression tricks: ex_guidedblur(), ex_ANguidedblur(), XDoG() (WIP), etc

Other: Some other scripts have received the ExTools treatment; FillMissing(), FastLineDarkenPlus(), SPresso(), DeStripe(), etc


ExTools main functions:
Code:
# EXPRESSIONS    
ex_lut()         - Single variable (1 clip)  expressions
ex_lutxy()       - Double variable (2 clips) expressions
ex_lutxyz()      - Triple variable (3 clips) expressions
ex_lutxyza()     - Quadruple variable (4 clips) expressions
ex_makediff()    - Clip based differentiation. Also calculates similarity/residual metrics via cost functions
ex_adddiff()     - Sum clips, specially useful to add back the result of differentiation
ex_makeadddiff() - ex_makediff() and ex_adddiff() in one step
ex_logic()       - Logical operations between 2 clips with logic ops (MIN, MAX, OR, AND, etc)
ex_merge()       - Merging. Performs a linear interpolation between 2 clips based on mask (3rd clip)
ex_clamp()       - Clamps first clip between the maximum of the second clip and the minimum of the third
ex_binarize()    - Performs binary type segmentation or thresholding
ex_athres()      - Adaptive Threshold. Special binary thresholding for uneven brightness images (ie. extracting letters from a shaded area)
ex_invert()      - Invert the clip pixel values
ex_lutspa()      - Relative or absolute pixel-location based expressions
ex_motion()      - Computes a very primitive motion mask akin to MaskTools2's mt_motion()
ex_hysteresis()  - Proof of concept Expr() port of mt_hysteresis(). Uses 'for' loops so very slow
                 
# MORPHOLOGICAL  
ex_expand()      - Morphological dilation/expansion of pixel-value based on structuring element given by the kernel window
ex_inpand()      - Morphological erosion/contraction of pixel-value based on structuring element given by the kernel window
ex_inflate()     - Expansion via outward blurring given structuring element of pixel values of the kernel window
ex_deflate()     - Contraction via inward blurring given structuring element of pixel values of the kernel window
ex_hitormiss()   - Structuring elements based morphological transforms for binary images
ex_edge()        - Gradient magnitude. Edge detection via (partial) local derivatives
ex_luts()        - Moving window relative pixel-location based expressions. A convolution do-it-all filter
ex_shape()       - Helper filter for ex_luts() (and other expression based filters) to fetch kernel-window pixels into a string
                 
# BLURS          
ex_boxblur()     - Discreet local neighborhood blur convolutions
ex_blur()        - Gaussian (or Butterworth) weighted blur convolutions
ex_gaussianblur()- Optimized Gaussian filter for large sigma
ex_kawase()      - Kawase optimized blur filter (still slower than ex_gaussianblur() ). Accepts different strides so good for exponential blur
ex_blur3D()      - Spatio-temporal blur filter
ex_bilateral()   - Bilateral blur filter (respects edges)
ex_smartblur()   - Like Bilateral filter but more performant (mimics Photoshop's Surface Blur)
ex_smooth()      - Savitzky-Golay smoothing filter. Halfway between blur and antialiasing
ex_FluxSmoothT() - Minimum change between a temporal weighted blur and temporal median. Informal port of FluxSmoothT filter via Didée's description
ex_FluxSmoothST()- Spatio-Temporal minimum change between weighted blur and median. Uses ex_FluxSmoothT() and its spatial equivalent ex_MinBlur()
ex_median()      - Median (rank order) based blur filtering. Also includes some alternative mean average algorithms
ex_repair()      - Median (rank order) based repair filter
STTWM()          - Spatio-Temporal Thresholded Weighted Median (STPresso() inspired / not a port)

Last edited by Dogway; 1st November 2023 at 01:24.
Dogway is offline   Reply With Quote
 

Tags
avisynth, dogway, filters, hbd, packs

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.