Adaptive Lens Blur Repair

[fvisagie]

Hi All,

This function is for adaptively repairing video damaged by lens blur, using a frame-adaptive repair mask, a selection of sharpeners and multi-stage motion-compensated artifact removal. It also provides blanket sharpening and denoising-only modes, the latter mainly for applying an equivalent amount of denoising to footage that did not need repairing. A sample of the type of footage this was created for: http://www.mediafire.com/file/0lym5r...ing_sample.wmv

Edit: updated for version 1.0.2

ALBR-1.0.2.zip

#############################################
#                                           #
#      Adaptive Lens Blur Repair 1.0.2      #
#            by Francois Visagie            #
#                                           #
#  Selective motion-compensated deblurring  #
#   using frame-adaptive repair mask and    #
#       multi-stage artifact removal        #
#                                           #
#############################################


#-------------------------------------------
# Rationale
#
#   With heavily and/or variably blurred (e.g. progressive centre-to-edge) video, Unsharp Mask sharpening usually has the following limitations:
#       *   The very first thing it does is further blurring, destroying already fragile motion vectors and creating distorted motion
#       *   It sharpens the whole frame equally much with unattractive results for variably blurred video
#   Variable blur is an especially big problem when it also moves around temporally, e.g. luma flares caused by bright objects in the frame.
#   In such cases even a selective sharpening mask is inadequate because it remains static between frames.
#
#   In an attempt to address these issues, this script
#       *   Works with motion vectors carefully, saving them before anything else
#       *   Uses edge detection to create a frame-adaptive repair mask for identifying flat vs. detailed areas
#       *   Provides a choice of sharpeners
#       *   Inserts motion compensation/denoising between repair stages using motion vectors refined with MRecalculate()
#
#   Various operating modes are available
#       *   Adaptive MC sharpening as described
#       *   Blanket (repair mask inactive) MC sharpening with denoising
#       *   MC temporal denoising only
#
#   To assist with tuning motion vector detection and repair mask parameters, various output modes are available.
#   Settings can be displayed in each output mode.
#
#
# Prototype, parameters and defaults
#
#   ALBR(clip a, string "mode", \
#        int "rc", int "thshlow", int "thshhi", float "vmdb", \
#        string "sharpnr", bool "shrpchr", int "rshy", int "rshc", float "strsh", int "thL", int "thU", int "type", int "map", int "lim", \
#        float "hue", float "sat", float "bright", float "cont", \
#        int "degrain", int "hpad", int "vpad", int "pel", int "rfilter", \
#        int "blksizA", int "ovrlapA", bool "temporl", int "lambda", int "pnew", \
#        int "blksizR", int "ovrlapR", int "thSADR", \
#        string "output", int "mvscale", bool "shosets")
#
#   Mode Setting
#   ------------
#
#   Available modes:
#       "AdSharp"   :   adaptive repair mask sharpening with denoising, all functionality and settings active
#       "Sharpen"   :   blanket sharpening (no repair mask) and denoise, Repair Mask Settings inactive
#       "Denoise"   :   temporally denoise only, Repair Mask and Sharpening Settings inactive
#
# mode    = Default(mode,   "AdSharp")
#
#   Repair Mask Settings
#   --------------------
#
# rc      = Default(rc,     50        )   # Radius for consolidating repair mask detailed and flat areas
# thshlow = Default(thshlow,22        )   # Repair mask low threshold - sharpen all detail BELOW this level
# thshhi  = Default(thshhi, 34        )   # Repair mask high threshold - sharpen no detail ABOVE this level
#                                         # - intermediate levels are sharpened proportionately
# vmdb    = Default(vmdb,   16.0      )   # Blur variance for debanding repair mask
#
#   Sharpening Settings
#   -------------------
#
#   Available sharpeners:
#       "Unsharpen" :   unsharpen() from VariableBlur.txt, almost identical to next, 3.1 fps
#       "Unsharp"   :   unsharp() from VariableBlur.txt, almost identical to previous, 3.1 fps
#       "SMSSQME"   :   Based on simplified multi-scale sampled quasi-median enhancer at http://forum.doom9.org/showthread.php?p=1595531#post1595531,
#                       less good than unsharp/en() on badly blurred source, 2.8 fps
#       "QMENoTopo" :   Above without topological reconstruction, slightly sharper on badly blurred source, 2.8 fps
#       "TUnSharp"  :   Does not work well for badly blurred source, 2.8 fps
#
# sharpnr = Default(sharpnr,"Unsharp" )   # Sharpener to use
# shrpchr = Default(shrpchr, true)        # Sharpen also chroma (true) or luma only (false), ignored by TUnSharp
#     Sharpener presets:                     Unsharpen   Unsharp     SMSSQME     QMENoTopo   TUnSharp
#         (NOTE: radius settings are internally converted to variance values for functions using those)
# rshy    = Default(rshy,   Select(shrpnum,  31,         31,         12,         12,         3        ))  # Unsharp mask Y radius/TUnSharp radius
# rshc    = Default(rshc,   Select(shrpnum,  31,         31,         8,          8,          0        ))  # Unsharp mask C radius, unused by TUnSharp
# strsh   = Default(strsh,  Select(shrpnum,  1.13,       1.13,       1.62,       1.62,       100      ))  # Sharpening strength
# thL     = Default(thL,    Select(shrpnum,  0,          0,          0,          0,          1        ))  # TUnSharp lower edge magnitude threshold,
#                                                                                                         # unused by others
# thU     = Default(thU,    Select(shrpnum,  0,          0,          0,          0,          5        ))  # TUnSharp upper edge magnitude threshold,
#                                                                                                         # unused by others
# type    = Default(type,   Select(shrpnum,  0,          0,          0,          0,          1        ))  # TUnSharp sharpening/unsharpmasking operator,
#                                                                                                         # unused by others
# map     = Default(map,    Select(shrpnum,  0,          0,          0,          0,          0        ))  # TUnSharp output map type, unused by others
# lim     = Default(lim,    Select(shrpnum,  0,          0,          0,          0,          2        ))  # TUnSharp sharpening limit, unused by others
# hue     = Default(hue,    Select(shrpnum,  0.0,        0.0,        0.0,        0.0,        0.0      ))  # Post-sharpening hue normalisation
# sat     = Default(sat,    Select(shrpnum,  0.65,       0.65,       1.0,        1.0,        1.16     ))  # Post-sharpening saturation normalisation
# bright  = Default(bright, Select(shrpnum,  35.3,       35.3,       23.5,       23.5,       0.0      ))  # Post-sharpening brightness normalisation
# cont    = Default(cont,   Select(shrpnum,  0.7,        0.7,        0.8,        0.8,        1.0      ))  # Post-sharpening contrast normalisation
#
#   Motion Vector Settings
#   ----------------------
#
# degrain = Default(degrain,3         )   # temporal radius for denoising/structural consolidation
# hpad    = Default(hpad,   16        )   # Need hpad and vpad this size for blksizA below
# vpad    = Default(vpad,   16        )
# pel     = Default(pel,    2         )   # MSuper() motion estimation accuracy
# rfilter = Default(rfilter,4         )   # Gives better results than default of 2
# blksizA = Default(blksizA,16        )   # Block size for analysing MVs
# ovrlapA = Default(ovrlapA,blksizA/2 )   # Overlap for analysing MVs
# temporl = Default(temporl,false     )   # Seems not to affect output, incompatible with SetMTMode when true
# lambda  = Default(lambda, 500       )   # For blurry source choose lambda and pnew values that slightly relax coherence,
# pnew    = Default(pnew,   25        )   # to prevent too many MVs being discarded and causing motion distortion
# blksizR = Default(blksizR,8         )   # Block size for refining MVs
# ovrlapR = Default(ovrlapR,blksizR/2 )   # Overlap for refining MVs
# thSADR  = Default(thSADR, 400       )   # Ditto relax re-estimation threshold
#
#   Tuning Aids
#   -----------
#
#   Available outputs:
#       "Result"    :   script result
#       "MVs"       :   motion vector overlay
#       "Mask"      :   repair mask
#       
# output  = Default(output, "Result"  )   # What to output
# mvscale = Default(mvscale,2         )   # Scale at which to display motion vectors
# shosets = Default(shosets,false     )   # Shows settings when true, also shows pixel values for output == "Result"
#
#
# Usage
#
#   ALBR(output="MVs", shosets=true)                      # Display motion vectors and settings for tuning purposes
#   ALBR(output="Mask", shosets=true)                     # Display repair mask and settings for tuning purposes
#   ALBR(sharpnr="TUnSharp", map=2, shosets=true)         # Display TUnSharp sharpening map (1/2) for tuning purposes
#   ALBR(shosets=true)                                    # Display result and settings for sharpener tuning
#   a=ALBR()                                              # Adaptively MC-sharpen with default sharpener and settings,
#   b=ALBR(mode="Denoise")                                #   denoise original by same amount,
#   splicelist="75:25, 150:25, 200:25, 400:25, 426:25"    #   specify splices (blends in this case) as needed,
#   SpliceFiltered(b, a, splicelist)                      #   selectively blend in sharpened material
#   ALBR(mode="Sharpen", sharpnr="SMSSQME", strsh=0.5)    # Blanket MC-sharpen cleaner source with high-quality enhancer
#
#
# Limitations
#
#   Repair mask -   although edge detection is much better than nothing, it also includes actual detail-less areas for sharpening
#   Not intended for sharpening motion blur
#
#
# Requirements
#
#   YV12 colourspace
#   From http://avisynth.org/mediawiki/External_filters:
#       MaskTools
#       MedianBlur
#       MVTools v2
#       RemoveGrain
#       TUnSharp
#       VariableBlur
#   Optional
#       SpliceFiltered from http://forum.doom9.org/showthread.php?t=166627
#
#-------------------------------------------

# Version history
#
#   1.0.2
#       Sharpening artifact removal now skipped when only denoising
#       MV recalculation between stages now disabled when only denoising
#
#   1.0.1
#       Expanded usage scenarios
#
#   1.0.0
#       Wrapped in function
#       Documented parameters and use
#       Added additional operating modes for blanket MC sharpening as well as MC temporal denoising
#
#   0.2.3
#       Added QMENoTopo sharpener
#       Added soure frame plus frame pixel values to Result settings display
#       Completed evaluation & characterisation of provided sharpeners & updated sharpener defaults accordingly
#       Various small improvements
#
#   0.1.1
#       Initial version
#

Toggling between the repair mask tuning and source displays:



Sample before-and-after screenshots:



Hopefully this is enough to get going anyone who wants to play with it, but shout if you have any questions.

All input, comments and suggestions gratefully welcomed. Many thanks to the collective assistance from this forum so far, in particular to poisondeathray who gave me direction initially.

Thanks and kind regards,
Francois