Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th August 2014, 03:51   #1  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 85
Real-time game footage postprocessing

I'm trying to apply Avisynth for upscaling and antialiasing of a live gameplay video to enhance video quality. Basically, the game runs in a small window on my second monitor, where I capture its video output, process it in Avisynth and then output it to my main monitor in fullscreen.

The reason is that the game is being run on the PS2 emulator and this particular game has very poor hardware mode support, so I have to resort to software, which is limited to console's native resolution - 512x448. So in order to play it in fullscreen without bleeding my eyes out, I decided to give a try at such an exotic Avisynth application.
Considering that software mode uses only CPU, the upscaling and spatial antialiasing are done using the OpenCL version of NNEDI3 that runs on GPU. Before that I do some temporal antialiasing by averaging source video with its motioncompensated version at the edges.
The script as a whole is being run via Avisynth tab of the FFDShow raw video filter.

Here is my script so far:
Code:
#-Settings-------------------
#Spatial anti-aliasing, runs on GPU. 2x for resolutions around 1280x720, 4x for larger screens if you have spare power.
FSAA=2 
#Temporal anti-aliasing, runs on CPU. Reduces flickering in motion, possible values 2 or 3. 0 disables it.
TXAA=3
#HDR dynamic contrast. Increases visibility in overly bright or dark areas. Values list: 0, 0.25, 0.5, 0.75, 1(>0.5 is not recommended).
HDR=0.25
#/Settings------------------

last=(HDR>0)?raveragew(last, 1-HDR, coloryuv(autogain=true ), HDR, mode=4, u=3, v=3, lsb_out=true).ditherpost():last

uv=last

super = MSuper(pel=2, chroma=false) 
backward_vectors = MAnalyse(super, isb=true, blksize=8, overlap=2, search=4, truemotion=true, chroma=false, temporal=true, badsad=1000) 
forward_vectors  = MAnalyse(super, isb=false, blksize=16, blksizev=8, overlap=0, overlapv=2, search=4, truemotion=false, chroma=false, temporal=true, badsad=1000) 
backward_compensation = MCompensate(super, backward_vectors, thSCD1=600)
forward_compensation  = MCompensate(super, forward_vectors, thSCD1=400)  

AA=(TXAA==3)?merge(merge(forward_compensation, backward_compensation).sharpen(0.2),0.67):merge(backward_compensation) 

me=mt_edge(mode="laplace", thy1=0, thy2=10, u=1, v=1).mt_inflate()
mm=mt_motion(thy1=0, thy2=15, tht=100, u=1, v=1).mt_inflate()
m=mt_logic(me, mm, mode="min", u=1, v=1).mt_expand(mode="both")  

last=(TXAA>0)?mt_merge(AA, m, u=1, v=1):last
uv=uv.bicubicresize(1870, 1050)
nnedi3x_rpow2(rfactor=FSAA,  u=false, v=false, nns=2)
spline36resize(1870, 1050)

mergechroma(uv)
sharpen(min(0.33*FSAA, 1.0))
I'm using the modified version of MVTools 2 from Dither package.

Since I have NVidia, I'm only aiming at achieving at most 30 fps of speed. This script already runs at ~30fps but this is at the limits of my system's capabilities (Core i5 2550k @ 4.4GHz, GTX 560 @ 950MHz).

Could you please advise, what can be improved in my script to raise quality or maybe spare some processing power?

So far the most troubling part is the temporal antialiasing, it causes chroma ghosting due to it not being processed together with luma to save time. Another thing that worries me is that I have no way of measuring the lag that this processing causes on the game (responsiveness to controls input). As such I will have to rely only on the theoretical knowledge of what kinds of filters can introduce lag during live processing. For example, what will introduce more lag, forward motion estimation and compensation or backward one? What about motion mask or shifting the video 1 frame forwards or backwards (in case I would need to smooth masks in time, for example).

Here is the recorded gameplay video that I was using for tuning my script: https://dl.dropboxusercontent.com/u/...mgs3motion.mkv.

Last edited by Seedmanc; 4th September 2014 at 19:24. Reason: Parts in gray color are now obsolete since I'm using Madvr for same processing
Seedmanc is offline   Reply With Quote
Old 29th August 2014, 20:11   #2  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
I hate that "twinkling/shimmering" effect as well. Your solution is very good, considering how hard the problem is (whether at real time or not), but I have a few thoughts:

* adding the results of mt_motion from the 2 previous frames seems to help catch more of the movement. Note the functions calls I used to do this were only used because I had these functions on hand; there are certainly other (better) ways to do the same thing.

* for temporal smoothing, take a look at FFT3DFilter in place of MCompensate. It may be only a personal preference (ie, which artifacts are more annoying for you), but I think FFT3DFilter looks a little better. Not sure how speed might be affected on your system. Again, I used a function I had worked up previously.

Code:
# mt_masktools-26.dll
# nnedi3.dll
# fft3dfilter211\FFT3DFilter.dll

AviSource("D:\VideoProjects\raw\mgs3motion uty7.avi")
ConvertToYV12(matrix="Rec709", chromaresample="spline16")
out_wid=1870
out_hgt=1050

#-Settings-------------------
## Spatial anti-aliasing, runs on GPU. 
## 2x for resolutions around 1280x720, 4x for larger screens if you have spare power.
FSAA=2 
## Temporal anti-aliasing, runs on CPU. 
## Reduces flickering in motion, possible values 2 or 3. 0 disables it.
TXAA=2
## HDR dynamic contrast. Increases visibility in overly bright or dark areas. 
## Values list: 0, 0.25, 0.5, 0.75, 1(>0.5 is not recommended).
HDR=0 /* not tested */
#/Settings------------------

Last = (HDR<=0) ? Last
\ : raveragew(Last, 1-HDR, ColorYUV(autogain=true ), HDR, mode=4, u=3, v=3, lsb_out=true)
\     .ditherpost()

uv = BicubicResize(out_wid, out_hgt)

#super = MSuper(pel=2, chroma=false) 
#backward_vectors = MAnalyse(super, blksize=8, 
#\                       overlap=2, search=4, 
#\                       truemotion=true,  chroma=false, temporal=true, 
#\                       badsad=1000, isb=true) 
#forward_vectors  = MAnalyse(super, blksize=16, blksizev=8, 
#\                       overlap=0, overlapv=2, search=4, 
#\                       truemotion=false, chroma=false, temporal=true, 
#\                       badsad=1000, isb=false) 
#backward_compensation = MCompensate(super, backward_vectors, thSCD1=600)
#forward_compensation  = MCompensate(super, forward_vectors,  thSCD1=400)  
#AA = (TXAA==3)
#\      ? Merge(Merge(forward_compensation, backward_compensation).Sharpen(0.2), 0.67)
#\      : Merge(backward_compensation) 

AA = AAff3d_10(temporal_size=4, sigma=6.0, sharpen=0.0) /* FFT3DFilter? */

me = mt_edge(mode="laplace", thy1=0, thy2=10, u=1, v=1).mt_inflate

mm = mt_motion(thy1=0, thy2=15, tht=100, u=1, v=1).mt_expand.mt_inflate
mm  = mm.UUDelayBlend(1, 1.0, "lighten").UUDelayBlend(2, 1.0, "lighten") /* multiple mt_motion? */

m  = mt_logic(me, mm, mode="min", u=1, v=1).mt_expand(mode="both").Blur(0.5)  

Last = (TXAA<=0) ? Last
\    : mt_merge(AA, m, u=1, v=1)
 
nnedi3x_rpow2(rfactor=FSAA,  u=false, v=false, nns=2)

Spline16Resize(out_wid, out_hgt) /* less ringy; a little faster? */

MergeChroma(uv)

Sharpen(Min(0.33*FSAA, 1.0)).Blur(0.1)

return Last 

#######################################
### FFT 3D antialising: decreases certain aliasing artifacts, 
##  such as moire and shimmer; spatial & temporal smoothing. 
##
## @ temporal_size: default 5; try 1-4 for faster processing 
##            0 - all previous frames (switch Kalman filter mode);
##            1 - only current frame (spatial 2D Wiener filter);
##            2 - previous and current frame (3D Wiener filter);
##            3 - previous, current and next frame (3D Wiener filter)
##          * 4 - two previous, current and next frame (3D Wiener filter)
##          * 5 - two previous, current and two next frames (3D Wiener filter)
##           -1 - sharpen only (2D);
##
## @ sigma - given noise value (float>0, default=12)
##           1.5 = subliminal
##           2.2 = slightly soft
##           3.0 = soft
##           5.0 = softer
##           (default 12 = very strong)
##
## @ sharpen - sharpening strength; good values about 0.3 to 1.0; 
##            negative values results in reverse effect.
##            (default=0.5 - slight sharpening)
##
## suggested settings: 
##   AAff3d_10()       ##  strong
##   AAff3d_10(4, 10)  ##  less strong
##   AAff3d_10(4, 2.2) ## slightly soft
##
function AAff3d_10(clip C, int "temporal_size", float "sigma", float "sharpen")
{
    Assert(C.IsYV12 || C.IsYUY2, 
    \     "AAff3d_10: invalid source format")
    sigma   = Float(Default(sigma, 12))
    sigma4  = sigma+20 ## given noise value at lowest frequencies
    
    sharpen = Float(Default(sharpen, 0.5))
    
    temporal_size = Default(temporal_size, 5)
    Assert((-2<temporal_size && 6>temporal_size),
    \     "AAff3d_10: invalid temporal_size argument")

    ## bw, bh - block width, height
    blksiz = 24
    ## ow, oh - overlap width, height
    ovrlap =  8 ## blksiz / 3

    C ## Last=C
    FFT3DFilter(sigma=sigma, sigma4=sigma4, bw=blksiz, bh=blksiz, 
    \           bt=temporal_size, ow=ovrlap, oh=ovrlap, sharpen=sharpen, 
    \           ncpu=2)
}

#######################################
### delay blend
#
#@param frames  - amount of delay (1 to 3 frames)
#@param opacity - amount of delayed video (from 0.0 to 1.0; default=0.5)
#@param mode    - Overlay mode (default "blend")
#
function UUDelayBlend(clip C, int "frames", float "opacity", string "mode")
{
    frames  =       Min(Max(1, Default(frames,  1  )), 3) 
    opacity = Float(Min(Max(0, Default(opacity, 0.5)), 1))
    mode    =                  Default(mode,    "blend")

    return Overlay(C, C.UUSlip(-frames), opacity=opacity, mode=mode)
}

#######################################
### "slip" a clip in time.
##
## @ C - clip to be advanced or delayed 
## @ offset - if positive, clip is advanced;
##            if negative, clip is delayed
## @ P - clip to be used for padding (default = BlankClip) 
##
function UUSlip(clip C, int offset, clip "P")
{
    lenTrim = (offset > 0) ? offset : 0
    lenPad  = (offset < 0) ? -offset : 0

    P = IsClip(P) ? P : BlankClip(C)

    C = (lenPad==0) ? C 
    \ : P.Trim(0, -lenPad) + C

    C = (lenTrim==0) ? C 
    \ : C.Trim(lenTrim, 0) + P.Trim(P.Framecount-lenTrim, -lenTrim) 

    return C
}

__END__

Last edited by raffriff42; 29th August 2014 at 20:19.
raffriff42 is offline   Reply With Quote
Old 2nd September 2014, 03:09   #3  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 85
Several days of observations in text ahead, beware.

Thanks for the reply.

Perhaps I should tell more about the way I tried to organize processing. At first I went this way: in GraphStudioNext there is a chain of three filters, Scree Capture Source, followed by FFDShow raw filter, that contains the Avisynth script, its output is then fed to Enhanced Video Rendered. Now I'm thinking about getting rid of GraphStudio part altogether, because it turns out that in players like MPC or Pot you can open a directshow video source filter as a device and still have avisynth filtering capabilities either via FFDshow or by inbuilt Avisynth processor (Potplayer has that). This way I'm also able to use Madvr for rendering instead of EVR, which is kinda cool, considering that not only Madvr has vertical sync enabled, it can actually replace the whole spatial antialiasing and upscaling part of my script by its own functionality (I couldn't make madvr work in graphstudio as output).
This way is tricky though, in the new MPC there's a deinterlacing filter in the way when you open a device and there's no way to disable it while it blurs the picture. Also sometimes there's huge lag involved (I'm talking whole seconds of lag here). As a whole, madvr tends to introduce lots of lag, probably due to its queuing and buffers everywhere, it was meant to watch "offline" content. I tried to lower those settings, still, there's around 300 ms of lag sometimes. But then there seems to be around 200 ms of lag even without any processing, in emulator alone. I measured the lag by recording the video from the screen with my phone and calculating the delay between the recorded sound of me pushing the button on the controller and the action happening on the screen. No idea how accurate that is. I'm still looking for ways of lowering madvr's output delay.

Realtime is the key factor in my case, if the processing results in fps lower than 30 or in a huge lag, I might as well watch a walkthrough on Youtube instead. This is why I am asking about possible optimization of my script, to free up some resources and either spend them to increase quality or reserve for possible future resource-intensive areas in game, to have some room.

Interesting that you mentioned FFT3d, I've never thought of it as an antialiaser, let alone temporal one. Although I did notice its "line straightening" effect when using it as denoiser. However it is very heavy to be used in my case. There is a GPU version of it and considering that there are still some spare cycles left on GPU even with the upscaling, I gave it a try. It blurs the picture a lot, even when applied via masks, and its antialiasing effect is mostly expressed in spatial domain rather than temporal. For spatial I already have nnediocl and FXAA shaders that come with the emulator.
I think that denoisers aren't very suited as antialiasers because the idea behind them is to actually avoid affecting the edges, blurring the textures instead, quite the opposite of what AA does. I tried using MDegrain, because it basically incorporates averaging two sets of MCompensates with the source video, but was left disappointed. As a denoiser, MDegrain averages clips with weights according to their difference from the source (more different - less weight), so the edges again are left behind since the most of difference happens to be there. Thus I was left with MCompensate anyway. As you can see I even had to use larger block size for the second set of vectors because there's no performance left to use better settings.

There's also that problem with masking motion, I need to have it kinda inverse to the way mt_motion does. Most shimmering is noticeable during very slow movements when the distance traveled by an object on screen is comparable to its size in pixels. It's especially noticeable in first-person view when the whole view moves slightly up and down due to character breathing, and flickers. During swift motion you wouldn't notice temporal aliasing anyway, while at the same time most of motion compensation artifacts get very intense this way, and FPS drops are more noticeable as well. So I'm looking for a way to select slow motion and give less weight in the mask for fast movements. Maybe I should try inverting the motion mask and then applying something like medianblur or set of mt_expands followed by set of mt_inpands to fill small holes in it, caused by slow motion, while still having fast motion deselected.
I'm also thinking that the upper part of the screen usually contains faraway objects that are shimmering the most during movement, due to their size, again, being comparable to pixel size in the source (thus the interference effect). At the same time lower part of screen usually contains large objects, which can be efficiently antialiased in spatial domain alone, while having motioncompensated artefacts there is noticeable the most. I wanted to compensate for that by overlaying a vertical gradient (white top, black bottom) with the existing edge mask in "soft light" mode, but overlay is too heavy, no resources left for it. Mere merge() or mt_average() are kinda lame.

I also had an idea of removing another kind of shimmering that mostly happens as pixels alternating colors between two states... in a way you can see it on the ground under steep angles of view. On PC games there would be mipmapping, LOD and isotropic filtering involved, blurring the textures so no flickering would've happened, but here everything is sharp and pixelated. I thought I can get rid of that by mixing an average of two frames via mask that takes the difference of a pixel between two adjacent frames, but only if same pixel is not very different from itself two frames apart. In other words, it would catch a pattern like "b w b w" ((b)lack and (w)white) for example, while leaving other kinds of differences untouched. However I couldn't quite build that kind of mask.
Seedmanc is offline   Reply With Quote
Old 6th September 2014, 04:02   #4  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
Re: motion, agree, the artifacts are most objectionable when there is little movement. I like to temporally-blur the carp out of anything that's not moving very much.

Re: alternating frames, I think if you look closely, the pattern will become more complex than a b a b. The nature of aliasing is the injection of random frequencies (spatial and temporal) into the desired signal. The only way to (partly, with luck) get rid of them is the aforementioned "blurring the carp out of it," with a mask of some type to hide the worst artifacts this creates.

I've got a script, posted below, that I think is a pretty good compromise between speed and quality, erring on the side of speed. The mask is alright, but there is room for improvement in the smoothing department.

Among smoothers, TemporalSoften is the speed king, but it can be very blurry. If the mask is working well, the blurriness is fairly well hidden. There is a selection of smoothers available in this script, including your MVTools code.

I checked it on a variety of footage, in addition to the footage you supplied. The default look is a bit soft overall, with occasional unwanted motion blur. Acceptable maybe, if you don't look at it too critically. (EDIT added wrapper function DeMoire42)
Code:
## requires MaskTools
## requires RemoveGrain
## requires FFT3DFilter, if strength=4-5 
## requires MVTools2, if strength=6

######## final output size
out_wid=1280 #1680
out_hgt=720 #1050

######## intermediate size, pre crop/letterbox (set per source below)
#size_wid=
#size_hgt=

AviSource("D:\VideoProjects\raw\mgs3motion uty7-2.avi")
#Info ## 512x448, 29.97
size_wid=1152 #1870
size_hgt=720 #1050

ConvertToYV12(matrix="Rec709", chromaresample="spline16")
#return Last

DeMoire42(strength=2, showmask=0>0)

#[[ choose one output resizer:
#return Spline36Resize(size_wid, size_hgt).UUSize(out_wid, out_hgt) ## (sharper)
return BilinearResize(size_wid, size_hgt).UUSize(out_wid, out_hgt) ## (slightly faster)
#return BilinearResize(size_wid, size_hgt) ## (don't letterbox)
#return Last ## (don't resize or letterbox)
#]]

#######################################
### Decreases certain aliasing artifacts, such as 
##  moire and shimmer; masked spatial & temporal smoothing.
##
## @ strength - main argument: "smoother" chooser
##                   1,2,3 = TemporalSoften (fast)
##                     4,5 = FFT3D         (slow)
##                       6 = MVTools
##                       0 = bypass
##                   (default 2)
## @ grnsens  - mask sensitivity to "grain": (0.0 to 10.0, default 1.0)
##                   (more grain sensitivity = softer output)
## @ mmlo     - motion mask "lo" threshold (0 to 11, default 10)
##                   (max "motion" where mask=0)
## @ mmhi     - motion mask "hi" threshold (12 to 99, default 60)
##                   (min "motion" where mask=255)
## @ subsamp  - if true, mask created from half-sized source
##                   (broader mask, also much faster)
##                   (default true if source width>600)
## @ expand   - mask expand (0 to 5, default 0)
## @ avgflashframes - if true, do mask frame averaging
##              (set to true for sources with lots of duplicate frames)
## @ sharpen  -  final sharpening: -1 to +1; default 0 
## @ showmask -  if true, make mask visible for tuning
##
## @version 0.42 raffriff42 6-Sep-2014
##
function DeMoire42(clip C, int "strength", float "grnsens", 
\                  int "mmlo", int "mmhi", bool "subsamp", 
\                  int "expand", bool "avgflashframes", 
\                  float "sharpen", bool "showmask")
{
    strength = Default(strength, 2)
    grnsens  = Min(Max(0.1, Float(Default(grnsens, 1.0))), 10.0) 
    mmlo     = Min(Max(0, Default(mmlo, 10)), 11)
    mmhi     = Min(Max(5, Default(mmhi, 60)), 99)
    subsamp  = Default(subsamp, C.Width>600)
    expand   = Default(expand, 0)
    avg      = Default(avgflashframes, false)
    sharpen  = Float(Min(Max(-1, Default(sharpen, 0)), 1))
    showmask = Default(showmask, false)

    S = (strength <1) ? C
    \ : (strength==1) ? C.TemporalSoften(1, 191, 191, 250, 2).Blim(0.6-sharpen)
    \ : (strength==3) ? C.TemporalSoften(3, 191, 191, 250, 2).Blim(1.0-sharpen)
    \ : (strength==4) ? C.AAff3d_10(temporal_size=4, sigma=29.0, sharpen=sharpen+0.5)
    \ : (strength==5) ? C.AAff3d_10(temporal_size=5, sigma=59.0, sharpen=sharpen+0.5)
    \ : (strength >5) ? C.mm_aa(bidi_comp=false).Blim(0.0-sharpen)
    \ : C.TemporalSoften(2, 191, 191, 250, 2).Blim(1.5-sharpen)
    #return S

    M = C.UUMoireMask5(mmlo, mmhi, grnsens, 
    \       subsamp, expand, avg)
    #return M

    return (strength<1) ? C
    \    : (showmask)   ? C.UUVisualMask(M, mode="luma")
    \    : C.mt_merge(S, M)
}

# http://avisynth.org.ru/fft3dfilter/fft3dfilter.html
#######################################
### FFT 3D antialising: decreases certain aliasing artifacts, 
##  such as moire and shimmer; spatial & temporal smoothing. 
##
## @ temporal_size: default 5; try 1-4 for faster processing 
##            0 - all previous frames (switch Kalman filter mode);
##            1 - only current frame (spatial 2D Wiener filter);
##            2 - previous and current frame (3D Wiener filter);
##            3 - previous, current and next frame (3D Wiener filter)
##            4 - two previous, current and next frame (3D Wiener filter)
##            5 - two previous, current and two next frames (3D Wiener filter)
##           -1 - sharpen only (2D);
##
## @ sigma - given noise value (float>0, default=12)
##
## @ sharpen - sharpening strength; good values about 0.3 to 1.0; 
##            negative values results in reverse effect.
##            (default=0.5 - slight sharpening)
##
## suggested settings: 
##   AAff3d_10()       ## aka AAff3d_10b strong
##   AAff3d_10(4, 10)  ## aka AAff3d_10c less strong
##   AAff3d_10(4, 2.2) ## slightly soft
##
function AAff3d_10(clip C, int "temporal_size", float "sigma", float "sharpen")
{
    Assert(Defined(C) && IsClip(C), 
    \     "AAff3d_10: invalid clip argument")
    Assert(C.IsYV12 || C.IsYUY2, 
    \     "AAff3d_10: invalid source format")
    sigma   = Float(Default(sigma, 12))
    sigma4  = sigma+20 ## given noise value at lowest frequencies
    
    sharpen = Float(Default(sharpen, 0.5))
    
    temporal_size = Default(temporal_size, 5)
    Assert((-2<temporal_size && 6>temporal_size),
    \     "AAff3d_10: invalid temporal_size argument")

    ## bw, bh - block width, height
    blksiz = 24
    ## ow, oh - overlap width, height
    ovrlap =  8 ## blksiz / 3

    C ## Last=C
    FFT3DFilter(sigma=sigma, sigma4=sigma4, bw=blksiz, bh=blksiz, 
    \           bt=temporal_size, ow=ovrlap, oh=ovrlap, sharpen=sharpen, 
    \           ncpu=2)
}

#requires MVTools2
#######################################
## @ Seedmanc, rr42 modded as a function
## http://forum.doom9.org/showthread.php?p=1691533#post1691533
##
function mm_aa(clip C, bool "bidi_comp")
{
    C ## Last=C
    bidi_comp = Default(bidi_comp, false)

    super = MSuper(pel=2, chroma=false) 
    backward_vectors = MAnalyse(super, isb=true, blksize=8, overlap=2, search=4, 
    \           truemotion=true, chroma=false, temporal=true, badsad=1000) 
    forward_vectors  = MAnalyse(super, isb=false, blksize=16, blksizev=8, 
    \           overlap=0, overlapv=2, search=4, truemotion=false, chroma=false,
    \           temporal=true, badsad=1000) 
    backward_compensation = MCompensate(super, backward_vectors, thSCD1=600)
    forward_compensation  = MCompensate(super, forward_vectors, thSCD1=400)  

    AA = (bidi_comp)
    \ ? merge(
    \       merge(
    \           forward_compensation, 
    \           backward_compensation).sharpen(0.2),
    \           0.67)
    \ : merge(backward_compensation) 
    return AA
} 

#requires RemoveGrain
#requires MaskTools2
##################################
### create mask from areas with "grain" (heavy random pixel activity)
##
## @ grnsens - sensitivity to "grain": (0.0 to 10.0, default 1.3)
## @ expand  - call mt_expand (0 to 5, default 3)
##
function UUGrainMask42(clip C, float "grnsens", int "expand")
{
    Assert(C.IsYUV, "source must be YUV") 

    grnsens = Min(Max(0.1, Float(Default(grnsens, 1.3))), 10.0) 
    expand  = Default(expand, 3)

    R = C.RemoveGrain(5, 0, 0) 
    \    .mt_lut(
    \       "x 127 - abs 8 * 32 - ", 
    \       u=-128, v=-128) 

    M = R.mt_lutxy(
    \       R.Blur(1.0).TemporalSoften(1, 191, 191, 250, 2),
    \       "x y - abs "+String(grnsens / 2.0 * 3.0)+" * ",
    \       u=-128, v=-128) 
    \    .mt_expandx(expand, 0.0)

    return M
}

#requires MaskTools2
#######################################
### create a mask for aliasing/moire/twinkle reduction
##
## @ mmlo    - motion mask "lo" threshold (0 to 11, default 0)
##                   (max "motion" where mask=0)
## @ mmhi    - motion mask "hi" threshold (12 to 99, default 5)
##                   (min "motion" where mask=255)
## @ grnsens - grain mask sensitivity (0.0 to 10.0, default 1.3)
##                   (more grain sensitivity = softer output)
## @ subsamp - if true, work with source 'C' half-sized 
##                   (broader mask, also 40% faster processing)
## @ expand  - call mt_expand (0 to 5, default 3)
## @ avgflashframes - if true, do mask frame averaging; about 10-15% slower
##                   (set to true for sources with lots of duplicate frames)
##
function UUMoireMask5(clip C, int "mmlo", int "mmhi", float "grnsens",
\                     bool "subsamp", int "expand", bool "avgflashframes")
{
    mmlo    = Min(Max(0, Default(mmlo, 0)), 11)
    mmhi    = Min(Max(5, Default(mmhi, 5)), 99)
    expand  = Min(Max(0, Default(expand, 0)),  5)
    grnsens = Float(Min(Max(0.1, Default(grnsens, 1.3)), 10))
    subsamp = Default(subsamp, false)
    avg     = Default(avgflashframes, false)

    S = (subsamp==false) ? C
    \ : C.BilinearResize(C.Width/2, C.Height/2)

    GM = S.UUGrainMask42(grnsens, expand)
    #return GM.BilinearResize(C.Width, C.Height)  

    MM = S.mt_motion(thy1=mmlo, thy2=mmhi, tht=100)
    \     .mt_inpand(u=-128, v=-128)
    \     .TemporalSoften(2, 191, 191, 250, 2)
    #return MM.BilinearResize(C.Width, C.Height)  

    MM = (avg==false) ? MM
    \  : MM.ConditionalFilter(
    \           MM.UUSlip(-1, MM), MM, "AverageLuma", "<", "16")
    M  = mt_logic(GM, MM.Invert, mode="min").Blur(1.0)

    return (subsamp==false) ? M
    \ : M.BilinearResize(C.Width, C.Height)  
}

# see also: Dither\mt_xxpand_multi.avsi
#requires MaskTools2
#######################################
### variable mt_expand with blur
## @ width - 0 to 5
##
function mt_expandx(clip C, int width, float "blur")
{
    b = Float(Min(Max(0, Default(blur, 0.3)), 1))

    M = C.mt_inflate
    M = (width<0) ? M : M.mt_expand.Blur(b)
    M = (width<1) ? M : M.mt_expand.Blur(b)
    M = (width<2) ? M : M.mt_expand.Blur(b)
    M = (width<3) ? M : M.mt_expand.Blur(b)
    M = (width<4) ? M : M.mt_expand.Blur(b)

    return M.Grayscale
}

##################################
### show mask overlay for visualization
##
## @ mode ("add"|"luma") default "add"
##
function UUVisualMask(clip C, clip M, String "mode", float "opacity")
{
    mode    = Default(mode, "add")
    opacity = Float(Default(opacity, 0.7))

    return (StrCmpi(mode,"luma")==0) 
    \ ? C.Overlay(M, mode="Luma", opacity=0.6*opacity) 
    \ : C.Overlay(M, mode="Add",  opacity=0.9*opacity)
}

#######################################
### constrained arguments to Blur
function Blim(clip C, float a) {
    return C.Blur(Min(Max(-0.99, a), 1.55))
}

#######################################
### "slip" a clip in time.
##
## @ C - clip to be advanced or delayed 
## @ offset - if positive, clip is advanced;
##            if negative, clip is delayed
## @ P - clip to be used for padding (default = BlankClip) 
##
function UUSlip(clip C, int offset, clip "P")
{
    lenTrim = (offset > 0) ? offset : 0
    lenPad  = (offset < 0) ? -offset : 0

    P = IsClip(P) ? P : BlankClip(C)

    C = (lenPad==0) ? C 
    \ : P.Trim(0, -lenPad) + C

    C = (lenTrim==0) ? C 
    \ : C.Trim(lenTrim, 0) + P.Trim(P.Framecount-lenTrim, -lenTrim) 

    return C
}

##################################
### crop or add borders to ensure clip is a certain size
#
function UUSize(clip C, int wid, int hgt, int "mod")
{
    mod = Max(1, Default(mod, 2))

    bdrWid = (wid - C.Width)
    bdrHgt = (hgt - C.Height)

    bdrLt = Abs(Round(Float(bdrWid) / 2.0))
    bdrLt = Sign(bdrWid) * (bdrLt + (bdrLt % mod))
    
    bdrTp = Abs(Round(Float(bdrHgt) / 2.0)) 
    bdrTp = Sign(bdrHgt) * (bdrTp + (bdrTp % mod))
    
    bdrRt = (bdrWid - bdrLt) 
    bdrBt = (bdrHgt - bdrTp)

    C = (bdrLt>0 || bdrTp>0 || bdrRt>0 || bdrBt>0) 
    \ ? C.AddBorders(
          \ ((bdrLt>0) ? bdrLt : 0), 
          \ ((bdrTp>0) ? bdrTp : 0), 
          \ ((bdrRt>0) ? bdrRt : 0), 
          \ ((bdrBt>0) ? bdrBt : 0))
    \ : C

    C = (bdrLt<0 || bdrTp<0 || bdrRt<0 || bdrBt<0)
    \ ? C.Crop(
          \ ((bdrLt<0) ? -bdrLt : 0), 
          \ ((bdrTp<0) ? -bdrTp : 0), 
          \ ((bdrRt<0) ?  bdrRt : wid), 
          \ ((bdrBt<0) ?  bdrBt : hgt))
    \ : C

    return C
}

__END__

Last edited by raffriff42; 6th September 2014 at 14:22. Reason: wrapper function DeMoire42
raffriff42 is offline   Reply With Quote
Reply

Tags
antialiasing, motion compensation, realtime, upscaling

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.