Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th October 2021, 01:05   #1  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Detecting the amount of noise

Is there a way to detect the amount of noise in a frame? Either that, or get the average luma of the 85% darkest pixels on the frame.

The idea is to run 2 clips of KNLMeans with different 'h' parameter, and selecting between them based on the noise threshold or luma threshold.
MysteryX is offline   Reply With Quote
Old 16th October 2021, 01:39   #2  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 439
Quote:
Originally Posted by MysteryX View Post
Is there a way to detect the amount of noise in a frame?
A crude but surprisingly effective way is to run an edge detector -> threshold -> AverageLuma. The edge detector should be a sensitive one, such as VapourSynth's g41.EdgeDetect(clip, mode='TEdge') (not sure which filter represents that on AviSynth side).

Quote:
Originally Posted by MysteryX View Post
Either that, or get the average luma of the 85% darkest pixels on the frame.
If you mean the darkest selection of pixels covering at least 85% of the screen then you need a luma histogram for that. I think a histogram would be very useful for many algorithms so hopefully someone implements a histogram plugin which writes the histogram data as an array frameProp.
zorr is offline   Reply With Quote
Old 16th October 2021, 01:54   #3  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 1,089
Here:
http://avisynth.nl/index.php/Externa...Edge_Detection
kedautinh12 is offline   Reply With Quote
Old 16th October 2021, 03:08   #4  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Quote:
Originally Posted by zorr View Post
A crude but surprisingly effective way is to run an edge detector -> threshold -> AverageLuma. The edge detector should be a sensitive one, such as VapourSynth's g41.EdgeDetect(clip, mode='TEdge') (not sure which filter represents that on AviSynth side).
It detects all the moving trees and barely detects the noisy areas. If I select only values below 50, there are still way too much everywhere and not in noisy areas.
MysteryX is offline   Reply With Quote
Old 16th October 2021, 03:18   #5  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
DUH! My code is already doing noise-diff on several denoising passes.

Noise-diff... HELLO!??

Now... mt_makediff gives a mask with values around 128... how do I calculate the amount of noise from that? Need to calculate the amount of deviation from the center. Noise is up and down so AverageLuma will remain 128.

Is this formula correct, or is there a better way to calculate the noise from a noise mask? (*5 is just to make it visual for testing)

Code:
clean = core.std.Expr([core.std.Expr(noise, "x 128 -"), core.std.Expr(noise.std.Invert(), "x 128 -")], "x y + 5 *").std.PlaneStats().text.FrameProps(scale=4)
EDIT: the inverted noise mask is wrong.

then I can store this AverageLuma as frame property "Noise" and re-use it on the next denoising pass.

Last edited by MysteryX; 16th October 2021 at 04:13.
MysteryX is offline   Reply With Quote
Old 16th October 2021, 03:57   #6  |  Link
Julek
Registered User
 
Julek's Avatar
 
Join Date: Dec 2020
Posts: 50
Instead of using mt_makediff and AverageLuma, why not just LumaDifference?
Julek is offline   Reply With Quote
Old 16th October 2021, 04:31   #7  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Quote:
Originally Posted by Julek View Post
Instead of using mt_makediff and AverageLuma, why not just LumaDifference?
LumaDifference would work in Avisynth. What about VapourSynth?
MysteryX is offline   Reply With Quote
Old 16th October 2021, 04:42   #8  |  Link
Julek
Registered User
 
Julek's Avatar
 
Join Date: Dec 2020
Posts: 50
Quote:
Originally Posted by MysteryX View Post
LumaDifference would work in Avisynth. What about VapourSynth?
core.std.PlaneStats(clipa, clipb)
Julek is offline   Reply With Quote
Old 16th October 2021, 04:50   #9  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 1,089
Quote:
Originally Posted by MysteryX View Post
LumaDifference would work in Avisynth. What about VapourSynth?
You can use PlaneCompare to replace LumaDifferenc
https://github.com/AmusementClub/mvs...sfunc.py#L1361

Example scripts:
https://github.com/WolframRhodium/mu...sfunc.py#L5441

Last edited by kedautinh12; 16th October 2021 at 06:41.
kedautinh12 is offline   Reply With Quote
Old 16th October 2021, 06:27   #10  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
OK I'm surprised... MVTools2 and BM3D don't necessarily remove more noise on dark noisy scenes than on lighter scenes, so the noise mask isn't a reliable way to detect noisy scenes... In fact, the removal of dark noise may count less than the loss of bright textures with MVTools2. And BM3D is just very constant in what it removes.

Other approach: getting average luma of the 85% darkest pixels? Or measuring how many pixels above 50% luma, and if lower than 25% of the frame, calculate AverageLuma of the remaining pixels.

Here's the VapourSynth code that I used to calculate the noise from noise_diff as a frame property.

Code:
def WriteNoiseStat(n, f: list[vs.VideoFrame]):
    noise = f[0].props["PlaneStatsAverage"] if "PlaneStatsAverage" in f[0].props else 0
    f_out = f[1].copy()
    f_out.props["xClean_Noise"] = noise
    return f_out
noise_stats = core.std.Expr(noise_diff.fmtc.bitdepth(bits=8, fulls=True, fulld=True, dmode=1), "x 128 - abs").std.PlaneStats()
clean2 = clean2.std.ModifyFrame([noise_stats, clean2], WriteNoiseStat)

Last edited by MysteryX; 17th October 2021 at 14:46.
MysteryX is offline   Reply With Quote
Old 16th October 2021, 07:39   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,087
Quote:
average luma of the 85% darkest pixels
How does this sound ?

Find luma level that partitions 85% [Y85=YPlaneMin(threshold=85.0)]
[EDIT: YPlaneMedian() is implemented as YPlaneMin(threshold=50.0) ]

Create mask using Masktools to set luma >= Y85 [EDIT: Should be <= Y85 ] to pure white, and use RT_Stats Masked function for average luma.

YPlaneMin [or 8 bit only RT_YPlaneMin]

Code:
RT_YPlaneMin(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      float "threshold"=0.0, int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1, if no valid pixels in Mask clip.
  Returns int value minimum luma (0 -> 255) in frame(n+delta) for area x,y,w,h.
  Threshold is a percentage, stating how many percent of the pixels are allowed below minimum (ignore extreme pixels ie noise).
  Threshold is % of valid mask pixels processed if Mask supplied. The threshold is optional and defaults to 0.0.
Actually, I suppose you could cut out the mask making thingy altogether and use the frame luma itself as the mask,

Again, 8 bit only.
Code:
RT_AverageLuma(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
    int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1, if no valid pixels in Mask clip.
  Returns FLOAT value average luma (0.0 -> 255.0) in frame(n+delta) for area x,y,w,h.
Mask=SourceClip MaskMin=0 MaskMax=Y85


Code:
****************************************
******* MASKED Luma Y Functions ********
****************************************

Compile time/runtime functions, Planar, YUY2, RGB24 & RGB32. (RGB internally converted to YUV-Y).

The compiletime/runtime clip functions share some common characteristics:-

The 'n' arg is an optional frame number and defaults to 'current_frame' if not specified.
The x,y,w,h, coords specify the source rectangle under scrutiny and are specified as for Crop(), the default 0,0,0,0 is full frame.
If 'interlaced' is true, then every other line is ommited from the scan, so if eg y=1, then scanlines 1,3,5,7 etc are scanned,
if eg y=4 then scanlines 4,6,8,10 etc are scanned. The 'h' coord specifies the full height scan ie same whether interlaced is true
or false, although it will not matter if the 'h' coord is specified as eg odd or even, internally the height 'h' is reduced by 1
when interlaced=true and 'h' is even.

Matrix: Conversion matrix for conversion of RGB to YUV-Y Luma.  0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,
  Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used

The optional mask clip, governs which pixels are processed by the functions. Where a luma pixel in selected area of the the mask clip is
in range "MaskMin" to "MaskMax" inclusive, then those pixels will be processed. The Mask must also be Planar but not
necessarily the same colorspace as clip c, also must be same dimensions and have at least the same number of frames as clip c.
Calling without mask clip OR with MaskMin=0,MaskMax=255 will effectively ignore the mask and scan full x,y,w,h area.
EDIT: A few more peculiar ones that might be of use one day.

Code:
RT_YPlaneStdev(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
    int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1, if no valid pixels in Mask clip.
  Returns FLOAT value luma Standard Deviation (0.0 -> 255.0) in frame(n+delta) for area x,y,w,h.
  Standard Deviation (Changed, from Sample Standard Deviation with Bessels Correction).
  http://en.wikipedia.org/wiki/Standard_deviation

***
***
***

RT_YInRange(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      int "lo"=128,int "hi"=lo,int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1, if no valid pixels in Mask clip.
  Returns FLOAT value (0.0 -> 1.0) being the amount of pixels in the range "lo" to "hi" (inclusive), 1.0 is equivalent to 100%.
  Implemented as requested by Martin53 (thankyou), NOTE, differs from other funcs that return range 0.0 to 255.0.
  NOTE, lo defaults to 128, "hi" defaults to "lo".

***
***
***

RT_YPNorm(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Return int -1 if no valid pixels in Mask clip.
  Returns FLOAT value greater or equal to 0.0, being the "Minkowski P-norm" (range depends upon values of 'd' and 'u') for frame(n+delta)
  area x,y,w,h.
  mu, Float, default 0.0 (0.0 -> 255.0)
  d,  int,   default 1 (1 -> 255)       # downscale
  p,  int,   default 1 (1 -> 16)        # power
  u,  int,   default 1 (1 -> 255)       # final upscale before returning result (experimental)
  Formula is: sum_over_pixels[ ((pixel-mu)/d)^p ]^(1/p) * u
  or in words: d and u are scaling aids. The differences between the pixel values and mu are scaled, taken to the power of p and added up
  over the frame. The sum is taken to the p-th root and finally rescaled.
  mu=0, d=1, p=1, u=1 yields the average.
  mu=average, d=1, p=2, u=1 yields the standard deviation (uncorrected sample standard deviation).
  Implemented as requested by Martin53 (thankyou). *** EXPERIMENTAL ***
  http://en.wikipedia.org/wiki/P-norm#The_p-norm_in_finite_dimensions
  http://en.wikipedia.org/wiki/Minkowski_distance
EDIT: And theres also this
Code:
RT_Ystats(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
      float "threshold"=0.0,int "lo"=128,int "hi"=lo,int "flgs"=255,string "prefix"="YS_",
        float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1, int "Matrix"=(Width>1100||Height>600?3:2),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
  Returns multiple results as for above single frame Luma sampling functions as Local Variables (prefixed with the prefix string arg).
  The args up to "interlaced", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
  "RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
  "lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
  "mu" and "d" and "p" and "u" are used only with the "RT_YPNorm" equivalent routine with same functionality.
  The new arg "Flgs" selects which results you want returned and the string "Prefix" that is prepended
  to the returned Local variable names.
  The actual return result is a copy of the flgs arg with any non valid bits reset, ie the Local variables that were set.
  Local variables are NOT altered for any function not selected in flgs.
  Returns 0 if no pixels found in search area of mask within MaskMin and MaskMax.

  Flgs_Bit_Number   Add_To_Flgs     Equivalent_Function             Local_Var_Set_Excluding_Prefix
     0                 1($01)        RT_YPlaneMin()                     "yMin"        (0->255)
     1                 2($02)        RT_YPlaneMax()                     "yMax"        (0->255)
     2                 4($04)        RT_YPlaneMinMaxDifference()        "yMinMaxDiff" (0->255)
     3                 8($08)        RT_YPlaneMedian()                  "yMed"        (0->255)
     4                16($10)        RT_AverageLuma()                   "yAve"        (0.0->255.0)
     5                32($20)        RT_YPlaneStdev()                   "yStdev"      (0.0->255.0)
     6                64($40)        RT_YInRange()                      "yInRng"      (0.0->1.0)
     7               128($80)        RT_YPNorm()                        "yPNorm"      (0.0->??? depends upon d and u)
  RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
  routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
  The Default flgs=255($FF) are all bits set and so sets ALL Local vars at once.
  RT_Ystats(flgs=1+2+16) would set Local vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.

  In addition to above Local Variables, RT_YStats() sets an int Local variable (where default prefix) of "YS_PixelCount" being
  the number of pixels in mask area X,Y,W,H between MaskMin and MaskMax inclusive, or pixels scanned in X,Y,W,H area where mask
  not used.
  NOTE, If no valid flg bits set (eg $FF00), then returns 0, YS_PixelCount and all other variables remain as before call.
  Assuming some valid flg bits, if no valid pixels were found in mask then function returns 0, and only YS_PixelCount would be set
  to 0, no other variables are touched (remain as before call, undefined if not previously existing).
  Example usage:
    ScriptClip("""
      got = RT_Ystats(c,mask=Mask,flgs=$10) # AverageLuma
      (got != 0) ? RT_debug("AverageLuma = " + String(YS_yAve) + "PixelCount = " + String(YS_PixelCount)) : RT_Debug("NO VALID PIXELS FOUND")
    """)
Also has similar-ish funcs for RGB.

EDIT: Damn!, Euromillions 185.5 Millions went to France, what a senseless waste.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 16th October 2021 at 09:58.
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.