mClean spatio/temporal denoiser v3.2 (01 March 2018)

[MysteryX]

Here's a comparison with a HD video. This clip mostly has noise in the dark leather and a little bit in the hair.

Original / mClean 1.3c / mClean 1.5c enh=0 / mClean 1.5c enh=5


Here, the leather doesn't benefit as much from renoise and looks more flat. Which is why I think working in Linear Light would be a better idea. You can't do MDegrain in Linear Light because it requires HBD to not loose a lot of data, and MDegrain in HBD is a lot slower. However, you can probably convert into Linear Light right after to do the rest of the processing.

Is this script with MDegrain4 sharper than the previous? With enh=0 it looks almost just as sharp. With enh=5, however, it's way too sharp. It might look fine now, and after encoding it will still be decent, but if you re-encode a second time it will get ugly.

Originally Posted by burfadel

The enh settings is fine if you are doing only a small amount of image enlargement afterwards, and definitely fine for reducing image size.

If it's too sharp for upscaling, then it's definitely out-of-balance, and will get ugly after encoding it twice.

[MysteryX]

I'm looking at your script. Instead of excluding U and V planes of a Y12 clip on all commands, why don't you just work with a Y8 clip? This would simplify the syntax quite a bit.

To convert to linear light, something like mt_lut(x, "x 2.2 ^") should do it. Not sure where it would have to go though, or whether it would work with your logic. I'm a bit confused as to what mt_binarize is doing in your script.

[burfadel]

I plan to change it to Y8, I just want to make sure everything is working like it should, then use Y8. The results should be identical, if not then the error can be directed at the proper filter. The mt_binarize is in there so renoise isn't applied back to areas with less than 32 luma. If there is a better way of doing it... The leather is black so it isn't being applied there. As for enh, it really depends on the basal quality of the clip. If any sharpen filter looks bad on the clip enh won't work, since it is a detail orientated unsharp mask. It's why it can be disabled with 0, it can only enhance what is there! There is something I can do to for it that can improve it though, it may make it usable at a lowered setting in those cases. It's something I always intended to do but wanted to get the base done right first . There is also something else I can try for situations such as in those screenshots.

EDIT: Not sure if there is any benefit converting to Y8, apart from making the script look a bit neater. If in Y8 and none of the 'do not process chroma' flags are specified, do the filters work just on the Y plane, or do they 'process' blank chroma channels? I guess the only other reason would be if it were faster, but then again you have to consider the conversion to and from Y8 format to the original YV12/YV16/YV24 format.

[burfadel]

Updated first post with soothe function.

[StainlessS]

Originally Posted by burfadel

Not sure if there is any benefit converting to Y8, apart from making the script look a bit neater. If in Y8 and none of the 'do not process chroma' flags are specified, do the filters work just on the Y plane, or do they 'process' blank chroma channels? I guess the only other reason would be if it were faster, but then again you have to consider the conversion to and from Y8 format to the original YV12/YV16/YV24 format.

I think that for ConvertToY8,
Avs standard uses subframe, ie sort of forgets that it has chroma. Results of further filter ops comes only from the luma plane, chroma no longer considered part of frame by following filters [EDIT: which produce Y8 luma only].
Avs+, copies luma plane into new frame buffer (I think this may happen only if not already aligned to some optimal memory boundary, for eg SSE2/SIMD or whatever, if already aligned, then may do same as avs, not sure).
Both above methods have their advantages, and both sets of devs prefer their method.

For Convert back to YV12 (using eg MergeChroma), new frame will be created and planes copied/blitted into it.
Copying will be quite fast and not really worth considering as a bottle neck.

For non Y8 Planar, some ops in your script may not have a 'do not process chroma' type flag, and so those would process chroma even if nothing useful
exists there, conversely, some ops may not support Y8. Choose your poison.

[burfadel]

I guess ultimately it will come down to which method is faster. If they perform the same, then I'll probably still use Y8 for script neatness. It will be interesting to see the results of this!

[MysteryX]

It will be faster and use less memory to use Y8. Otherwise you're allocating and moving useless space around.

The only "downside" is a memcpy (or however they implement it) of a single plane for conversion, which is totally irrelevant performance-wise.

[StainlessS]

Originally Posted by MysteryX

It will be faster and use less memory to use Y8. Otherwise you're allocating and moving useless space around.

I dont see nottin wrong there, I could be mistaken. (EDIT: bit pissed).

[MysteryX]

Frame data need to allocated and passed for every filter. Y12 clip requires twice as much memory as Y8 clips. Since allocation is no more of a performance issue than memcpy, it probably won't have any performance effect other than taking twice as much memory (3x with Y24), unless some filter in the chain doesn't exclude the chroma planes.

There's just no downside to using Y8 (unless you want to support AVS 2.5.8)

If you're wondering how it works behind the scene, Y12 stores its data in 3 separate memory areas (planes). The simplest filter would take each plane a memcpy into the destination frames. ConvertToY8 simply memcpy the first plane. Not really any conversion there; it just drops unnecessary planes.

[burfadel]

That makes sense, I'll change it just to Y8 on the weekend as well as the next feature. Memory efficiency is also important.

[MysteryX]

To work in Linear Light. When you get the Diff mask containing the noise, reverse the 2.2 gamma curve and you're in Linear Light. Do your noise processing on that, then apply the 2.2 gamma curve back before adding the noise to the clip. This might solve a bunch of things.

[StainlessS]

Originally Posted by MysteryX

Y12 stores its data in 3 separate memory areas (planes). The simplest filter would take each plane a memcpy into the destination frames.

Actually, I think that only 1 memory block is allocated (large enough for all three planes + padding to ensure planes aligned on some boundary) , the plane pointers are merely set to point to an offset within that memory block.

[MysteryX]

Correct; so the only difference is double(or triple) memory usage then; and more complex syntax.

[burfadel]

I used ConvertToY8 for the Luma processing, memory use seems the same, and if anything it's fractionally slower? I have to convert the Y8 only back to YV12/YV16/YV24 to merge with chroma otherwise the merge commands gives the error it must be the same format. Is there are way of doing that without conversion? Does the conversion do anything more than just add two blank chroma channels until the chroma/luminance merge?

Below is a test script with function mCleanY8. It's identical to the latest mClean apart from processing luma in Y8. If it is faster or uses less RAM that's good, but if it isn't it would be good to try and track down the reason why since it should be faster due to less memory data copied etc, or at least, absolutely not slower .

What are the encoding speeds for mClean and everything exactly the same except using mCleanY8? Does the latest RGTools functions properly support Y8, in that if it is Y8 that it automatically doesn't even consider phantom chroma channels?

# mClean spatio/temporal denoiser
# Version: 1.5e (06 September 2017)
# By burfadel

### *** For TESTING only *** ###
### *** Function mCleanY8 *** ###


function mCleanY8(clip c, int "thSAD", int "blksize", int "blksizeV", int "overlap", int "overlapV", int "enh", int "rn", int "outbits", int "cpu")
{
    defH        = Max (C.Height, C.Width/4*3)     # Resolution calculation for auto blksize settings
    thSAD       = Default (thSAD, 450)     # Denoising threshold
    blkSize     = Default (blkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 16 : DefH<1600 ? 24 : 32)     # Horizontal block size for MDegrain4
    blksizeV    = Default (blksizeV, blksize)     # Vertical block size for MDegrain4, default same as horizontal
    overlap     = Default (overlap, blksize>4?(blksize/4+1)/2*2:0)     # Horizontal block overlap
    overlapV    = Default (overlapV, blksize>4?(blksizeV/4+1)/2*2:0)     # Vertical block overlap
    enh         = Default (enh, 20)     # Detail enhancement (detail orientated sharpen) strength
    rn          = Default (rn, 13)     # ReNoise strength from 0 (disabled) to 20
    outbits     = Default (outbits, c.BitsPerComponent)     # Output bits, default input depth
    calcbits    = c.BitsPerComponent == 8 ? 12 : c.BitsPerComponent
    calcbits    = outbits > calcbits ? calcbits : outbits
    cpu         = Default (cpu, 4)     # Threads for fft3dfilter


    Assert(isYUV(c)==true, """mClean: Supports only YUV formats (YV12, YV16, YV24)""")
    Assert(isYUY2(c)==false, """mClean: Supports only YUV formats (YV12, YV16, YV24)""")
    Assert(isYV411(c)==false, """mClean: Supports only YUV formats (YV12, YV16, YV24)""")
    Assert(enh>=0 && enh<=54, """mClean: "enh" ranges from 0 to 54""")
    Assert(rn>=0 && rn<=20, """mClean: "rn" ranges from 0 to 20""")
    Assert(outbits>=8 && outbits<=16, """mClean: "outbits" ranges from 8 to 16""")

padX = c.width%4 == 0 ? 0 : (4 - c.width%4)
padY = c.height%4 == 0 ? 0 : (4 - c.height%4)
c    = padX+padY<>0 ? c.pointresize(c.width+padX, c.height+padY, 0, 0, c.width+padX, c.height+padY) : c
cy   = c.ConvertToY8()

# Spatio/temporal chroma noise filter
filt_chroma  =  fft3dfilter (c, bw=blksize*2, bh=blksizeV*2, ow=overlap*2, oh=overlapV*2, sharpen=0.12, bt=3, ncpu=cpu, dehalo=0.3, sigma=2.35, plane=3)

# Temporal luma noise filter
super      =  cy.MSuper (hpad=16, vpad=16)
bvec4      =  MAnalyse (super, isb = true, delta = 4, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=13)
bvec3      =  MAnalyse (super, isb = true, delta = 3, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=8)
bvec2      =  MAnalyse (super, isb = true, delta = 2, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=5)
bvec1      =  MAnalyse (super, isb = true, delta = 1, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=3)
fvec1      =  MAnalyse (super, isb = false, delta = 1, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=3)
fvec2      =  MAnalyse (super, isb = false, delta = 2, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=5)
fvec3      =  MAnalyse (super, isb = false, delta = 3, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=8)
fvec4      =  MAnalyse (super, isb = false, delta = 4, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=13)
clean      =  cy.MDegrain4(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, bvec4, fvec4, thSAD=thSAD)
clean      =  clean.dctfilter (1,1,1,1,1,1,0.50,0)

clean      =  calcbits != clean.BitsPerComponent ? clean.ConvertBits(calcbits) : clean
cy         =  calcbits != c.BitsPerComponent ? cy.ConvertBits(calcbits) : cy

# Masks for spatial noise reduction and noise independent detail enhancement
noised      =  mt_makediff (clean, cy)
noise       =  mt_binarize (clense(mt_makediff(mt_binarize(noised), mt_edge(sharpen(clean, 0.82), "prewitt"))))

# Spatial luma denoising
clean2      =  mt_merge (clean, removegrain(clean, 18), noise)

# Unsharp filter for spatial detail enhancement
clsharp     = (enh>=51<=54) ? mt_adddiff (blur(mt_makediff(clean, gblur(clean2, (enh-19), sd=3)),0.8,0), clean2) :
            \ (enh>0<=50) ? mt_adddiff (blur(mt_makediff(clean, blur(clean2, 1.58*(0.03+(0.97/50)*enh))),0.5,0), clean2) : clean2
diff        = (enh>0) ? mt_makediff(clean2, clsharp) : nop()
diff2       = (enh>0) ? diff.temporalsoften(1,255,0,32,2) :nop()
clsharp     = (enh>0) ? mt_makediff(clean2, mt_lutxy(diff,diff2,  "x 128 - y 128 - * 0 < x 128 - 100 / " + string(40) 
            \ + " * 128 + x 128 - abs y 128 - abs > x " + string(40) + " * y 100 " + string(40) + " - * + 100 / x ? ?")) : clsharp

# If selected, combining ReNoise
renoise     =  (rn==0)    ? nop() : tweak(temporalsoften (noised, 3, 160, 0, scenechange=0, mode=2), cont=1.025+(0.022*(rn/20)))
clean2      =  (rn>0<=20) ? mt_merge(clean2, mergeluma (clean2, mt_adddiff(clean2, renoise), 0.3+(rn*0.035)),
            \  mt_logic(noised, mt_binarize(clean, 24), mode="andn")) : clean2 

# Combining spatial detail enhancement with spatial noise reduction using prepared mask
filt_luma   =  mt_merge (clean2, clsharp, mt_invert(noise))

# Converting bits per channel and luma format
filt_luma   =  outbits < filt_luma.BitsPerComponent ? ConvertBits(filt_luma, outbits, 1) : filt_luma
filt_luma   =  c.isYV12() == true ? ConvertToYV12(filt_luma) :
            \  c.isYv16() == true ? ConvertToYV16(filt_luma) : ConvertToYV24(filt_luma)
filt_chroma =  filt_chroma.BitsPerComponent <> filt_luma.BitsPerComponent ? ConvertBits(filt_chroma, BitsPerComponent(filt_luma)) : filt_chroma

# Combining result of luma and chroma cleaning
output      =  mergechroma (filt_luma, filt_chroma)
return padX+padY<>0 ? output.pointresize(c.width-padX, c.height-padY, 0, 0, c.width-padX, c.height-padY) : output
}

[StainlessS]

You could try this stuff here (swap et al):- http://avisynth.nl/index.php/Swap
I cant try, not got avs+ set up.

EDIT:
Does FFT3DFilter require padding ?

Think AVS+ requires alignment mod 32, your padding might require additional blit/copy.

Would crop be quicker than point resize at end ?

[burfadel]

Originally Posted by StainlessS

You could try this stuff here (swap et al):- http://avisynth.nl/index.php/Swap
I cant try, not got avs+ set up.

EDIT:
Does FFT3DFilter require padding ?

Think AVS+ requires alignment mod 32, your padding might require another blit.

Would crop be quicker than point resize at end ?

The swap stuff looks like it could work! I borrowed the pointresize from deblock_QED as mentioned in the first post, apparently they changed from using addborders and crop to pointresize, so I presumed it was done for a reason. I'll have to look up where the padding occurs in point resize and try it out with addborders/crop borders.

If AVS+ requries mod 32, that's easy, since mod 32 is also mod 4.

[StainlessS]

Didee Padding

function Padding(clip c, int left, int top, int right, int bottom) {
    # Didee: http://forum.doom9.org/showthread.php?p=1596804#post1596804
    # eg, Padding(32,32,0,0).Padding(0,0,32,32)
    w = c.width()
    h = c.height()
    c.pointresize( w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom )
}

EDIT: Crop defo faster if at end of chain on avs standard (but your func dont work on that so is moot).

EDIT: Dont use Addborders, Padding edge mirroring [EDIT: duping] better.

Avisource("F:\v\Cabaret.avi")
Robocrop()
Padding(32,32,32,32)
return last

EDIT: MSuper will be padding again

MSuper

MSuper (
	clip,
	int  hpad (8),
	int  vpad (8),
	int  pel (2),
	int  levels (0),
	bool chroma (true),
	int  sharp (2),
	int  rfilter (2),
	clip pelclip (undefined),
	bool isse,
	bool planar,
	bool mt (true)

EDIT:

Avisource("F:\v\Cabaret.avi")
Robocrop()
MOD=64 # Or whatever
padW = ((width+MOD-1)/MOD*MOD) - Width   # Required additional width mod MOD
LeftW=PadW/4*2
RightW=padW-LeftW
Padding(LeftW,0,RightW,0)
return last.Info

[burfadel]

That's good! I can change it so it pads to 32 on for instance, bottom and right, and crops that at the end.

[StainlessS]

Above EDIT, is not really padding (neither is yours, is only achieving mod MOD width).

Below will I think ensure padding and also produce mod MOD width.

Avisource("F:\v\Cabaret.avi")
Robocrop() # source has big borders
ORGW=Width
ORGH=Height
PADW=16                                             # Minimum Padding required either side (0 or more)
PADH=16                                             # Minimum Padding required top/bot (0 or more)
MODW=32                                             # width must be modulo MODW
MODH=2                                              # height must be modulo MODH (32 probably excessive, some encoders require modulo 8, I think, )
AddW = ((width+PADW*2+MODW-1)/MODW*MODW) - Width    # Required additional width mod MODW
LeftW=AddW/4*2  RightW=AddW-LeftW
AddH = ((height+PADH*2+MODH-1)/MODH*MODH) - Height  # Required additional height mod MODH
TopH=AddH/4*2   BotH=AddH-TopH
Padding(LeftW,TopH,RightW,BotH)
RT_debugF("PADW=%d MODW=%d PADH=%d MODH=%d LeftW=%d RightW=%d TopH=%d BotH=%d Width=%d(Orig Width=%d) Height=%d(Orig Height=%d)",
    \ PADW,MODW,PADH,MODH,LeftW,RightW,TopH,BotH,Width,ORGW,Height,ORGH)
return last.Info

EDIT: Without proper padding, can result in weird disappearing reappearing things on edges of frame.
(only noticed left and right, never seen on top or bottom).

EDIT: Added ORGW

EDIT: Fiddled with it a bit more.

EDIT: Reduced MODH to 2, Fiddled with it a bit more. Can also set MSuper(hpad=0,vpad=0) as we already did it.

[burfadel]

Updated the first post with version 1.6.

If you extract the luminance channel such it is just the Y channel, obviously it is a different format to the YUV clip such as output from the chroma filtering. You first have to convert back to the same format before combining. This is fine, however when you are in more than 8 bits, the commands such as isYV12() etc do not work since it is expecting a YUV420Y8 clip for it to be true. Since it is a YUV420P10 clip (for example) the command isYV12() seems to give back false. If you run convertToYV12(), it keeps the higher bitdepth of 10 etc, so isYV12() still doesn't work. This, and possibly other little issues, are problematic when you want to have the output clip the same format as the input clip and it isn't 8-bit etc. This is why I put back the 'do not process' flags to the luma commands, it was either that or dropping support for YV16 and YV24, which obviously isn't desirable.

It works fine as is because the bit depths can be matched easily and the pixel type hasn't changed. I could do a convertbits() just so isYV12() etc works, but then it's an extra unnecessary bit of processing that will probably affect speed more than what was gained by converting it to Y8 and going back to the desired pixel type.