Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
7th December 2021, 14:13 | #41 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,823
|
Quote:
anton_foy, I just noticed a difference between my script and the version of FastDegrain on the wiki in respect to LSFMod sharpening, and I don't remember if it was deliberate or just a silly. It's probably the latter but I rarely use FastDegrain with sharpening enabled so I can't remember. Anyway.... The version on the wiki sharpens the video before MSuper. Mine doesn't. I'm not sure how much difference it'd make or if one way is better than the other, but lines 75-79 in the version I posted are this (or they were, as I'll edit my previous post to change it to match the original script): Code:
Input = !Sharp || !SFirst ? Source : \ Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot) SuperClip = Source.MSuper(pel=Pel, mt=MT) Code:
Input = !Sharp || !SFirst ? Source : \ Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot) SuperClip = Input.MSuper(pel=Pel, mt=MT) The default is for LFSMod to sharpen the denoised clip as it would if it followed FastDegrain in a script. ie FastDegrain() LSFMod() Oh, and to be pedantic, I changed the first two lines of the Output section too. This: Code:
Output = \ (Degrain < 1) ? Source : \ Code:
Output = \ (Degrain < 1) ? Input : \ Last edited by hello_hello; 7th December 2021 at 15:05. |
|
7th December 2021, 15:00 | #42 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,823
|
anton_foy,
Something else you might care to try. I was going to add MDegrain=4/5/6 to FastDegrain, but as I've never known if MDegrainN produces the same result as the other MDegrains I compared them today, and it appears when all else is equal, it does. So for a greater temporal range I used this script instead, It's called MDegrainNL (MDegrain Light) as the defaults do fairly light degraining, but it's just the defaults that are a little different, and MDegrainNL doesn't include sharpening. These should produce the same result. MDegrainNL(TR=3, thSAD=400, BLKSize=16, Overlap=4, Pel=2) FastDegrain(Degrain=3, thSAD=400, BLKSize=16, Overlap=4, Pel=2) Edit: By the way, when you said FastDegrain gives you 0.0222x speed is that correct, because for a 24fps source I'd interpret that to mean something like 0.53fps, which is painfully slow. Edit: Removed the default thSAD2 value from the MDegrainNL function. See post #46. Code:
# =============================================================================== # =============================================================================== # MDegrainNL (MDegrainN Light) 2021-12-08 # =============================================================================== # =============================================================================== function MDegrainNL(clip Source, \ int "TR", int "thSAD", int "thSAD2", int "BLKSize", int "Overlap", int "Pel", bool "Precise", \ bool "TrueM", bool "GlobalM", int "lambda", int "pnew", bool "MT", bool "LSB") { TR = default(TR, 1) # Temporal radius thSAD = default(thSAD, 150) # Denoising strength BLKSize = default(BLKSize, 16) # Block size Overlap = default(Overlap, BLKSize/2) # Block overlap Pel = default(Pel, 2) # MVAnalyse pel (1, 2, 4) Precise = default(Precise, false) # Recalculate TrueM = default(TrueM, false) # Enable/disable truemotion GlobalM = default(GlobalM, true) # Enable/disable global motion MT = default(MT, false) # Internal multithreading LSB = default(LSB, false) # Stacked 16-bit # Uncommenting the two lines below results in the same MVTools2 settings as QTGMC when truemotion=false # lambda = default(lambda, (TrueM ? 1000 : 100 ) * BLKSize * BLKSize / 64) # pnew = default(pnew, TrueM ? 50 : 25) Super = MSuper(Source, pel=Pel, mt=MT) MultiVec1 = MAnalyse(Super, multi=true, delta=TR, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) MultiVec2 = !Precise ? MultiVec1 : \ MRecalculate(Super, MultiVec1, tr=TR, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD2, truemotion=TrueM, lambda=lambda, pnew=pnew) MDegrainN(Source, Super, MultiVec2, TR, thSAD=thSAD, thSAD2=thSAD2, mt=MT, lsb=LSB) } # =============================================================================== # =============================================================================== Last edited by hello_hello; 8th December 2021 at 07:37. |
7th December 2021, 16:06 | #43 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
"Is there some method of detecting if Prefetch is being used ?"
It is a good question to AVS developers - may be exist or can be simply added some 'property' like _num_avs_threads. If it > 1 - the internal avstp-based mt in mvtools may be disabled. Or may be adjusted to num_cores - num_avs_threads using avstp_set_threads(). " it's permanently enabled? " Hmm - it looks like forgotten place to optimize. If enabled it start process of estimating global motion vector (more or less time consuming). But if disabled - the motion vector still checked at the search process (may be with some default data like zero). If users frequently do not use global motion analysis - it can be disabled in processing and speed up things a bit. But it will be +1 branching in the frequently used part of program and possible +slight degradation of speed for users of global motion. It looks it was expected global motion predictor will be most of time used. So its disabling 'global' may be sort of user-side debug option. Though it is auto-disabled with disabling of 'truemotion' group of settings. But 'truemotion' is enabled in default settings. " I've never known if MDegrainN produces the same result as the other MDegrains" In latest official 2.7.45 MDegrainN fallback to (faster (possibly)) MDegrainX if thSAD=thSAD2. In later testbuilds MDegrainN have additional wpow param and pinterf wrote it need to be propagated to MDegrainXs for compatibility. But it still delayed. The speed of MDegrainXs may be a bit better in compare with MDegrainN with same tr. Do SVP GPU-based MAnalyse can support 'large' tr like up to 128 (current MDegrainN max) ? "thSAD2 = default(thSAD2, thSAD/2)" It is strange to see such great limit of thSAD2 param. It will mostly waste CPU cycles for 'useless' frames at the end of 'tr'. The main idea of thSAD2 is looks like to lower some visible bugs (errors in averaging too different blocks and so blurring) with large enough tr values. If these bugs ever happens at all. It may be if thSAD is set too high for current content (its noise level). So if thSAD2 = thSAD/2 it for MDegrain3 mean about only MDegrain2 or MDegrain1 is actually used for denoising because at the tr=3 the thSAD falls to really thSAD/2 and below noise-deviated SAD so that blocks will get 0 averaging weight in MDegrainN. Typical default for enabling MDegrainN processing is thSAD2 = thSAD-1 . With precise adjustment (bewteen detail blurring and noise decreasing) the thSAD value is just small above 'noise level' and there is almost no space left to lower thSAD2. So thSAD2 may be 75..90% of thSAD or just equal. In best case 'degrain script' must not have any thSAD default (or set it to extra low like 100 or 50) so user must adjust it (increase) to its content to set and understand the relation between degraining strength and low contrast details lost. For the current user's use case. Last edited by DTL; 7th December 2021 at 16:57. |
7th December 2021, 21:01 | #44 | Link |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
@DTL
Yes I guess it is something weird with my script or my computer? The footage I use is in UHD. @hello_hello I use UHD-footage from my Sony A6300 camera. My BASIQ-script is almost as fast as using only fastdegrain with mp_pipeline and very effective (the dancing noise in my footage gives a wobbly appearance with spatial and most temporal denoisers aswell but not with fastdegrain/mdegrain + my neo_vaguedenoiser-mocomped) Comp: Nvidia GeForce GTX 970 Intel Core i5-3570K 3.40GHz 32GB RAM Script: Code:
MP_Pipeline(""" SetFilterMTMode("DEFAULT_MT_MODE", 2) #Index LSMASHVideoSource("F:...mp4") # ### prefetch: 32, 16 ### ### ### platform: Win32 SetFilterMTMode("LSmashVideoSource", MT_SERIALIZED) convertbits(16) Deblock_QED_mt(quant1=26, aOff1=1, aOff2=1, bOff1=2, bOff2=2, uv=3) Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false) ### platform: win32 ### branch 4 ### ### ### platform: Win32 ### ### # DENOISE # #basiq(preset="high") Fastdegrain(degrain=3) #Fastdegrain_svp(degrain=3) ### ### ### platform: Win32 neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true) ### platform: Win32 ### ### """) trim(40,60) #Chroma Upscaling ConverttoYUV444() ConvertBits(bits=10, dither=1) Prefetch(4) |
7th December 2021, 21:20 | #45 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
"it is something weird with my script or my computer?"
I do not know what mean Deblock_QED_mt and neo_f3kdb and what is their speed. How speed increases if disable Fastdegrain ? Also it is better not leave thSAD param to default - it should be adjusted to the current footage noise level. If there is a combined footage with different 'ISO' settings and different noise levels - it is better to process each part with different thSAD value. If your camera do not use auto-ISO at time of shooting I think it is possible to make a table of thSAD values for each ISO setting (that mean for some fixed noise levels from that hardware). And assign the required thSAD value based in the ISO setting from metadata. Or at least select it as starting value for fine tuning. |
8th December 2021, 07:33 | #46 | Link | |||
Registered User
Join Date: Mar 2011
Posts: 4,823
|
Quote:
thSAD2 = default(thSAD2, thSAD/2) but as you made me look I repeated one of yesterday's encodes with thSAD2=thSAD, and there's differences, both good and bad. Yesterday's encode came in at 4169.06 kb/s, and after running it again today with thSAD2=thSAD, the bitrate dropped to 3679.27 kb/s, so yes obviously more noise was removed. The source is very noisy animation so I'm using: MDegrainNL(Tr=6, thSAD=600, Pel=4) For animation, making thSAD2 the same as thSAD caused some lines to be blurred a tad more or reduced in size where there's movement. Not a lot, but it was there. That additional blurring increased a little when the video was encoded (compared to looking at the script output) but I assume that's because the extra noise also caused the encoder to keep a bit of extra detail too. So after a few cigarettes and some time contemplating my naval, as I can only see differences where there's motion (in respect to blurring), and even then, only when I compare individual frames, I think I'll just remove the thSAD2 line from the script. Thanks for that. Quote:
Quote:
Last edited by hello_hello; 8th December 2021 at 08:11. |
|||
8th December 2021, 08:46 | #48 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
To make 'FastDegrain' faster at 16bit inputs it may be added converting input to 8bit for MAnalyse. It should be faster and new testbuilds have only 8bit MAnalyse SIMD accelerated functions for 8x8 and 16x16 blocks only.
|
8th December 2021, 09:25 | #49 | Link | ||
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
Code:
# Create our Super clip for MV search. Then, search for motion vectors. super = src.MSuper(pel=pel) super8 = src.convertbits(8).MSuper(pel=pel) bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() # Degraining the video using MVDegrain. Nothing special here. src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src return src } Quote:
|
||
8th December 2021, 10:54 | #50 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
Is that mainly the trade off for higher blocksizes or can another parameter compensate it? |
|
8th December 2021, 12:29 | #51 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
"get more motion blur added in moving parts"
In theory the thSAD is scaled internally to at the changing of block size. So the disadvantages of 16x16 block should be mostly on the angled luma/chroma transients. But large flat enough areas should be identical with 8x8 block. Have you adjust the thSAD value for the best ratio of blur vs degraining ? For some 'drafting' work the 'PredictorType=4' mode may be used. It is sort of 'checked/refined pel=0.5 (or 0.25?)/ interpolated (predicted) pel = 1' and simply outputs prediction of level 1 without any search at largest and slowest level 0. So it close to use full-HD sized MAnalyse with UHD-1 (4K) sized source for MDegrain. The speed is in theory about 4 times faster. But unfortunately speed of MDegrain itself also not very great yet and with infinitely fast MAnalyse it become main limiting. I hope its stable enough example available at build - https://drive.google.com/file/d/1kMc...H8JBx3Pl1/view . Using optPredictorType=4 require re-adjusting thSAD to about 1.5 times lower because SAD from level 1 (half HV sized) typically lower in compare with level 0. Last edited by DTL; 8th December 2021 at 12:31. |
8th December 2021, 15:07 | #52 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,823
|
anton_foy,
Something else you might want to try... Expanding a limited range source to full range for MAnalyse seems to be common in functions these days. I've been meaning to add it for a while but never got round to it. It might compensate for reducing the bitdepth though. I don't know, but I added some new arguments to FastDegrain and MDegrainNL for MAnalyse bitdepth and range, and while I was at it I added Degrain4/5/6 to FastDegrain, in case you want to encode even slower. Code:
# =============================================================================== # =============================================================================== # MDegrainNL (MDegrainN Light) 2021-12-09 # =============================================================================== # =============================================================================== function MDegrainNL(clip Source, \ int "TR", int "thSAD", int "thSAD2", int "BLKSize", int "Overlap", int "Pel", bool "Precise", \ bool "TrueM", bool "GlobalM", bool "Exp", bool "SBits", int "BitD", int "lambda", int "pnew", bool "MT", bool "LSB") { TR = default(TR, 1) # Temporal radius thSAD = default(thSAD, 150) # Denoising strength BLKSize = default(BLKSize, 16) # Block size Overlap = default(Overlap, BLKSize/2) # Block overlap Pel = default(Pel, 2) # MVAnalyse pel (1, 2, 4) Precise = default(Precise, false) # Recalculate TrueM = default(TrueM, false) # Enable/disable truemotion GlobalM = default(GlobalM, true) # Enable/disable global motion Exp = default(Exp, true) # Limited->Full range expansion for MAnalyse search SBits = default(SBits, false) # Use source bitdepth for MAnalyse search BitD = default(BitD, 8) # Bit depth for MAnalyse search (max = source bits) MT = default(MT, false) # Internal multithreading LSB = default(LSB, false) # Stacked 16-bit # Uncommenting the two lines below results in the same MVTools2 settings as QTGMC when truemotion=false # lambda = default(lambda, (TrueM ? 1000 : 100 ) * BLKSize * BLKSize / 64) # pnew = default(pnew, TrueM ? 50 : 25) # ------------------------------------------------------------------------------- AvsVerStrLC = LCase(VersionString()) IsAvsPlus = (FindStr(AvsVerStrLC, "avisynth+") > 0) IsAvsNeo = (FindStr(AvsVerStrLC, "avisynth neo") > 0) AvsStrNum = IsAvsPlus || IsAvsNeo ? FindStr(AvsVerStrLC, "(r") + 2 : 0 AvsBuildNum = (AvsStrNum > 2) ? int(value(MidStr(AvsVerStrLC, AvsStrNum))) : 0 IsAvsHBD = (AvsBuildNum > 2294) # ------------------------------------------------------------------------------- Super = MSuper(Source, pel=Pel, mt=MT) Search = Exp ? Super.ColorYUV(levels="TV->PC") : Super Search = IsAvsHBD && !SBits && (BitD < BitsPerComponent(Source)) ? Search.ConvertBits(BitD) : Search MultiVec1 = MAnalyse(Search, multi=true, delta=TR, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) MultiVec2 = !Precise ? MultiVec1 : \ MRecalculate(Search, MultiVec1, tr=TR, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD2, truemotion=TrueM, lambda=lambda, pnew=pnew) MDegrainN(Source, Super, MultiVec2, TR, thSAD=thSAD, thSAD2=thSAD2, mt=MT, lsb=LSB) } # =============================================================================== # =============================================================================== Code:
# =============================================================================== # =============================================================================== # FastDegrain 2021-12-09 # =============================================================================== # =============================================================================== # # FastDegrain by Sagekilla # Easy to use alias for MDegrain 1, 2, or 3 # Features optional sharpening if you should desire # Required plugins: LSFMod (to enable sharpening) & MVTools2.dll # # Changes by Hello_Hello # Updated the required plugins list # Added missing "thSAD" argument for MDegrain1, MDegrain2 & MDegrain3 # Added "TrueM" argument and made truemotion=false the default # Added "GlobalM" argument and made global=true the default # Added "MT" argument # Added "Precise" argument # Added "Exp" argument # Added "SBits" argument # Added "BitD" argument # LimitedSharpenFaster variable "oshot" corrected to "OShoot" # Replaced LimitedSharpenFaster with LSFMod # Added "SFirst" argument and moved the sharpening to after the degraining by default # Changed the default for "Sharp" from 160 to 75 # # ------------------------------------------------------------------------------- # # Degrain: MVDegrain 1, 2, or 3. Each is increasingly stronger and slower. # thSAD: Affects degraining: Low values reduce degraining, high values raise it. # BLKSize: MVAnalsye blksize. See MVTools documentation for details. # Overlap: MVAnalyse overlap. Must be even & <= BLKSize/2. Higher values are better & slower. # Pel: MVAnalyse subpixel accuracy. Higher values give better MVs and lowers speed. # Precise: Recalculate. It's quite a bit slower. # Limit: Limits max MVDegrain change. Decrease if you experience artifacts. # TrueM: Enables truemotion. # GlobalM: Disables global motion. # Exp: Limited->Full range expansion for MAnalyse search. Set false for full range source. Default true. # SBits: For bit depths > 8, use the same bit depth for MAnalyse search as the source. Default false. # BitD: Bit depth for MAnalyse search when SBits=false. Max = source bits. Default 8. # MT: Enables internal multithreading. # Sharp: Enables LSFMod sharpening. # Str: LSFMod's sharpening strength. Increase for more sharpening. # SS: Supersampling for LSFMod, values over 1 decrease edge aliasing. # Sft: Dampens LSFMod sharpening (it weakens the sharpening) # OShoot: Sharpening limit, increase it too much and it causes haloing. # SFirst: SFirst=true sharpens before denoising. The default is false. # # =============================================================================== # =============================================================================== function FastDegrain(clip Source, \ int "Degrain", int "BLKSize", int "thSAD", int "Overlap", int "Pel", bool "Precise", int "Limit", \ bool "TrueM", bool "GlobalM", bool "Exp", bool "SBits", int "BitD", bool "MT", int "lambda", int "pnew", \ bool "Sharp", int "Str", int "SS", int "Sft", int "OShoot", bool "SFirst") { # ------------------------------------------------------------------------------- Degrain = default(Degrain, 2) # Degraining method (1 - 3) thSAD = default(thSAD, 400) # MVDegrain thSAD BLKSize = default(BLKSize, 8) # MAnalyse block size (4, 8, 16) Overlap = default(Overlap, BLKSize/2) # MVAnalyse block overlap Pel = default(Pel, 1) # MVAnalyse pel (1, 2, 4) Precise = default(Precise, false) # Recalculate Limit = default(Limit, 255) # MVDegrain limit (0 - 255) TrueM = default(TrueM, false) # Enable/disable truemotion GlobalM = default(GlobalM, true) # Enable/disable global motion Exp = default(Exp, true) # Limited->Full range expansion for MAnalyse search SBits = default(SBits, false) # Use source bitdepth for MAnalyse search BitD = default(BitD, 8) # Bit depth for MAnalyse search (max = source bits) MT = default(MT, false) # Internal multithreading (through avstp.dll) Sharp = default(Sharp, false) # Toggles LSFMod Str = default(Str, 75) # LSFMod strength SS = default(SS, 1.5) # LSFMod supersampling Sft = default(Sft, 30) # LSFMod soft OShoot = default(OShoot, 1) # LSFMod overshoot SFirst = default(SFirst, false) # Enable sharpening before denoising # Uncommenting the two lines below results in the same MVTools2 settings as QTGMC when truemotion=false # lambda = default(lambda, (TrueM ? 1000 : 100 ) * BLKSize * BLKSize / 64) # pnew = default(pnew, TrueM ? 50 : 25) # ------------------------------------------------------------------------------- AvsVerStrLC = LCase(VersionString()) IsAvsPlus = (FindStr(AvsVerStrLC, "avisynth+") > 0) IsAvsNeo = (FindStr(AvsVerStrLC, "avisynth neo") > 0) AvsStrNum = IsAvsPlus || IsAvsNeo ? FindStr(AvsVerStrLC, "(r") + 2 : 0 AvsBuildNum = (AvsStrNum > 2) ? int(value(MidStr(AvsVerStrLC, AvsStrNum))) : 0 IsAvsHBD = (AvsBuildNum > 2294) # ------------------------------------------------------------------------------- Input = !Sharp || !SFirst ? Source : \ Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot) Super = Input.MSuper(pel=Pel, mt=MT) Search = Exp ? Super.ColorYUV(levels="TV->PC") : Super Search = IsAvsHBD && !SBits && (BitD < BitsPerComponent(Source)) ? Search.ConvertBits(BitD) : Search bvec1 = (Degrain >= 1) ? MAnalyse(Search, isb=true, delta=1, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() bvec2 = (Degrain >= 2) ? MAnalyse(Search, isb=true, delta=2, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() bvec3 = (Degrain >= 3) ? MAnalyse(Search, isb=true, delta=3, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() bvec4 = (Degrain >= 4) ? MAnalyse(Search, isb=true, delta=4, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() bvec5 = (Degrain >= 5) ? MAnalyse(Search, isb=true, delta=5, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() bvec6 = (Degrain >= 6) ? MAnalyse(Search, isb=true, delta=6, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() fvec1 = (Degrain >= 1) ? MAnalyse(Search, isb=false, delta=1, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() fvec2 = (Degrain >= 2) ? MAnalyse(Search, isb=false, delta=2, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, lambda=lambda, pnew=pnew, global=GlobalM, mt=MT) : nop() fvec3 = (Degrain >= 3) ? MAnalyse(Search, isb=false, delta=3, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() fvec4 = (Degrain >= 4) ? MAnalyse(Search, isb=false, delta=4, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() fvec5 = (Degrain >= 5) ? MAnalyse(Search, isb=false, delta=5, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, lambda=lambda, pnew=pnew, global=GlobalM, mt=MT) : nop() fvec6 = (Degrain >= 6) ? MAnalyse(Search, isb=false, delta=6, blksize=BLKSize, overlap=Overlap, \ truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop() # ------------------------------------------------------------------------------- Precise ? Eval(""" bvec1 = (Degrain >= 1) ? MRecalculate(Search, bvec1, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() bvec2 = (Degrain >= 2) ? MRecalculate(Search, bvec2, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() bvec3 = (Degrain >= 3) ? MRecalculate(Search, bvec3, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() bvec4 = (Degrain >= 4) ? MRecalculate(Search, bvec4, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() bvec5 = (Degrain >= 5) ? MRecalculate(Search, bvec5, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() bvec6 = (Degrain >= 6) ? MRecalculate(Search, bvec6, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec1 = (Degrain >= 1) ? MRecalculate(Search, fvec1, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec2 = (Degrain >= 2) ? MRecalculate(Search, fvec2, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec3 = (Degrain >= 3) ? MRecalculate(Search, fvec3, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec4 = (Degrain >= 4) ? MRecalculate(Search, fvec4, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec5 = (Degrain >= 5) ? MRecalculate(Search, fvec5, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() fvec6 = (Degrain >= 6) ? MRecalculate(Search, fvec6, blksize=BLKSize/2, overlap=Overlap/2, \ thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop() """) : nop() # ------------------------------------------------------------------------------- Output = \ (Degrain < 1) ? Input : \ (Degrain == 1) ? \ MDegrain1(Input, Super, bvec1, fvec1, \ thSAD=thSAD, limit=Limit) : \ (Degrain == 2) ? \ MDegrain2(Input, Super, bvec1, fvec1, bvec2, fvec2, \ thSAD=thSAD, limit=Limit) : \ (Degrain == 3) ? \ MDegrain3(Input, Super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, \ thSAD=thSAD, limit=Limit) : \ (Degrain == 4) ? \ MDegrain4(Input, Super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, bvec4, fvec4, \ thSAD=thSAD, limit=Limit) : \ (Degrain == 5) ? \ MDegrain5(Input, Super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, bvec4, fvec4, bvec5, fvec5, \ thSAD=thSAD, limit=Limit) : \ (Degrain >= 6) ? \ MDegrain6(Input, Super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, bvec4, fvec4, bvec5, fvec5, bvec6, fvec6, \ thSAD=thSAD, limit=Limit) : nop() Output = \ !Sharp || SFirst ? Output : Output.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot) return Output } # =============================================================================== # =============================================================================== # FastDegrainSharp # =============================================================================== # =============================================================================== function FastDegrainSharp(clip Source, \ int "Degrain", int "BLKSize", int "thSAD", int "Overlap", int "Pel", bool "Precise", int "Limit", \ bool "TrueM", bool "GlobalM", bool "Exp", bool "SBits", int "BitD", bool "MT", int "lambda", int "pnew", \ bool "Sharp", int "Str", int "SS", int "Sft", int "OShoot", bool "SFirst") { Sharp = default(Sharp, true) FastDegrain(Source, Degrain, BLKSize, thSAD, Overlap, Pel, Precise, Limit, \ TrueM, GlobalM, Exp, SBits, BitD, lambda, pnew, MT, Sharp, Str, SS, Sft, OShoot, SFirst) } # =============================================================================== # =============================================================================== |
9th December 2021, 00:15 | #53 | Link | ||
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
Quote:
Code:
Fastdegrain(degrain=3,bitd=8,globalm=false,thsad=300) |
||
9th December 2021, 00:26 | #54 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Is it used in MAnalyse?"
Yes. Archive have compiled version SO1_PT4 with hardcoded PredictorType=4 option for users without ability to edit complicated scripts. So you can simply load this dll. It also have some simple optimizations for CPUs up to SSE4.1. |
9th December 2021, 00:44 | #55 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
Code:
# FastDegrain - MOD # function FastDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel", \ int "thSAD", int "limit", bool "MT") { degrain = default( degrain, 2 ) # Degraining method (1 - 3) blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16) overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2) pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4) thSAD = default( thSAD, 400 ) # MVDegrain thSAD limit = default( limit, 255 ) # MVDegrain limit (0 - 255) MT = default( MT, false) # Internal multithreading (through avstp.dll) # Create our Super clip for MV search. Then, search for motion vectors. super8 = src.convertbits(8).ex_KNLMeansCL(D=1, A=2, h=6, s=2, device_type="gpu").MSuper(pel=pel) super = src.MSuper(pel=pel) bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT, PredictorType=4) : NOP() # Degraining the video using MVDegrain. Nothing special here. src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src return src } Code:
# FastDegrain - MOD # function FastDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel", \ int "thSAD", int "limit", bool "MT") { degrain = default( degrain, 2 ) # Degraining method (1 - 3) blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16) overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2) pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4) thSAD = default( thSAD, 400 ) # MVDegrain thSAD limit = default( limit, 255 ) # MVDegrain limit (0 - 255) MT = default( MT, false) # Internal multithreading (through avstp.dll) # Create our Super clip for MV search. Then, search for motion vectors. super8 = src.convertbits(8).ex_KNLMeansCL(D=1, A=2, h=6, s=2, device_type="gpu").MSuper(pel=pel) super = src.MSuper(pel=pel) bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() # Degraining the video using MVDegrain. Nothing special here. src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src return src } For a speed check, using my script commenting out everything but convertbits(16), Levels, ConverttoYUV444() and convertbits(10) has a speed of 0.0562x Last edited by anton_foy; 9th December 2021 at 02:23. |
|
9th December 2021, 03:11 | #56 | Link | ||
Registered User
Join Date: Mar 2011
Posts: 4,823
|
Quote:
I downloaded your original script from post # 26. The only changes I made were to add the missing thSAD arguments and the MT argument so I could enable/disable MT easily, and I changed the name to FDegrain so I could put it in the auto-loading folder and run both versions, then I did a comparison with the version I posted yesterday. When the settings are the same they're exactly the same. Code:
A = FastDegrain(degrain=3, TrueM=true, thsad=300, Sbits=true, Exp=false) B = FDegrain(degrain=3, thsad=300) ShowDiff(A, B, Amp=true) As a side note, enabling internal multithreading definitely changes the output, although when it's enabled for both versions the result is still the same. Code:
A = FastDegrain(degrain=3, TrueM=true, thsad=300, Sbits=true, Exp=false, MT=true) B = FDegrain(degrain=3, thsad=300, MT=false) ShowDiff(A, B, Amp=true) I ran a couple of test encodes with Exp=true. Expanding the levels for MAnalyse obviously makes a difference. I think it does retain a tad more detail during motion but it probably removes a little less noise during motion too. The bitrate of the encode was a little lower than with Exp=false for both tests, so I think I'll keep using Exp=true. Code:
A = FastDegrain(degrain=3, TrueM=true, thsad=300, Sbits=true, Exp=true, MT=false) B = FDegrain(degrain=3, thsad=300, MT=false) ShowDiff(A, B, Amp=true) Quote:
Code:
function ShowDiff(clip Clip1, clip Clip2, val "Amp", val "Comp", val "Show") { Amp = defined(Amp) && IsInt(Amp) ? ((Amp == 0) ? false : true) : default(Amp, false) Show = defined(Show) && IsInt(Show) ? ((Show == 0) ? false : true) : default(Show, false) Comp = defined(Comp) && IsInt(Comp) ? ((Comp == 0) ? false : true) : default(Comp, false) assert((IsInt(Amp) || IsBool(Amp)) && (IsInt(Comp) || IsBool(Comp)) && (IsInt(Show) || IsBool(Show)), \ " ShowDiff " + chr(10) + """ "Amp", "Comp" & "Show", can only be integer or bool. """ + chr(10) + \ " For Integer: Zero specifies false, and any other integer specifies true. " + chr(10) + \ " For Bool: They be true or false. " + chr(10)) C1 = Clip1.levels(96, 1.0, 160, 96, 160).GreyScale() C2 = Subtract(Clip1, Clip2) C3 = !Amp ? C2 : C2.levels(124, 1.0, 131, 0, 255) C4 = !Show ? C3 : Merge(C1, C3) C5 = !Comp ? C4 : Compare(Clip1, Clip2, show_graph=true) return C5 } Code:
function FDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel", \ int "thSAD", int "limit", bool "MT") { degrain = default( degrain, 2 ) # Degraining method (1 - 3) blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16) overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2) pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4) thSAD = default( thSAD, 400 ) # MVDegrain thSAD limit = default( limit, 255 ) # MVDegrain limit (0 - 255) MT = default( MT, false ) # Internal multithreading (through avstp.dll) # Create our Super clip for MV search. Then, search for motion vectors. super = src.MSuper(pel=pel, mt=MT) bvec3 = (degrain>=3) ? super.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec2 = (degrain>=2) ? super.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() bvec1 = (degrain>=1) ? super.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec1 = (degrain>=1) ? super.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec2 = (degrain>=2) ? super.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP() fvec3 = (degrain>=3) ? super.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP() # Degraining the video using MVDegrain. Nothing special here. src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit, mt=MT) : src src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit, mt=MT) : src src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit, mt=MT) : src return src } Last edited by hello_hello; 9th December 2021 at 10:15. |
||
9th December 2021, 10:06 | #57 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
THSAD=200 makes it freeze"
Something very strange happens. Adjusting thSAD should not cause any visible change in speed or freeze. Also typical adjustments steps may be 50..25 or less to finetune of noise removing vs detail preservation. The speed still very low - may you do not run avs+ mt with Prefetch() at the end of script ? May your system is too low on RAM to process UHD and system running in swapping pages to slow HDD ? Last edited by DTL; 9th December 2021 at 14:14. |
9th December 2021, 13:44 | #58 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,718
|
With UHD resolutions, MVTools functions and Prefetch, you may need to increase the maximum cache size from the default to avoid any performance hits.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
9th December 2021, 15:38 | #59 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
My RAM is 32gb ddr3 or do you refer to VRAM? |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|