New DirectX 12 Video APIs

[Boulder]

AVX mean it do not have fast enough integer operations with increased size register file. Also operations are limited to SSE2 128bit integer per op, AVX2 allow 256bit integer ops that is virtually twice faster. It is better to upgrade to AVX2 CPU at least in 202x years. Intel promises AVX1024 in the mid of 202x already.
This.

You can get a very decent second hand AVX2 system on the cheap thanks to AMD. The 5xxx series Ryzens can be used on very old mobos like several X370 or B350 based ones after a BIOS update. The previous generation is also not bad at all, I'm very happy with the 3900X. I'm contemplating on upgrading to 5900X since people seem to be switching to 5800X3D and there's a supply of used CPUs.

[DTL]

niche cases like TR=>10 with pel=2 or 4 at 4K.

'Big' tr-values really more naturally for 4K and more because with the same amount of light passed via input aperture of camera the samples at 4K tesselation of image receive 4 times less light in compare with 1080. Also if fill-factor of sensors is not great - even less.
So for 4K we need about 4 times more tr to have same relative shot noise in compare with 1080.

Also the new motion vectors low pass processing become more effective with not very small tr values. At least >5..6. So the number of position samples may be covered with impulse kernel of about 10 size. Also if I understand right to make good low-pass filter with cut-off of about 0.1 of the max (Nyquist) band we also need to process not very few samples (may be about 10).

The value noise of samples really cause 'position noise' because the position of image area (patch/block) defined by values of its samples only. So noised values of completely static block start to be translated in the position noise with current motion-search algoriphms (and may be with any motion search algo). After receiving not-zeroed coordinates for completely static block the blending engine start to motion-compensate with errors-noised vectors and it will cause blurring of block. For slow moving blocks it also cause not-perfect motion compensation and blurring. Also blending with shifted blocks with non-zero average value cause some shifting of block in the output frame so performed conversion of value noise in the position noise and MPEG encoder need to find and encode this error-movement too.
The effect of false noise-caused movement may become more visible with decreasing of block size. So the MVLPF processing may be more effective at smaller block sizes (like 8x8 or even 4x4).
So low-pass filtering of motion vectors after search for current pair of src+ref frames in big enough time domain using the nature of real images not have too fast and random moving of blocks can decrease these negative effects. But to perform filtering good enough it need to gather big enough number of motion vectors values of the block in consecutive frames.

The need of creating AVS-interface (script-accessible of samples data for read and write) to the MV data passed from MAnalyse to MDegrain really high. It will allow script-writers to create more advanced algoriphms of intermediate motion search data processing before MDegrain blending in the script form. And after slow experiments the best methods may be encoded into compiled functions of MDegrain or some new compiled filters of mvtools-pack for fast execution.
I think of creating motion-data clip in the RGB16 format with mapping of 16bit (dx,dy,SAD) data to R,G,B channels. But real current motion-data 'frame' have also some header of about 256 bytes inside with analysis data. So need to find if this header can be transfered as metadata unchanged and not in 'active' part of frame.

[takla]

'Big' tr-values really more naturally for 4K and more because with the same amount of light passed via input aperture of camera the samples at 4K tesselation of image receive 4 times less light in compare with 1080. Also if fill-factor of sensors is not great - even less.
So for 4K we need about 4 times more tr to have same relative shot noise in compare with 1080.


Oh wow. I didn't think about this but it makes perfect sense.

[takla]

AVX512 is coming you said @DTL?
https://i.imgur.com/kNyftsO.jpg

[DTL]

AVX1024 is coming in mid of 202x if this civilization will not die too fast. I hope it will have even larger register file size to perform more operations in fastest available memory in chip. AMD chips with at least AVX512 support is also good.

From https://clickthis.blog/en/sluhi-o-proczessorah-intel-xeon-sleduyushhego-pokoleniya-10-nm-emerald-rapids-7-nm-granite-rapids-5-nm-diamond-rapids-do-144-yader-lion-cove-k-2025-godu/
https://i0.wp.com/clickthis.blog/wp-content/uploads/2022/01/intel-xeon-future-table-2.jpg?w=640&ssl=1

Currently expected the most of performance boost from AVX2/AVX512 operations strategic redesign of MVtools are internal sub-sample shift and scale for both MAnalyse (onCPU processing) and MDegrain (onCPU processing) for pel 2 and 4. It will greatly reduce memory requirement for pre-calculated super clip and it looks very important for new massive multicore chips and current cache AVS design. Also as core become more and more faster in compare with main host memory it should finally make internal scale/shift processing faster in compare with pre-calculated upscaled planes for pel 2 and 4.

I also interested when intel hardware will provide DX12-ME API. It looks intel UHD Graphics 750 still do not have either drivers designed or required hardware features to expose this interface from MPEG encoder.

"I didn't think about this but it makes perfect sense."

Same is about HFR feature of new video systems - the max possible frame accumulation time with HFR is lower so number of accumulated photons per frame per object's view also lower.

So MDegrainN works as 'secondary level video camera' allowing to extend photon's flux accumulation time over the frame's exposure time in 'first level time sampling video camera'. It is only about real optical video cameras (not about digitally synth images). The more image data carriers (photons) accumulated per object's view - the more precision got. The object's view less distorted (noised) by natural photon's shot noise. So the possible tr value is in best case unlimited. Practically may be limited to typical cutscene duration that is several seconds minimum and with even old 24/25 fps it is about many 10'ths or 100'ths.

So the host RAM-based solution allow much higher tr values in compare with full onAccelerator processing (the max tr multiplied to threads number is very limited by accelerator's memory size to hold all required frames). Only sending current+ref pairs to accelerator to ME-process as it work now. Actually it now upload to accelerator current frame once and only send new refs per one output frame of MDegrainN.

[StainlessS]

Methinks you take on monster task DTL, mucho respecto, grassy ass :)

[DTL]

The more 'monster task' is to add more transforms support in MVtools - like rotate, scale, lighting (additive and multiplied levels change) and may be more. MVtools with only translate transform support are still at the very beginning of development.

[guest]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.10

Hi DTL, I had an error with this build with SMDegrain. (In RipBot264)

I can't tell you what the error was other than it was something to do with Line 645, of SMDegrain 3.5.0d

https://github.com/Dogway/Avisynth-Scripts/blob/master/SMDegrain%20v3.5.0d/SMDegrain%20v3.5.0d.avsi#L645

Or maybe this isn't meant to be used with SMDegrain

[takla]

Same is about HFR feature of new video systems - the max possible frame accumulation time with HFR is lower so number of accumulated photons per frame per object's view also lower.


Yep. But the higher frame rate also allows your brain to process more informations. This basically cancels that downside. (When comparred to the same video with a lower frame rate, anyway)

[DTL]

"higher frame rate also allows your brain to process more informations. "

Nowdays before viewing we typically need to very highly compress moving picture data to broadcast or stream via IP or torrents. So MDegrain is mostly part of MPEG compression to make the MPEG output rate and filesize as low as possible with given quality level. I now can make rips from HD1080i sources with x264 crf=18 at medium output rate down to about 4000 kBit/s for low-movement documentaries.

"Line 645"

It is just end of large 'messages output'. Need real AVS error message to see what may be wrong. As pinterf ask it should be compatible with all old scripts when no new params are used (internal defaults should be equal to 2.7.45 release). But to use new features it is required to manually edit scripts like SMDegrain to add new params in no-default values.

[guest]

"Line 645"

It is just end of large 'messages output'. Need real AVS error message to see what may be wrong. As pinterf ask it should be compatible with all old scripts when no new params are used (internal defaults should be equal to 2.7.45 release). But to use new features it is required to manually edit scripts like SMDegrain to add new params in no-default values.

That "message output" IS SMDegrain v3.5.0d !!!

I don't understand what you are saying here, it's a LOT over my head...I would think that Dogway would need to rectify this, so as to use your Mvtools2 build(s).

[kedautinh12]

You need show more detail about error. That not only said line 645. What is error in line 645?? You need sceenshot when error show with ripbot

[DTL]

If your error is with command-line AVSmeter: What is the error text ? Where this 645-line data come from ?

The 645 line at github is 2 }} - this either end of "SHOW PANEL" block or end of all SMDegrain() function. Anyway it is not command or filter or function call line so error may be anywhere above this line (or in any function call from above block).

[guest]

You need show more detail about error. That not only said line 645. What is error in line 645?? You need sceenshot when error show with ripbot

I can't give anymore info, as all AVSMeter displays is the path where SMDegrain v3.5.0d is, and that Line 645 is where the error is, and AVSMeter stops, at that point.

I don't know why it chooses that line which is simply }}, and I guess that's where Dogway needs to have a look.

That's ALL I can do...sorry.

[takla]

@DTL newest release (https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.10) gives me this error (https://i.imgur.com/DAEJOCt.png) which goes away if I delete line 5. Is dithering no longer valid?

[DTL]

Oh - I need to check it. Dithering in AVS ConvertBits() from 16 to 8 - is not in MDegrain. But may be output of MDegrain somehow cause this error. If you feed MDegrain with 16bit it should output 16bit too (with right params or defaults ?) and it can not not perform dithering internally. The DX12-ME mode MAnalyse only can use 8bit YV12 now so you need to make conversion in the script EZdenoise internally if you use 16bit input.

Where is the version of EZdenoise that you use in that script ?

[takla]

@DTL fixed it like this:

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)

Super = Input.ConvertBits(16).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super.ConvertBits(8).MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.ConvertBits(16).MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}

[DTL]

I catch this crash with 16bit source: It happens only in release build in MAnalyse in SAD 16bit SSE function. Need some time to find what is wrong. I not touch that part from beginning.

Current workaround may be to feed 8bit to MAnalyse and 16bit to MDegrain.

This looks like working:

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Super8 = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super8.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}

[takla]

@DTL new issues
mvtools2 from printerf MvTools2 2.7.45 and your newest release is no longer bit identical with same settings (post 117)

MvTools2 2.7.45 pinterf
time=21.380s
501.153 KB
ProRes

mvtools2 DTL (https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.10)
time=19.116s
500.510 KB
ProRes

The last build that was bit identical was https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.05

[ChaosKing]

@DTL new issues
mvtools2 from printerf MvTools2 2.7.45 and your newest release is no longer bit identical with same settings (post 117)


Maybe because some bugs were fixed!?

[takla]

Maybe because some bugs were fixed!?

Good point. Maybe. I'd like DTL to comment on it in more detail.


One more thing, the script from post #117 has 510.140 KB while the script from post #118 has 510.021 KB. Why is that? Shouldn't they both do the exact same thing? Encoded with ffmpeg -y -benchmark -i 01.avs -c:v ffv1 TEST.mkv

[DTL]

@DTL fixed it like this:


Super = Input.ConvertBits(16).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super.ConvertBits(8).MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)




I think it should not work at all. The 'super' clip from MSuper is only semi-compatible with AVS so it must be directly feed to other filters of MVtools. If you try to 'convert 16bit super into 8bit' it may be damaged.
MSuper and MAnalyse outputs are semi-clips and I think can not be processed with any other filters (at todays) and designed to be used only as inputs for other MVtools filters. The only MSuper output can be outputted as AVS data and rendered as a viewable images. But I think it is not mean it can be easily 'converted' from 16bit to 8bit.

Same is about question "the script from post #117 has 510.140 KB while the script from post #118 has 510.021 KB. Why is that? Shouldn't they both do the exact same thing?". Yes - I think 'super' 8bit converted from 16bit 'super' is not the same as 'super' clip created from 16to8bit downconverted 'standard AVS clip' with MSuper().

I hope syntax

Super8 = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super8.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)


is 'auto' compatible with any bitdepth input because ConvertBits(8) will pass 8bit formats and downconvert any >8 to 8. So MAnalyse will always receive 8bit 'super8' semi-clip created from 8bit MSuper() input.

It looks

Multi_Vector = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma).MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

also works.

[DTL]

@DTL new issues
mvtools2 from printerf MvTools2 2.7.45 and your newest release is no longer bit identical with same settings (post 117)



And same is with workaround from post 118 ?

Also about >8 bit: It useful only if output of the denoising is also used with increased bitdepth. Like 8bit input and 10bit of more output to appropriate codec with >8bit support. Increasing bitdepth from 8 to 16 before MDegrain and converting back to 8 after MDegrain output possibly makes nothing useful but wastes RAM and speed. MDegrain can use 8bit input naturally and output 16 if it required. Simply set 'out16=true'. I currently do not use HEVC with 10bit for my work so do not use this feature.

[takla]

And same is with workaround from post 118 ?


Yes. I just double checked:

Note: I used ProRes before, but this here is FFV1.

2.7.46-a.10
510:021 KB

2.7.46-a.05
513.247 KB

2.7.45
513.247 KB

Here are the exact settings:

LWLibavVideoSource("C:\Users\Admin\Documents\01.mkv")
Trim(0, 600)
EZdenoise(HBD=true)
ConvertBits(10, dither=1)
Prefetch(12, 48)

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma", bool "HBD")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)
HBD = default(HBD, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Super8 = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super8.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2, out16=HBD)
}

ffmpeg -y -benchmark -i 01.avs -c:v ffv1 EZ.mkv

And thanks for the explanation on msuper & manalyze. Makes sense that they do not care about bit depth, except for compatibility.

[DTL]

Here is a test build for testing : (Updated )

It have optimizations disabled for 16bit (>8bit) SAD functions. Currently it is the only fast way to save from crash in release build. Debug build works but too slow and can not show the source of crash. May process slower. I hope no other (8bit) processing speed of MAnalyse is touched. It is fast but not perfect fix. Also as a bonus it have 2 C++ builds for SSE2 and AVX2 CPUs. May be AVX2 is a bit faster at AVX2 CPU.

If testing will be acceptable I will place next pre-release to github.

About not bit-exact results with old 2.7.45 - it may be one of many small changes to MDegrainN finally works a bit different to old release. Will try to look later.

Update: https://drive.google.com/file/d/1lkmkuvP8mncb0CuOwzUq5pAL2KoQlyjq/view?usp=sharing

It is even more strange - the crash happens only with SSE2-targeted build inside SSE2 intrinsic-based function. With AVX2-targeted it looks working with all optimizations enabled. It may be some complex bug inside mvtools or in current used version of MSVS compiler ? So in updated archive link the SSE2 build is with optimizations for SAD >8 bit disabled and AVX2 build is 'normal'.

"thSAD2=thSAD/2"

I do not think it is good internal default. Typically thSAD should be 'just a small above noise level' . So setting thSAD2 too low by default you either lost many useful frames in tr-scope or force user to increase tr-value too high to have more neibour frames being covered with high enough thSAD near current frame (and it will cause slower speed processing). If user will raise thSAD high enough to get thSAD2 not too small - it may cause additional blurring or detail lost. I typically set thSAD2 manually just a few below thSAD. In float math it is about thSAD2=0.9*thSAD. I not test how it processed by AVS+ scripting. May be ToInt(0.9*thSAD) required if MDegrain will not accept float value or something else. Or let user to enter thSAD2 and thSADC2 too. I think the feature to allow lower thSAD at the edges of tr-range is mostly to decrease artifacts if artifacts happens. If not - the best for denoising and speed with current tr-value is thSAD2 close to thSAD.

[takla]

What do you want me to test with the test build exactly? Here are the encoding times:

AVX2
time=15.412s
SSE2
time=40.264s

Unfortunately thSAD2=0.9*thSAD is not a valid parameter. I changed thSAD2's to 135 manually, which is the same value. It made the encoding faster by 0.5 seconds on a 25 seconds clip (not applied in the encodings measured above) which is nice.

function EZdenoise(clip Input, int "thSAD", int "thSAD2", int "thSADC", int "thSADC2", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma", bool "HBD")
{
thSAD = default(thSAD, 150)
thSAD2 = default(thSAD, 135)
thSADC = default(thSADC, thSAD)
thSADC2 = default(thSADC, thSAD2)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)
HBD = default(HBD, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Super8 = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super8.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD2, thSADC=thSADC, thSADC2=thSADC2, out16=HBD)
}

[DTL]

"What do you want me to test with the test build exactly?"

If it not crash at your system with your 16bit (>8bit) clips (frame size, colour format, etc).

"AVX2
time=15.412s
SSE2
time=40.264s"

Oh - it is so great difference. The non-optimized SSE2 really very slow. I think it will be only a bit slower because it uses SSE2 intrinsics internally. I hope not many users will run nowdays at SSE2 only CPUs with >8bits MAnalyse. To found what is wrong with 'normally optimized' SSE2 builds may take unknown time.

"Unfortunately thSAD2=0.9*thSAD is not a valid parameter. "

I think more comfortable to user to enter short 'far end thSAD multiplier' as script param and it can be applied to both thSAD2 and thSADC2 equally. Something like


function EZdenoise(clip Input, int "thSAD", float "far_thSAD_mul", int "TR", int "thSADC", int "BLKSize", int "Overlap", int "Pel", bool "Chroma", bool "HBD")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
far_thSAD_mul = default(far_thSAD_mul, 0.9)
thSAD2 = Int(thSAD * far_thSAD_mul)
thSADC2 = Int(thSADC * far_thSAD_mul)


In real use it is easy to call like EZdenoise(200, 0.8, 10). Not set 4 params of th-s in some fixed ratio between values every time when need to adjust 'base thSAD'.

Name far_thSAD_mul is not short and nice - may be something shorter possible.

Also HBD is not clear about 'internal conversion' of 8bit input to 16bit output. May be better name 'toHBD'.

[takla]

Thanks. It no longer crashes.
And I take the 0.5seconds speed gain back. Just modern CPU boosting which caused the variance...
On HBD: I only added it because of the crash with my usual settings. But since that is fixed now I'll remove it again.
Also I never wanted to expose thSAD2's anyways, for the same reason you mentioned.

And thank you for showing me how to add the falloff multiplier.

This is what I'm using now:

LWLibavVideoSource("C:\Users\Admin\Documents\01.mkv")
Trim(0, 600)
ConvertBits(16)
EZdenoise()
ConvertBits(10, dither=1)
Prefetch(12, 48)

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma", float "Falloff")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)
Falloff = default(Falloff, 0.9)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=Int(thSAD*Falloff), thSADC=thSADC, thSADC2=Int(thSADC*Falloff))
}

[takla]

@DTL
What is your opinion on Vulkan based video processing (https://www.khronos.org/blog/an-introduction-to-vulkan-video)? Are you aware of it? And yes I realize it does not explicitly talk about motion estimation (because it is probably missing?).

My reason for bringing it up is you talked about (https://forum.doom9.org/showpost.php?p=1969840&postcount=51) MVtools based denoising on a capture device and I too thought about this before. The thing is, such a device would probably not support DX12. Realistically, the camera footage is send to a smartphone. And that platform would support Vulkan on Android or Metal on iOS.

There is also vkFFT (https://github.com/DTolm/VkFFT), which could also be used for denoising.

[DTL]

"(because it is probably missing?"

Yes - it looks Vulkan developers still not reach the ME-API as a service of hardware video encoder exposed to other applications. It looks very rarely need by anyone even todays. Only Microsoft understand it and add to DirectX API. Also I think no other (using general purpose computation units in accelerator) implementation of ME available as free to use library in Vulkan.

"on a capture device"

Not on a capture hardware interface card directly. But as a feature of live capture host or pass-through host with a function of live MDegrain. Practically its data flow will be from capture interface card to accelerator card and via CPU to output card. Or with industry transition to IP-based streams - from software API to receive stream - to accelerator and back to API so send stream via IP. Using hardware IP adapters.

"that platform would support Vulkan on Android or Metal on iOS."

I think current smartphones uses same methods of noise reduction as mvtools in hardware. But I not read if any API is exposed to user applications. It may be deeply in the hardware units for camera data processing. So in a perfect world it may be even possible to connect set/rig of smartphones as hardware accelerators to some PC host via USB and use as external hardware accelerated denoisers. The most of money now looks put in the quality of hardware accelerated denoising in smartphones so they quickly got nice results. But it may be covered by patents and not exposed as API for external user applications.

In the current phase of dying civilization the end-users home desktop PCs are dying too. And current buyers can simply put money to smartphone with good denoising if required to shoot new footage. The pro broadcast cameras looks also progress with internal denoisers as I see in 2022 from Tallinn Europe Skate championship broadcast. So it looks any investment in hardware accelerated denoising for home desktop PCs is not any profitable nowdays.

[takla]

IIRC denoising depends on the camera app itself. There are some apps which let you disable most postprocessing. But yes, it is unclear if they use fixed functions or CPU.

And yes, if you can do good denoising internally, I can see why no one wants to spent dev time for a desktop solution.

[DTL]

Finally some working tech demo of 2 different processing of MSuper/MDegrainN for pel > 1 : https://drive.google.com/file/d/1E8iAWho9yf9svQq32FzsrPp13mU2Z_Q7/view?usp=sharing . Only for chroma=false processing or RAM usage/speed testing. Chroma=true with UseSubShift>0 in MDegrainN still outputs some buggy blocks sometime. AVX2 build only.

Test scripts :

LoadPlugin("mvtools2.dll")

ColorBars(3840,2160, pixel_type="YV12")

Trim(0,1000)

tr=12
super=MSuper(last,chroma=true, mt=false, pel=4, hpad=16, vpad=16, levels=1, pelrefine=false)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, overlap=0, chroma=false, mt=false, optSearchOption=5, optPredictorType=0,levels=1)
MDegrainN(last,super, multi_vec, tr, thSAD=175, thSAD2=160, mt=false,wpow=7, UseSubShift=1)

Prefetch(6)


vs


LoadPlugin("mvtools2.dll")

ColorBars(3840,2160, pixel_type="YV12")

Trim(0,1000)

tr=12
super=MSuper(last,chroma=true, mt=false, pel=4, hpad=16, vpad=16, levels=1, pelrefine=true)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, overlap=0, chroma=false, mt=false, optSearchOption=5, optPredictorType=0,levels=1)
MDegrainN(last,super, multi_vec, tr, thSAD=175, thSAD2=160, mt=false,wpow=7, UseSubShift=0)

Prefetch(6)


At i5-9600K + GTX1060 the second (standard mvtools MSuper/MDegrainN) takes about 7+ GB RAM and runs about 1.5 fps. The new internal sub-shifting method for MDegrainN of single full-size frame takes about 1.8 GB RAM and run at about 7.5 fps. Unfortunately the decreasing of RAM usage with pel=4 is not 16x times lower but only about 4 times. May be super clips are not largest in caching in AVS+.

[takla]

tested with 720p (I'll test 4K in a bit)

pelrefine=false with UseSubShift=1 looks like this (https://imgsli.com/MTEyNzg2) (after). Uses 1860MB

pelrefine=true with UseSubShift=0 looks normal. Uses 2880MB

And please do not use ColorBars for testing. I'll share a 4K sample in an hour or so...

[DTL]

"UseSubShift=1 looks like this (after)."

I know the output quality is completely bad. It is only test for RAM usage and speed at the use cases like https://forum.doom9.org/showthread.php?p=1962965#post1962965 . Where it exhaust 32 GB of RAM with tr about 10.

[takla]

Oh, ok.

Edit:
PelRefine=true & UseSubShift=0 uses 5908MB

PelRefine=false & UseSubShift=1 uses 3731MB

Here is the sample (https://www.mediafire.com/file/pu0tqruqpklqcd4/SAMPLE.mkv/file)

Not full "4K", cropped to 1600 pixels in height, but still.

by the way, can you tell me why my script crashes? I ended up testing with your settings instead.

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 12)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 0)
Pel = default(Pel, 4)

Super = Input.MSuper(Pel=Pel, Chroma=true, Levels=1, PelRefine=false)
Multi_Vector = Super.MAnalyse(Multi=true, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=false, Levels=1, optSearchOption=5, optPredictorType=0)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=int(float(thSAD*0.9)), thSADC=thSADC, thSADC2=int(float(thSADC*0.9)), UseSubShift=1)
}

[DTL]

"can you tell me why my script crashes? "

That was not good debugged build and real far outside frame motion vectors may cause crash. So the first test was with not-noised colourbars and larger padding of 16 to save from crash better. Though it was not very good test because not-noised colourbars should produce zero motion vectors and sub-sample shifting is not used so speed may be better. Though the test shows significant difference in speed even in with static colourbars (may be 4K pel=4 super shifted planes creation with old method pelrefine=true also loads CPU/memory a lot too).

Here is possibly better protected from that crash version so should run with real content with default padding of 8 -
https://drive.google.com/file/d/1oIyVmqo8gYRsZFeinZ-7l46zwerbw-Hk/view?usp=sharing

"PelRefine=true & UseSubShift=0 uses 5908MB
PelRefine=false & UseSubShift=1 uses 3731MB"

What was the real frame size and threads number ? Can you adjust Prefetch() to match your CPU cores number ? What is the fps difference ?

"Not full "4K", cropped to 1600 pixels in height,"

If you do not have full-frame 4K source you can put simple fast resize like BilinearResize(3840,2160) before degraining.

"Here is the sample"

I test with your source file sample and FFMS2 source with a full script:

LoadPlugin("mvtools2.dll")
LoadPlugin("ffms2.dll")


function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 12)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 0)
Pel = default(Pel, 4)

Super = Input.MSuper(Pel=Pel, Chroma=true, Levels=1, PelRefine=false)
Multi_Vector = Super.MAnalyse(Multi=true, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=false, Levels=1, optSearchOption=5, optPredictorType=0)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=int(float(thSAD*0.9)), thSADC=thSADC, thSADC2=int(float(thSADC*0.9)), UseSubShift=1)
}

FFmpegSource2("sample.mkv")

ConvertBits(8)
ConvertToYV12()

EZdenoise(TR=12)

Prefetch(6)


At my CPU i5-9600K with 6 cores with 6 threads it run AVSmeter with about 5600M RAM and about 2.6 fps. Without crash (at least at first about 100 frames) with latest build. The total letterboxed frame size in a sample I see is 3840x2160 that is enough for test.
With pelrefine=true and UseSubShift=0 it looks start to swap taking about 10..11+ GB RAM and so fps drops to about 0.3.

It looks with filesource filter and some simple intermediate like convertbits and converttoyv12 the used RAM for AVS+ caching difference is even smaller. Though still reach about 2x.

I also tried to play with combination of SetCacheMode(1) and different Prefetch(6,N):

With lowest possible Prefetch(6,1) and pelrefine=true usesubshift=0 I got RAM usage about stable 8400M but still low fps about 0.3.

With Prefetch(6,2) and pelrefine=false usesubshift=1 RAM usage is about 4700M and speed about 1.2 fps.
With not-defined N in Prefetch(6,N) the RAM usage is just a bit higher about 5200M and speed quickly reach 2.5+ fps.

[takla]

No longer crashing, thanks.

With
LWLibavVideoSource("C:\Users\Admin\Downloads\SAMPLE.mkv")
ConvertBits(8)
Crop(0, 280, -0, -280)
BilinearResize(3840, 2160)
EZdenoise()
Prefetch(12, 12)

I get 4082MB & 6293MB (only encoding the first 12 frames in avspmod). During actual encoding, 2.6 FPS and 1.3 FPS respectively. Memory usage is much higher during encoding in ffmpeg, with UseSubShift=0 randomly spiking from 8GB to 22GB.

With SetCacheMode(1) memory usage is halved but so is the speed (FPS)

With just CPU, RAM usage is much more stable and stays below 7GB at all times. FPS is about 2.7

[DTL]

Thank you for testing and providing results.

There also an idea about simulating of overlap MDegrainN processing with single non-overlap output of MAnalyse - using interpolating of motion vectors with half-block offset. May be simple mean MV of the 4 surrounding MVs and also mean of SAD. It will not be as precise as true shifted motion search but may good enough simulate overlap processing to hide currently sometime visible blockiness while keeping speed at good value and not require to use 2 hardware accelerators to keep speed.

For speed of development the overlapping may be made inside AVS scripting (or BlockOverlap fizik's filter).

The interpolation may be made inside MDegrainN. Possible processing may be like

mvs_clip=MAnalyse(overlap=0)
std=MDegrainN(mvs_clip)
shifted=MDegrainN(mvs_clip, interpolateoverlap=true)
BlockOverlap(std, shifted)

[DTL]

New build: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.11 SSE2/AVX2 builds by VisualStudio2019 16.3.

Added UseSubShift param to MDegrainN and pelrefine=true/false to MSuper. Default = 0, set to 1 to enable.

If pelrefine=false in MSuper - all other filters must use UseSubShift=true (or optSearchOption=5 for MAnalyse).

Also redesigned MDegrainN no-overlap processing to single pass YUV formats processing - looks like also added to performance. Old MDegrainN uses each plane separate processing that cause 3 times more DegrainWeight() and norm_weights() calls with equal data. Also separate planes processing not allow to reuse subshifted blocks in both SAD re-check in MVLPF processing followed by MDegrainN processing efficiently

Currently only YV12 format for block size 8x8 (8x8 Y and 4x4 UV) is fully accelerated with AVX2 for sub-shifting. All other will fallback to C-reference that is slow. Also only 8bit is now fully supported.

There is very experimental UseSubShift=1 for MAnalyse to test possible speed (may be unstable and crash with chroma=true , only valid for optPredictorType=1) in 'onCPU' MAnalyse with pel > 1. I still not expect it may be faster in compare with pre-computed planes form MSuper (at least before more fast AVX512 subshift implementations will be designed) but may help to run with larger threads count or tr-value at the limited RAM hosts.

Also to make speed of MVLPF better by reusing of subshifted blocks from MVPlane - the redesigned to single pass processing use_block_yuv() function in MDegrainN is added. It looks helps also in other processing modes performance. Currently only non-overlap single pass colour formats processing in MDegrainN is implemented (no external switches - auto detecting if YUV format input and overlapH=overlapV=0). For overlapped processing single-pass mode is also possible but need more time and more complex design.

If hardware ME accelerator is not available it is possible to enable some 'fastest' mode of MAnalyse to check pel=4 processing without badly limited speed by full-processing mode in MAnalyse:

tr=15
super=MSuper(last, mt=false, chroma=true, pel=4)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=4, chroma=false, mt=false)
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, UseSubShift=1)

At i5-11600 it runs at 15.6 vs 20.4 fps with UseSubShift=0 or 1 for MDegrainN for about FullHD frame size.

Updated: _2 version from 02.07.22 with fixed bug.

[anton_foy]

New build: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.11 SSE2/AVX2 builds by VisualStudio2019 16.3.

Added UseSubShift param to MDegrainN and pelrefine=true/false to MSuper. Default = 0, set to 1 to enable.

If pelrefine=false in MSuper - all other filters must use UseSubShift=true (or optSearchOption=5 for MAnalyse).

Also redesigned MDegrainN no-overlap processing to single pass YUV formats processing - looks like also added to performance. Old MDegrainN uses each plane separate processing that cause 3 times more DegrainWeight() and norm_weights() calls with equal data. Also separate planes processing not allow to reuse subshifted blocks in both SAD re-check in MVLPF processing followed by MDegrainN processing efficiently

Currently only YV12 format for block size 8x8 (8x8 Y and 4x4 UV) is fully accelerated with AVX2 for sub-shifting. All other will fallback to C-reference that is slow. Also only 8bit is now fully supported.

There is very experimental UseSubShift=1 for MAnalyse to test possible speed (may be unstable and crash with chroma=true , only valid for optPredictorType=1) in 'onCPU' MAnalyse with pel > 1. I still not expect it may be faster in compare with pre-computed planes form MSuper (at least before more fast AVX512 subshift implementations will be designed) but may help to run with larger threads count or tr-value at the limited RAM hosts.

Also to make speed of MVLPF better by reusing of subshifted blocks from MVPlane - the redesigned to single pass processing use_block_yuv() function in MDegrainN is added. It looks helps also in other processing modes performance. Currently only non-overlap single pass colour formats processing in MDegrainN is implemented (no external switches - auto detecting if YUV format input and overlapH=overlapV=0). For overlapped processing single-pass mode is also possible but need more time and more complex design.

If hardware ME accelerator is not available it is possible to enable some 'fastest' mode of MAnalyse to check pel=4 processing without badly limited speed by full-processing mode in MAnalyse:

tr=15
super=MSuper(last, mt=false, chroma=true, pel=4)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=4, chroma=false, mt=false)
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, UseSubShift=1)

At i5-11600 it runs at 15.6 vs 20.4 fps with UseSubShift=0 or 1 for MDegrainN for about FullHD frame size.

Updated: _2 version from 02.07.22 with fixed bug.

What you do with mvtools is very interesting and I love the progress you make, much respect. Yesterday I tried your latest build and it worked quite well with the one clip I tested. Some details were smeared/blurred compared to my tests with to my prefiltered tests with the pinterf mvtools but I only tested one problematic clip yet. Overall I am very positive to this build you made since even without prefiltering the lines and objects in high grain clip did not get the usual dancing/wobblyness that is very annoying. I cannot wait until you release it for HBD and blocksize above 8. The source I have been testing it with is 8-bit slog2 4K (3840x2160). With your script above I get about 0.9fps with my intel i5 3570 3.4ghz, 32gb ram, nvidia geforce GTX 970. Mostly I found with pinterf mvtools to experiment with different prefiltering techniques since I have 4 quite different test clips in 4k that are very hard to denoise with the same script. After many months I have found something after trial and error that seems to work pretty well for everything but with your build I think it will improve alot in the future. Thanks again for your great efforts!

[DTL]

"I cannot wait until you release it for HBD and blocksize above 8. "

It is only for better speed and lower RAM usage at some use cases. All new quality features already should work with all 2.7.45 bitdepth and blocksize values supported. The DX12-ME mode can not support > 8 bit input because it is not supported by current Microsoft DX12 API. And the only supported input format for hardware ME is NV12 that is internally converted from YV12 AVS format. So to process HBD with hardware ME you need to downconvert source for MAnalyse to YV12. And you can feed 16bit source to MDegrainN using different super clip.

Example was already shown here like

Super = MSuper(levels=1...)
Super8 = ConvertToYV12.MSuper(levels=1...)
Multi_Vector = Super8.MAnalyse(optSearchOption=5, levels=1...)

MDegrainN(Super, Multi_Vector,...)


"the lines and objects in high grain clip did not get the usual dancing/wobblyness that is very annoying."

It is with MVLPF options enabled for MDegrainN ?

[DTL]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.12

Added single pass colour overlapped processing in MDegrainN. Fixed regression of not using thSADC/thSADC2 in single pass processing.
Added tweaking param adjSADLPFedmv to MDegrainN to adjust SAD of MVs passed thSAD check after filtering. Float param. Default 1.0 - no correction. Recommended value about 0.8. Typically SAD of the filtered MVs positions is a bit higher in compare with initial after ME processing (so ME engine points to best SAD). So this adjustment allow to add some boost to weighting of blocks after interfiltering of MVs.

Added optSearchOption=6 to MAnalyse. In this mode DX12-ME only used for getting MVs from HW accelerator and SAD calculation performed on host CPU. Compute.cso shader is not used. Also for 8x8 8bit block available UseSubShift=1 for MAnalyse to use sub-shifting (allow to run with pelrefine=false at MSuper and save RAM).
May be faster at some combinations of host/accelerator. Also the SAD calcultation of shader for pel=2 and pel=4 still not completely correct (higher in compare with original mvtools).

So for onCPU SAD calculation (as 'reference' quality mode untill shader not completely fixed):

super=MSuper(mt=false, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, overlap=0, chroma=true, optSearchOption=6, mt=false, levels=1, UseSubShift=1)
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=400, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.8, UseSubShift=1)


On i5-9600K CPU with GTX1060 MAnalyse (SO=6 and USS=1) is still a bit slower with MPEG encoding (about 6 vs 6.7 fps) but produces a bit smaller file.

The RAW performance of MAnalyse with different options of search and SAD calcultaion looks may be tested with AVSmeter without MDegrain like

super=MSuper(mt=false, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)
multi_vec=MAnalyse (super, blksize=8, chroma=true, optSearchOption=6, mt=false, levels=1, UseSubShift=1)
MStoreVect(multi_vec)


It looks MStoreVect not support multi=true ? So the resulted fps need to be divided to tr_x_2 to estimate real processing speed (without MDegrain) with different tr-values. This script produces performance in pairs frames per second (src+ref).

The 16bit subshifting in MDegrainN still not work completely correctly and still only slow C-reference.

[DTL]

Finally the most long awaited feature to MDegrainN with hardware acceleration in 2022 - fully internal MDegrainN interpolated overlap mode : First working release - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.13

Added interpolated overlap mode to MDegrainN. Only 'max' overlap of blocksize/2 mode currently implemented.

New param of MDegrainN: IntOvlp (int).
Values:
0 - standard mode (default).
1 - internally interpolate input MVs to blocksize/2 overlap mode.

Added block size 16x16 for subshifting with AVX2 implementation. Fixed bug with chroma=false in MDegrainN no copy of chroma planes to output from previous release.

The new param is int and not bool because it is planned to test different interpolated overlap modes in the future. Currently it have more fail-safe design with SAD re-check for interpolated MVs to decrease probability of bad blends. But it is slower. It is possible to run interpolation-only faster mode with SAD interpolation too without re-check but it may decrease quality. Also it is possible to move MVLPF processing before interpolation to test speed/quality.

At the i5-9600K with GTX1060 it runs with x264 encoding about 50% slower but the quality is visibly better. No more blockiness on flames/fogs/fades should be. Also small blockiness on moving objects mostly removed.
It also may run with 'onCPU' MAnalyse with no-overlap MVs search to make some performance gain without HW accelerator.
The subshifting may be used in this mode but may or not be faster in compare with 'precalculated' sub planes in MSuper - looks may depend on host CPU. At i5-9600K with IntOvlp=1 a bit faster run with no-use of subshifting feature.

Now about 4 different quality/speed modes avaialable for overlapping:
1. Old onCPU MAnalyse overlap (full true up to 4x blocks number overlapping search) - possibly best 'reference' quality. Slowest mode.
2. 2 separated clips diagonally shifted at half-block sized processed with hardware-accelerated MAnalyse (single or dual accelerators should be supported if available for each MAnalyse) and overlapped in AVS using different internal or external filters. Uses a 2 sets of really analysed full frame MVs. May be a bit lower in quality in compare with 1. Require additional scripting and/or plugins. The Fizick's BlockOverlap pluging is still C-reference only so may be slow. Speed depend on host performance.
3. Hardware MAnalyse (SO=5 or 6) and interpolated overlap in MDegrainN based on single non-overlapped MVs array. Faster but may be lower in quality in compare with 1 and 2. Possibly the most RAM-saving mode (also support minimal RAM usage with UseSubShift option).
4. No overlap processing with hardware accelerated MAnalyse and standard MDegrain in no overlapped mode. Lowest quality - may produce visible blockiness on flames/fogs/fades. Fastest mode.

I hope the speed penalty from no-overlapped MDegrainN with interpolated overlap may be decreased in the future releases - still not look with profilter what may be optimized more. But in the interpolated overlap mode it also processes 4x blocks number so the host CPU load is high.

Some sad news - the block size 16x16 runs unstable at least at some test modes with hardware acceleration at my remote test host and remote debugger can not catch exception about divide by zero. So it may be a NVIDIA driver issue of Windows or this software.

If you like my software - you may donate me or join my team in OZON promo platform to support my growing family with several kids. Write me a private message for details.

[mastrboy]

Finally the most long awaited feature to MDegrainN with hardware acceleration in 2022 - fully internal MDegrainN interpolated overlap mode : First working release - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.13


Does it not support YUV420 in 8bit?
I can only get it to work with 10,12,16bit:

Working tests:
ColorBarsHD().crop(4,0,-4,0)
ConvertToYUV420().ConvertBits(16)
#ConvertToYUV420().ConvertBits(12)
#ConvertToYUV420().ConvertBits(10)
tr = 3
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr, blksize=16)
MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150, IntOvlp=0)

Not working:
ColorBarsHD().crop(4,0,-4,0)
ConvertToYUV420().ConvertBits(8)
#ConvertToYUV420()
tr = 3
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr, blksize=16)
MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150, IntOvlp=0)

Error I get from AVSmeter on 8bit content:
Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: C:\Program Files (x86)\AviSynth+\plugins64+\mvtools2.dll
Address: 0x00007FFD41996618

[DTL]

"Does it not support YUV420 in 8bit?"

Practically the only mostly tested format is the YV12 that is separated planes YUV420 in 8bit. That I typically use at my encodings. And block size of 8x8. If you got crash with block size 16 - try to increase padding in MSuper to 16 or more. I think it was fixed in some old versions (may be in the pinterf 2.7.45 source) but if appear again - the current first workaround to try is to increase padding.

So the better MSuper for blocksize=16 is
MSuper(hpad=16, vpad=16)

I even think of making it something like auto-adjust of padding from block size but unfortunately the data flow is from MSuper to downstream filters so MSuper can not get the block size from MAnalyse (in the easy way of current frames sending via AVS environment). And padding of 8 is internal default in MSuper. May be it can be safely enough increased to 16 or even 32 because current PCs typically have more memory. Will try to do it in next builds.

It looks it is old issue of mvtools so in some scripts I see auto-increasing padding to the block size may be added for fail-safety - it it easy in script but may be not possible in a separated filters execution:

Myblksize = 16
sc=MSuper(hpad=Myblksize, vpad=Myblksize, ...)
MAnalyse(sc, blksize=Myblksize,...)


I know users like block size of 16 because it typically faster onCPU (and with overlap it make not very visible blockiness) but I typically use 8x8 because it give better quality (also as I see 16 is unstable with HW modes at least at my current only avaialble test hardware setup).

Other known issue that SSE2 builds may run unstable with bitdepth >8 on new CPUs. So AVX2 build is recommented where possible.
Also the very few frame sizes were tested - so it is recommended to start from 'standard' of 1920x1080 for FullHD and 3840x2160 for UHD4K. If HW mode will create several buggy blocks lines at the bottom of frame - current workaround is to pad frame from the bottom to several block size lines (I typically use 72 for FullHD and blocksize of 8x8).

The padding is required to keep good quality at the edges of frames because all search, SAD check and blend engines can not operate with parts of blocks (paranoid check of borders will decrease processing speed over all the frame) so for correct and best quality work the padding of at least blocksize size is good. Too large default padding will waste of RAM and may decrease speed. But if 0xC__5 exception occur and increasing padding to some 'large value' like blocksize x10 solves it - it is a mark that some more debug and adjusting of clipping MVs or other bugfix required.

[takla]

@DTL
Great update!

LWLibavVideoSource("C:\Users\Admin\Downloads\newSAMPLE.mkv")
Crop(0, 280, -0, -280)
BilinearResize(1920, 1080)
ConvertBits(8, dither=1).ConvertToYV12()
EZdenoise(thSAD=300, TR=8, Chroma=true)
Prefetch(12, 48)

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 0)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma, Levels=1)
Multi_Vector = Super.MAnalyse(Multi=true, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma, Levels=1, optSearchOption=5)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=int(float(thSAD*0.9)), thSADC=thSADC, thSADC2=int(float(thSADC*0.9)), IntOvlp=1)
}

CPU
v13 - Levels=1, Overlap=BLKSize/2, 8-bit & YV12
time=74.898s
time=75.007s
857.119 KB

GPU
v13 -optSearchOption=5 & IntOvlp=1
time=71.913s
time=73.643s
time=72.687s
826.089 KB

GPU
v13 -optSearchOption=5 & IntOvlp=0
time=76.360s
785.041 KB

GPU
v13 -optSearchOption=6 & IntOvlp=1
time=72.494s
824.445 KB

All tested using the previously posted 4K sample, with this command:

ffmpeg -y -benchmark -i 01.avs -c:v prores_ks -qscale:v 4 v13.mkv

Hardware used:
AMD Ryzen 3900X
AMD Radeon RX5700

Quality difference to CPU is now much closer and speed is now faster even at 1080P. I'll most likely post an update for EZdenoise in my thread soon, with some instructions.

[DTL]

It is more interesting to test with max quality at pel=4. CPU only vs DX12-ME assisted. Default pel=1 is sort of 'draft' quality only. Same as IntOvlp=0 - fastest but low quality mode.

Also it is good to test if 'large cache' AMD Ryzen will be faster or slower with new interpolated overlap mode for MDegrainN (prefferably with MVLPF enabled also that adds one more SAD re-checking pass and full ref frames reload to dispatch ports of CPU) and UseSubShift true/false. For both 1080p and 4K with pel=4.

So combinations to test:

MSuper(pelrefine=false, pel=4)
MAnalyse(optSearchOption=5) (optSearchOption=6 require UseSubShift=1 in this case)
MDegrainN(MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.8, UseSubShift=1, IntOvlp=1)

and

MSuper(pelrefine=true, pel=4)
MAnalyse(optSearchOption=5 or 6)
MDegrainN(MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.8, UseSubShift=0, IntOvlp=1)

"v13 -optSearchOption=5 & IntOvlp=1
time=71.913s
v13 -optSearchOption=5 & IntOvlp=0
time=76.360s"

It is even strange - at my 'old' intel CPU of 9-series the (interpolated) overlap mode of MDegrainN is about 2 times slower. May be here something else limits speed so results are close or even much more complex overlap processing in MDegrainN even faster ? The overlap processing in MDegrainN is at minimum 2 passes over the frame - first pass accumulates partial weighted blocks (may be even in float or short 16bit at least) and second pass blends and convert to output bitdepth. Though I typically work with pel=4 only so it require either large RAM planes fetching or many sub-sample shift computing in CPU.

"CPU
v13 - Levels=1,"

Running MAnalyse onCPU is better to use all levels (so levels=0). It is typically only a bit slower but may catch good long MVs if there is fast movement in footage.

[magnetite]

So I tried this new update with the OnCPU SAD calculation in this post (https://forum.doom9.org/showpost.php?p=1971553&postcount=142), and it still asks me for the Compute.cso shader file. Is that normal, or I thought it was supposed to be CPU only?

[DTL]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.14

Added mode 2 for IntOvlp for MDegrainN: It do not check real SAD of the interpolated blocks positions. So it is faster but may be lower in quality.

Fixed buffer overrun bug in InterpolateOverlap in MDegrainN.

Added AVX2 (8 bit output), SSE2 and SSE4 ( >8 bit output) second pass processing to output format into MDegrainN.

Disabled loading of shader file Compute.cso in optSearchOption=6 mode of MAnalyse.

Added different builds - for Win10 and later with DX12, for Win7 and others without DX12. Also some IntelC++ builds available for AVX2 CPUs.

It is possible to move (copy) interpolation of overlap to MAnalyse and also put its mode=1 computing of SAD to accelerator. But as I test with IntOvlp=2 at my CPU without re-check of SAD the speed benefit is small enough (about 12%). And that redesign need more time.

As current profiling shows the most of time for overlap processing in MDegrainN with high pel precision is in ref data fetching from memory (USS=0) or sub-shifting computing (USS=1). At the i5-9600 CPU both processes is about balanced. But at faster chips and AVX512 subshifting may be finally USS=1 mode will be visibly faster. Though it depends on cache size and speed and task size. The overlap blend computing and data conversion/storing is very fast already. So putting of post-overlap 16bit to 8bit conversion from C-ref to AVX2 makes almost zero speed addition. At least at my tested config.

IntelC++ SSE2 builds require some syntax redesign and development time so not included in this release. At i5-9600 the speed decreases in a sequence IntelC AVX2 -> MSVC AVX2 -> MSVC SSE2 as 3.55 -> 3.4 -> 3.2 fps with UseSubShift=1 and IntOvlp=1.

Update: Finally add new options descriptions to documentation. See updated file https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Documentation/mvtools2.html . Still no documented lots of limitations of new options. Like block size, bitdepth and so on (supported yes/no, SIMD accelerated yes/no). It looks need a table form.

[DTL]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.15

Added diagonal interpolated overlap mode to MDegrainN of 2x blocks number to process. IntOvlp=3 with SAD re-check and IntOvlp=4 with interpolated SAD.
Added more error messages if non-compatible options provided for MSuper/MAnalyse/MDegrainN.
Updated documentation with new options. Updated file is https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Documentation/mvtools2.html
Added meander scan in the combined luma+chroma overlapped processing - may be better reuse of cached ref planes data.

Now the IntOvlp=3 is the typical everyday usage mode because it is much better in speed and very close to the quality as IntOvlp=1. Only about 30% slower in compare with no-overlap processing at i5-9600 with SO=5. It is close or equal to old BlockOverlap plugin operation.

Now for the future possible to make many combined modes of speed/quality:
1. Diagonal overlap search onCPU in MAnalyse.
2. Diagonal overlap is compatible with DX12ME and can double load of accelerator with 'real' search - may be used when host CPU speed is low and accelerator is underloaded.
Some internal flags may be added to MVs clip to indicate if it contain diagonal overlapped MVs data.
MDegrainN may also accept 2 MVs clips from 2 MAnalyse for original and diagonally shifted blocks seach data using any combinations of onHWAcc (single or dual) or onCPU search. To balance loading between host CPU and a single or multiply accelerators. As I see after-mining secondhand headless cards with may be equal to GTX1060 chip are now avaialble at Aliexpress at about $35..50.
Though the quality between real searched 2 MVs planes for diagonal overlap mode and second interpolated MVs set need to be examined - may be too low difference. May be only worth is host CPU too slow for MDegrainN and many free accerelators resources available.