Dogway's Filters Packs

[LeXXuz]

@LeXXuz: The way you are cascading SMDegrain is for MDegrain cascading. I only do this for grain fest like 300 or 21 Grams, otherwise I use prefiltering cascading.
For LFR it's the same concept, initial LFR values can be somewhat higher than last LFR (wouldn't go over 300 or 400 for 1080p).
For chroma it depends how damaged it is, otherwise use @anton_foy's suggestion.


Thanks again Dogway. :) I've adjusted the settings for LFR and a first test looks promising so far regarding detail retention in difficult areas.

[Dogway]

Finally updated SMDegrain. Adds RGB support to ex_luma_rebuild() and fixes the issue in ex_retinex(), also added option to disable highlight compression to ex_retinex(), so in essence working as a shadow enhancer, it's built inline so for lvl=1 it's faster than before.
Also for GradePack fixed saturation in ex_contrast() as it was a regression I didn't notice.

[DTL]

About high thSAD:

As I see in current version of SMDegrain:
thSCD1 = Default( thSCD1, round(0.35*thSAD+260)) # Typically between 330 and 400

So if user want to use thSAD of 1000 the default internal thSCD1 will be about 610 only. So if blocks SAD generally > 610 it will quickly overload the counter of too bad blocks per frame and frame will be marked as not-usable and any increasing of tr (and thSAD) will be useless (also the total MDegrain processing will be disabled if all ref frames will be marked as non-usable). So for large noise level content is it better to manually set thSCD1 param or increasing of thSAD above about 500 will quickly become useless and will not take more blocks/frames into processing.

It is good to add as a note to script users: if thSAD > 500 it is recommended to set thSCD1 manually about thSAD (not less). With a caution about possibly more errors of bad blocks blending at scene change. Or may be added into script as a condition.

So the formula thSCD1=0.35*thSAD+260 stop working good with thSAD about 500..600+. And user will silently got no-degrained result. May be it is a design feature of SMDegrain to save user from more distorted result but with more noise left ? If so it is good to be documented.

The function for marking total frame as non-usable is https://github.com/pinterf/mvtools/blob/d8bdff7e02c15a28dcc6e9ef2ebeaa9d16cc1f56/Sources/FakePlaneOfBlocks.cpp#L72

IsSceneChange(sad_t nTh1, int nTh2) const
{
int sum = 0;
for ( int i = 0; i < nBlkCount; i++ )
sum += ( blocks[i].GetSAD() > nTh1 ) ? 1 : 0;

return ( sum > nTh2 );
}


Where nBlkCount - total number of blocks per frame, Th1 and Th2 is scaled to SAD and normalized to blocks number thSCD1 and thSCD2 params. 'sum' is counter of 'too bad' blocks in the frame.

So 'scene change' detector may trigger frame-non-usable flag also with too noisy content (too high interframe SAD of blocks) and too low thSCD1 param. When frame is marked as non-usable - no any one block of frame is used for useful denoising of output. Whole frame is failed.

[Dogway]

The formula was modeled after scene change detection. For noisy sources anything higher than ~600 was not detecting any scene change at all and hence blending scenes. It's typically always above default of 400 so already improving on defaults. Maybe we should decouple it from pure SC detection.

[tormento]

The formula was modeled after scene change detection.
Do you plan to support newer DTL development in MVTools?

It could boost his development a lot, as SMDegrain and your tools are so widely used.

[DTL]

"Maybe we should decouple it from pure SC detection."

There is no other 'scene detect' in MAnalyse+MDegrain system. So the only high quality solutions for very noisy movies may be to cut scenes manually or some automatically using AVS plugins to separated clips and process each clip separately and concatenate back. It will save from different scenes frames blending inside falling in 2_x_tr group of neibour frames. With any high thSAD values and with completely removed SCD functionality with setting thSCD1 to some high value like 10000 and thSCD2 to 255.

[Dogway]

Do you plan to support newer DTL development in MVTools?

It could boost his development a lot, as SMDegrain and your tools are so widely used.

I think block size of 16 is not supported, which is the default for HD. Should look into other changes though. Current defaults are modeled after vanilla mvtools so I would need to account for the changes. In any case I still haven pendant of the final Zopti optimization run with synthetic grain. What I'm seeing more and more is playback film grain addition in codecs like JXL, AV1, HEVC, etc. So the trend is to encode clean and add synthetic grain on top on playback, although I'm suspicious of such real-time grain quality.

[DTL]

" block size of 16 is not supported"

It is supported as max current block size of DX12-ME HW accelerated mode and also in most of already working MDegrainN new modes. It simply not tested well because it produces lower quality and I currently use 8x8 only for HD. As for speed in hardware ME I not remember if using 16x16 makes significantly better speed. There planned some 'fastest' hacking mode of ME processing with supplying half sized image to ME engine (level 1 of MSuper clip instead of level 0) and search with block size 8x8 and interpolate MVs to double sized with 16x16 block size. But still not finished. It is also already should work with 'onCPU' MAnalyse with optPredictorType about 5 (output only interpolated prediction at level 0 and SAD recalculation).

[tormento]

It is supported as max current block size of DX12-ME HW accelerated mode
Perhaps it's better to have testing on both sw and hw MVTools versions, so that we can understand if it's a DirectX issue or a programming one.

[LeXXuz]

I'm playing around with prefilters at the moment. I can't decide between DGDenoise and BM3DCuda.

DGDenoise is very fast even on older graphic cards but I read quite a few times now it should be used mainly on anime and not films because it is very soft?

I don't know how much this is valid if it's running as a prefilter for MDegrain. Comments? :)

[anton_foy]

I'm playing around with prefilters at the moment. I can't decide between DGDenoise and BM3DCuda.

DGDenoise is very fast even on older graphic cards but I read quite a few times now it shoould be used mainly on anime and not films because it is very soft?

I don't know how much this is valid if it's running as a prefilter for MDegrain. Comments? :)

I have been using it on film/slog-footage with low settings (str=0.3) with good results but if its low light I compensate with levels to brighten up the lift and gamma a bit before degraining and then darken it back with levels immediately after otherwise it seems to miss alot of motion vectors in the shadows.

Edit: although I only used it on 4K resolution material, I would think it needs different settings for lower resolutions.

[LeXXuz]

I have been using it on film/slog-footage with low settings (str=0.3) with good results but if its low light I compensate with levels to brighten up the lift and gamma a bit before degraining and then darken it back with levels immediately after otherwise it seems to miss alot of motion vectors in the shadows.

Edit: although I only used it on 4K resolution material, I would think it needs different settings for lower resolutions.

That is interesting. Could you post an example on raising the levels maybe?

[anton_foy]

That is interesting. Could you post an example on raising the levels maybe?

This you can try:

Convertbits(16)

#Levels shadow compensate:
Levels(0, 1, 255*256, 36*256, 255*256, coring=false)

#Prefilter:
p=levels(12*256,1.0,100*256,0,255*256).temporalsoften(3,20,255,15,2).ex_DGDenoise(str=0.3)

#Denoise:
SMDegrain(prefilter=p)

#levels reset:
Levels(36*256,1.0,255*256,0,255*256)

I found that adding temporalsoften before DGDegrain stabilize the "wobblyness" that the heavy noise creates on lines and objects. Tweak it to your taste.

[anton_foy]

ex_reduceflicker(strength=1) is great but it gives me strange halos and sometimes like double exposed offset lightsources on 4K sources. Can post before after images if needed. It seems to only happen in bright areas.

[DTL]

I'm playing around with prefilters at the moment. I can't decide between DGDenoise and BM3DCuda.

DGDenoise is very fast even on older graphic cards but I read quite a few times now it should be used mainly on anime and not films because it is very soft?

I don't know how much this is valid if it's running as a prefilter for MDegrain. Comments? :)

Typically prefilter is not for MDegrain but for MAnalyse. So the output of MDegrain is not affected directly by the output of prefilter. The only direct way to modify MDegrainX/N output is to provide custom per-refining clip for pel > 1 (and for old method of fetching samples of pel-refined clip for pel > 1 and not integer sample coordinates). The idea of prefilter is to reduce MVs noise caused by luma/chroma noise and the way of MAnalyse detect the motion coordinates of the block. But it also may degrade the quality of motion search (small position errors will cause detail smoothing) and cause up to gross position errors (cause visible bad blocks blends of thSAD too high) because after removing noise with details the difference between blocks also become smaller and it is possible to run far from original block position if running full tree of hierarchical search with too much search radius (so with prefiltering the use of lots of hacks in MAnalyse like penalties and other 'truemotion' stuff may be more important, also may be not good to set too big search radius and even use not all levels).

In new approach of 2.7.45+ builds the noise from MVs is more or less removed after MAnalyse search and before MDegrain blending using the properties of long enough time-domain vectors. Like for static block the MV in each frame of the tr-scoped group of frames should be zero and with some real life motion the coordinate of MV in a sequence of frames can not change too fast.

I still not found significant benefit from prefilter while running hardware ME from GTX1060 and applying MVLPF in the new builds of mvtools as intermediate processing before ME and MDegrain blending to fix some noise from MVs. Currently the some implementation of MVLPF filtering is intergated into MDegrainN (of 2.7.45+ builds) and not available for direct tweaking by script writers. I really interesting why so useful and not very complex processing was still not implemented many years ago at the active development of mvtools. Or may be it is somehow working at MRecalculate or other parts of mvtools ? The only idea that first MDegrainX of X 1,2,3 runs too slow on old CPUs so not efficient time-domain filtering may be applied visibly profitable. Nowdays with MDegrainN and tr > 10 running at acceptable speed and ME-acceleration where available it become more usable and allow to use more complex transfer functions filters in time domain for motion of each block.

It really very science-loaded processing (may not have some single best for all implementation of linear or even non-linear filter) and may be need to be tweaked for per-movie or per-scene basis. So it planned to add some long-requested interface for script-accessible MVs format of datastream between MAnalyse and MDegrain so script-writers may make lots of testing with intermediate MVs data additional processing to get best results. As well as lots of work was put for pre-processing before MAnalyse in the past. Though any anti-noise processing of MVs is not image-processing task but DSP processing in time-domain and can not be performed fast with existing 2D processing filters and direct data access scripts in interperting mode may be much slower. So it mostly for finding some profitable filtering methods for future addition in the compiled executable form.

[LeXXuz]

Typically prefilter is not for MDegrain but for MAnalyse. So the output of MDegrain is not affected directly by the output of prefilter. The only direct way to modify MDegrainX/N output is to provide custom per-refining clip for pel > 1 (and for old method of fetching samples of pel-refined clip for pel > 1 and not integer sample coordinates). The idea of prefilter is to reduce MVs noise caused by luma/chroma noise and the way of MAnalyse detect the motion coordinates of the block. But it also may degrade the quality of motion search (small position errors will cause detail smoothing) and cause up to gross position errors (cause visible bad blocks blends of thSAD too high) because after removing noise with details the difference between blocks also become smaller and it is possible to run far from original block position if running full tree of hierarchical search with too much search radius (so with prefiltering the use of lots of hacks in MAnalyse like penalties and other 'truemotion' stuff may be more important, also may be not good to set too big search radius and even use not all levels).

In new approach of 2.7.45+ builds the noise from MVs is more or less removed after MAnalyse search and before MDegrain blending using the properties of long enough time-domain vectors. Like for static block the MV in each frame of the tr-scoped group of frames should be zero and with some real life motion the coordinate of MV in a sequence of frames can not change too fast.

I still not found significant benefit from prefilter while running hardware ME from GTX1060 and applying MVLPF in the new builds of mvtools as intermediate processing before ME and MDegrain blending to fix some noise from MVs. Currently the some implementation of MVLPF filtering is intergated into MDegrainN (of 2.7.45+ builds) and not available for direct tweaking by script writers. I really interesting why so useful and not very complex processing was still not implemented many years ago at the active development of mvtools. Or may be it is somehow working at MRecalculate or other parts of mvtools ? The only idea that first MDegrainX of X 1,2,3 runs too slow on old CPUs so not efficient time-domain filtering may be applied visibly profitable. Nowdays with MDegrainN and tr > 10 running at acceptable speed and ME-acceleration where available it become more usable and allow to use more complex transfer functions filters in time domain for motion of each block.

It really very science-loaded processing (may not have some single best for all implementation of linear or even non-linear filter) and may be need to be tweaked for per-movie or per-scene basis. So it planned to add some long-requested interface for script-accessible MVs format of datastream between MAnalyse and MDegrain so script-writers may make lots of testing with intermediate MVs data additional processing to get best results. As well as lots of work was put for pre-processing before MAnalyse in the past. Though any anti-noise processing of MVs is not image-processing task but DSP processing in time-domain and can not be performed fast with existing 2D processing filters and direct data access scripts in interperting mode may be much slower. So it mostly for finding some profitable filtering methods for future addition in the compiled executable form.

Thank you very much DTL for another detailed insight. :thanks:

This you can try:

Convertbits(16)

#Levels shadow compensate:
(0, 1, 255*256, 36*256, 255*256, coring=false)

#Prefilter:
p=levels(12*256,1.0,100*256,0,255*256).temporalsoften(3,20,255,15,2).ex_DGDenoise(str=0.3)

#Denoise:
SMDegrain(prefilter=p)

#levels reset:
Levels(36*256,1.0,255*256,0,255*256)

I found that adding temporalsoften before DGDegrain stabilize the "wobblyness" that the heavy noise creates on lines and objects. Tweak it to your taste.

Thanks anton_foy for sharing that script. That may prove useful for some material I have with lots of troublesome dark areas. :)

[DTL]

The sad truth may be that mvtools was not designed as denoise tool and the degrain part is only small part. Its main task was the frame interpolation for motion smoothing and it cause the main hacks and tweaks in MAnalyse for better motion interpolation (less visible bugs) and not for best possible degraining for video compression. So mvtools old versions do not have built-in prefiltering and interfiltering of data for best possible denoising and users need to test different handmade approaches around the given filters set in mvtools project.

*After again reading mvtools2 doc* : Do the MStoreVect and MRestoreVect functions work for converting MVs internal format to 'standard clip' and back so script writer can get access to MVs data and make intermediate filtering of MVs too (between MAnalyse and MDegrain) and continue to pass the changed data to next mvtools filters ? So may be this functionality already exist (but not initially designed for this process) and no need to be developed again ? It is very few documented and do not document the mapping of each block MV to sample data of the 'normal clip' but it may be checked. A question to Dogway: Was it tested as a way to access MAnalyse MVs data (of both dx, dy data for inter-filtering and so frequently asked SAD for script-based automation like auto-thSAD assignment) ? Using MStoreVect and 'pixel-accessing' functions from a script (like Eval ?) ?

Also about MFlow: documentation says it can be used for pixel-based and not block-based compensation and for denoising too (without block artefacts and may be not required overlap mode directly). So in theory it can create less detail smoothing at the small and complex moving images like facial animation where even mid-sized blocks like 8x8 is still too large (and typical 16x16 and 32x32 are awfully large) and downsampling to 4x4 for the total clip is too slow in processing: Was it used in the known degrain scripts already ? Or it was too slow so the old single-filter MDegrain block-based runs faster ? Are there plans to look for MFlow-based degrain in Dogway scripts to test it quality compared to MDegrain-block-based execution engine ? Unfortunately it looks not currently support 'wrapped' multi-tr data passing from MAnalyse to degraining so need long hand-written scripts for passing +-forward/backward compensated groups of frames to blending engine like old MDegrainX for every tr-value (or may be it is possible to create a function in script ?).

The simulation of simplest MDegrain1 with MFlow in documentation is already awfully large:

super = MSuper()
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
forward_compensation = MFlow(super, forward_vectors, thSCD1=500) # or use MCompensate
backward_compensation = MFlow(super, backward_vectors, thSCD1=500)
# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

And still do not have complete example with exact blending engine. Also not passes SAD to blending engine to protect from too badly compensated areas blends.
Or it is a task for some future MDegrainNFlow() function or a option to MDegrainN like (UseFlowInterpolation=0/1).


Also as addition to new 2.7.45+ builds with supporting hardware ME acceleration and different block sizes: Currently Microsoft DX12-ME API support only 8x8 and 16x16 block size. But as I see from mvtools2 doc there are existing special functions for 'scale' of MVs between different block sizes (for faster processing). So if blocksize of 32x32 and higher is required the downsampled clip can be feed to MAnalyse with blocksize 16x16 and post-processed with MScaleVect to 32x32 block size. It same idea as optSearchOption=6 for the internals of MAnalyse but looks not required and may be removed if it already designed as developed function of mvtools. It may be important in the era of HD/UHD/UHD2 mix formats or for 8K that is not currently directly covered by old ME-accelerators of mid-201x years. So blocksize of 8x8 may be processed for 4K downsampled clip from 8K source and after MScaleVect to 16x16 being supplied to MDegrain of 8K processing full size input clip. The reserve of precision default down to qpel at 8x8 and 16x16 block sizes from ME-accelerators at full speed is very enough for 4K and 8K processing - even after scaling to double block size the precision is only lowered to half pel.

[anton_foy]

The sad truth may be that mvtools was not designed as denoise tool and the degrain part is only small part. Its main task was the frame interpolation for motion smoothing and it cause the main hacks and tweaks in MAnalyse for better motion interpolation (less visible bugs) and not for best possible degraining for video compression. So mvtools old versions do not have built-in prefiltering and interfiltering of data for best possible denoising and users need to test different handmade approaches around the given filters set in mvtools project.

*After again reading mvtools2 doc* : Do the MStoreVect and MRestoreVect functions work for converting MVs internal format to 'standard clip' and back so script writer can get access to MVs data and make intermediate filtering of MVs too (between MAnalyse and MDegrain) and continue to pass the changed data to next mvtools filters ? So may be this functionality already exist (but not initially designed for this process) and no need to be developed again ? It is very few documented and do not document the mapping of each block MV to sample data of the 'normal clip' but it may be checked. A question to Dogway: Was it tested as a way to access MAnalyse MVs data (of both dx, dy data for inter-filtering and so frequently asked SAD for script-based automation like auto-thSAD assignment) ? Using MStoreVect and 'pixel-accessing' functions from a script (like Eval ?) ?

Also about MFlow: documentation says it can be used for pixel-based and not block-based compensation and for denoising too (without block artefacts and may be not required overlap mode directly). So in theory it can create less detail smoothing at the small and complex moving images like facial animation where even mid-sized blocks like 8x8 is still too large (and typical 16x16 and 32x32 are awfully large) and downsampling to 4x4 for the total clip is too slow in processing: Was it used in the known degrain scripts already ? Or it was too slow so the old single-filter MDegrain block-based runs faster ? Are there plans to look for MFlow-based degrain in Dogway scripts to test it quality compared to MDegrain-block-based execution engine ? Unfortunately it looks not currently support 'wrapped' multi-tr data passing from MAnalyse to degraining so need long hand-written scripts for passing +-forward/backward compensated groups of frames to blending engine like old MDegrainX for every tr-value (or may be it is possible to create a function in script ?).

The simulation of simplest MDegrain1 with MFlow in documentation is already awfully large:

super = MSuper()
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
forward_compensation = MFlow(super, forward_vectors, thSCD1=500) # or use MCompensate
backward_compensation = MFlow(super, backward_vectors, thSCD1=500)
# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

DeGrainMedian() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

And still do not have complete example with exact blending engine. Also not passes SAD to blending engine to protect from too badly compensated areas blends.
Or it is a task for some future MDegrainNFlow() function or a option to MDegrainN like (UseFlowInterpolation=0/1).


Also as addition to new 2.7.45+ builds with supporting hardware ME acceleration and different block sizes: Currently Microsoft DX12-ME API support only 8x8 and 16x16 block size. But as I see from mvtools2 doc there are existing special functions for 'scale' of MVs between different block sizes (for faster processing). So if blocksize of 32x32 and higher is required the downsampled clip can be feed to MAnalyse with blocksize 16x16 and post-processed with MScaleVect to 32x32 block size. It same idea as optSearchOption=6 for the internals of MAnalyse but looks not required and may be removed if it already designed as developed function of mvtools. It may be important in the era of HD/UHD/UHD2 mix formats or for 8K that is not currently directly covered by old ME-accelerators of mid-201x years. So blocksize of 8x8 may be processed for 4K downsampled clip from 8K source and after MScaleVect to 16x16 being supplied to MDegrain of 8K processing full size input clip. The reserve of precision default down to qpel at 8x8 and 16x16 block sizes from ME-accelerators at full speed is very enough for 4K and 8K processing - even after scaling to double block size the precision is only lowered to half pel.

Interesting. But I see you use degrainmedian/"use your favourite spatial/temporal denoiser" there, is this like motion compensation without mdegrain?

[DTL]

" I see you use degrainmedian/"use your favourite spatial/temporal denoiser""

It is example copied from mvtools2 doc directly with comments. Now I see the DeGrainMedian() is possibly some real external denoiser plugin/script.

" is this like motion compensation without mdegrain?"

In this example of simulating MDegrain1 processing the motion compensation is performed by MFlow. But for final blending some external engine is used (may be with protection from too bad blending based on absolute difference of samples values ?).
Or with even number of frames and/or with equal weigthing may be AVS internal blenders may be used like Overlay() or Layer() with 1/3 weight mask for each of 3 clips (current, -1 frame compensated, +1 frame compensated).

The MDegrainX/N is compiled executable for several operations:
1. Motion compensation of blocks based on MAnalyse data
2. Blending of compensated blocks in the tr-scoped pool of surrounding frames using some weighting based on SAD values relative to thSAD params (also scene change detection). Simple or overlapped mode blending based on service data from MAnalsye.

[LeXXuz]

I'm looking for a preset between 'Low' and 'Normal' for ex_BM3D in SMDegrain.

I'm asking because the low profile still leaves a lot of headroom for my GPUs on 1080p sources. The 'Normal' profile however drops performance significantly and makes the GPU the bottleneck. Both with radius=1, of course.

I just don't want to tweak numbers here as long as I don't understand what they are really doing and maybe risk to worsen quality. :o

I also don't quite understand why the RTX 30 GPU performs just slightly better than an old GTX 10 GPU, although it has several times the CUDA processing power according to Geekbench.

[DTL]

About accessing MVs from MAnalyse via MStoreVect:

Made a small test:

LoadPlugin("mvtools2.dll")

BlankClip(pixel_type="YV12")

super=MSuper()
mvs=MAnalyse(super)

MStoreVect(mvs, "RGB24")

FlipVertical()

It shows some vertical lines for 0 frame (the VECTOR structure is filled with 'invalid' values that is 0,0 for dx,dy and max/invalidSAD like 65535) because the MVs for the 0-1 frame are invalid and not calculated. And 1 and later frames are black (the MVs for static black frame are dx=dy=SAD=0) after header data.

Vertical Flip is looks to reverse lines scan order for RGB imaging so to start from top left corner. It have a small header (may be 256 bytes total of AnalysisData structure) and a sequence of VECTOR structures for all blocks in sequential scan order (from top to bottom and from left to right I hope). VECTOR structure of MVs data for each block is 12bytes of dx, dy and SAD as 32bit integers (looks like all signed). The mapping to RGB24 pixel data is not very easy to read because the starting offset to the VECTOR structures may be not integer part of RGB24 pixel array of 3 bytes per addressable pixel. So the function to get MVs data for some x,y of block or blocknum=y*blocks_per_row+x is not very simple. Also the width and height of the output frame by MStoreVect is in some form calculated from blocks X,Y count to form some not very narrow frame I think to be good to view as rectangle and not thin line.

" why the RTX 30 GPU performs just slightly better than an old GTX 10 GPU,"

May be it is limited by memory transfter speed or some other processing not offloaded to accelerator.

[LeXXuz]

" why the RTX 30 GPU performs just slightly better than an old GTX 10 GPU,"

May be it is limited by memory transfter speed or some other processing not offloaded to accelerator.

I don't think it's memory related. Bandwith is much higher.

RTX 3060ti: 8GB GDDR6 / 256bit / 448GB/s
GTX 1050ti: 4GB GDDR5 / 128bit / 112GB/s

Maybe code is not optimized for Ampere architecture. Something probably only Wolfram knows.

[DTL]

Memory transfer between accelerator and host CPU internals. It may be very great bottleneck for attempt to 'offload' processing of some small chunks of data between host CPU executed program and some external accelerator support via PCI-E bus (also the already awful memory subsystem of current PC architecture chips of multi level caches and so on). So any real data transfer between host and accelerator it is a very long task nowdays. Like scheduling of special host memory buffer and putting command to a queue of some dispatcher for DMA transfer from host RAM to accelerator when its time come to the scheduled task. So speed optimization of 'offload' processing of small sequential pieces of work here and there at host CPU and accelerator is hardly possible. Only offloading large pieces of work (using upload to accelerator) and waiting to complete and getting result back from some host RAM area after download back from accelerator. Best case is full frame uploading or a set of frames and download ready result back to AVS caching system.

You may look at the performance counters of accelerator in Win10 task manager - its CUDA, Compute, Copy graphs. May be accelerator CUDA simply sit and wait for data transfer to/from host to complete. So RTX 3060 will simply show more free CUDA resources available. Also the upload/download tasks to/from accelerator is so important so it have already separate performance counter Copy (or even Copy_0, Copy_1 and other).

"/ 256bit / 448GB/s"

It is peak onboard RAM to chip of accelerator transfer speed. The real task data transfer between register file or L1D cache of CPU and internals of accelerator may be 1000x times slower and more. The scheduling overhead quickly become the main limiting factor if trying to operate small enough chunks of data. The typical random RAM byte fetching performance of current PCs with total latency of about 100ns is still only about 10 MB/s.

[LeXXuz]

You may look at the performance counters of accelerator in Win10 task manager - its CUDA, Compute, Copy graphs. May be accelerator CUDA simply sit and wait for data transfer to/from host to complete. So RTX 3060 will simply show more free CUDA resources available. Also the upload/download tasks to/from accelerator is so important so it have already separate performance counter Copy (or even Copy_0, Copy_1 and other).

For the GTX 1050ti Windows 10 task manager is showing usage under the 'Cuda' section of the GPU.

On the other system with the RTX 3060ti there is no Cuda section in Windows 10 task manager. The usage is shown in 3d section. Odd.

Both have latest NV drivers.

Just checked another system with Windows 11 and a GT 710. There is no Cuda section either and computing usage is shown in the 3d section as well.

The usage under any of the Copy sections is so minimal on all systems that I have to full screen Windows task manager to even barely see some tiny peaks.

[DTL]

"RTX 3060ti there is no Cuda section in Windows 10 task manager. The usage is shown in 3d section. Odd."

May you not add required compute-related performance counters views to task manager 'GPU' window ? Or the CUDA is not available in the drop-down list of available performance counters ? May be it is named as some other Compute (may be added to Compute_0 or Compute_1) in more new accelerators. The DX12 compute shaders execution is displayed in Compute_ performance counters for GTX1060.

[LeXXuz]

It's not available under performance counters. When running BM3DCUDA the GPU load is shown by the 3d performance counter.

There is close to none usage shown on any of the other counters.

https://abload.de/thumb/gpu_taskman_1050tirnkdf.png (https://abload.de/image.php?img=gpu_taskman_1050tirnkdf.png)
https://abload.de/thumb/gpu_taskman_3060ti8pjdw.png (https://abload.de/image.php?img=gpu_taskman_3060ti8pjdw.png)

[DTL]

Oh - it is strange. No Compute, no cuda. It looks they collapsed all computing and other 3D texturing to single 3D. But 3D shows 95% load.

[LeXXuz]

Dogway would it be possible to get DGSharpen() as another sharpener for the Contrasharp routine into SMDegrain?

It's very fast and quality wise somewhere between Contrasharp() and LSFplus().

[Dogway]

You can simply use it after SMDegrain. The reason Contrasharp and LSFplus are integrated is because they use CClip and other complex correlated settings, but DGSharpen has just one argument.

By the way, I wanted to share a sneak peak of coming CMF (Color Matching Function) for TransformsPack, will update soon when I fix an issue on CAT().

https://i.imgur.com/1WTpzrOl.png (https://i.imgur.com/1WTpzrO.png)

[LeXXuz]

The reason Contrasharp and LSFplus are integrated is because they use CClip and other complex correlated settings


That's why I asked, because I thought the result may be a little different when calling a sharpener after SMD script. Just trying to gain a little speed but don't want to lose quality of course. :)

[LeXXuz]

I've noticed a deviation in colours when comparing the source and filtered video in MeGUI's Avisynth preview window. See screenshots.

I was wondering if this is due the massiv picture processing with SMDegrain or maybe a bug in MeGUI?

https://abload.de/thumb/2022-07-171w6jrx.png (https://abload.de/image.php?img=2022-07-171w6jrx.png)https://abload.de/thumb/2022-07-176lkvb.png (https://abload.de/image.php?img=2022-07-176lkvb.png)

(It's 4k content with BT2020 SMTPE2084 static HDR. But I noticed this with BT709 SDR content too.)

Anyone else noticed slight colour/luma differences?

[Dogway]

LeXXuz, did you feed tonemapped as prefilter? It's suggested on the header of SMDegrain, something like DGHDRtoSDR(mode="pq",white=100) will do.
SMDegrain shouldn't do any color manipulation.

[LeXXuz]

LeXXuz, did you feed tonemapped as prefilter? It's suggested on the header of SMDegrain, something like DGHDRtoSDR(mode="pq",white=100) will do.

... if you compare those two screenshots closely you will already notice a slight deviation in colour. Which becomes clearly visible when playing the encoded video on an HDR capable screen. That only occurs when I use SMDegrain on that input.

I've used these settings for some subtle filtering:

SMDegrain(tr=6, thSAD=250, thSCD1=250, contrasharp=100, limitS=true, LFR=round(width()/6), DCTFlicker=false, refinemotion=true, truemotion=false, pel=2, Str=2.0, prefilter=7, chroma=true, plane=4)
Prefetch(32,32)
Return(Last)

EDIT: removed some wrong statements

[Dogway]

You don't tonemap the source but the prefilter clip to compute MV.
MAnalyse is going to have a hard time finding good vectors on such a compressed space.

You can do this:

tm = DGHDRtoSDR(mode="pq",white=100)
pr = tm.ex_DGDenoise(str=0.10,LFR=width()/2.,UV=3).ex_boxblur(0.5, mode="weighted", UV=3)

SMDegrain(tr=6, thSAD=250, thSCD1=250, contrasharp=100, limitS=true, LFR=round(width()/6.), DCTFlicker=false, refinemotion=true, truemotion=false, pel=2, Str=2.0, prefilter=pr, chroma=true, plane=4)


There might be a point of failure though, ex_DGDenoise() requires YV12 input, so if yours is 422 or 444 it's going to read primaries from frame properties, depending on your source loader it might be not retrieving this. I use latest DGSource.

[LeXXuz]

You don't tonemap the source but the prefilter clip to compute MV.
MAnalyse is going to have a hard time finding good vectors on such a compressed space.


Now I understand what you mean. :o I use latest DG tools as well. Thanks, I will add this to my script when processing HDR content. But are you sure about white=100? That is almost pitch black. :)

Anyway, I have to apologize to you. It seems neo_f3kdb() is the root of evil here and not SMDegrain. I totall forgot I usually add this later as the final filter to my scripts. :o

Was quite happy with its results and speed but I guess it's time to look for another debanding filter.

EDIT:

There might be a point of failure though, ex_DGDenoise() requires YV12 input, so if yours is 422 or 444 it's going to read primaries from frame properties, depending on your source loader it might be not retrieving this. I use latest DGSource.

Well I chose DGDenoise() as it's an animation movie. Would it be beneficial to use BM3d instead?

[Boulder]

Do you actually need debanding for 10bit sources?

[LeXXuz]

Do you actually need debanding for 10bit sources?

I came across quite a few movies with banding even in 10bit.

Technically it shouldn't be necessary, you're right. But I like to have a debanding filter running for those scenes that are poorly mastered.

Maybe I overexaggerate, but I absolutely hate banding. Especially in scenes like underwater and stuff. Completely ruins the movie experience for me. :(

[Dogway]

Well I chose DGDegrain() as it's an animation movie. Would it be beneficial to use BM3d instead?

Both DGDenoise and KNLMeansCL ar nlmeans based so they are fine for anime. Although your source looked clean from the screenshot.
For debanding you can try my mod GradFun3plus, it's on the fast side with default mode.

[Boulder]

But are you sure about white=100? That is almost pitch black. :)

That's definitely a small mistake there. The default is 1000 which should be a good value to use.


Anyway, I have to apologize to you. It seems neo_f3kdb() is the root of evil here and not SMDegrain. I totall forgot I usually add this later as the final filter to my scripts. :o
I noticed some strange happenings with neo_f3kdb with HDR encodes a long time ago (https://forum.doom9.org/showthread.php?p=1912378#post1912378), but I've never seen any changes in color in SDR or HDR. Is it reproducable by some bare minimum script? To me it looks more like an issue with the colormatrix.

[Dogway]

That's definitely a small mistake there. The default is 1000 which should be a good value to use.


I don't know, I can go as low as 48 and still shows fine. Check here (https://forum.doom9.org/showthread.php?p=1941217#post1941217).
Anyway, it's a prefilter so it doesn't need to be perfect, I guess it depends on source master level, simply use the value that saturates the bit range.

[Boulder]

I don't know, I can go as low as 48 and still shows fine. Check here (https://forum.doom9.org/showthread.php?p=1941217#post1941217).
Anyway, it's a prefilter so it doesn't need to be perfect, I guess it depends on source master level, simply use the value that saturates the bit range.

That's odd. The range is 1-2000 I think, and a regular rather bright movie like Green Book was pitch black with 100..

EDIT: Oh, a pointer regarding GradFun3plus, I think the default threshold for PQ sources should be higher. The default value is a bit too low, I'd go maybe for 0.75 or something like that.

[LeXXuz]

I noticed some strange happenings with neo_f3kdb with HDR encodes a long time ago (https://forum.doom9.org/showthread.php?p=1912378#post1912378), but I've never seen any changes in color in SDR or HDR. Is it reproducable by some bare minimum script? To me it looks more like an issue with the colormatrix.

I was still using neo_f3kdb_r6 version. Tried out _r7 and no problems with colour deviations so far. :)

But if that doesn't work, I'll try out Dogways mod of Gradfun.

[guest]

Just use SMDegrain(prefilter=6, gpuid=-1). SMDegrain will choose BM3DCPU is prefilter

Hi ked,

I have a very belated question about this gpuid "switch" in SMDegrain..

It has been bought to my attention that with the changes Dogway did to this function a few builds ago, that if it's left to default, any SMDegrain filter will use the GPU for filtering / pre-filtering.

Would that be the case ???

Do I need to change it to gpuid=-1 for CPU instead, for my scripts ??

Cheers

[kedautinh12]

Hi ked,

I have a very belated question about this gpuid "switch" in SMDegrain..

It has been bought to my attention that with the changes Dogway did to this function a few builds ago, that if it's left to default, any SMDegrain filter will use the GPU for filtering / pre-filtering.

Would that be the case ???

Do I need to change it to gpuid=-1 for CPU instead, for my scripts ??

Cheers

You need use gpuid=-1 for CPU. If not, Smdegrain will choose GPU is default

[guest]

You need use gpuid=-1 for CPU. If not, Smdegrain will choose GPU is default

Thankyou for that, I will need to change all my script's then :(

And another question, with KNLMeansCL as a pre-filter, how do you change that to use the CPU, in SMDegrain....it looks like it's written a little differently to BM3D "instruction's".

Cheers

[kedautinh12]

Thankyou for that, I will need to change all my script's then :(

And another question, with KNLMeansCL as a pre-filter, how do you change that to use the CPU, in SMDegrain....it looks like it's written a little differently to BM3D "instruction's".

Cheers

Just use SMDegrain(prefilter=5, gpuid=-1)

[guest]

Just use SMDegrain(prefilter=5, gpuid=-1)
:thanks:

[madey83]

Now I understand what you mean. :o I use latest DG tools as well. Thanks, I will add this to my script when processing HDR content. But are you sure about white=100? That is almost pitch black. :)

Anyway, I have to apologize to you. It seems neo_f3kdb() is the root of evil here and not SMDegrain. I totall forgot I usually add this later as the final filter to my scripts. :o

Was quite happy with its results and speed but I guess it's time to look for another debanding filter.

EDIT:



Well I chose DGDenoise() as it's an animation movie. Would it be beneficial to use BM3d instead?


HI LeXXuz,

i use a lot neo_f3kdb() and you mentioned that this is the root cause for color "changing". Do you know what would be the best alternative for neo_f3kdb?

this is my basic command for this filter:
neo_f3kdb(preset="veryhigh/nograin", dynamic_grain=false, output_depth=10)

Could you share yours for alternative?

[LeXXuz]

HI LeXXuz,

i use a lot neo_f3kdb() and you mentioned that this is the root cause for color "changing". Do you know what would be the best alternative for neo_f3kdb?

this is my basic command for this filter:
neo_f3kdb(preset="veryhigh/nograin", dynamic_grain=false, output_depth=10)

Could you share yours for alternative?

I was still using r6 version of neo_f3kdb().

With version r7 I could not reproduce this problem so far:
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/releases/tag/r7

[LeXXuz]

Dogway I noticed some faint distortions in my encodes, barely noticeable.
So I played around with the preview mode a little and saw a lot of erroneous frames of the prefiltered clip. See pics below:

https://abload.de/thumb/2022-07-291tjjv3.png (https://abload.de/image.php?img=2022-07-291tjjv3.png)https://abload.de/thumb/2022-07-292b4kjm.png (https://abload.de/image.php?img=2022-07-292b4kjm.png)https://abload.de/thumb/2022-07-2931lknh.png (https://abload.de/image.php?img=2022-07-2931lknh.png)https://abload.de/thumb/2022-07-29452j8t.png (https://abload.de/image.php?img=2022-07-29452j8t.png)https://abload.de/thumb/2022-07-2958qk14.png (https://abload.de/image.php?img=2022-07-2958qk14.png)https://abload.de/thumb/2022-07-290uk5b.png (https://abload.de/image.php?img=2022-07-290uk5b.png)

EDIT: I turned off DGDenoise and the errors are gone. Tried with no prefilter (-1) and BM3DCUDA (6) with no errors.

Btw. about "speed, memory and quality" shown on the right with different colours. Are they displayed anywhere? All I see is the gamma curve in the preview.