New Filters: TTempSmooth and TBilateral

[real.finder]

Has somebody made a 64-bit DLL of TTempSmooth? Thanks in advance.

EDIT: Yeah ok I found it here https://forum.doom9.org/showthread.php?p=1821762#post1821762 ; this is not a totally useless post, I guess it's useful to have it linked in the original TTempSmooth thread :)

that for vs not avs

anyway, this one https://github.com/Asd-g/AviSynth-vsTTempSmooth is uptodate and has HBD

[bruno321]

that for vs not avs

You sure? It's working for me with avisynth+ 64bit.

[real.finder]

You sure? It's working for me with avisynth+ 64bit.

don't know, but the Thread said "TTempsmooth port or equivalent for vapoursynth? "

there are old one here https://www.mediafire.com/download/zv0jm3mtmzf/TTempSmooth_3-20-2010.rar

[Reel.Deel]

that for vs not avs

It's for avs, it was updated to the 2.6 api since VS cannot load 2.5 plugins.

[DTL]

Some new toy to play with - pure statistical temporal degrain mode: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.01

Added new param:
pmode (integer).
pmode=0 - old type of samples averaging
pmode=1 - new statistical mostly probable sample fetching mode (FIR-type, no memory).
Only work with 8bit formats and C-ref only (opt=0).

Usage example:
vsTTempSmooth(pmode=1, Ythresh=5, Uthresh=5, Vthresh=5, maxr=6, opt=0) (other params not used)

Currently FIR-type only. IIR-type addition expected in next builds. Only C-ref implementation for now and very slow. The SIMD optimization is mostly probably possible but subject of some future design.

FIR-type filter leave some residual temporal nosie and may look more 'life like' but it is also less MPEG-friendly.

[anton_foy]

Some new toy to play with - pure statistical temporal degrain mode: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.01

Added new param:
pmode (integer).
pmode=0 - old type of samples averaging
pmode=1 - new statistical mostly probable sample fetching mode (FIR-type, no memory).
Only work with 8bit formats and C-ref only (opt=0).

Usage example:
vsTTempSmooth(pmode=1, Ythresh=5, Uthresh=5, Vthresh=5, maxr=6, opt=0) (other params not used)

Currently FIR-type only. IIR-type addition expected in next builds. Only C-ref implementation for now and very slow. The SIMD optimization is mostly probably possible but subject of some future design.

FIR-type filter leave some residual temporal nosie and may look more 'life like' but it is also less MPEG-friendly.

Actually I think it delivers very pleasant results with the new parameter even without motion compensation. Yet as you say leaving residual noise.
Have to experiment more with it. Not possible to make it available for HBD use any time soon?

[DTL]

The new mode algorithm still not finally tuned (so need testing with MPEG encoder and/or visual quality or may be required to add one more thresholds group). Currently threshold check is performed only with final best candidate for output value. And it is possible to skip 'too bad' input samples (replace its dismetric to 'verybig diff' in DM-table ?).

16bit integer is easily possible but I need to solve the complex templating program design - currently it uses very simplified form of main processing function to make build of program possible. I not good in high level C++ programming and even less understand new C++ language syntax (of C++ 17 and more). And Asd-g like to rewrite all plugins to highest possible C++ syntax so it is hard to add changes to it. May be it helps to performance with very new compilers like LLVM but make software harder to understand and change.

Also to make it run at good speed the converting to SIMD computing is required. It may be benefitical of SIMD because it make many computing over a small dataset.

[DTL]

Some short info on current attempt of IIR-type addition to FIR-only in pmode=1:

First simple port from MDegrainN shows it is significantly worse simple FIR-type. It was first unnatural but it really have sources: If we simply use update-threshold and try to increase it to save from update with more worse samples - it also cause locking of first not very best sample and IIR-update threshold not allow to update stored value with better value. It is really bad for both denoise and MPEG ecoding.

So I tried to modify algorithm: Add also memory of 'best metric' (lowest DM table sum indicating most probable value) and in IIR-update it compare both current frame best metric with stored and update threshold. If current frame sample best metric is better in compare with stored - the IIR-memory also updated and this sample selected as best to output. But this still also can not any significantly outperform the 'simple FIR' part of algorithm in the reduction of MPEG-encoder output bitrate. So it looks some more ideas required.

Good part of this findings - the total algorithm implemented in both MDegrainN for blocks and in vsTTempSmooth for samples still not perfect and need some additions and it can make quality better.

[anton_foy]

Good part of this findings - the total algorithm implemented in both MDegrainN for blocks and in vsTTempSmooth for samples still not perfect and need some additions and it can make quality better.

Thanks. So when it is bad block matching it can use samples(FIR), or will it be too much demanding calculations to use both?

[DTL]

In theory it is possible - if too low data collected from current tr-pool of frames for current block (low weights of all blocks except current frame) - try to fallback to samples-level processing for this block area. Sort of combining block-based and samples-based temporal processing. But it really will make processing even slower and not known how much benefit will add (at least when using typical 8x8 blocks and 'full' overlapping the spatial granularity is already not very bad). Though if evaluate 200..400% crops it is still sometime visible how differently degrained neibours blocks areas (typically near object's edges and with low enough thSAD values). If users for FullHD and UHD like to use 16x16 and lager blocks for better performance - it may be better visible. So per-sample processing may in theory decrease these processing issues. It may be put into 'todo-list' of 'next-gen' MDegrainN design. I put it to my list of new notes for post-2.7.45 mvtools as feature number 42.

Also for die-hard perfectionists it is possible to go into sub-sample spatial precision like in half-pel or even q-pel (and downsampling back to full-pel after processing at finer levels). And yes - it will be about 4 and 16 times slower. Though it can be already simulated with simply upsizing 2x or 4x before processing. The only performance benefit of internal sub-sample processing with runtime scaling of small areas (blocks) is lower host RAM usage and better performance with on-chip CPU caches.

[DTL]

New release: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.02

Fixed samples fetching from tr-pool.

Added IIR mode (somehow working).

New params for IIR:
ythupd
uthupd
vthupd
(for y, u, v channels separately), integer, default=0 (no IIR). IIR mem update thresholds. If current best sample SAD below 'thupd' and lowest sum in DM table not less of stored sample lowest sum - stored in memory sample used. Typically expected to be below 'thresh' params values (about 1/2 of thresh).

Current IIR-mode is not visibly helps to MPEG encoder so may be need more logic on update strategy of stored sample. But it is somehow working implementation.

It looks all this processing only good for completely static areas. With motion the temporal aliasing quickly cause significant distortions. Or it looks need to be tested after motion compensation (as I see in some denosie script with MCompensate).

[DTL]

New release: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.03

Finally part (main ?) of the pmode=1 going to SIMD and it makes processing performance about good (about 20 fps in both 8/16 bit modes at 1920x1080 frame at i3-9100T CPU with maxr=7). opt=2 for AVX2 processing need to set manually if AVX512 avaialble.

New params of update penalty internally scaled to bitdepth >8 (as other vsTTempSmooth params). Values are absolute and directly applied to the 'similarity metric' stored in IIR memory for current stored sample. So it may be difficult to describe its measurement units. Some testing shows it may be at about 2..10 useful range. The idea is somehow close to 'pnew' param in MAnalyse MV search.

[anton_foy]

New release: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.03

Finally part (main ?) of the pmode=1 going to SIMD and it makes processing performance about good (about 20 fps in both 8/16 bit modes at 1920x1080 frame at i3-9100T CPU with maxr=7). opt=2 for AVX2 processing need to set manually if AVX512 avaialble.

New params of update penalty internally scaled to bitdepth >8 (as other vsTTempSmooth params). Values are absolute and directly applied to the 'similarity metric' stored in IIR memory for current stored sample. So it may be difficult to describe its measurement units. Some testing shows it may be at about 2..10 useful range. The idea is somehow close to 'pnew' param in MAnalyse MV search.

Wow must try! No HDB support yet?

[kedautinh12]

Wow must try! No HDB support yet?

HBD supported, you need click to link and read changelogs

[anton_foy]

HBD supported, you need click to link and read changelogs

Thanks my mistake.

[real.finder]

New release: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.03

Finally part (main ?) of the pmode=1 going to SIMD and it makes processing performance about good (about 20 fps in both 8/16 bit modes at 1920x1080 frame at i3-9100T CPU with maxr=7). opt=2 for AVX2 processing need to set manually if AVX512 avaialble.

New params of update penalty internally scaled to bitdepth >8 (as other vsTTempSmooth params). Values are absolute and directly applied to the 'similarity metric' stored in IIR memory for current stored sample. So it may be difficult to describe its measurement units. Some testing shows it may be at about 2..10 useful range. The idea is somehow close to 'pnew' param in MAnalyse MV search.

will you PR these changes to https://github.com/Asd-g/AviSynth-vsTTempSmooth ?

[DTL]

It is expected to be PR to Asd-g branch. But looks sometime later. Described in https://github.com/Asd-g/AviSynth-vsTTempSmooth/issues/7 . Currently it is still in development of better SIMD implementations. Started to make full-SIMD based AVX2 version (only AVX2 and later have hardware gathering instructions allowing to not fallback to scalar domain in gathering best sampels pointed by indices). Made first performance test - it is about 2x faster at i3-9100T in compare with partial-SIMD in release from 22.06.23. But it need more debug and finishing. May be next weeks. So I think Asd-g will wait for next commits and more finished form of new mode.

[DTL]

New release: https://github.com/DTL2020/AviSynth-vsTTempSmooth/releases/tag/1.3.0_a.04

Finished full-SIMD AVX2 processing (about 30x faster of C-scalar with maxr > 20).

maxr in pmode=1 may be up to 128 (same as in MDegrainN/mvtools2).

It is finally ready for PR to Asd-g. But it was found in current Asd-g sources also some non-compatibility with x86 32bit building. So still no 32bit build. Waiting for reply from Asd-g about 32bit building.

Tests shows the IIR mode still produce worse MPEG bitrate and may be more artifacts. At least with natrual content. May be better with flat colours anime. Also any non-motion compensated processing looks like quickly make lots of artifatcs even with ythresh close to 'noise level'. So expected to use mostly in motion-compensated scripts (MCTD.avsi ?) . I again was too lazy to collect all plugins required to run MCTD and made simple mvtools-only based MC script (up to tr=12). Also it was found the DX12-ME mode of MAnalyse may have some limitaiton on number of MAnalyse in the script (at least with GTX1060 card and its drivers) so the multi-mode and extracting MVs for each MCompensate is very useful in such cases. It allow to run 1 (or 2 for interlaced) MAnalyse with DX12 objects and use any tr required.

For interlaced footage:


# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")
LoadPlugin("vsTTempSmooth.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 3)

FFmpegSource2("mv.mpg", fpsnum=25)


SeparateFields()

fields_orig=last

odd_f=SelectOdd()
even_f=SelectEven()

super_o = MSuper(odd_f, chroma=true)
super_e = MSuper(even_f, chroma=true)

tr=12

/*
# better MVs MAnalyse + full real 4x overlap, about 5x slower
mv_m_o=MAnalyse(super_o, multi=true, delta=tr, search=3, trymany=true, global=true, pzero=10, pnew=10, pglobal=10, overlap=4, chroma=true)
mv_m_e=MAnalyse(super_e, multi=true, delta=tr, search=3, trymany=true, global=true, pzero=10, pnew=10, pglobal=10, overlap=4, chroma=true)
*/


#Simple MAnalyse
mv_m_o=MAnalyse(super_o, multi=true, delta=tr)
mv_m_e=MAnalyse(super_e, multi=true, delta=tr)


/*
#DX12 HW ME
mv_m_o=MAnalyse(super_o, multi=true, delta=tr, optSearchOption=6, levels=1)
mv_m_e=MAnalyse(super_e, multi=true, delta=tr, optSearchOption=6, levels=1)
*/

bv1_o = SelectEvery(mv_m_o, tr*2, 0)
bv1_e = SelectEvery(mv_m_e, tr*2, 0)
bv2_o = SelectEvery(mv_m_o, tr*2, 2)
bv2_e = SelectEvery(mv_m_e, tr*2, 2)
bv3_o = SelectEvery(mv_m_o, tr*2, 4)
bv3_e = SelectEvery(mv_m_e, tr*2, 4)
bv4_o = SelectEvery(mv_m_o, tr*2, 6)
bv4_e = SelectEvery(mv_m_e, tr*2, 6)
bv5_o = SelectEvery(mv_m_o, tr*2, 8)
bv5_e = SelectEvery(mv_m_o, tr*2, 8)
bv6_o = SelectEvery(mv_m_o, tr*2, 10)
bv6_e = SelectEvery(mv_m_o, tr*2, 10)
bv7_o = SelectEvery(mv_m_o, tr*2, 12)
bv7_e = SelectEvery(mv_m_o, tr*2, 12)
bv8_o = SelectEvery(mv_m_o, tr*2, 14)
bv8_e = SelectEvery(mv_m_o, tr*2, 14)
bv9_o = SelectEvery(mv_m_o, tr*2, 16)
bv9_e = SelectEvery(mv_m_o, tr*2, 16)
bv10_o = SelectEvery(mv_m_o, tr*2, 18)
bv10_e = SelectEvery(mv_m_o, tr*2, 18)
bv11_o = SelectEvery(mv_m_o, tr*2, 20)
bv11_e = SelectEvery(mv_m_o, tr*2, 20)
bv12_o = SelectEvery(mv_m_o, tr*2, 22)
bv12_e = SelectEvery(mv_m_o, tr*2, 22)

fv1_o = SelectEvery(mv_m_o, tr*2, 1)
fv1_e = SelectEvery(mv_m_e, tr*2, 1)
fv2_o = SelectEvery(mv_m_o, tr*2, 3)
fv2_e = SelectEvery(mv_m_e, tr*2, 3)
fv3_o = SelectEvery(mv_m_o, tr*2, 5)
fv3_e = SelectEvery(mv_m_e, tr*2, 5)
fv4_o = SelectEvery(mv_m_o, tr*2, 7)
fv4_e = SelectEvery(mv_m_e, tr*2, 7)
fv5_o = SelectEvery(mv_m_o, tr*2, 9)
fv5_e = SelectEvery(mv_m_e, tr*2, 9)
fv6_o = SelectEvery(mv_m_o, tr*2, 11)
fv6_e = SelectEvery(mv_m_e, tr*2, 11)
fv7_o = SelectEvery(mv_m_o, tr*2, 13)
fv7_e = SelectEvery(mv_m_e, tr*2, 13)
fv8_o = SelectEvery(mv_m_o, tr*2, 15)
fv8_e = SelectEvery(mv_m_e, tr*2, 15)
fv9_o = SelectEvery(mv_m_o, tr*2, 17)
fv9_e = SelectEvery(mv_m_e, tr*2, 17)
fv10_o = SelectEvery(mv_m_o, tr*2, 19)
fv10_e = SelectEvery(mv_m_e, tr*2, 19)
fv11_o = SelectEvery(mv_m_o, tr*2, 21)
fv11_e = SelectEvery(mv_m_e, tr*2, 21)
fv12_o = SelectEvery(mv_m_o, tr*2, 23)
fv12_e = SelectEvery(mv_m_e, tr*2, 23)


fwc1_o = MCompensate(super_o, fv1_o)
fwc1_e = MCompensate(super_e, fv1_e)
fwc2_o = MCompensate(super_o, fv2_o)
fwc2_e = MCompensate(super_e, fv2_e)
fwc3_o = MCompensate(super_o, fv3_o)
fwc3_e = MCompensate(super_e, fv3_e)
fwc4_o = MCompensate(super_o, fv4_o)
fwc4_e = MCompensate(super_e, fv4_e)
fwc5_o = MCompensate(super_o, fv5_o)
fwc5_e = MCompensate(super_e, fv5_e)
fwc6_o = MCompensate(super_o, fv6_o)
fwc6_e = MCompensate(super_e, fv6_e)
fwc7_o = MCompensate(super_o, fv7_o)
fwc7_e = MCompensate(super_e, fv7_e)
fwc8_o = MCompensate(super_o, fv8_o)
fwc8_e = MCompensate(super_e, fv8_e)
fwc9_o = MCompensate(super_o, fv9_o)
fwc9_e = MCompensate(super_e, fv9_e)
fwc10_o = MCompensate(super_o, fv10_o)
fwc10_e = MCompensate(super_e, fv10_e)
fwc11_o = MCompensate(super_o, fv11_o)
fwc11_e = MCompensate(super_e, fv11_e)
fwc12_o = MCompensate(super_o, fv12_o)
fwc12_e = MCompensate(super_e, fv12_e)


bwc1_o = MCompensate(super_o, bv1_o)
bwc1_e = MCompensate(super_e, bv1_e)
bwc2_o = MCompensate(super_o, bv2_o)
bwc2_e = MCompensate(super_e, bv2_e)
bwc3_o = MCompensate(super_o, bv3_o)
bwc3_e = MCompensate(super_e, bv3_e)
bwc4_o = MCompensate(super_o, bv4_o)
bwc4_e = MCompensate(super_e, bv4_e)
bwc5_o = MCompensate(super_o, bv5_o)
bwc5_e = MCompensate(super_e, bv5_e)
bwc6_o = MCompensate(super_o, bv6_o)
bwc6_e = MCompensate(super_e, bv6_e)
bwc7_o = MCompensate(super_o, bv7_o)
bwc7_e = MCompensate(super_e, bv7_e)
bwc8_o = MCompensate(super_o, bv8_o)
bwc8_e = MCompensate(super_e, bv8_e)
bwc9_o = MCompensate(super_o, bv9_o)
bwc9_e = MCompensate(super_e, bv9_e)
bwc10_o = MCompensate(super_o, bv10_o)
bwc10_e = MCompensate(super_e, bv10_e)
bwc11_o = MCompensate(super_o, bv11_o)
bwc11_e = MCompensate(super_e, bv11_e)
bwc12_o = MCompensate(super_o, bv12_o)
bwc12_e = MCompensate(super_e, bv12_e)


# create interleaved frames sequences

int_o = interleave(fwc12_o, fwc11_o, fwc10_o, fwc9_o, fwc8_o, fwc7_o, fwc6_o, fwc5_o, fwc4_o, fwc3_o, fwc2_o, fwc1_o, odd_f, bwc1_o, bwc2_o, bwc3_o, bwc4_o, bwc5_o, bwc6_o, bwc7_o, bwc8_o, bwc9_o, bwc10_o, bwc11_o, bwc12_o)
int_e = interleave(fwc12_e, fwc11_e, fwc10_e, fwc9_e, fwc8_e, fwc7_e, fwc6_e, fwc5_e, fwc4_e, fwc3_e, fwc2_e, fwc1_e, even_f, bwc1_e, bwc2_e, bwc3_e, bwc4_e, bwc5_e, bwc6_e, bwc7_e, bwc8_e, bwc9_e, bwc10_e, bwc11_e, bwc12_e)


my_Yth=10
my_thupd=0
my_pnew=2
my_opt=2
my_tr=tr

int_o=vsTTempSmooth(int_o, pmode=1, Ythresh=my_Yth, opt=my_opt, ythupd=my_thupd, uthupd=my_thupd, vthupd=my_thupd, ypnew=my_pnew, upnew=my_pnew, vpnew=my_pnew, maxr=my_tr)
int_e=vsTTempSmooth(int_e, pmode=1, Ythresh=my_Yth, opt=my_opt, ythupd=my_thupd, uthupd=my_thupd, vthupd=my_thupd, ypnew=my_pnew, upnew=my_pnew, vpnew=my_pnew, maxr=my_tr)

odd_f=selectevery(int_o, tr*2+1, tr) # return filtered central (not-compensated) frames only
even_f=selectevery(int_e, tr*2+1, tr) # return filtered central (not-compensated) frames only

Interleave(odd_f, even_f)

Weave()

Prefetch(..)


The 'best' MAnalyse run about 5x slower but produce best quality (and lowest MPEG file size at fixed crf). Also for even better MVs quality (and details saving) the multu-generation MAnalyse+MDegrainN can be used. The DX12-ME at GTX1060 produce slightly larger MPEG file (same as with MDegrainN). Using motion-compensated sources allow to use Ythresh about 2x larger of 'noise level' and still have low enough artifacts.

The 'denoise power' mostly depends on tr (with enough Ythresh) and comparable to MDegrainN. The performance mostly limited by MPEG encoder (x264) at my CPU. To save from lots of scripting with bir tr-values (and also to use already working interpolated overlap) it is expected to addition to MDegrainN as separate or combined processing mode. Quality is about good at low motion areas and mostly static. At fast moving areas it looks at least some overlap is required to make quality better (slower with old MAnalyse+MCompensate - need at least interpolating overlap addition to MCompensate too). Or may be some motion-masking (as in MCTD ?).

Update: Asd-g wrote it is no more possible to do 32bit builds of this plugin because of the limitaiton of the VCL used for SIMD functions. Also looks all other plugins by Asd-g with same VCL-based SIMD functions will no more have 32bit builds.

[DTL]

will you PR these changes to https://github.com/Asd-g/AviSynth-vsTTempSmooth ?

Some good news: Asd-g started to port some new features to his repository.

But at more testing it was found the main issue with IIR-mode: It can only run well with MT_SERIALIZED MT mode of AVS+ (require sequential frame request). But in this mode it looks like all multithreading is disabled around such a filter. And if the filter is slow - all performance drops to single threaded. So it was found to make performance good again - the internal multithreading must be added. Currently it is started via OpenMP but needs time to finish.
If IIR-mode is enabled (y/u/vthupd >0) - filter will register itself as MT_SERIALIZED and the user will need to set 'threads' param > 1 to make use of many CPU cores available.

The same processing ideas in the mvtools/MDegrainN also require MT_SERIALIZED MT mode auto-switch and mt=true. So it was great luck that pinterf found and fixed a bug in the AVSTP because it is currently the only way to run internal multithreading in the mvtools.

[DTL]

It looks like an implemented 'statistical' algorithm for pmode=1 calculate typical known math 'average' as median (as I see from wiki typically used 3 types of 'averages' - mean, median, mode https://en.wikipedia.org/wiki/Median ). The pmode=0 of vsTTempSmooth is close to mean average, pmode=1 is (equal ?) to median average (the 'mode' average not yet implemented).

So the latest version with pmode=1 is close (or equal) to MedianBlurTemporal with temporal processing only.

vsTTempSmooth(pmode=1, Ythresh=255, maxr=3) expected to be close or equal to
MedianBlurTemporal(radiusy=0, radiusu=0, radiusv=0, temporalradius=3)

Performance with tr 1 and 2 is better with 'special' modes of MedianBlurTemporal and Clense and with tr 3 and more (may be to 15..20 and more) is better with vsTTempSmooth(pmode=1). Also vsTTempSmooth provides additional service of thresholding vs current input frame (equal to 'limit' post-processing in MDegrainN ?) and some IIR-type optional processing. Also the float32 samples in vsTTempSmooth in pmode=1 processed in full precision (as described with constant time mode of MedianBlurTemporal it is quantized to 16bit integer because of limitation of algorithm used). Used algorithm support not only numbers but also other comparable/measurable 'objects' - like blocks of samples in MDegrainN.

I sent a message to pinterf via github about possible addition of this temporal-median implementation to MedianBlur2 (for tr >=3 and float32 samples) for compatibility with old scripts based on MedianBlurTemporal.

[kedautinh12]

vsTTempSmooth 1.2.4
https://github.com/Asd-g/AviSynth-vsTTempSmooth/releases
Added parameters pmode, y/u/vthupd, y/u/vpnew, threads. (@DTL2020)

[DTL]

Default internal multithreading enabled for max logical CPUs for any params of pmode=1 may be not best with also AVS+ multithreading enabled. With not using IIR processing mode (y/u/vthupd >0) it may be better to check if performance better with threads=1 and AVS+ multithreading (Prefetch(N>1)).

[kedautinh12]

vsTTempSmooth 1.2.5
Changed back default threads to 1.
https://github.com/Asd-g/AviSynth-vsTTempSmooth/releases

[kedautinh12]

vsTTempSmooth 1.2.6
Fixed undefined behavior when float input (pmode=1, regression from 1.2.5).
https://github.com/Asd-g/AviSynth-vsTTempSmooth/releases