Welcome to Doom9's Forum, THE inplace to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. 
14th July 2021, 21:09  #121  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

Yes I got like 2 or 3 more fps (82fps) vs mine (79fps, CPU now is in lazy state). It's like an optimized version of the variable syntax.
EDIT: woow so fast (your edit), now 94fps vs 85fps EDIT2: I mainly use 5x5 sorting networks for median5 and adaptive, but I'm thinking on adding it also to weighted 50% percentile, because it kinda makes more sense there than 3x3, although output wasn't what I expected...
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 14th July 2021 at 21:19. 
14th July 2021, 21:58  #122  Link 
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,478

I'm realising just now that although these sorting networks are optimised for sorting, they are not necessarily optimised for finding. My generator makes the best optimisation it can out of them (mathematically speaking), but it can lead to nonoptimal results. This may only affect oddnumbered networks. I'll continue to investigate.

14th July 2021, 22:35  #123  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

Thanks for the effort. Probably there might be finding algos around but I didn't research that. In any case you can use reverse elimination as we have been doing.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
19th July 2021, 14:52  #124  Link  
Registered User
Join Date: Nov 2009
Posts: 2,338

@wonkey_monkey: I could achieve a 5% improvement in median5 by realizing that my median algo was the same than undot4 (clip to closest). This requires one less input for the sorting network.
I'm also adding vertical median, matching verticalcleaner mode=1. Code:
"x[0,1] x[0,1] dup1 dup1 max swap2 min x swap2 clip" Quote:
Code:
x[0,2] T2^ x[0,1] T1^ x[0,2] B1^ x[0,1] B2^ B2 B1 > T1 T2 > & B2 B2 B1  + T1 T1 T2  + min T1 ? MA^ B2 B1 < T1 T2 < & B2 B1 B2   T1 T2 T1   max B2 ? MI^ x B1 MA min x T1 max max T1 MI max x min B2 max clip ################### On another note, I'm also developing a dot crawl dechecker filter. I will make use of your fantastic kernel here, but that will be the last discretion of three, SAD, spatial match and temporal match. About SAD I'm not sure how to implement it, per block or per pixel. Per pixel(?) Code:
a=selectevery(1,0) b=selectevery(1,1) Expr(a,b,"x[1,1] XA^ x[1,0] XB^ x[1,1] XC^ x[1,1] XD^ x[1,0] XE^ x[1,1] XF^ x[0,1] XG^ x[0,1] XH^ y[1,1] YA^ y[1,0] YB^ y[1,1] YC^ y[1,1] YD^ y[1,0] YE^ y[1,1] YF^ y[0,1] YG^ y[0,1] YH^ XA YA  abs XB YB  abs XC YB  abs XD YB  abs XE YE  abs XF YF  abs XG YG  abs XH YH  abs + + + + + + + ","") EDIT: By the way mt_motion(last, thy1=0,thy2=255,tht=255,U=128,V=128) pretty much matches the next Code:
thSAD=300 / 7.111 Expr(a,b,"x[1,1] XA^ x[1,0] XB^ x[1,1] XC^ x[1,1] XD^ x[1,0] XE^ x[1,1] XF^ x[0,1] XG^ x[0,1] XH^ y[1,1] YA^ y[1,0] YB^ y[1,1] YC^ y[1,1] YD^ y[1,0] YE^ y[1,1] YF^ y[0,1] YG^ y[0,1] YH^ XA YA  abs XB YB  abs XC YB  abs XD YB  abs XE YE  abs XF YF  abs XG YG  abs XH YH  abs + + + + + + + 8 / W@ 9 * {thSAD} scalef < 0 W ?","")
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 19th July 2021 at 15:47. 

20th July 2021, 21:39  #126  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

Thanks. Found a source I could read.
ExTools updated to v4.0. ex_median() new modes; vertical medians (ported from verticalcleaner), temporal medians (medianT and medianT5) and spatiotemporal medians (medianST, ML3D, ML3Dex, and BDM). Some modes overlap to Clense and MedianBlur. ex_kawase(), add logarithmic multipass ex_repair(), port half the modes from removegrain repair() ex_smooth(), 2 small bugfixes
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 21st July 2021 at 00:39. 
22nd July 2021, 17:18  #127  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

ExTools updated to v4.1.
ex_bilateral() new filter. Performs (much) slower than Dither_bilateral16() not sure why since it's just a few operators (Edit: yes, exp operator is uber slow, pow is slower though). Anyhow the output is cleaner and HBD native (and includes a dejaggie arg). Won't make a SmoothGrad filter unless the speed issue is solved. ex_median() new mode MMF. Fixed weightedp (for real)
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 22nd July 2021 at 22:12. 
22nd July 2021, 23:38  #128  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

ExTools updated to v4.2.
ex_median() new modes; SixNN (6 NearestNeighbours), PML (Planar MultiLevel), and removegrain modes 26 and 27. I plan to implement Kalman motion estimation (if possible) to avoid using mvtools. ex_bilateral() optimized a bit, still runs at around 93fps mainly due to the exp operator. I will be uploading to OP a comparison chart for all the modes in a few minutes.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
23rd July 2021, 10:11  #129  Link 
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,439

Would you mind to create a greyscale mode for SMDegrain and, if needed, ExTools?
With old movies, where more intensive filtering is required, I tried some times ago to set plane=0, chroma=false but I had worst results, mostly with higher bitrate when encoding, than leaving on "color mode".
__________________
@turment on Telegram 
23rd July 2021, 10:51  #130  Link 
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,673

Greyscale() at the end of your script would do the same if those two parameters already bypass any chroma processing for speedups.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... 
23rd July 2021, 20:37  #132  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

If the clip doesn't have colors then the UV planes should be mostly blank, so analyzing vectors from them looks like a bad idea.
At least for ExTools you can set UV=1 and chroma planes will be copied, I would need to check out SMDegrain but I think it does the same. I have yet to rework the Contrasharpening area, there are at least three methods for antiringing that I have to study, closely related to limiting so I also need to assess spatial and temporal limiting, it's not my forte so it will take some time.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 24th July 2021 at 02:42. 
23rd July 2021, 21:06  #133  Link  
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,583

Quote:
__________________
See My Avisynth Stuff 

24th July 2021, 10:55  #134  Link  
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,439

Quote:
__________________
@turment on Telegram 

24th July 2021, 11:06  #135  Link 
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,583

remove UV with ConvertToY/ConvertToY8 after source call and you are safe and you should get more speed as well
__________________
See My Avisynth Stuff 
24th July 2021, 16:24  #136  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

If the grain is very high solutions have been given before. You can try with this. It's the feature I've been long wanting to implement to SMDegrain.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
25th July 2021, 18:12  #137  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

ExTools updated to v4.3.
ex_kawase() fixed a stupid mistake and added 2 new modes, fibonacci and tribonacci sequence. ex_repair() implemented 4 temporal repair modes, this replaces TemporalRepair from RGTools which by the way, is broken. GradePack updated to v2.5. ex_glow() new filter to add glow, 2 modes fibonacci and tribonacci, it's very slow, is not optimized, only for proof of concept. I also updated OP with a ranking benchmarks of all the modes (ex_median, blurs and ex_edge). I want to upload also a comparison chart for all blend modes.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
27th July 2021, 00:17  #138  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

ExTools updated to v4.4.
ex_smartblur(). Filter for blurring similar to Photoshop's filter. Less gentle blur than ex_bilateral() but much faster. ex_median() new mode EMF (Extended Median Filter) ex_vinverse() removed the monolithic script and included into ExTools with some additional updates. ex_boxblur() updated "weighted" mode weights to use binomial coefficients. Now it should match ex_blur() (but faster) and removegrain(12), blur(), etc QTGMC updated to v3.5 based on 3.382s Practically same speed for 8bits (even with Expr unsupported LUT based calculations) and about 15% to 20% more performant for HBD. See OP for benchmarks. Changelog:  Replaced masktools2 with Expr calls via ExTools pack wrappers (except mt_merge)  Merged redundant expressions  Replaced blurs and medians with ExTools equivalents  Replaced Vinverse_avsi() with ex_vinverse() (HBD) and vinverse() (8bit)  Removed Dither support  Formatting and small optimizations I also want to post a list of all the median filters I could find. They happen to be much more than I thought, so I'm not including them all, although people can help out if interested to make a good library, despite some of them protected under a paywall or not possible with Expr expressions (recursions, etc) From all of them the ones I'm interested most are: IDBA, TSND, RAA, ADKI, and AUTMF. Convolution algorithms: Code:
# ATMF  Asymmetric Trimmed Median Filter # SMF  Switching Median Filter # WMF  Weighted Median Filter # CWMF  Center Weighted Median Filter # DWMF  Directional Weighted Median Filter # MDWMF  Modified Directional Weighted Median Filter # RWMF  Recursive Weighted Median Filter # PSMF  Progressive Switching Median Filter # SMFBDND  Switching Median Filter with Boundary Discriminative Noise Detection # BDSMF  Boundary Discriminative Switching Median Filtering # DBF  Decision Based Filter # DBA  Decision Based Algorithm Median Filter # DBA2  Decision Based Algorithm 2 Median Filter # MDBUTMF  Modified Decision Based Unsymmetric Trimmed Median Filter # DBUTMF  Decision Based Unsymmetric Trimmed Median Filter # ACWMF  Adaptive Center Weighted Median Filter # NID  New Impulse Detector # AMF  Adaptive Median Filter # FUF  Fuzzy Filter # UF  # ADKI  Adaptive Decision based Kriging Interpolation # IMF  Improved adaptive Median Filter ( Ha et al. (2016)) # IAMF  Improved Adaptive Median Filtering ( (Wang and Li, 2010)) # SAMF  Selective Adaptive Median Filter # MDBSMF  Multiple Decision Based Switching Median Filter # TSMF/TSF  TriState Median Filter # NTSMF  New TriState Switching Median Filter # NASMF  Noise Adaptive Softswitching Median Filter # FRMF  Fuzzy Rulebased Median Filtering # RAA  Removal of salt and pepper an Adaptive Approach # IDBA  Improved decision based algorithm # HFFLND  Hybrid Filter and Fuzzy Logic Noise Detector # NSPF  New Salt and Pepper Filter. (Biswal and Bohi, 2013) # FEMF  Fast and Efficient Median Filter # AMFLPD  Adaptive Median Filtering on Local Pixel Distribution # SSMF  Sorted Switching Median Filter # TSND  TwoStage Noise Detector # IEMF  Improved Extremum and Median value Filter # ADPSMF  Absolute Difference based Progressive Switching Median Filter # IMFA  Improved Median Filtering Algorithm # MBDND  Modified Boundary Discriminative Noise Detection # MSSMF  Multistate Switching Median Filter # RVINE  Random Valued Impulse Noise Elimination # RVINENF  Random Valued Impulse Noise Elimination using Neural Filter # RVINENFF  Random Valued Impulse Noise Elimination using NeuroFuzzy Filter # EPF  Edge Preserving Filter # NBPPFT  Neural Based Post Processing Filtering Technique # FTPHR  Filtering Technique for Preserving Homogeneous Region # FTPHRAER  Filtering Technique for Preserving Homogeneous Region and Edge Region # FTHCDI  Filtering Technique for Highly Corrupted Digital Images # CNFT  Combined Neural based Filtering Technique # FFBPAMNEM1 Feed Forward Back Propagation Algorithm for Multiple Noise Elimination Method 1 # FFBPAMNEM2 Feed Forward Back Propagation Algorithm for Multiple Noise Elimination Method 2 # ANFIS  Adaptive Neurofuzzy Inference System # HNFF  Hybrid NeuroFuzzy Filter # TSHNFF1  Two Stage Hybrid NeuroFuzzy Filter 1 # TSHNFF2  Two Stage Hybrid NeuroFuzzy Filter 2 # MNENFF1  Multiple Noise Elimination using NeuroFuzzy Filter 1 # MNENFF2  Multiple Noise Elimination using NeuroFuzzy Filter 2 # INEDI1  Intelligent Network for Enhancing Digital Images 1 # INEDI2  Intelligent Network for Enhancing Digital Images 2 # BF  Bilateral Filter # FBF  Fuzzy Bilateral Filter # INR  Impulse Noise Reduction # FRINR  Fuzzy Random Impulse Noise Reduction # FRINRM  Fuzzy Random Impulse Noise Reduction Method # FIDRM  Fuzzy Impulse noise Detection and Reduction Method # FIDRMC  Fuzzy Impulse noise Detection and Reduction Method for Color Images # HFRMC  Histogrambased Fuzzy Restoration Method for Color Images # FISF  Fuzzy Inference System Filter # AUTMF  Adaptive Unsymmetrical TrimBased Morphological Filter # MMEM  MinimumMaximum Exclusive Mean Filter # DBA  DecisionBased Asymmetrical trimmed median filter # FRF  Fuzzy Rankordered Filter # LABSVMF  cieLAB Switching Vector Median Filter # FCG  Fuzzy Color preserving Gaussian filter # Non Linear (for RGB images) # VMF  Vector Median Filter # TVMF  Trimmed Vector Median Filter # EVMF  Extended Vector Median Filter # FMVMF  Fast Modified Vector Median Filter # FSVF  Fast Similarity Vector Filter # DVMF  Directional Vector Median Filter # GVMF  Generalized Vector Median Filter # RCVMF  Rank Conditioned Vector Median Filter # RCTVMF  Rank Conditioning and Threshold Vector Median Filter # CLMMF  Crossing Level Median Mean Filter # VDF  Vector Directional Filter # BVDF  Basic Vector Directional Filter # GVDF  Generalized Vector Directional Filter # DDF  Directional Distance Filter # WVMF  Weighted Vector Median Filter # TWVMF  Trimmed Weighted Vector Median Filter # EXWVMF  Extended Weighted Vector Median Filter # ROWVMF  Rank Order Weighted Vector Median Filter # WVDF  Weighted Vector Directional Filters # GA WVDF  Genetic Algorithm Weighted Vector Directional Filter # CWVMF  CenterWeighted Vector Median Filter # AVMF  Adaptive Vector Median filter # ABVDF  Adaptive Basic Vector Directional Filter # MSVMAF  Multiclass Support Vector Machine based Adaptive Filter # ATCC  Adaptive Threshold and Color Correction # RSVF/RSVMF Robust Switching Vector Filter # AMMF  Adaptive Marginal Median filter # MSMF  Modified Switching Median Filter # RSDDF  Robust Switching Directional Distance filter # ACWVMF  Adaptive CenterWeighted Vector Median Filter # ARWSF  Adaptive rank weighted switching filter # PGA  Peer Group Averaging # PGSF  Peer Group Switching Filter # MPG  Modified Peer Group # PGVF  Peer Group Vector Filter # FPGF  Fast Peer Group Filters # NPGF  Novel Peer Group Filter # FWAF  Fuzzy Weighted Average Filter # FVMF  Fuzzy Vector Median Filter # FVDF  Fuzzy Vector Directional Filter # FOVF  Fuzzy Ordered Vector Filter # FHF  Fuzzy Hybrid Filter # ANNF  Adaptive NearestNeighbor Filter # ANNMF  Adaptive NearestNeighbor Multichannel Filter # AHMF  Adaptive Hybrid Multichannel Filter # FIDRM  Fuzzy Impulse Detection and Reduction Method # VMRHF  Vector Median Rational Hybrid filter # SAHVF  StructureAdaptive Hybrid Vector Filter # SWAV  Structure Weighted Average Filter # SVMF  Sigma Vector Median Filter # SBVDF  Sigma Basic Vector Directional Filter # SDDF  Sigma Directional Distance Filter # ASVMF  Adaptive Sigma Vector Median Filter # ASBVDF  Adaptive Sigma Basic Vector Directional Filter # ASDDF  Adaptive Sigma Directional Distance Filter # ASVMF  Adaptive Sigma Vector Median Filter # EVMF  Entropy Vector Median Filter # QSF  Quaternion Switching Filter # QVMF  Quaternion Vector Median Filter # QSVMF  Quaternion Switching Vector Median Filter
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
30th July 2021, 10:49  #139  Link 
Registered User
Join Date: Nov 2009
Posts: 2,338

ExTools updated to v4.5.
ex_median() ported spatiotemporal DeGrainMedian modes ex_repair() ported repair(16) Logo updated to v12.0. For some reason I couldn't load DeGrainMedian with MP_Pipeline win32 mode so I couldn't double check if the output matched, nor benchmark it against. The filter is slowish but I'm not sure how to optimize it further. I also wanted to port GrapeSmooth but closed source it seems. I'm also still porting Adaptive Sharpen, I came to a little halt so may ask later for some help.
__________________
i74790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread 
30th July 2021, 11:56  #140  Link 
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,583

__________________
See My Avisynth Stuff 
Tags 
avisynth, dogway, filters, hbd, packs 
Thread Tools  Search this Thread 
Display Modes  

