Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
14th July 2021, 21:09 | #121 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Yes I got like 2 or 3 more fps (82fps) vs mine (79fps, CPU now is in lazy state). It's like an optimized version of the variable syntax.
EDIT: woow so fast (your edit), now 94fps vs 85fps EDIT2: I mainly use 5x5 sorting networks for median5 and adaptive, but I'm thinking on adding it also to weighted 50% percentile, because it kinda makes more sense there than 3x3, although output wasn't what I expected...
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 14th July 2021 at 21:19. |
14th July 2021, 21:58 | #122 | Link |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,532
|
I'm realising just now that although these sorting networks are optimised for sorting, they are not necessarily optimised for finding. My generator makes the best optimisation it can out of them (mathematically speaking), but it can lead to non-optimal results. This may only affect odd-numbered networks. I'll continue to investigate.
|
14th July 2021, 22:35 | #123 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Thanks for the effort. Probably there might be finding algos around but I didn't research that. In any case you can use reverse elimination as we have been doing.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
19th July 2021, 14:52 | #124 | Link | |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
@wonkey_monkey: I could achieve a 5% improvement in median5 by realizing that my median algo was the same than undot4 (clip to closest). This requires one less input for the sorting network.
I'm also adding vertical median, matching verticalcleaner mode=1. Code:
"x[0,1] x[0,-1] dup1 dup1 max swap2 min x swap2 clip" Quote:
Code:
x[0,2] T2^ x[0,1] T1^ x[0,-2] B1^ x[0,-1] B2^ B2 B1 > T1 T2 > & B2 B2 B1 - + T1 T1 T2 - + min T1 ? MA^ B2 B1 < T1 T2 < & B2 B1 B2 - - T1 T2 T1 - - max B2 ? MI^ x B1 MA min x T1 max max T1 MI max x min B2 max clip ################### On another note, I'm also developing a dot crawl dechecker filter. I will make use of your fantastic kernel here, but that will be the last discretion of three, SAD, spatial match and temporal match. About SAD I'm not sure how to implement it, per block or per pixel. Per pixel(?) Code:
a=selectevery(1,0) b=selectevery(1,-1) Expr(a,b,"x[1,1] XA^ x[1,0] XB^ x[1,-1] XC^ x[-1,1] XD^ x[-1,0] XE^ x[-1,-1] XF^ x[0,1] XG^ x[0,-1] XH^ y[1,1] YA^ y[1,0] YB^ y[1,-1] YC^ y[-1,1] YD^ y[-1,0] YE^ y[-1,-1] YF^ y[0,1] YG^ y[0,-1] YH^ XA YA - abs XB YB - abs XC YB - abs XD YB - abs XE YE - abs XF YF - abs XG YG - abs XH YH - abs + + + + + + + ","") EDIT: By the way mt_motion(last, thy1=0,thy2=255,tht=255,U=-128,V=-128) pretty much matches the next Code:
thSAD=300 / 7.111 Expr(a,b,"x[1,1] XA^ x[1,0] XB^ x[1,-1] XC^ x[-1,1] XD^ x[-1,0] XE^ x[-1,-1] XF^ x[0,1] XG^ x[0,-1] XH^ y[1,1] YA^ y[1,0] YB^ y[1,-1] YC^ y[-1,1] YD^ y[-1,0] YE^ y[-1,-1] YF^ y[0,1] YG^ y[0,-1] YH^ XA YA - abs XB YB - abs XC YB - abs XD YB - abs XE YE - abs XF YF - abs XG YG - abs XH YH - abs + + + + + + + 8 / W@ 9 * {thSAD} scalef < 0 W ?","")
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 19th July 2021 at 15:47. |
|
20th July 2021, 21:39 | #126 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Thanks. Found a source I could read.
ExTools updated to v4.0. -ex_median() new modes; vertical medians (ported from verticalcleaner), temporal medians (medianT and medianT5) and spatio-temporal medians (medianST, ML3D, ML3Dex, and BDM). Some modes overlap to Clense and MedianBlur. -ex_kawase(), add logarithmic multipass -ex_repair(), port half the modes from removegrain repair() -ex_smooth(), 2 small bugfixes
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 21st July 2021 at 00:39. |
22nd July 2021, 17:18 | #127 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
ExTools updated to v4.1.
-ex_bilateral() new filter. Performs (much) slower than Dither_bilateral16() not sure why since it's just a few operators (Edit: yes, exp operator is uber slow, pow is slower though). Anyhow the output is cleaner and HBD native (and includes a dejaggie arg). Won't make a SmoothGrad filter unless the speed issue is solved. -ex_median() new mode MMF. Fixed weightedp (for real)
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 22nd July 2021 at 22:12. |
22nd July 2021, 23:38 | #128 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
ExTools updated to v4.2.
-ex_median() new modes; SixNN (6 Nearest-Neighbours), PML (Planar Multi-Level), and removegrain modes 26 and 27. I plan to implement Kalman motion estimation (if possible) to avoid using mvtools. -ex_bilateral() optimized a bit, still runs at around 93fps mainly due to the exp operator. I will be uploading to OP a comparison chart for all the modes in a few minutes.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
23rd July 2021, 10:11 | #129 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Would you mind to create a greyscale mode for SMDegrain and, if needed, ExTools?
With old movies, where more intensive filtering is required, I tried some times ago to set plane=0, chroma=false but I had worst results, mostly with higher bitrate when encoding, than leaving on "color mode".
__________________
@turment on Telegram |
23rd July 2021, 10:51 | #130 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,804
|
Greyscale() at the end of your script would do the same if those two parameters already bypass any chroma processing for speedups.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
23rd July 2021, 20:37 | #132 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
If the clip doesn't have colors then the UV planes should be mostly blank, so analyzing vectors from them looks like a bad idea.
At least for ExTools you can set UV=1 and chroma planes will be copied, I would need to check out SMDegrain but I think it does the same. I have yet to rework the Contrasharpening area, there are at least three methods for antiringing that I have to study, closely related to limiting so I also need to assess spatial and temporal limiting, it's not my forte so it will take some time.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 24th July 2021 at 02:42. |
23rd July 2021, 21:06 | #133 | Link | |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,597
|
Quote:
__________________
See My Avisynth Stuff |
|
24th July 2021, 10:55 | #134 | Link | |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Quote:
__________________
@turment on Telegram |
|
24th July 2021, 11:06 | #135 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,597
|
remove UV with ConvertToY/ConvertToY8 after source call and you are safe and you should get more speed as well
__________________
See My Avisynth Stuff |
24th July 2021, 16:24 | #136 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
If the grain is very high solutions have been given before. You can try with this. It's the feature I've been long wanting to implement to SMDegrain.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
25th July 2021, 18:12 | #137 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
ExTools updated to v4.3.
-ex_kawase() fixed a stupid mistake and added 2 new modes, fibonacci and tribonacci sequence. -ex_repair() implemented 4 temporal repair modes, this replaces TemporalRepair from RGTools which by the way, is broken. GradePack updated to v2.5. -ex_glow() new filter to add glow, 2 modes fibonacci and tribonacci, it's very slow, is not optimized, only for proof of concept. I also updated OP with a ranking benchmarks of all the modes (ex_median, blurs and ex_edge). I want to upload also a comparison chart for all blend modes.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
27th July 2021, 00:17 | #138 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
ExTools updated to v4.4.
-ex_smartblur(). Filter for blurring similar to Photoshop's filter. Less gentle blur than ex_bilateral() but much faster. -ex_median() new mode EMF (Extended Median Filter) -ex_vinverse() removed the monolithic script and included into ExTools with some additional updates. -ex_boxblur() updated "weighted" mode weights to use binomial coefficients. Now it should match ex_blur() (but faster) and removegrain(12), blur(), etc QTGMC updated to v3.5 based on 3.382s Practically same speed for 8-bits (even with Expr unsupported LUT based calculations) and about 15% to 20% more performant for HBD. See OP for benchmarks. Changelog: - Replaced masktools2 with Expr calls via ExTools pack wrappers (except mt_merge) - Merged redundant expressions - Replaced blurs and medians with ExTools equivalents - Replaced Vinverse_avsi() with ex_vinverse() (HBD) and vinverse() (8bit) - Removed Dither support - Formatting and small optimizations I also want to post a list of all the median filters I could find. They happen to be much more than I thought, so I'm not including them all, although people can help out if interested to make a good library, despite some of them protected under a paywall or not possible with Expr expressions (recursions, etc) From all of them the ones I'm interested most are: IDBA, TSND, RAA, ADKI, and AUTMF. Convolution algorithms: Code:
# ATMF - Asymmetric Trimmed Median Filter # SMF - Switching Median Filter # WMF - Weighted Median Filter # CWMF - Center Weighted Median Filter # DWMF - Directional Weighted Median Filter # MDWMF - Modified Directional Weighted Median Filter # RWMF - Recursive Weighted Median Filter # PSMF - Progressive Switching Median Filter # SMF-BDND - Switching Median Filter with Boundary Discriminative Noise Detection # BDSMF - Boundary Discriminative Switching Median Filtering # DBF - Decision Based Filter # DBA - Decision Based Algorithm Median Filter # DBA2 - Decision Based Algorithm 2 Median Filter # MDBUTMF - Modified Decision Based Un-symmetric Trimmed Median Filter # DBUTMF - Decision Based Un-symmetric Trimmed Median Filter # ACWMF - Adaptive Center Weighted Median Filter # NID - New Impulse Detector # AMF - Adaptive Median Filter # FUF - Fuzzy Filter # UF - # ADKI - Adaptive Decision based Kriging Interpolation # IMF - Improved adaptive Median Filter ( Ha et al. (2016)) # IAMF - Improved Adaptive Median Filtering ( (Wang and Li, 2010)) # SAMF - Selective Adaptive Median Filter # MDBSMF - Multiple Decision Based Switching Median Filter # TSMF/TSF - Tri-State Median Filter # NTSMF - New Tri-State Switching Median Filter # NASMF - Noise Adaptive Soft-switching Median Filter # FRMF - Fuzzy Rule-based Median Filtering # RAA - Removal of salt and pepper an Adaptive Approach # IDBA - Improved decision based algorithm # HFFLND - Hybrid Filter and Fuzzy Logic Noise Detector # NSPF - New Salt and Pepper Filter. (Biswal and Bohi, 2013) # FEMF - Fast and Efficient Median Filter # AMF-LPD - Adaptive Median Filtering on Local Pixel Distribution # SSMF - Sorted Switching Median Filter # TSND - Two-Stage Noise Detector # IEMF - Improved Extremum and Median value Filter # ADPSMF - Absolute Difference based Progressive Switching Median Filter # IMFA - Improved Median Filtering Algorithm # MBDND - Modified Boundary Discriminative Noise Detection # MSSMF - Multistate Switching Median Filter # RVINE - Random Valued Impulse Noise Elimination # RVINENF - Random Valued Impulse Noise Elimination using Neural Filter # RVINENFF - Random Valued Impulse Noise Elimination using Neuro-Fuzzy Filter # EPF - Edge Preserving Filter # NBPPFT - Neural Based Post Processing Filtering Technique # FTPHR - Filtering Technique for Preserving Homogeneous Region # FTPHRAER - Filtering Technique for Preserving Homogeneous Region and Edge Region # FTHCDI - Filtering Technique for Highly Corrupted Digital Images # CNFT - Combined Neural based Filtering Technique # FFBPAMNEM1- Feed Forward Back Propagation Algorithm for Multiple Noise Elimination Method 1 # FFBPAMNEM2- Feed Forward Back Propagation Algorithm for Multiple Noise Elimination Method 2 # ANFIS - Adaptive Neuro-fuzzy Inference System # HNFF - Hybrid Neuro-Fuzzy Filter # TSHNFF1 - Two Stage Hybrid Neuro-Fuzzy Filter 1 # TSHNFF2 - Two Stage Hybrid Neuro-Fuzzy Filter 2 # MNENFF1 - Multiple Noise Elimination using Neuro-Fuzzy Filter 1 # MNENFF2 - Multiple Noise Elimination using Neuro-Fuzzy Filter 2 # INEDI1 - Intelligent Network for Enhancing Digital Images 1 # INEDI2 - Intelligent Network for Enhancing Digital Images 2 # BF - Bilateral Filter # FBF - Fuzzy Bilateral Filter # INR - Impulse Noise Reduction # FRINR - Fuzzy Random Impulse Noise Reduction # FRINRM - Fuzzy Random Impulse Noise Reduction Method # FIDRM - Fuzzy Impulse noise Detection and Reduction Method # FIDRMC - Fuzzy Impulse noise Detection and Reduction Method for Color Images # HFRMC - Histogram-based Fuzzy Restoration Method for Color Images # FISF - Fuzzy Inference System Filter # AUTMF - Adaptive Unsymmetrical Trim-Based Morphological Filter # MMEM - Minimum-Maximum Exclusive Mean Filter # DBA - Decision-Based Asymmetrical trimmed median filter # FRF - Fuzzy Rank-ordered Filter # LABSVMF - cieLAB Switching Vector Median Filter # FCG - Fuzzy Color preserving Gaussian filter # Non Linear (for RGB images) # VMF - Vector Median Filter # TVMF - Trimmed Vector Median Filter # EVMF - Extended Vector Median Filter # FMVMF - Fast Modified Vector Median Filter # FSVF - Fast Similarity Vector Filter # DVMF - Directional Vector Median Filter # GVMF - Generalized Vector Median Filter # RCVMF - Rank Conditioned Vector Median Filter # RCTVMF - Rank Conditioning and Threshold Vector Median Filter # CLMMF - Crossing Level Median Mean Filter # VDF - Vector Directional Filter # BVDF - Basic Vector Directional Filter # GVDF - Generalized Vector Directional Filter # DDF - Directional Distance Filter # WVMF - Weighted Vector Median Filter # TWVMF - Trimmed Weighted Vector Median Filter # EXWVMF - Extended Weighted Vector Median Filter # ROWVMF - Rank Order Weighted Vector Median Filter # WVDF - Weighted Vector Directional Filters # GA WVDF - Genetic Algorithm Weighted Vector Directional Filter # CWVMF - Center-Weighted Vector Median Filter # AVMF - Adaptive Vector Median filter # ABVDF - Adaptive Basic Vector Directional Filter # MSVMAF - Multiclass Support Vector Machine based Adaptive Filter # ATCC - Adaptive Threshold and Color Correction # RSVF/RSVMF- Robust Switching Vector Filter # AMMF - Adaptive Marginal Median filter # MSMF - Modified Switching Median Filter # RSDDF - Robust Switching Directional Distance filter # ACWVMF - Adaptive Center-Weighted Vector Median Filter # ARWSF - Adaptive rank weighted switching filter # PGA - Peer Group Averaging # PGSF - Peer Group Switching Filter # MPG - Modified Peer Group # PGVF - Peer Group Vector Filter # FPGF - Fast Peer Group Filters # NPGF - Novel Peer Group Filter # FWAF - Fuzzy Weighted Average Filter # FVMF - Fuzzy Vector Median Filter # FVDF - Fuzzy Vector Directional Filter # FOVF - Fuzzy Ordered Vector Filter # FHF - Fuzzy Hybrid Filter # ANNF - Adaptive Nearest-Neighbor Filter # ANNMF - Adaptive Nearest-Neighbor Multichannel Filter # AHMF - Adaptive Hybrid Multichannel Filter # FIDRM - Fuzzy Impulse Detection and Reduction Method # VMRHF - Vector Median Rational Hybrid filter # SAHVF - Structure-Adaptive Hybrid Vector Filter # SWAV - Structure Weighted Average Filter # SVMF - Sigma Vector Median Filter # SBVDF - Sigma Basic Vector Directional Filter # SDDF - Sigma Directional Distance Filter # ASVMF - Adaptive Sigma Vector Median Filter # ASBVDF - Adaptive Sigma Basic Vector Directional Filter # ASDDF - Adaptive Sigma Directional Distance Filter # ASVMF - Adaptive Sigma Vector Median Filter # EVMF - Entropy Vector Median Filter # QSF - Quaternion Switching Filter # QVMF - Quaternion Vector Median Filter # QSVMF - Quaternion Switching Vector Median Filter
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
30th July 2021, 10:49 | #139 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
ExTools updated to v4.5.
-ex_median() ported spatio-temporal DeGrainMedian modes -ex_repair() ported repair(16) Logo updated to v12.0. For some reason I couldn't load DeGrainMedian with MP_Pipeline win32 mode so I couldn't double check if the output matched, nor benchmark it against. The filter is slowish but I'm not sure how to optimize it further. I also wanted to port GrapeSmooth but closed source it seems. I'm also still porting Adaptive Sharpen, I came to a little halt so may ask later for some help.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
30th July 2021, 11:56 | #140 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,597
|
__________________
See My Avisynth Stuff |
Tags |
avisynth, dogway, filters, hbd, packs |
Thread Tools | Search this Thread |
Display Modes | |
|
|