Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th January 2011, 02:51 | #1 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
STAMPEDE : My new Degraining filter
STAMPEDE is my latest filter, and it is my favorite by far.
STAMPEDE stands for Spacio Temporal And Masked Precision Eludes Damage Exquisitely It uses a threshold mask that is created by the difference of two filters the first is a spacial smoother and the second is a spaciotemporal filter it is actually the same filter but it is told to process 2 frames instead of 1 (reducefluctuations with repair to reduce the chance of blending) It takes the difference of those two processed frames, and uses that as the threshold. Another mask is created to define what the general outline of the frame looks like, and that part is not filtered. The reason that it takes the difference and not the average, is because if you take the difference, between 1 spatially filtered frame and then a spaciotemporally filtered frame, the spaciotemporal filter will remove fluctuating noise and grain, and the spatial filter will remove some small noise on its own. So by using the difference of the two filters, certain things like noise, but also some detail might be removed in the spatial filtering step, but that detail will also be removed in the spatial part of the spaciotemporal step, meaning that the difference will be the temporal noise also known as fluctuating noise or grain, basically creating a mask for temporal noise which in itself seems pretty impossible but the end result is that the filter does not target details as details do (read should) not fluctuate (usually) Now that all sounds find and dandy but to give a visual representation it might make more sense. So let me do just that. Here is a comparison of the filtered to the mask, the black parts are not filtered, also on the first picture and the second are pretty much duplicates. That gives the filter the maximum accuracy and power. http://screenshotcomparison.com/comparison/20086 Here is the plethora of examples I have taken http://screenshotcomparison.com/comparison/16978 http://screenshotcomparison.com/comparison/16979 http://screenshotcomparison.com/comparison/16980 http://screenshotcomparison.com/comparison/16981 http://screenshotcomparison.com/comparison/16982 http://screenshotcomparison.com/comparison/16985 http://screenshotcomparison.com/comparison/16988 http://screenshotcomparison.com/comparison/16989 http://screenshotcomparison.com/comparison/16992 http://screenshotcomparison.com/comparison/16993 http://screenshotcomparison.com/comparison/16995 http://screenshotcomparison.com/comparison/16996 http://screenshotcomparison.com/comparison/20105 and here are a couple of live action comparisons I took http://screenshotcomparison.com/comparison/20314 Notes about usage I created this filter by modding and changing my sharpening filter, which was a mod of SSSharpfaster which itself is a mod of SuperSlowSharpen. I will probably make a post about my sharpener later , here is an example of that The purpose of this filter is not for denoising or smoothing, it is solely meant for the reduction of grain and artifacts. It tends to help with compression by making the video easier for x264 to encode, as x264 does not really like high level noise or grain. On a couple of tests I did with compression, with Nausicaa I was able to reduce the file size by 23% with little to no decrease in quality (as seen in the screenshots above). This is not a filter you should use for every source, as you will end up being disappointed by the results, it works best on HD content, but does work on SD content too. Luckly it is pretty damn fast due to a lot of optimizing I have done. Do not use on clean sources IE ones that lack grain or noise because you will either not gain much compression or have issues with the mask not having enough to accurately be generated. Now for the download, I included all of the required filters (hopefully) I might of forgotten one or two, but I don't think so http://www.mediafire.com/download.php?ze19lqi99echc8p Feedback and input is greatly appreciated, any ideas to improve the filter would also be greatly appreciated
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 16th January 2011 at 22:35. |
16th January 2011, 06:02 | #2 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,352
|
Before completely reading your post, yes, I thought it was good for more efficient encodings, very subtle effect but most noticeable on HD sources with nervious grain. I must be the only one, but IMHO I like to clean (and watch) anime just like if they were the original cels (reads noise free flat colors), specially on old ones. Needless to say I appreciate the contribution since filtering in 1080p can become a nightmare of slowness/space : D
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
16th January 2011, 09:34 | #3 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
No problem and thankyou for the feedback
I know exactly how you feel about wanting to clean the anime, and yea filtering BD shit is a pain in the ass , that is part of the reason I have been making filters for HD in mind, because I can't really find any that suit my needs This filter is best when used as a prefilter by removing the nervous grain so that you can use other filters to smooth the video
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 16th January 2011 at 09:39. |
16th January 2011, 16:15 | #5 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
It would be fun to comment on this filter. Someone untie me?
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
16th January 2011, 22:14 | #6 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
@Gser
I have not tried it on any live action stuff, I will give it a shot on some live action things I have in a bit Edit: I add some comparisons from a BD encode I had of Memento http://screenshotcomparison.com/comparison/20314 The results are pretty damn good, much better than I expected them to be @Didee *unties* Comment away! I am very interested in what you think of this filter :3
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 16th January 2011 at 22:37. |
16th January 2011, 22:35 | #7 | Link |
LaTo INV.
Join Date: Jun 2007
Location: France
Posts: 701
|
Yes, this filter is suboptimal, here is an updated version:
Code:
function STAMPEDE2(clip c, float "strength", int "hthr", int "hbias", int "limit1", int "limit2", int"ss") { #Spacio Temporal And Masked Precision Eludes Damage Exquisitely II #Modded version for more efficiency and speed #Do not touch these values unless you have done graduate work in math limit1 = default(limit1, 1) limit2 = default(limit2, 2) ss = default(ss, 2) hthr = default(hthr, 256) hbias = default(hbias, -128) strength = default(strength, 3.0) #WARNING: do not touch the code below or you will break everything in the filter w = c.width() h = c.height() s = c.spline36resize((w*ss)/(limit2-limit1),(h*ss)/(limit2-limit1)) a = s.awarpsharp(cm=0, depth=3, blurlevel=1, thresh=0.99) b = s.removegrain(-1,0,0) MT_merge( mt_lutxy(s,a,Uexpr="x x - "+string(strength/3.0)+" *",U=1,V=1), mt_lutxy(a,s,Vexpr="x x - "+string(strength/3.0)+" *",U=1,V=1), blankclip(b), U=3, V=3 ) return(last.spline64resize((-2*w*hbias)/(htr),(-2*h*hbias)/(htr)).mergechroma(c)) } |
16th January 2011, 22:49 | #8 | Link | |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Ooh Lato let me take a look
Edit: Lato I think you meant this Quote:
Ok so Lato, it seems your version of the filter does even less than mine http://screenshotcomparison.com/comparison/20328 http://screenshotcomparison.com/comparison/20329 No idea about how the speed increased , but it seems for such minimal results (and I mean the original was pretty minimal) there really is no benefit
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 16th January 2011 at 23:14. |
|
16th January 2011, 23:24 | #9 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
@LaTo: oh, how you mean you are!
The basic idea is to combine a spatial and a spatio-temporal filter, so that the spatio-temporal does not do what the spatial filter would do. Or s'th similar. This script does not do that. What it does is to perform the SAME spatio-temporal filter two times. One time the temporal filter (recursively-calmed temporal median) is limited to a maximum change of +/-1, then comes spatial limiting. The other one is the same, just the temporal filter is limited to +/-2 before spatial limiting. Later in the script, what happens is that in those places where the 2nd filter does more than the 1st filter (i.e. where the 2nd causes a +/-2 instead of a +/-1), the difference is blown up to +/-3. This is a) not what the synopsis claims, and b) this is not a generally valid operation. You can't (safely) blow a filter effect to >100%. (E.g.: if you take blur(1) and blow the effect to 300%, you'll only partly get "more blurring", but get also lots of "inverse haloing". Simply because the operation is not valid to do, in regard to "correct" filtering.) Funny nitpicks from the script: Code:
s.removegrain(0,-1,-1).srf(limit2) Code:
mt_lutxy(s, a, yexpr="y x x - "+string(strength/3.0)+" * +",U=1,V=1) In essence, the script does several "highly questionable" operations. Some will reduce noise, some will introduce artifacts out of thin air. The only thing that keeps things from falling apart is that the script works with "very small" differences from the start. Because of this, in the result it is not immediately obvious that the whole script is a botch. Here's a mini-script in the spirit of the original idea (protect a temporal filter to act on those bits that a spatial filter would act on) Code:
a = last b = a.RemoveGrain(11) f = b.FluxSmoothT().merge(b,0.49) a.mt_makediff(mt_makediff(b,f,U=3,V=3),U=3,V=3)
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
16th January 2011, 23:32 | #10 | Link | |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Didee you are right about most of what you said, there are issues like when you raise the strength to above 6 or so you start creating noise. I think that mostly has to do with my relative ineptitude with regards to scripting and full syntax. I pretty much only had an idea, and a general understanding of how the original script worked. I just adapted it from that. BTW for reference I adapted it from this script that I created.
Quote:
I really would like input and/or suggestions for ways to improve the filter and fix the parts that are useless/broken
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 17th January 2011 at 00:15. |
|
17th January 2011, 09:22 | #11 | Link | ||
LaTo INV.
Join Date: Jun 2007
Location: France
Posts: 701
|
Quote:
mt_lut/mt_lutxy/... are written in reverse polish notation, look at this line for example: Code:
mt_lutxy(s, a, yexpr="y x x - "+string(strength/3.0)+" * +",U=1,V=1) x is the first clip "s" y is the second clip "a" So: (x-x)*(strength/3.0) will always be equal to 0, the final is always "y" and the result is the clip "a". U=1/V=1 are set, so chroma will be garbage. Quote:
There are full details about scripting here: Scripting Reference and here: Avisynth Synthax Having understood this, you can read the documentation of filters. For example: Masktools and RemoveGrain I hope this will help you, good luck for your work! (PS: my modded version was a little joke, do not start from it to continue your script) |
||
17th January 2011, 09:56 | #12 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
lol Lato do not worry, it took me a bit because I did not actually read what you put in the filter, I just sort of saved it to avsi then took screens, but once I actually looked at it I got that it was a joke and there was no way I would just blindly work off of your filter lol
I will definitely give those links a look I am still sort of hoping that Didee or you will be able to salvage my script, of course I will be taking a look at redoing it too , but it would be nice to have experienced people giving input as well.
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
17th January 2011, 10:41 | #13 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Salvaging ... sure, but in which direction? My problem is the rationale behind, which is not quite clear to me.
If I got everything right, the current outline is "to combine a spatial filter with a spatio-temporal filter so that the spatio-temporal filter does not do those changes that the spatial filter would do on its own". Is that correct? If yes, then the next question is: why first use a spatio-temporal filter, then jump through loops in order to eliminate the spatial part out of it ..... instead of simply using a pure temporal filter from the start, without any spatial component? Summed up: I'm a bit lost with "here's my script, can you salvage it?" The core question is: what exactly should the script do? And also, "why" is it meaningful to do so? (2nd question aims at: creating a filter that "makes sense".)
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
17th January 2011, 22:50 | #14 | Link | |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Didee I did not really mean to ask you guys to salvage it, I just hoped that you would be able to. Also the point of the script is as I described in the synopsis
Quote:
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 18th January 2011 at 00:58. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|