Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
30th November 2021, 17:50 | #661 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Is there a special scene or frame you have issues with? I tested with the Carrie call (I use tr 1 and 2 and thSAD=400 for the 2nd call) and found it good enough.
Also keep in mind this is a BW film so plane=0 and chroma=false. The film shows "fizz" grain. It's in between the Carrie example and something stronger like 300. I found that a blksize of 32 is good to avoid catching these kind of big grains, along refinemotion this is a good approach. Filters like SPresso or STTWM are not suited here because they only filter minimal change noise, these are more for DVDs or faint grain. My general approach with strong grain is a rather strong prefilter (with LFR), and a rather not so strong filter without LFR. Personally for my encodes I don't like the clean look because they either look psychovisually blurry or sometimes it shows the "screen" effect, like grain stuck over a glass, so I apply some artificial grain after denoising.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 30th November 2021 at 17:59. |
30th November 2021, 21:13 | #662 | Link | |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Quote:
The best results are from SPresso+STPresso+SMDegrain but the speed is in the order of 0.5 fpsL too slow. I am eager to see your porting of STPresso if it could achieve better speeds and the same quality. Plus, I have found that, strangely, if you apply plane=0 and chroma=false, you obtain a lower noise cleanup.
__________________
@turment on Telegram |
|
1st December 2021, 09:29 | #663 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,804
|
BW movies are often very grainy. Also, chroma information is a much smaller portion of data than luma.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
1st December 2021, 09:33 | #664 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Results vary by the "type" of grain, i.e. silver particle size, distribution, etc. Some BW movies are really easy to compress and you can achieve really high compression ratio.
The strange thing is that using plane=0 and chroma=false, lowers the noise reduction, even using Greyscale() to cleanup chroma completely. That's a thing I will never understand.
__________________
@turment on Telegram |
1st December 2021, 11:25 | #665 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
The chroma plane which is plain grey will be considered for the SAD calculation so it lowers the overall SAD value, moreover when I use a scaleCSAD of 1 for HD sources. Chroma planes are always greyish so the difference is not much but it might be noticeable. You have to counteract this with a higher thSAD.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
2nd December 2021, 17:46 | #666 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
@dogway
Do you see anything wrong with SMDegrain (tr=6,PreFilter=4,thSAD=900,contrasharp=false,refinemotion=true, plane=0, chroma=false, Globals=2) SMDegrain (tr=6, thSAD=900,contrasharp=false,refinemotion=true, plane=0, chroma=false, Globals=1) instead of using higher tr and thSAD? Which choice is, quality wise, better?
__________________
@turment on Telegram |
2nd December 2021, 18:56 | #667 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Ah, I noticed some issues, well Globals is something I haven't touched in a long time.
I recommend you for the time being to not use Globals. Other than that I see the output is very clean, I'm not accustomed to using such high values, maybe you can add some LFR recovery. Maybe the first pass can use blksize of 32 to speed up processing, but this depends on what details are being wiped or not, it depend on grain amount. Using two passes is fine, it indirectly multiplies the effect for a fraction of the performance. (-6) -6 0 6 (6) = 24 frames for the current frame output.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
2nd December 2021, 20:32 | #668 | Link | ||
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Quote:
So should I use 2 lines of SMDegrain without passing vectors? I just can see some issues right before scene change, where details get somehow lost. Quote:
Do you suggest me to put LFR in the first line, in the second or both? 12 angry men, which I sent you a clip of, is really a bloody mess. With x264 crf 20, I am sweating to keep bitrate unter 4000kb/s and for a BW movie it is really a lot.
__________________
@turment on Telegram Last edited by tormento; 2nd December 2021 at 20:47. |
||
3rd December 2021, 08:53 | #669 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
I already fixed the issue with Globals. There are already many changes for new SMDegrain, I was waiting to add RGB support and cumulative YPlaneMin/YPlaneMax to ex_retinex but I lost part of that work plus I was having issues with runtime variables readback.
I only need to finish an issue with UHDhalf prefiltering (switched position so prefiltering happens on dowscaled version) and I do the release. Next v3.4.0 is a good candidate for long stable release. Running 2 passes is a good idea for several reasons, as you might know 2 mean averages approximates a binomial weight: ex_boxblur(3,mode="weighted") <-> ex_boxblur(1,mode="mean").ex_boxblur(1,mode="mean") This is called tent blur and in the context of motion vectors this is more performant. Besides you calculate new vectors from a cleaner source so its a finer refinement of MV. For MDegrain this is like running a median twice, so it approximates a weighted median. Together you perform a stronger denoising but with stronger center weighting, in other words better protection and speed than a single call with higher 'tr'. With Globals this is not possible because you are reusing the "dirty" Globals of the first pass (the output Globals of the first prefiltering) and the only benefit is the weighted median. (In contrast using the first pass as a prefiltering only you do refine the MV but don't perform the weighted median) Globals was designed when combining filters that use MV in rather clean sources, for example for temporal limiting, frame interpolation, etc. I do like old films, actually I hate new digital recorded movies but lately I have been only developing rather than encoding. If you use such high thSAD or tr, LFR and DCTFlicker by extensions are a must. By Didée words grain is a high frequency only property but DCT encoding propagates this to low frequency. 'DCTFlicker' is not perfect so I recommend running both at first pass and lower thSAD and tr for the second pass to don't destroy what was recovered. You can optimize x264 bitrate by using preset 'slower' or 'very slow'. Using 'zones' for the opening and credits. Also using a range limiter in avisynth. And a light blur where motion vectors fail (the 'mfilter' arg)
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 3rd December 2021 at 09:05. |
3rd December 2021, 10:16 | #670 | Link | |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Quote:
I am using slow preset as my CPU is the real bottleneck of my computer. What about introducing a "pass" or "recursion" parameter, to internally apply the consecutive denoising? Perhaps it could lead to more speed and less roundings.
__________________
@turment on Telegram Last edited by tormento; 3rd December 2021 at 10:18. |
|
3rd December 2021, 16:52 | #671 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Ok, I just uploaded a new version. It needs latest avisynth, test32 I think. By the way I included a new prefilter, ex_FluxSmoothST(), this is MinBlur + FluxSmoothT, which share the same concept but one is spatial and the other temporal. So now DFTTest is 4, KNLMeanCL 5 and BM3D 6.
I'm reworking the whole packs to be numerically stable and fast by using the latest updates by pinterf (we should build him a monument or at least a xmas present :D ) You might notice some speed improvements when I update ExTools, currently I'm finishing QC but it might take more days to be sure I don't screw it. I mainly have to assure that the new f32 scale_inputs works fine across the board along the new range_max constant (as full scale). ex_retinex() needs more work to include cumulative stats, they will be in next version. Do you mean recursion for MDegrain only (not MV)? That's possible. I will run tests to see how it improves things. It fits in the scope of current recursion solution by prepending an "r", so it would work as mode="rMDegrain". EDIT: sneakingly added recursion to MDegrain, test it out to find if it fills your needs. I might give it a revision for the next version, as Didée was doing something similar back then.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 3rd December 2021 at 17:18. |
3rd December 2021, 18:39 | #672 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
I'd prefer to have an added number than to change years of habits.
I meant a way to replicate 2 calls of SMDegrain consecutively but internally and optimized, as we were talking about the opportunity to have 2 calls with minor THSAD instead of 1 only with bigger one.
__________________
@turment on Telegram |
6th December 2021, 12:02 | #673 | Link | |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Quote:
I can't see any parameter to add but only a check about the "r" letter on MDegrain and I can't understand how to make it work.
__________________
@turment on Telegram |
|
6th December 2021, 13:32 | #674 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Yes, declare mode="rMDegrain" in the call and it will use recursion, but only for the MDegrain part not the vectors. As for to replicate 2 consecutive calls, I think it's best to do it explicitly as currently is done, I prefer simple things that can be seen and there wouldn't be speed gain at all (you only save a single TV->PC range).
Bu the way I'm reworking ex_gaussianblur() (used in LFR and ex_retinex()) as it was designed with a single resolution size, but it changes depending on resolution so I had to make a multi-regression fit. The good part is that it will fit even better than before. I will release ExTools in 2 days.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
6th December 2021, 14:46 | #675 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
I hoped to save some video memory, as the dual line recursion eats it like peanuts and I have to increase the number of threads: AVS doesn't like 2xSMDegrain so much and sits idle for the most of machine time.
__________________
@turment on Telegram |
6th December 2021, 15:21 | #676 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
Well yes, that's an issue on mvtools, running manalyze several times has a great impact on performance, that's why among other things I don't like creating more MV in scripts like STPresso. DCFlicker uses it too, so it's a bit on the heavy size.
Back in the time I used to split the first pass of SMDegrain into a lossless file and then run the second call on this, this usually was faster than in a single script. MPPipeline also helped on this regard. Now for HD I don't see lossless being an option, maybe pseudo-lossless can cut it, a x264 profile at 12-bit with pseudo-lossless settings.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
6th December 2021, 17:20 | #677 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
x264 CRF 0 is of great use but unfortunately Nvidia can read 8 bit AVC 4:2:0 only, leaving me with the option to index it with some software only. HEVC is out of question, as it is too much CPU intensive.
__________________
@turment on Telegram |
11th December 2021, 14:56 | #678 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,367
|
I have been making a SAD sampler for the last 2 days without success, so I will leave this here in case someone sees an issue.
Code:
# ex_Luma_Rebuild(s0=6) # SAD_sampler(show=2) ## Dependencies: ## ExTools ## FredAverage ## ResizersPack ## Grade (for show) ## MasksPack (for show) ## TransformsPack (for show) # show: 0: no debug, 1: show sampled area 2: show sampled area in diff mode function SAD_sampler(clip src, int "show") { show = Default(show, 0) mmod(src,128,mode="crop") b=ConvertBits(8, dither=-1).ExtractY() w = width() h = height() davg=0 grey=0 id = 0 fr = 0 # x = 1 y = 1 OK = false while (!OK) { for (x = 1, w/128, 1) { stridex = 128*x for (y = 1, h/128, 1) { stridey = 128*y c = b.Crop(stridex-128,stridey-128,-w+stridex,-h+stridey) a = c.FredAverage() d = c.TemporalSoften(10,255,0,30,2) # e = c.ex_edge("frei-chen",0,255) diff = ex_makediff(a, d, aug=5, metric="none", UV=128) # diff = ex_logic(diff, e, "max", UV=128) ScriptClip(diff, function[a,d,x,y,c,show] () { davg = (AverageLuma(last,-5)+AverageLuma(last)+AverageLuma(last,+5))/3. grey = (AverageLuma(a, -5)+AverageLuma(a) +AverageLuma(a, +5))/3. OK = (davg < 40.) && (90. < grey < 126.) d = show > 1 ? last : show > 0 ? c : c OK ? propSet(d,"_x_id",-1,0).propSet( "_y_id",-1,0).propSet("_diff",davg,0).propSet("_grey",grey,0) : \ propSet(d,"_x_id", x,0).propSet( "_y_id", y,0).propSet("_diff",davg,0).propSet("_grey",grey,0) } ) yy = propGetInt("_y_id") xx = propGetInt("_x_id") EOL = (w+h)/128. t1 = id/EOL == 1.0 t2 = id/EOL == 2.0 t3 = id/EOL == 3.0 t4 = id/EOL == 4.0 fr = t4 ? fr+24*3 : fr b = t1 ? b.PadResize(w,h,biasw=64,biash=0) : \ t2 ? b.PadResize(w,h,biasw=0, biash=64) : \ t3 ? b.PadResize(w,h,biasw=-64,biash=0) : \ t4 ? b.PadResize(w,h,biasw=0,biash=-64).selectevery(1,fr) : b if (show > 0 && yy<0) { grey = round(propGetFloat("_grey")) davg = round(propGetFloat("_diff")) src = b.subtitle(Format("id:{id} fr:{fr}\nx:{x} y:{y}\ng:{grey} d:{davg}"),lsp=1) bg = BoxMask(src,x*128-128,x*128,y*128-128,y*128,invert=false,matchformat=true) ex_blend(src,bg,"multiply",opacity=0.3) } x = yy<0 ? w/128 : t1 || t2 || t3 || t4 ? 1 : xx y = yy<0 ? h/128 : t1 || t2 || t3 || t4 ? 1 : yy id = yy == h/128 ? id : id + 1 id = t4 ? 0 : id OK = yy<0 } } OK ? last : Undefined() } I will continue with last fixes for ExTools, SharpenersPack and SMDegrain.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 11th December 2021 at 15:04. |
14th December 2021, 00:05 | #679 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 720
|
Quote:
EDIT: also smdegrain at line 61. Last edited by anton_foy; 14th December 2021 at 00:17. |
|
14th December 2021, 00:44 | #680 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,823
|
Trying ResizersPack-7.1~Dogway.avsi
Script error: Invalid arguments to function 'width'. (D:/Programmi/Media/AviSynth+/plugins64/ResizersPack-7.1~Dogway.avsi, line 171) (D:/Programmi/Media/AviSynth+/plugins64/ResizersPack-7.1~Dogway.avsi, line 150) (D:/Programmi/Media/AviSynth+/plugins64/TransformsPack-1.0rc29~Dogway.avsi, line 136) (D:/Programmi/Media/AviSynth+/plugins64/TransformsPack-1.0rc29~Dogway.avsi, line 145) (D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.9d~Dogway.avsi, line 914) (D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.9d~Dogway.avsi, line 917) (D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.9d~Dogway.avsi, line 228)
__________________
@turment on Telegram |
Tags |
avisynth, dogway, filters, hbd, packs |
Thread Tools | Search this Thread |
Display Modes | |
|
|