Dogway's Filters Packs

[Dogway]

The performance to quality ratio is going to drop severely when raising the range (and not necessarily leads to better quality), there's no need here since we already have RefineMotion. The exact values were derived from Zopti optimized pareto fronts, you can read some of it here (https://forum.doom9.org/showthread.php?t=175723&page=14) but most of my tests aren't published.

[DTL]

For x264, the 'merange' defaults to around 16, and can be scaled up for (U)HD footage to values like 24, 32, etc.

Given this, using range values of 1, 2, 5, etc seem extremely small when using search=4.

MAnalyse use hierarchical search with many levels of details (each level is 2x smaller). So searchparam is the search range at each level. It is not directly equal to x264 search range. So with MAnalyse search range is about equal to x264 if limit levels to 1 and may be use pel=1. It _may_ be sometime useful to save from too long bad MVs (like at no detail noise only areas).

Typically search radius =2 at each level give good enough result and increasing it mostly drops performance with may be very low quality gain.

[ENunn]

Ok, I will update soon, meanwhile edit the first CombinePlanes line with:
a4 = CombinePlanes(y.BicubicResize(w/2, is422(a) ? h : h/2, -0.5, 0.3), a, planes="YUV", pixel_type="YUV444P"+string(bi==32 ? "S" : bi))

Then you have to preconvert the 422 to planar and call with CCD():
ConvertToYUV422()
CCD()

Try to play a bit with the CCD() values so the effect scales to your artifacts.

Thanks for your help! This worked!

You want fast like Directshowsource?? But it't almost error accurate frame and you're greed. I think more fast more cost error. Why you don't still waiting CPU can support it to below 1s in Index :D

I'm not expecting miracles but when you're dealing with 6-8 hour long tape transfers it can get really slow. Wish there was something as good as lwlibav but faster and supported FFV1.

Yuck to DirectShowSource

[Boulder]

I'm not expecting miracles but when you're dealing with 6-8 hour long tape transfers it can get really slow. Wish there was something as good as lwlibav but faster and supported FFV1.

Are you using an SSD or a regular HDD? I'd expect the hard drive read performance playing a big part if the file is big.

[ENunn]

Are you using an SSD or a regular HDD? I'd expect the hard drive read performance playing a big part if the file is big.

HDD's, don't got the space to put them on my SSD sadly. File sizes can be 150-180gb at most.

[StainlessS]

Well if its that slow, why not take a nap, take dog for a walk, or go feed the ducks.
You dont have to sit and time it. :)
(I've just ordered a 1TB 2.5" sata SSD for £61.00, prices are coming down) :- https://www.amazon.co.uk/dp/B07YD579WM/ref=pe_27063361_485629781_TE_item_image

EDIT: Or here a 1TB Crucial P2 nvme @ £60.00:- https://www.amazon.co.uk/Crucial-CT1000P2SSD8-Internal-2400-NAND/dp/B089DNM8LR/ref=sr_1_2?crid=37HYTNISNX7GY&keywords=crucial+p2&qid=1667418761&qu=eyJxc2MiOiIyLjMyIiwicXNhIjoiMS4zNyIsInFzcCI6IjEuMDkifQ%3D%3D&s=computers&sprefix=crucial+p2%2Ccomputers%2C139&sr=1-2

I've got a P1 (not quite as good as P2) and two P5 PLUS [4th gen PCI-e nvme], which may be a bit overkill on 3rd gen PCI-e as you only get half gen 4 speed.

Here a REALLY GOOD report on nvme SSD, and which to buy [EDIT:2019]:- https://www.youtube.com/watch?v=GlGjd1GZWdo
For most people it dont matter much which one you buy, so long as you have one.

I only bought 4th gen nvme crucial P5 Plus, in hope that I would one day have 4gen PCI-e, and just yesterday I ordered a
(2nd hand) Lenovo M70Q Gen 2 mini /i5-11400T/8GB Ram/256GB SSD/W10/B :- https://uk.webuy.com/product-detail/?id=sdeslenm70q96b&categoryName=desktops-windows&superCatName=computing&title=lenovo-m70q-gen-2-mini-i5-11400t-8gb-ram-256gb-ssd-w10-b&referredFrom=search&queryID=a4265986dbfa8ba48634a9fcbd5f6090&position=1
which does have 4th gen PCI-e, and also AVX512 which really persuaded me to get it. (its probably a bit slow due to being a tiny little thing and low clock speed),
but I also wanted to try out the AVX512 when it becomes commonplace.

Lenovo M70Q Gen 2 mini /i5-11400T: Tiny little thing (about 7.5" square by 1.75 inches)
https://www.amazon.co.uk/Lenovo-ThinkCentre-M70q-Intel-Windows/dp/B09M43H7X1/ref=sr_1_1?crid=20BRERMF1VSWV&keywords=Lenovo+M70Q+Gen+2+mini&qid=1667419550&qu=eyJxc2MiOiIxLjUxIiwicXNhIjoiMC4wMCIsInFzcCI6IjAuMDAifQ%3D%3D&sprefix=lenovo+m70q+gen+2+mini%2Caps%2C300&sr=8-1
EDIT: I just had a little bit of panic, the amazon link says i5-11400T is "processor count = 2" whereas it is actually 6 core / 12 thread,
also says "Processor Speed = 2.1GHz" whereas it is "Processor Base Frequency = 1.3GHz - burst to 3.7GHz", so panic over.
Intel doc on i5-11400T:- https://www.intel.co.uk/content/www/uk/en/products/sku/212273/intel-core-i511400t-processor-12m-cache-up-to-3-70-ghz/specifications.html


The 1TB SATA SSD and one of the 1TB 4th gen Crucial P5 Plus's is going into the Lenova tiny little thing, and also gonna add either 8 or 16GB RAM to the existing 8GB.

DO WATCH the youtube report, it'll defo help make up your mind about nvme's [gen 4 PCI-e was not 'a thing' when youtube report was created - 2019].

EDIT:
Here an update to the above Youtube report [2022], the same guy [he's VERY GOOD]:- https://www.youtube.com/watch?v=9PA0DrV_kiw
I have not watched it yet, only just found it and downloading as we speak.
Suggest that you ALL watch both youtube videos.

EDIT: The first Youtube is longer in-depth review of about 7 or 8 3rd gen nvme's is the better one, 2nd youtube clip is much shorter
and not so interesting but also worth a watch.

EDIT: Got the tiny little thing, a bit pissed off that PCI-e gen 4.0 only used for graphics on the Lenovo M70q Tiny Gen 2, nvme uses Gen 3.0 Pci-e,
you need the M70q Gen 3 (with 12th gen processor) to get PCI-e 4.0 for nvme. Waste of time and cash putting the crucial P5 Plus in it :(

[anton_foy]

Is there a way to make ex_edge motion sensitive? What I mean is that atm it finds detail/edges too much where it is higher motion and motionblur.

[Dogway]

Is there a way to make ex_edge motion sensitive? What I mean is that atm it finds detail/edges too much where it is higher motion and motionblur.

Sure, raise the 'lo' cutoff. And use it with a HQ kernel like frei-chen. If that's still picking too much you might want to prefilter, or mask with a spatial or motion mask.

[anton_foy]

Sure, raise the 'lo' cutoff. And use it with a HQ kernel like frei-chen. If that's still picking too much you might want to prefilter, or mask with a spatial or motion mask.

Great thanks!

[madey83]

Sure, raise the 'lo' cutoff. And use it with a HQ kernel like frei-chen. If that's still picking too much you might want to prefilter, or mask with a spatial or motion mask.

@Dogway

could you please explain what is benefit of using this and when it could be used?

:thanks:

[tormento]

I have an anime with mixed 23.976p, 29.97p and 60i (on purpose not because of laziness, perhaps to maintain details?) with 29.97p video tagging. I'd like to give your QTGMC+ a try to port everything to 23.976p.

Do you have some hints not to change video duration and have a good result with 60i? In the 60i part (CGI) every field has different details, so it would be a bit hard to have proper deinterlacing without flickering.

P.S.: How is possible that I have Scenes Pack v4.4 (24-09-2022) when on GitHub there is version 4.2? xD

[guest]

P.S.: How is possible that I have Scenes Pack v4.4 (24-09-2022) when on GitHub there is version 4.2? xD

No, it's v4.4 (24-09-2022)

https://github.com/Dogway/Avisynth-Scripts/blob/master/ScenesPack.avsi#L4

[Dogway]

Do you have some hints not to change video duration and have a good result with 60i? In the 60i part (CGI) every field has different details, so it would be a bit hard to have proper deinterlacing without flickering.

You can deinterlace to double rate (with option for flicker reduction), then decimate to 23.976 and interpolate where appropriate.

To expand a little:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
0 I 2 I 4 I 6 I 8 I 10 I 12 I 14 I 16 I 18 I 20 I 22 I 24


So deinterlace to 60 progressive, then interpolate to 120fps, and then pick every 5 frames.

QTGMCp()
FrameInterpolation
SelectEvery(5,0)

[tormento]

You can deinterlace to double rate (with option for flicker reduction), then decimate to 23.976 and interpolate where appropriate.
Thanks! Simply creating the table needed some effort! :cool:

You mean, bring everything to 120p or just the 60i part?

Can you be more clear about function and your suggestions about parameters?

I am a complete noob about fields and related :p

[tormento]

What (https://www.youtube.com/watch?v=uFpUJ3cHkNY) I'd like to achieve.

[Dogway]

Yes, I can see the aircraft at 1:00 being interpolated really badly. That won't happen with my above method.

You want everything at 23.976 constant framerate, right. Trim out the 60i portion, apply the above and chain it back to the IVTCed (or whatever) 23.976 anime content.

I personally like a sense of craftsmanship on what I see, the kinds of waifu upscales without hallucination... I don't like.

[tormento]

You want everything at 23.976 constant framerate, right. Trim out the 60i portion, apply the above and chain it back to the IVTCed (or whatever) 23.976 anime content.
That's the easiest way to achieve something good but I should have the frame ranges of the fps changings. Anything useful to do it automatically?

I saw that using TIVTC is a hit and miss to get those variations.

[Dogway]

Automatically I don't know, maybe AnimeIVTC but I never used it, I never did episodes only OVA/Films so I did manually. An alternative is to export scene changes to a log, maybe with ScenesPack, convert that to bookmarks, and F4 iteratively to go through scene changes so you analyze the content much faster.

[tormento]

Automatically I don't know
Is there any other way for AVS+ to work differently based on current fps? I have thought about treating every fps differently.

What do you think about?

[Dogway]

If you mean VFR, I have no experience with that.

[Ryushin]

I've been using SMDegrain with RipBot264 using Pauly Dunne's Tools folder that is using Dogway's filters.

I have a 4K source that I'm trying to use blksize=4 on to try minimize some banding/blotching but it seems to be erroring out. I had good success with blksize=8 before on another source that I had that really helped minimize the blotching that I was seeing. Setting blksize=8 works fine, along with leaving the default of 16.

My SMDegrain script for RB:
#Custom
LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\MedianBlur2\MedianBlur2.dll")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\LSF-PLUS\LSFplus.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\RESIZERS-PACK\Resizers Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\SHARPENERS-PACK\Sharpeners Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\SMDEGRAIN\SMDegrain cpu.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\Zs_RF_Shared\Zs_RF_Shared.avs")
Video=SMDegrain(video,tr=4,thSAD=400,thSADC=200,contrasharp=true,prefilter=1,refinemotion=true,blksize=4)

Is there something I'm missing in order for blksize=4 to work?

[Boulder]

Set refinemotion=false?

[Ryushin]

Set refinemotion=false?

Afraid that did not work. Odd. Everything I read said blksize=4 should work.

[Boulder]

The exact error message would definitely be very helpful ;)

[DTL]

4*4 in theory can be refined to 2*2 even with 4:2:0 chroma subsampling but overlap may not work. Do overlap is default enabled ?
Disabling overlap may additionally damage quality so to process with blocksize 2*2 and overlap it may be need feed 4:4:4 to MDegrain.
Other possible solution of 'supersampling' is to upsize 2x before degrain and use blocksize=8 and downsize 2x after.

[kedautinh12]

The exact error message would definitely be very helpful ;)

I think no error messages cause ripbot264 don't show it. Do you remember TDS only show scripts with ripbot264 with no error messages?? :D

[guest]

I think no error messages cause ripbot264 don't show it. Do you remember TDS only show scripts with ripbot264 with no error messages?? :D

Mr K,

I'm right here :D

I know Ryushin, and I have sent him a PM, shouldn't be a problem :sly:

BTW, it was an SMDegrain error at Line 214, Refinemotion setting options, with low blksize.

[Boulder]

Well, blocksize 4 is simply not supported. This is the content of line 214. Anyways, blocksize 4 with a 4K source is... not very common :D

RefineMotion ? Assert(blksize > 7, "SMDegrain: For RefineMotion you need a blksize of at least 8")

[guest]

Well, blocksize 4 is simply not supported. This is the content of line 214. Anyways, blocksize 4 with a 4K source is... not very common :D

RefineMotion ? Assert(blksize > 7, "SMDegrain: For RefineMotion you need a blksize of at least 8")

It worked for me with Refinemotion=false, or when Refinemotion wasn't in the call at all.

I guess I will see what Ryushin has to say, when he replies to my PM.

[Boulder]

It worked for me with Refinemotion=false, or when Refinemotion wasn't in the call at all.

I guess I will see what Ryushin has to say, when he replies to my PM.

Yes, probably some version conflict or something like that. There doesn't seem to be any other restrictions for size 4 than with Refinemotion enabled.

[DTL]

Very low size blocks require very sharp source to produce any good quality MVs. Also 4K and blocksize=4 (and full overlap if possible) may run at very low speed. SAD-based search with 2 random fields at denoising work do not always provide true-best MVs - it provide SAD-best MVs for current random input blocks. Refined to 2x2 blocks and with large search radius enabled may produce many really bad MVs. So it may be disabled to save user from slow and may be not good results.

"There doesn't seem to be any other restrictions for size 4 than with Refinemotion enabled."

For 4:2:0 chroma subsampling may be possible either refine to 2x2 blocks with 1x1 chroma samples or overlap with size of 2. Not refine + overlap at the same time. The overlap may be more valuable in compare with refining to 2x2.

[StainlessS]

Any testing with YUV444 ?

[tormento]

If you mean VFR, I have no experience with that.
It's not VFR, each part has its own fps. It's sort of a collage video among different fps.

[Dogway]

That also applies as VFR I think. I have no experience with that really.
VFR compatibility is low so I personally would fill with dups to maintain the perceived FPS but with CFR.

On another note finally yesterday I fixed the "devil that chases me" chroma location, I'm doing some other tweaks and releasing an updated ResizersPack and TransformsPack soon, before WC. After that I will be out for a while installing and tuning Win10.

[poisondeathray]

That is considered a type of "VFR" . It's "VFR content in a CFR stream" (or "VFR in CFR") , where different effective content FPS have repeated fields or frames in various patterns to achieve a CFR stream such as 29.97i in order statisfy some standard like broadcast, or DVD-video, or BD. e.g. a 12fps (actually ~ 11.98) anime pan in a section might have duplicate frames in a 23.976 animation. And if that were hard telecined it would have repeated fields (standard 3:2 pulldown) to achieve 29.97i (59.94 fields/s)

Timecode (timestamp, refers same thing) VFR is a 2nd different type of "VFR" , where you have unique frames, but the frame display times are specified in the timecode file .

Usually the 1st type is transformed into the 2nd, because the 1st is the distribution type (e.g. DVD, BD) , where the 2nd is more efficient (you're not encoding extra fields or frames)

[LeXXuz]

I may need a new debanding filter. No idea why I didn't see this much sooner. :rolleyes:

I've used neo_f3kdb() for a long time and was quite happy with the results when it comes to debanding and speed.

But now I realized that it kills A LOT of details in dark/ low contrast scenes. It also enhances the smearing effect in moving objects, again in dark or low contrast textures when using SMDegrain.

Although I always encode everything to 10bit, I still need a debanding filter as my TV is sadly very picky on very clean films and shows off banding in backgrounds very quickly. :(

I played around with Grainfactory() but it increases filesizes quite a lot until I get satisfying results on my TV. So simply adding grain back again doesn't cut it.

What do you guys use?

[Dogway]

As I observed f3kdb adds a nice amount of grain, specially on dark areas, even when grain is set to none, so it adds distortion to details.
Maybe use other debanding filters, there's mine Gradfun3mod, also cretindesalpes' Gradfun3 but doesn't support HBD, or more recently libplacebo_Deband (https://github.com/Asd-g/avslibplacebo#debanding).

If you need grain, unless it's for artistic purposes, add in the renderer in realtime.

[kedautinh12]

I may need a new debanding filter. No idea why I didn't see this much sooner. :rolleyes:

I've used neo_f3kdb() for a long time and was quite happy with the results when it comes to debanding and speed.

But now I realized that it kills A LOT of details in dark/ low contrast scenes. It also enhances the smearing effect in moving objects, again in dark or low contrast textures when using SMDegrain.

Although I always encode everything to 10bit, I still need a debanding filter as my TV is sadly very picky on very clean films and shows off banding in backgrounds very quickly. :(

I played around with Grainfactory() but it increases filesizes quite a lot until I get satisfying results on my TV. So simply adding grain back again doesn't cut it.

What do you guys use?

You can use F3KDB_3() to replace neo_f3kdb()
https://github.com/kedaitinh12/AVSPlus-Scripts/blob/master/F3KDB_sMOD.avsi

About add grain, you can use this scripts don't increases filesizes quite a lot
gr = 0.3
size = 0.8
GrainFactory3mod(g1str=6*gr,g2str=8*gr,g3str=5*gr,g1size=1.2*size,g2size=1.5*size,g3size=1.4*size,g1cstr=1.5*gr,g2cstr=1.5*gr,g3cstr=1.5*gr,temp_avg=0)

[LeXXuz]

Maybe use other debanding filters, there's mine Gradfun3mod, also cretindesalpes' Gradfun3 but doesn't support HBD, or more recently libplacebo_Deband (https://github.com/Asd-g/avslibplacebo#debanding).


Gradfun3mod looks interesting. Do you have a subtle setting for very clean sources at hand which I may use as a starting point for tweaking? :)

You can use F3KDB_3() to replace neo_f3kdb()
https://github.com/kedaitinh12/AVSPlus-Scripts/blob/master/F3KDB_sMOD.avsi


Thanks. I'll give that a try, too. :)

[LeXXuz]

As I observed f3kdb adds a nice amount of grain, specially on dark areas, even when grain is set to none, so it adds distortion to details.
Maybe use other debanding filters, there's mine Gradfun3mod, also cretindesalpes' Gradfun3 but doesn't support HBD, or more recently libplacebo_Deband (https://github.com/Asd-g/avslibplacebo#debanding).

If you need grain, unless it's for artistic purposes, add in the renderer in realtime.

I tried Gradfun.
This is the filtered film with the banding I'd like to get rid of:
https://abload.de/thumb/2022-11-17gqe7y.png (https://abload.de/image.php?img=2022-11-17gqe7y.png)

This is with default call GradFun3plus() as last filter:
https://abload.de/thumb/2022-11-171xpf5o.png (https://abload.de/image.php?img=2022-11-171xpf5o.png)

No, I did NOT accidently swap the screenshots. :D
It really made things much worse. What do I miss here? :o

[VoodooFX]

It really made things much worse. What do I miss here? :o

GradFun2DBmod()

[Dogway]

Probably?:
ConvertBits(16)
GradFun3plus()
ConvertBits(8,dither=1)

Without HBD your only option is to add grain like with GradFun2DBmod()

[LeXXuz]

Probably?:
ConvertBits(16)
GradFun3plus()
ConvertBits(8,dither=1)

Without HBD your only option is to add grain like with GradFun2DBmod()

It's running in 16bit. (I feed the source in 16bit to x265 which dithers it down to 10bit internally.)
That's why the output confuses me. Something is off.

[Dogway]

It mimics the original GradFun3 strength so maybe you need to increase radius? Without a small sample I don't know, it works for me for skies, etc

[LeXXuz]

Without a small sample I don't know, it works for me for skies, etc

That's quite alright.

One other thing. A lot of old movies have some kind of luminance flickering which of course becomes way more noticeable after the layer of grain was removed through denoising. Is there any filter that can soothe this a little?

[kedautinh12]

You can add grain after denoise with GrainFactory3mod :D

[anton_foy]

That's quite alright.

One other thing. A lot of old movies have some kind of luminance flickering which of course becomes way more noticeable after the layer of grain was removed through denoising. Is there any filter that can soothe this a little?

Ex_reduceflicker(strength=2) #or 1?

[LeXXuz]

Ex_reduceflicker(strength=2) #or 1?

I can't find that function. Which package does this belong to?

[Dogway]

It's called DCTFlicker, you can enable the option in SMDegrain (requires LFR>200) or simply use ex_LFR() from ExTools with DCTFlicker enabled. It's not perfect though. You can also try ex_reduceflicker() from QTGMC+

[LeXXuz]

It's called DCTFlicker, you can enable the option in SMDegrain (requires LFR>200) or simply use ex_LFR() from ExTools with DCTFlicker enabled. It's not perfect though. You can also try ex_reduceflicker() from QTGMC+

Is it enough to set LFR=true and DCTFlicker=true for SD sources or do I have to set it to 200 manually at least?
I remember a post where you mentioned LFR~150 as the default for SD.

Also I'm not so sure anymore if flickering is the right description of what I mean. It could also be described as a pumping effect of the luma in the backgrounds.
No idea what the correct expression for that effect may be. :o