Dogway's Filters Packs

[Boulder]

It's probably the source filter playing tricks on you. I think you can force it to output the correct format with specific parameters.

[simple_simon]

Originally Posted by Boulder

It's probably the source filter playing tricks on you. I think you can force it to output the correct format with specific parameters.

What format is required? What format do I force the output to?

[Boulder]

Originally Posted by simple_simon

What format is required? What format do I force the output to?

Hmm.. maybe format="YUV420P8" would be correct if MediaInfo is outputting the correct information.

[anton_foy]

@Dogway, any suggestions how to modify soothe or make a similar filter to temporally stabilize after purely spatial denoising? Getting rid of the "nervous" or "jittery" artifacts.

[LeXXuz]

Originally Posted by anton_foy

@Dogway, any suggestions how to modify soothe or make a similar filter to temporally stabilize after purely spatial denoising? Getting rid of the "nervous" or "jittery" artifacts.

I second that.

Also how to cope with wobbling backgrounds after degraining with SMDegrain. It usually shows up on older movies with flickering luma.
The problem is the denoised movie with the wobble looks worse than the original with noise and flickering.
I'm almost at the point of giving up denoising these films. As I can't find any setting that reduces this succesfully. Maybe degrain is the wrong denoiser for these.

A special prefilter which stabilizes luminance flicker may be helpful, idk. LFR and DCTFlicker don't help me here.

[anton_foy]

Originally Posted by LeXXuz

I second that.

Also how to cope with wobbling backgrounds after degraining with SMDegrain. It usually shows up on older movies with flickering luma.
The problem is the denoised movie with the wobble looks worse than the original with noise and flickering.
I'm almost at the point of giving up denoising these films. As I can't find any setting that reduces this succesfully. Maybe degrain is the wrong denoiser for these.

A special prefilter which stabilizes luminance flicker may be helpful, idk. LFR and DCTFlicker don't help me here.

Have you tried to use ex_reduceflicker in the prefiltering phase?

[Dogway]

Originally Posted by LeXXuz

Btw is there an overview of what prefilter may suit what source/noise type best if not better than others?

Generally I divide them between median like or gaussian blur like .
For example for 2D cel animation:

median / surface blur → bilateral → NLMeans

Then you have blurs with block matching:

MVTools → BM3D

And finally FFT based , which leave an ugly aftermark IMO unless your source is really blocky :

FFT3D → DFTTest

Finally you have mixes of them like Minblur , Fluxsmooth , IQM ...

IMO block matching is the superior for video but for anime median like filters seem move appropiate. The spatial - only prefilter variants are there solely for performance reasons .

Originally Posted by simple_simon

But now I'm getting this error with some videos:

ConvertYV24ToRGB: Unknown matrix

Any idea what might be causing this one? deep_resize is the only filter being used in the script.

Probably my filter is trying to retrieve source matrix information from a clip untagged by the source loader. I'm without computer today, so will have a look during the weekend. Meanwhile try with FFMS2 loader.

Originally Posted by anton_foy

Have you tried to use ex_reduceflicker in the prefiltering phase?

I also was confused at a time with "reduceflicker" plugin. In the above three posts four different kinds of flicker were described; luma flicker, motion flicker, DCTflicker and (de)interlacing flicker.
If the problem is luma flicker you need to stabilize it with ex_autolevels or other solution.

[simple_simon]

Originally Posted by Boulder

Hmm.. maybe format="YUV420P8" would be correct if MediaInfo is outputting the correct information.

Yeah I tried doing that already since that's what mediainfo said it was but it didn't change the error message. This is what I added:

LSMASHVideoSource("F:\USER\Desktop\Great.mov", format="YUV420P8")

Originally Posted by Dogway

Probably my filter is trying to retrieve source matrix information from a clip untagged by the source loader. I'm without computer today, so will have a look during the weekend. Meanwhile try with FFMS2 loader.

I tried FFMS2 which detected the correct colorspace but still getting the same error message.

Originally Posted by TDS

Just thinkin' out loud here, but I recently had some "indifferent" experience with deep_resize, but wouldn't you need some more dependencies to get this to work ??

Like, I dunno, ResizersPack, and whatever else ??

Yes I have all the required updated dependencies. I've successfully used deep_resize with some other videos files already but there are two that are throwing up this same error. The dimensions for both of the problematic files are below 480p so I think that might have something to do with it but I can't figure out what.

[Boulder]

Originally Posted by simple_simon

Yeah I tried doing that already since that's what mediainfo said it was but it didn't change the error message.

Try adding just Info() or propShow() after the source is loaded, see what is actually output.

[simple_simon]

Originally Posted by Boulder

Try adding just Info() or propShow() after the source is loaded, see what is actually output.

Strangely, Info() reports the colorspace as YV12. Adding format="YUV420P8" to LSMASHVideoSource doesn't seem to change it.

[kedautinh12]

YV12 is YUV420 in avs+
http://avisynth.nl/index.php/Planar

[Boulder]

Originally Posted by simple_simon

Strangely, Info() reports the colorspace as YV12. Adding format="YUV420P8" to LSMASHVideoSource doesn't seem to change it.

Then it definitely looks like a problem involving deep_resize.

[Dogway]

Originally Posted by simple_simon

Yes I have all the required updated dependencies. I've successfully used deep_resize with some other videos files already but there are two that are throwing up this same error. The dimensions for both of the problematic files are below 480p so I think that might have something to do with it but I can't figure out what.

Works fine here, even clearing all frame props for under 480p clips.

spline16resize(320,240)
propClearAll()
deep_resize(960,720,grain=0,qual=2,gpuid=-1)

[LeXXuz]

Originally Posted by Dogway

Generally I divide them between median like or gaussian blur like .
For example for 2D cel animation:

median / surface blur → bilateral → NLMeans

Then you have blurs with block matching:

MVTools → BM3D

And finally FFT based , which leave an ugly aftermark IMO unless your source is really blocky :

FFT3D → DFTTest

Finally you have mixes of them like Minblur , Fluxsmooth , IQM ...

IMO block matching is the superior for video but for anime median like filters seem move appropiate. The spatial - only prefilter variants are there solely for performance reasons .

Thanks Dogway. I agree with block matching being best suitable for most films. After countless tests over the last year I ended up with a combination of BM3D+FluxSmooth for prefiltering which gave me the best results so far. Depending on noise of the source I use both with modified parameters. Their default settings were too strong for my taste.

Originally Posted by Dogway

I also was confused at a time with "reduceflicker" plugin. In the above three posts four different kinds of flicker were described; luma flicker, motion flicker, DCTflicker and (de)interlacing flicker.
If the problem is luma flicker you need to stabilize it with ex_autolevels or other solution.

I've uploaded two samples here, the noisy source with the luma flicker and the cleaned version with the wobble.
Just watch the left and upper side of the archway the actor is walking through and you can't miss it. Looks awful on a big screen TV.
Original:
https://send.cm/d/JxeF
Denoised:
https://send.cm/d/JxeG

Will play around with reduceflicker and autolevels. Would appreciate some input on starting parameters from where to go as I have no experience with both filters.

[Dogway]

Interesting clip, interlaced? I think you might treat it as progressive, so set propSet("_FieldBased",0).
For prefilter I noticed issues for FluxSmooth on temporal domain, it causes some ghosting, probably better to go with it's spatial MinBlur which is fantastic as far as spatial prefilter, then BM3D, then MVTools.

Back to the clip I was thinking yesterday to include IQM to SceneStats since the function was added weeks ago to ExTools. This is a more robust metric.
I was playing now with ex_autolevels() but I didn't map the gamma but after that albeit the result is better the overall is not still good enough, median bad, average too dark for first scene, will try next with IQM.
Following that you might need to apply a strong temporal denoiser. Anyway videoFred is a specialist on these kind of sources, I would also ask him too since he has a lot of experience on old rusty films like this.

[LeXXuz]

Thanks Dogway for having a look at this.

Originally Posted by Dogway

Interesting clip, interlaced? I think you might treat it as progressive, so set propSet("_FieldBased",0).

Well this old film actually is progressive, but the DVD, like many low budget productions, suffers from many PAL phase shifts from poor AD transfer. So I use Ceppos tools to correct those errors. After that I use propset as you already suggested before.

Originally Posted by Dogway

For prefilter I noticed issues for FluxSmooth on temporal domain, it causes some ghosting, probably better to go with it's spatial MinBlur which is fantastic as far as spatial prefilter, then BM3D, then MVTools.

I only use FluxSmooth in spatial domain, 'cause I noticed that too, and BM3D with a temporal radius of up to 5. Although the differences between r=5 and r=1 are much more subtle when using BM3D as prefilter for Mvtools.

Furthermore I use BM3D first then FluxSmooth because block matching tends to produce some poor artefacts in rare cases on some forms of noise. FluxSmooth filters these a little and smoothes some residual low freq noise from block matching quite nicely.

But I rarely use ex_FluxSmooth with a radius >1 as it smoothes fine detail too much with 2 or even 3. Only if residual noise is still too present and the trade-off is worth it.

Originally Posted by Dogway

Back to the clip I was thinking yesterday to include IQM to SceneStats since the function was added weeks ago to ExTools. This is a more robust metric.
I was playing now with ex_autolevels() but I didn't map the gamma but after that albeit the result is better the overall is not still good enough, median bad, average too dark for first scene, will try next with IQM.

Please let me know about any suitable results. I have quite a lot of these kind of films to give them a test run.

Originally Posted by Dogway

Following that you might need to apply a strong temporal denoiser.

As general noise filter or as a prefilter? Any suggestions which ones to give a try maybe?

[simple_simon]

I noticed that deep_resize is changing the colors slightly. Is that intentional for some reason?

[Boulder]

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain? I've usually taken a precarious way like using a ex_MinBlur with radius 1 or 2 depending on the source. Running ex_BM3D on the clip definitely removes all the noise but much of the finer details as well. (By the way, I couldn't replicate the issue with DGSource cropping and BM3D so maybe it's fixed now.)

[DTL]

Originally Posted by Boulder

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain?

Typically noise removed from source to MAnalyse to attempt get more stable motion vectors. The input to MDegrain depends on processing script and may be input source (not prefiltered). The detail smoothing at the output of MDegrain depends on the motion vectors stability and precision (quality). So with prefiltering to MAnalyse you may get more details saved at the output of MDegrain with higher thSAD and more denoising effect.

If you get a sequence of stable photos with noise and provide zeroed motion vectors to MDegrain it will not smooth details at all with any high thSAD (it will work as simple static Average() processing). But with real footage the MDegrain do not know which blocks have zeroed motion vectors at real motion pictures camera shooting with noised blocks so motion vectors search with MAnalyse may provide more and more false non-zeroed motion vectors for real fixed parts of frame and it one of the reason to details lost at the MDegrain output. With prefiltering you remove part of noise from the source to MAnalyse and increase probability to compute more correct motion vector (zero for fixed blocks and correct pointing to moving). But it is not guaranteed because prefiltering also cause details lost for MAnalyse and it also may be the reason of non-correct motion vectors. In current versions of mvtools there is no implementation of multi-pass iterative motion search and degrain and next iteration motion vectors refining and so on. Only some attempts exist with script writers to make 'prefiltering' and some multi stages processing with possible data refining.

[LeXXuz]

Originally Posted by Boulder

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain? I've usually taken a precarious way like using a ex_MinBlur with radius 1 or 2 depending on the source. Running ex_BM3D on the clip definitely removes all the noise but much of the finer details as well. (By the way, I couldn't replicate the issue with DGSource cropping and BM3D so maybe it's fixed now.)

You're quite right. You can't overdo it with prefiltering. Finding the best balance between noise filtering and detail preservation is key. And it is ALWAYS a compromise. Setting up prefilters correctly is the most important part imho.

That's why I created myself 10 different profiles, ranging from sources with extreme high quality to extreme low quality.

So when denoising a film I pick one of these profiles as a general direction and then still fine-tune it more or less, because every source is different.