Dogway's Filters Packs

[poisondeathray]

So unless proven otherwise, this is what works for me :-

Seems to work with prefilter -1 thru 3 & 5 (didn't test any higher)

video=ConvertBits(video,16)
video=Tweak(video,hue=0,sat=0.8,bright=2,cont=1)
video=nnedi3wrap(video)
video=deep_resize(video,3840,grain=0,qual=2)
video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,thSCD1=400,thSCD2=130,contrasharp=40,prefilter=3,str=1.2,refinemotion=true)

Seems to work with prefilter -1 thru 3 & 5 (didn't test any higher)

video=ConvertBits(video,16)
video=Tweak(video,hue=0,sat=0.8,bright=2,cont=1)
video=deep_resize(video,1920,grain=0,qual=2)
video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,thSCD1=400,thSCD2=130,contrasharp=40,prefilter=3,str=1.2,refinemotion=true)

Hopefully this is the end of this saga.

I'll have to update most of my scripts, since the changes in SMDegrain, since 3.5.7d (currently using 3.5.9d), which may have been part of the problem.


deep_resize isn't actually used in those examples (no-op), because it's upscaled before to 3840 by nnedi3wrap(video) in the 1st example, or stays at the current 1920 resolution for the 2nd example. So they provide no addition information about your original deep_resize problem

[poisondeathray]

Well, that's interesting to know, but how does it know to upscale to 3840 ??

video=nnedi3wrap(video) , would take the input width *2 , input height*2 , because the default setting is 2.

eg. If source video was 640x480, output from that line would be 1280x960

[poisondeathray]

https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack.avsi#L317

There must be some "tweak's" to that...I don't understand all the hieroglyphics.

nnedi3resize (and related derivatives) scales by powers of 2 by default. It accepts xfactor and yfactor scaling parameters, but only valid values are powers of 2 ; so 2,4,8... If you enter a non power of 2, like 3, it will go down to 2. fwidth and fheight are for secondary scaling, so after the power scaling step, you can downscale to the final width and height (for cases where you need a specific non power of 2 scaling)


So I could drop the deep_resize line, then ??

deep_resize is not being used in those 2 examples

You can obviously do whatever you want, I thought you wanted to get the deep_resize function working

I'm just curious about the "hanging" without error message . It's not common for that to happen, and it's unexpected with that script .

[Reel.Deel]

Do you know what this is ??

https://forum.doom9.org/showthread.php?p=1982132#post1982132

It's now prefilter #3. (or part of it)

Prefilter 3 is ex_Median(mode="IQMV") in line 2355 of ExTools (https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi#L2355) it describes it as "Switches between IQM and IQM5 depending on a difference threshold for better edge protection. *Similar to ex_MinBlur(3) in output but near 3 times faster."

[Dogway]

First post on page 100 !!!!
Thanks! You made it possible! lol
Man I wouldn't mind all the posts involving your issue go to a single thread, since you seem to go in rounds without actually following my (and others) debug suggestions.
Well, I need to wait and see what Dogway is going to suggest...
I suggest this post (https://forum.doom9.org/showthread.php?p=1982099#post1982099). And yes, use FFMS2 always.
What is IQMV ??
It's a replacement of ex_MinBlur(3), output is very similar and speed 3 times faster, on par with ex_MinBlur(2). It's only spatial denoising so you might want to check if it's worth for you for its speed versus temporal variants. I might see an use in noisy Anime maybe, but KNLMeans also fits that role.

BTW Dogway, IDK if you're still on Win7 but BM3D test10 no longer works with Win7 due to it using the Cuda 12 SDK.
Edit: I reported the issue and WolframRhodium compiled a version with the older sdk, test10-cuda118 works in Win7
Ahh well, thanks for the info! Yes, AVS dev has me from focusing on OS change. That requires all my focus to clean the disaster of Win10 and want to leave all scripts in perfect condition before that.
I'm gonna finish SMDegrain docs as soon as possible too, it's up on my todo list.

[kedautinh12]

And @TDS help post reach 100 pages very much :D

[LeXXuz]

It's a replacement of ex_MinBlur(3), output is very similar and speed 3 times faster, on par with ex_MinBlur(2). It's only spatial denoising so you might want to check if it's worth for you for its speed versus temporal variants. I might see an use in noisy Anime maybe, but KNLMeans also fits that role.


Btw is there an overview of what prefilter may suit what source/noise type best if not better than others?

[Reel.Deel]

Hi guy's,

A pretty simple question...

Does it matter what order the call's are in a script ??

eg:-

.....

I do recall shuffling around some of the lines in one script a bit, made the difference of it not working, & working, so it must matter to some degree !!

For loading plugins/scripts what matters is that LoadPlugin/Import are above the call that uses the plugin(s)/filter(s). For example:

Import("somefunction.avsi") # this works
video = somefunction()

video = somefunction()
Import("somefunction.avsi") # this does not

By the way, you're question is not as simple as you may think. There are times when the order of the processing does matter and sometimes it does not. It all depends on what you're doing.

[simple_simon]

I figured out that my "Invalid Program" error was caused by my gpu not supporting opencl so I had to change gpuid=-1 and that fixed it.

But now I'm getting this error with some videos:

ConvertYV24ToRGB: Unknown matrix

Any idea what might be causing this one? deep_resize is the only filter being used in the script.

[simple_simon]

Can you post the script ?

Maybe some of your video's aren't the correct "structure" to start with.

http://avisynth.nl/index.php/Convert

https://mediaarea.net/en/MediaInfo

Like I said, deep_resize is the only filter being used.

LoadPlugin("F:\USER\My Applications\Video Tools\MeGUI x64\tools\lsmash\LSMASHSource.dll")
LSMASHVideoSource("F:\USER\Desktop\Great.mov")
propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1)

Mediainfo says the video is YUV 4:2:0 8bits

[Boulder]

It's probably the source filter playing tricks on you. I think you can force it to output the correct format with specific parameters.

[simple_simon]

It's probably the source filter playing tricks on you. I think you can force it to output the correct format with specific parameters.

What format is required? What format do I force the output to?

[Boulder]

What format is required? What format do I force the output to?

Hmm.. maybe format="YUV420P8" would be correct if MediaInfo is outputting the correct information.

[anton_foy]

@Dogway, any suggestions how to modify soothe or make a similar filter to temporally stabilize after purely spatial denoising? Getting rid of the "nervous" or "jittery" artifacts.

[LeXXuz]

@Dogway, any suggestions how to modify soothe or make a similar filter to temporally stabilize after purely spatial denoising? Getting rid of the "nervous" or "jittery" artifacts.

I second that.

Also how to cope with wobbling backgrounds after degraining with SMDegrain. It usually shows up on older movies with flickering luma.
The problem is the denoised movie with the wobble looks worse than the original with noise and flickering.
I'm almost at the point of giving up denoising these films. As I can't find any setting that reduces this succesfully. Maybe degrain is the wrong denoiser for these.

A special prefilter which stabilizes luminance flicker may be helpful, idk. LFR and DCTFlicker don't help me here.

[anton_foy]

I second that.

Also how to cope with wobbling backgrounds after degraining with SMDegrain. It usually shows up on older movies with flickering luma.
The problem is the denoised movie with the wobble looks worse than the original with noise and flickering.
I'm almost at the point of giving up denoising these films. As I can't find any setting that reduces this succesfully. Maybe degrain is the wrong denoiser for these.

A special prefilter which stabilizes luminance flicker may be helpful, idk. LFR and DCTFlicker don't help me here.

Have you tried to use ex_reduceflicker in the prefiltering phase?

[Dogway]

Btw is there an overview of what prefilter may suit what source/noise type best if not better than others?
Generally I divide them between median like or gaussian blur like .
For example for 2D cel animation:
median / surface blur → bilateral → NLMeans
Then you have blurs with block matching:
MVTools → BM3D
And finally FFT based , which leave an ugly aftermark IMO unless your source is really blocky :
FFT3D → DFTTest

Finally you have mixes of them like Minblur , Fluxsmooth , IQM ...

IMO block matching is the superior for video but for anime median like filters seem move appropiate. The spatial - only prefilter variants are there solely for performance reasons .
But now I'm getting this error with some videos:

ConvertYV24ToRGB: Unknown matrix
Any idea what might be causing this one? deep_resize is the only filter being used in the script.
Probably my filter is trying to retrieve source matrix information from a clip untagged by the source loader. I'm without computer today, so will have a look during the weekend. Meanwhile try with FFMS2 loader.
Have you tried to use ex_reduceflicker in the prefiltering phase?
I also was confused at a time with "reduceflicker" plugin. In the above three posts four different kinds of flicker were described; luma flicker, motion flicker, DCTflicker and (de)interlacing flicker.
If the problem is luma flicker you need to stabilize it with ex_autolevels or other solution.

[simple_simon]

Hmm.. maybe format="YUV420P8" would be correct if MediaInfo is outputting the correct information.

Yeah I tried doing that already since that's what mediainfo said it was but it didn't change the error message. This is what I added:

LSMASHVideoSource("F:\USER\Desktop\Great.mov", format="YUV420P8")


Probably my filter is trying to retrieve source matrix information from a clip untagged by the source loader. I'm without computer today, so will have a look during the weekend. Meanwhile try with FFMS2 loader.

I tried FFMS2 which detected the correct colorspace but still getting the same error message.

Just thinkin' out loud here, but I recently had some "indifferent" experience with deep_resize, but wouldn't you need some more dependencies to get this to work ??

Like, I dunno, ResizersPack, and whatever else ??

Yes I have all the required updated dependencies. I've successfully used deep_resize with some other videos files already but there are two that are throwing up this same error. The dimensions for both of the problematic files are below 480p so I think that might have something to do with it but I can't figure out what.

[Boulder]

Yeah I tried doing that already since that's what mediainfo said it was but it didn't change the error message.

Try adding just Info() or propShow() after the source is loaded, see what is actually output.

[simple_simon]

Try adding just Info() or propShow() after the source is loaded, see what is actually output.

Strangely, Info() reports the colorspace as YV12. Adding format="YUV420P8" to LSMASHVideoSource doesn't seem to change it.

[kedautinh12]

YV12 is YUV420 in avs+
http://avisynth.nl/index.php/Planar

[Boulder]

Strangely, Info() reports the colorspace as YV12. Adding format="YUV420P8" to LSMASHVideoSource doesn't seem to change it.

Then it definitely looks like a problem involving deep_resize.

[Dogway]

Yes I have all the required updated dependencies. I've successfully used deep_resize with some other videos files already but there are two that are throwing up this same error. The dimensions for both of the problematic files are below 480p so I think that might have something to do with it but I can't figure out what.

Works fine here, even clearing all frame props for under 480p clips.
spline16resize(320,240)
propClearAll()
deep_resize(960,720,grain=0,qual=2,gpuid=-1)

[LeXXuz]

Generally I divide them between median like or gaussian blur like .
For example for 2D cel animation:
median / surface blur → bilateral → NLMeans
Then you have blurs with block matching:
MVTools → BM3D
And finally FFT based , which leave an ugly aftermark IMO unless your source is really blocky :
FFT3D → DFTTest

Finally you have mixes of them like Minblur , Fluxsmooth , IQM ...

IMO block matching is the superior for video but for anime median like filters seem move appropiate. The spatial - only prefilter variants are there solely for performance reasons .


Thanks Dogway. I agree with block matching being best suitable for most films. After countless tests over the last year I ended up with a combination of BM3D+FluxSmooth for prefiltering which gave me the best results so far. Depending on noise of the source I use both with modified parameters. Their default settings were too strong for my taste.


I also was confused at a time with "reduceflicker" plugin. In the above three posts four different kinds of flicker were described; luma flicker, motion flicker, DCTflicker and (de)interlacing flicker.
If the problem is luma flicker you need to stabilize it with ex_autolevels or other solution.

I've uploaded two samples here, the noisy source with the luma flicker and the cleaned version with the wobble.
Just watch the left and upper side of the archway the actor is walking through and you can't miss it. Looks awful on a big screen TV.
Original:
https://send.cm/d/JxeF
Denoised:
https://send.cm/d/JxeG

Will play around with reduceflicker and autolevels. Would appreciate some input on starting parameters from where to go as I have no experience with both filters. :o

[Dogway]

Interesting clip, interlaced? I think you might treat it as progressive, so set propSet("_FieldBased",0).
For prefilter I noticed issues for FluxSmooth on temporal domain, it causes some ghosting, probably better to go with it's spatial MinBlur which is fantastic as far as spatial prefilter, then BM3D, then MVTools.

Back to the clip I was thinking yesterday to include IQM to SceneStats since the function was added weeks ago to ExTools. This is a more robust metric.
I was playing now with ex_autolevels() but I didn't map the gamma but after that albeit the result is better the overall is not still good enough, median bad, average too dark for first scene, will try next with IQM.
Following that you might need to apply a strong temporal denoiser. Anyway videoFred is a specialist on these kind of sources, I would also ask him too since he has a lot of experience on old rusty films like this.

[LeXXuz]

Thanks Dogway for having a look at this. :thanks:

Interesting clip, interlaced? I think you might treat it as progressive, so set propSet("_FieldBased",0).

Well this old film actually is progressive, but the DVD, like many low budget productions, suffers from many PAL phase shifts from poor AD transfer. So I use Ceppos tools to correct those errors. After that I use propset as you already suggested before.


For prefilter I noticed issues for FluxSmooth on temporal domain, it causes some ghosting, probably better to go with it's spatial MinBlur which is fantastic as far as spatial prefilter, then BM3D, then MVTools.

I only use FluxSmooth in spatial domain, 'cause I noticed that too, and BM3D with a temporal radius of up to 5. Although the differences between r=5 and r=1 are much more subtle when using BM3D as prefilter for Mvtools.

Furthermore I use BM3D first then FluxSmooth because block matching tends to produce some poor artefacts in rare cases on some forms of noise. FluxSmooth filters these a little and smoothes some residual low freq noise from block matching quite nicely.

But I rarely use ex_FluxSmooth with a radius >1 as it smoothes fine detail too much with 2 or even 3. Only if residual noise is still too present and the trade-off is worth it.


Back to the clip I was thinking yesterday to include IQM to SceneStats since the function was added weeks ago to ExTools. This is a more robust metric.
I was playing now with ex_autolevels() but I didn't map the gamma but after that albeit the result is better the overall is not still good enough, median bad, average too dark for first scene, will try next with IQM.

Please let me know about any suitable results. I have quite a lot of these kind of films to give them a test run.


Following that you might need to apply a strong temporal denoiser.

As general noise filter or as a prefilter? Any suggestions which ones to give a try maybe?

[simple_simon]

I noticed that deep_resize is changing the colors slightly. Is that intentional for some reason?

[Boulder]

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain? I've usually taken a precarious way like using a ex_MinBlur with radius 1 or 2 depending on the source. Running ex_BM3D on the clip definitely removes all the noise but much of the finer details as well. (By the way, I couldn't replicate the issue with DGSource cropping and BM3D so maybe it's fixed now.)

[DTL]

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain?

Typically noise removed from source to MAnalyse to attempt get more stable motion vectors. The input to MDegrain depends on processing script and may be input source (not prefiltered). The detail smoothing at the output of MDegrain depends on the motion vectors stability and precision (quality). So with prefiltering to MAnalyse you may get more details saved at the output of MDegrain with higher thSAD and more denoising effect.

If you get a sequence of stable photos with noise and provide zeroed motion vectors to MDegrain it will not smooth details at all with any high thSAD (it will work as simple static Average() processing). But with real footage the MDegrain do not know which blocks have zeroed motion vectors at real motion pictures camera shooting with noised blocks so motion vectors search with MAnalyse may provide more and more false non-zeroed motion vectors for real fixed parts of frame and it one of the reason to details lost at the MDegrain output. With prefiltering you remove part of noise from the source to MAnalyse and increase probability to compute more correct motion vector (zero for fixed blocks and correct pointing to moving). But it is not guaranteed because prefiltering also cause details lost for MAnalyse and it also may be the reason of non-correct motion vectors. In current versions of mvtools there is no implementation of multi-pass iterative motion search and degrain and next iteration motion vectors refining and so on. Only some attempts exist with script writers to make 'prefiltering' and some multi stages processing with possible data refining.

[LeXXuz]

A rather simple question regarding prefiltering, this is something I've often wondered.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain? I've usually taken a precarious way like using a ex_MinBlur with radius 1 or 2 depending on the source. Running ex_BM3D on the clip definitely removes all the noise but much of the finer details as well. (By the way, I couldn't replicate the issue with DGSource cropping and BM3D so maybe it's fixed now.)

You're quite right. You can't overdo it with prefiltering. Finding the best balance between noise filtering and detail preservation is key. And it is ALWAYS a compromise. Setting up prefilters correctly is the most important part imho.

That's why I created myself 10 different profiles, ranging from sources with extreme high quality to extreme low quality.

So when denoising a film I pick one of these profiles as a general direction and then still fine-tune it more or less, because every source is different.

[DTL]

I've uploaded two samples here, the noisy source with the luma flicker and the cleaned version with the wobble.
Just watch the left and upper side of the archway the actor is walking through and you can't miss it. Looks awful on a big screen TV.
Original:
https://send.cm/d/JxeF


It is spatial distortion in the source. To fix it some advanced 'area stabilization' processing required to keep real objects motion and to attempt to decrease this 'scale+translate' transform on more or less big ares of the frame. May be initial film frames were distorted so the filmscan already distorted too.

[poisondeathray]

I noticed that deep_resize is changing the colors slightly. Is that intentional for some reason?

In what situation ? What are the details of videos and script ? It does not change the colors here:

eg.

colorbarshd(1920,1080)


vs


colorbarshd(1920,1080)
deep_resize(1280,720, grain=0)

[Dogway]

As general noise filter or as a prefilter? Any suggestions which ones to give a try maybe?

As a general noise with good temporal prefilter (BM3D), because the luma flicker is not homogeneous so the ex_autolevels() is never going to fix it completely.

The update to SceneStats with IQM is finished except for an issue on the accumulator that I have to fix, it's challenging me.

If you remove most or all the noise from the prefilter clip, you are also bound to remove plenty of fine details as well. Is this not a problem when trying to keep the details but remove the noise with MDegrain?

The real case scenario for prefiltering is one where you use masks and smart filtering. The built-in prefilters are only a convenience for not advanced users. In any case the prefiltering is not the real output, it can be a bit more blurry than expected since you only need the structure for the motion vectors. Motion vectors from MVTools itself are neither as fine detailed as you think (blocks of 4x4 for refinemotion like a 480x270 clip for a 1080p source with blksize=8) so if there are finer elements you want to recover you will need to add them back later with custom masks (corner masks, detail masks, luma masks...). I chose to keep SMDegrain "simple", which suits all kind of sources, like an interface or agnostic extension of MVTools.


EDIT: ScenesPack updated, ex_autolevels() in GradePack also updated.

[simple_simon]

In what situation ? What are the details of videos and script ? It does not change the colors here:

eg.

colorbarshd(1920,1080)


vs


colorbarshd(1920,1080)
deep_resize(1280,720, grain=0)


Here's my sample.

https://send.cm/d/K1aD

Compare

spline36resize(960,720)

to

propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1)

[simple_simon]

Works fine here, even clearing all frame props for under 480p clips.
spline16resize(320,240)
propClearAll()
deep_resize(960,720,grain=0,qual=2,gpuid=-1)

Here's one of the clips giving me the "ConvertYV24ToRGB: Unknown Matrix" error

https://send.cm/d/K1b5

propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1)

[poisondeathray]

Here's my sample.

https://send.cm/d/K1aD

Compare

spline36resize(960,720)

to

propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1)

I don't see any color change, or it's negligible.

Also, you're upscaling "SD" to "HD", the usual convention is to apply colormatrix or similar transform from 601 to 709 , and flag the encode properly

"HD" material by default uses 709 , "SD" by default 601 (some rare exceptions) . Some players will use 709 to convert to RGB for display for HD (height > 576), so video colors will look shifted if it uses the wrong matrix



Here's one of the clips giving me the "ConvertYV24ToRGB: Unknown Matrix" error

https://send.cm/d/K1b5

propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1)

Works for me, no error messages with either LWLibavVideoSource, or FFVideoSource

Check your dependencies


Color primaries : Display P3
Transfer characteristics : BT.709
Matrix coefficients : BT.709


The primaries are atypical

Maybe try setting the matrix property

propSet("_Matrix",1) #709

[simple_simon]

I don't see any color change, or it's negligible.

Also, you're upscaling "SD" to "HD", the usual convention is to apply colormatrix or similar transform from 601 to 709 , and flag the encode properly

Strange that you aren't seeing the color change. But I did wonder if deep_resize was converting the colorspace to HD also but I couldn't find anywhere where that was stated. So that's good to know so I can apply the encoding settings correctly in x264.

Here's screenshots of what I'm seeing.

Spline36Resize
https://send.cm/oo8tlpqk5ey7

deep_resize
https://send.cm/4izbhjbte3rw

Works for me, no error messages with either LWLibavVideoSource, or FFVideoSource

Check your dependencies



The primaries are atypical

Maybe try setting the matrix property

propSet("_Matrix",1) #709

Well I'm stumped on this one. I doublechecked all dependencies and their dependencies and everthing is up to date as far as I can see.

I tried adding propSet("_Matrix",1) before deep_resize like you suggested and still get the error.

[poisondeathray]

Here's screenshots of what I'm seeing.


Screenshots definitely show a 709 vs. 601 mismatch. But I cannot reproduce it with deep_resize and that script

How are you taking screenshots ? Is this a preview or encode ? How are you viewing it ? How is the RGB conversion being done ?




Well I'm stumped on this one. I doublechecked all dependencies and their dependencies and everthing is up to date as far as I can see.

I tried adding propSet("_Matrix",1) before deep_resize like you suggested and still get the error.

What is the full error message for "ConvertYV24ToRGB: Unknown Matrix" ? Does it give a line number ?

What avs+ version ?

[simple_simon]

Screenshots definitely show a 709 vs. 601 mismatch. But I cannot reproduce it with deep_resize and that script

How are you taking screenshots ? Is this a preview or encode ? How are you viewing it ? How is the RGB conversion being done ?

Screenshots are window captures of the MeGUI Preview window using Greenshot. I'm not doing any RGB conversion.


What is the full error message for "ConvertYV24ToRGB: Unknown Matrix" ? Does it give a line number ?

What avs+ version ?

Nope, no line number. I gave all the details that were provided. It doesn't even say what filter is throwing the error. Here's a screenshot of the actual error message in MeGUI.

https://send.cm/ifax6um50ces

And using AviSynthPlus-3.7.3_r3877, which is the very latest revision that I'm aware of.

[poisondeathray]

Screenshots are window captures of the MeGUI Preview window using Greenshot. I'm not doing any RGB conversion.


MeGUI is doing the RGB conversion for the preview

If you don't know what it's doing for the preview, you can explicitly control the conversion by addding ConvertToRGB(matrix="rec709"), or ConvertToRGB(matrix="rec601") (just don't forget to remove them when encoding)



Nope, no line number. I gave all the details that were provided. It doesn't even say what filter is throwing the error. Here's a screenshot of the actual error message in MeGUI.

https://send.cm/ifax6um50ces

And using AviSynthPlus-3.7.3_r3877, which is the very latest revision that I'm aware of.

Maybe an issue with that version ? I was using r3835 . But I checked with r3877 - no problems either, I cannot reproduce either of your issues


Did you check the script in another application, such as mpchc, avspmod ? Maybe some specific issue with megui ?

[simple_simon]

MeGUI is doing the RGB conversion for the preview

If you don't know what it's doing for the preview, you can explicitly control the conversion by addding ConvertToRGB(matrix="rec709"), or ConvertToRGB(matrix="rec601") (just don't forget to remove them when encoding)

I checked the script in AvsPmod and didn't see the change in color when applying deep_resize. However AvsPmod has a setting to explicitly select which color matrix to use when previewing so that's probably why. What program did you use to preview?

I did use the Frame Properties tool in AvsPmod and the color primaries/matrix/tranfer are read as undefined until deep_resize is applied. Then they change to Rec.709. So I think what's happening in MeGUI is since the matrix properties are undefined it's just making an assumption based on frame size that the matrix is Rec.601 and using that to preview. But deep_resize explicitly sets the matrix properties to Rec.709 so when I apply that filter MeGUI changes it's preview matrix to match and that's why I'm seeing the difference.

Now I'm wondering, is deep_resize actually converting the color primaries/matrixes/transfers to Rec.709 or is it just flagging it as that in the frame properties. If it's just flagging it wouldn't that lead to improper color display at playback. But also if it's actually converting it, I noticed that it still changes it to Rec.709 if downsizing 480p video. Wouldn't that be incorrect? Shouldn't it stay at Rec.601 in that case?


Maybe an issue with that version ? I was using r3835 . But I checked with r3877 - no problems either, I cannot reproduce either of your issues


Did you check the script in another application, such as mpchc, avspmod ? Maybe some specific issue with megui ?

I also checked this script in AvsPmod. It didn't have any issues running it. No errors. According to the frame properties though, deep_resize is changing the matrix from (1 [709]) to (12 [Chroma ncl]). Like you noted earlier the source video has odd color primaries, (12 [ST 432-1]). Since the code numbers are the same I'm guessing deep_resize is changing the matrix to match the primaries. And I'm assuming MeGui doesn't recognize the nonstandard matrix so throws back the error I've been getting when trying to preview. So how do I convert the color primary for this video? Or force deep_resize to use 709 instead of Chroma ncl?

But it seems both problems were in the MeGui previewer, not deep_resize. Although in the first instance I think MeGui handled the matrix display correctly, make an assumption about matrix based on source frame dimensions until matrix is specified. I prefer that to the way AvsPmod does it.

[poisondeathray]

However AvsPmod has a setting to explicitly select which color matrix to use when previewing so that's probably why. What program did you use to preview?

avspmod, I have it set not to read from source, and resolution based . You can also explictly control the RGB conversion as mentioned earlier. It doesn't matter, as long as you keep in the back of your mind how it's being done


Now I'm wondering, is deep_resize actually converting the color primaries/matrixes/transfers to Rec.709 or is it just flagging it as that in the frame properties.


You can insert a propshow() any point to determine if a filter is changing frame properties. If you set up avspmod correctly, you can view the actual YUV values with the color picker to see if any actual pixel value changes are occurring due to a filter. For colors, you can also test colorbars (and colorbars videos with tags) to easily see what's happening

Avisynth can pass frame properties to some programs -e.g. ffmpeg - so it actually can affect the end result and tagging and other program behaviour. Beware of or learn the new behaviour. Some people like to clear the props before encoding, and/or after loading video - so one option is to clear some or all of the properties after the source filter, or set some or all of them specifically .

Props are good and bad - but in some cases they can screw up the results if you're not careful - if a filter makes assumptions or uses values based on props. Sometimes videos are tagged incorrectly . A common one is "PAL" area distribution e.g. progressive content DVD's or BD's , that are encoded interlaced. Vapoursynth users are used to these "props" problems (props have been there a few years longer) but it has invaded avisynth now. Beware and there will be growing pains

You should notify megui developers about the preview issue

[LeXXuz]

EDIT: ScenesPack updated, ex_autolevels() in GradePack also updated.

Should I give that a try? If so, where to put it best and what settings to start from? I don't expect to get rid of that wobble, but I take anything that helps to reduce it. :)

Also right now I'm trying with much smaller temporal radii which seems to reduce the effect, as expected, but at the cost of losing precision. Maybe a mix of denoising and leaving some residual noise is best here. It still beats that clean but wobbly look.

[DTL]

I don't expect to get rid of that wobble, but I take anything that helps to reduce it. :)


As a starting solution you can try 'blind fix' - cut frame to a sub-clips with a fixed grid like 2x2 or 3x3 or more and apply stabilization to each sub-clip and stack subclips back. But it will produce more or less visible issues at the stacking borders looking like a fixed grid.

Second part of solution may be combining first and second (shifted to half of grid step diagonally) clips with same method as overlapping blocks for denoise - you can use BlockOverlap plugin or Overlay with manually painted mixing mask (or may be masktools can help).

But shifting diagonally 1/2 grid step for very dense grids for small blocks of denoise work good - leaving only 1/2 of blocksize uncovered regions near the very edges of frame. And shifting very large 'blocks' of stabilization with only something like 3x3 cutting grid may still leave large enough unfixed lines at the edges of the frame and may need more edges of frame workarounds.

Next step may be pad original frame with 'stab blocksize/2' borders and so you can provide 2 sets of the stabilized clips to BlockOverlap covering all parts of original frame.

BlockOverlap is https://forum.doom9.org/showthread.php?t=101307 and https://github.com/DTL2020/BlockOverlap/releases/tag/v0.2-x64 x64 build.

[Dogway]

Should I give that a try? If so, where to put it best and what settings to start from? I don't expect to get rid of that wobble, but I take anything that helps to reduce it. :)

Yes, you can play with it, it will reduce the luma flicker. I just updated ScenesPack with a small fix to switch to plain mean metric when samples are too low.

Right now I'm playing with some ideas from Stab() using TemporalSoften to reduce the flicker further.


An example:
ffvideosource("noisy_original.mkv")

propSet("_FieldBased",0)
Stab()
SceneStats("Range+Stats")
pref = ex_autolevels()
temp = Pref.TemporalSoften(5,255,0,25,2) # SC thr to 25 otherwise pans will stutter
rep = temp.Repair(Pref.TemporalSoften(1, 255,0,25,2))

pref=ex_merge(rep, pref,MotionMask(rep,scale=10,sharpness=5))
SMDegrain(6, 1000, mode="TemporalSoften",prefilter=pref,LFR=false,RefineMotion=true)

ex_unsharp(0.3)
gr = 1
size = (720/2133.) * 3.3
GrainFactory3mod(g1str=7*gr,g2str=9*gr,g3str=7*gr,g1size=1.7*size,g2size=1.4*size,g3size=1.4*size,g1cstr=0.5*gr,g2cstr=0.3*gr,g3cstr=0.1*gr,temp_avg=1,tv_range=true)


Result not perfect but hopefully an improvement: link (https://www.mediafire.com/file/s48ohvwd8mnlczt/New_File_%2528328%2529_3.mp4/file)

[LeXXuz]

Thanks DTL and Dogway for your input. Really appreciate it. :thanks:
Something new to fiddle around with the next couple of days. :D

[simple_simon]

avspmod, I have it set not to read from source, and resolution based . You can also explictly control the RGB conversion as mentioned earlier. It doesn't matter, as long as you keep in the back of your mind how it's being done

I don't really care if the colorspace isn't displayed correctly in the temporary preview. I just want it to be correct in the final playback file.

You can insert a propshow() any point to determine if a filter is changing frame properties. If you set up avspmod correctly, you can view the actual YUV values with the color picker to see if any actual pixel value changes are occurring due to a filter. For colors, you can also test colorbars (and colorbars videos with tags) to easily see what's happening

propshow() is just going to tell me if the frame properties tags have changed. That still doesn't give me an answer about whether deep_resize is actually doing the colorspace conversion to go from SD --> HD colorspace when upsizing. Or just adding HD colorspace frame property tags. Maybe Dogway can chime in and specify what's actually going on behind the scenes.

You should notify megui developers about the preview issue

The developer of megui hasn't been very active for a number of years now, unfortunately. It's still possible to update the various external supporting tools but the actual framework gui pretty much is what it is.

[poisondeathray]

That still doesn't give me an answer about whether deep_resize is actually doing the colorspace conversion to go from SD --> HD colorspace when upsizing.

It's not currently. You get (essentially) the same results as Spline36Resize in terms of YUV values - Those are the actual values that get encoded

[simple_simon]

It's not currently. You get (essentially) the same results as Spline36Resize in terms of YUV values - Those are the actual values that get encoded

Well that really doesn't make sense for deep_resize to be changing the frame properties to 709 colorspace when it isn't. It should either assume the colorspace by the input frame resolution or leave it unspecified. Or require the user to explicitly state the input colorspace so it knows what frame property to output.

So what's the best way to handle this? Should I clear props after deep_resize to not confuse anything further along in the chain? And just encode it as 601? Or since deep_resize is flagging the video as 709 and since most playback software/hardware will assume 709 based on the HD frame resolution, should I already be converting the colorspace & range myself when upscaling to 720p or higher? Adding this before deep_resize:

ColorMatrix(mode="Rec.601->Rec.709",outputfr=true)

Or whatever the equivalent is in fmtconv. It's probably better but I haven't been able to decipher how to use it yet.

[poisondeathray]

Well that really doesn't make sense for deep_resize to be changing the frame properties to 709 colorspace when it isn't. It should either assume the colorspace by the input frame resolution or leave it unspecified. Or require the user to explicitly state the input colorspace so it knows what frame property to output.


Many growing pains in avs-land for props


So what's the best way to handle this? Should I clear props after deep_resize to not confuse anything further along in the chain? And just encode it as 601? Or since deep_resize is flagging the video as 709 and since most playback software/hardware will assume 709 based on the HD frame resolution, should I already be converting the colorspace & range myself when upscaling to 720p or higher? Adding this before deep_resize:

ColorMatrix(mode="Rec.601->Rec.709",outputfr=true)

Or whatever the equivalent is in fmtconv. It's probably better but I haven't been able to decipher how to use it yet.


I definitely would not encode and tag as 601. You have the highest chance of it playing correctly everywhere with the 601=>709 shift when going SD=>HD , and tagged as 709 in the encoder

Personally , I clear and do everything manually , because there are many "growing pains" with avisynth frame props right now. I also clear before the pipe out, because some filters screw up the props right now. I flag in the encoder instead of relying if avisynth will pass props correctly . (Some encoders do not accept, so do). You decide what works "best" for you

Yes, colormatrix, or zimg, or fmtc, or avisynthshader

avsresize would look like


z_convertformat(pixel_type=last.pixeltype, colorspace_op="170m:709:709:l=>709:709:709:l")