Help Inserting Resizing Between Denoising And Line Darkening

[LouieChuckyMerry]

Happy Saturday! I'm wanting to insert a resize after the denoising but before the line darkening in an existing script, but I'm having trouble negotiating the transition from 16-bit to 8-bit back to 16-bit. The existing script is:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,Chroma=False,PreFilter=2,Lsb=True,Lsb_Out=True)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

and the resize script is:

### Resize ###
NNEDI3_RPow2(4,CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp2(Depth=5)
Sharpen(0.2)

I understand that SMDegrain is processing in 16-bits then outputting 16-bits, I understand what "DitherPost(Mode=-1)" means, I understand what the FastLineDarkenMod-aWarpSharp2 line is doing, and I understand the "Dither_Convert_8_16()" call, but I can't wrap my head around the red text. This:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,Chroma=False,PreFilter=2,LSB=True,LSB_Out=False)
### Resize ###
NNEDI3_RPow2(4,CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp2(Depth=5)
Sharpen(0.2)
### Darken-Thin Lines ###
Dither_Convert_8_To_16()
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

works, but I don't know if it's the most elegant solution (or if it's even correct). I guess what I'm really wondering: is there a "proper" way to incorporate the resizing with the red text? Thanks for any clarity :) .

[Motenai Yoda]

you are trying to merge 2 clips with different resolutions.

also jpsdr's nnedi3 works on avs+ 16bit too https://forum.doom9.org/showthread.php?t=170083

[LouieChuckyMerry]

you are trying to merge 2 clips with different resolutions.

Thanks for your reply. Would you please be more specific? Maybe I can learn something ;) .

also jpsdr's nnedi3 works on avs+ 16bit too https://forum.doom9.org/showthread.php?t=170083

Thank you for the link; I'll check it out now.

[StainlessS]

You got a typo in first line of code (double comma).

Just curious, what is original frame size (bigger/equal 240x180 AND smaller than 480x360) ? ... [ NNEDI3_RPow2(4) ]

EDIT: Not sure I got them there bigger than smaller than numbers correct.

[LouieChuckyMerry]

You got a typo in first line of code (double comma).

Damn cut-and-paste...


Just curious, what is original frame size (bigger/equal 240x180 AND smaller than 480x360) ? ... [ NNEDI3_RPow2(4) ]

EDIT: Not sure I got them there bigger than smaller than numbers correct.

PAL DVD, 720x576. I went with "4" because the thread from which I stole the upscale bit of the script states, to paraphrase: use 2 but change it to 4 if you've the patience because it'll look better. Thanks jagabo :) .

[StainlessS]

nnedi3_rpow2:- http://avisynth.nl/index.php/Nnedi3/nnedi3_rpow2

int rfactor =

Image enlargement factor. Must be a power of 2 in the range [2 to 1024].

Use 2, ie double original size (and then uses CShift, fWidth, & fHeight to correct the image center shift and output correct downscale size).
In other words, nnedi3_rpow2(2, ...) to double size and then downscale. (original script would quadruple size, and then down scale which would be slower than necessary and maybe not as good qual [methinks]).

No idea about the rest of script, I gorra get a bit more into the hi bit stuff.

EDIT: Some stuff about src_left,src_top removed.

[LouieChuckyMerry]

nnedi3_rpow2:- http://avisynth.nl/index.php/Nnedi3/nnedi3_rpow2

Use 2, ie double original size (and then uses CShift, fWidth, & fHeight to correct the image center shift and output correct downscale size).
In other words, nnedi3_rpow2(2, ...) to double size and then downscale. (original script would quadruple size, and then down scale which would be slower than necessary and maybe not as good qual [methinks]).

No idea about the rest of script, I gorra get a bit more into the hi bit stuff.

EDIT: For center shift correct, I think that you may have to add this (see above linked doc examples, otherwise only resize, no center shift [methinks] )
src_left=-0.5, src_top=-0.5

I searched for many hours but I couldn't find the original post that led me to believe it's better to use "4" instead of "2", when possible. My memory won't win any awards but I still think this is correct. And from what I've read, the shift correct is covered by the "CShift"; I believe that "src_left=-0.5, src_top=-0.5" is for a special case.

[StainlessS]

EDIT: Ignore prev posted stuff about src_left/right, not arg in Nnedi3_rpow2

The CShift (resizer name) is just to resize to desired output size (and shift center correct).
I personally dont [usually] care enough about it to be so bothered about centering, Centering required due to way nnedi3_rpow2() works (keeps some original pixels and synthesizers those in between).
Nnedi3_rpow2 produces a shifted output (after upsize due to above synthesized pixels, before resize downscale).
Think this is how it works:::
If rFactor=4, then every output pixel coord that is evenly divisible by 4, is original, others synthesized.
If rFactor=2, then every output even pixel is original, every odd is synthesized.

You use rFactor = 2 where resizing larger than source and less or equal to 2* source dimension.
You use rFactor = 4 where resizing larger than 2xSource and less or equal to 4* source dimension.
Above rough estimate, eg if only a little over double size then might prefer to just use rfactor=2, significantly faster with little difference in result.
If upscaling by eg 5%, then hardly worth using Nnedi3_rpow2() at all, simple resize would be a lot faster and little difference.

If less than doubling size (as in your case), then rFactor=2 is already bigger than required and rpow2 upscale by 4 IS OVERKILL [if he is dead, stop wasting bullets].
EDIT: In your case, when downsize via CShift, if using rfactor=4, then would be using 1 src pixel for every group of 4 pixels [1 src and 3 synth],
when rfactor=2, would use 1 src pixel for every pair of pixels, obviously rfactor=2 is [in this case] better [and quicker].

EDIT:
Upsize shifted, O=Original Pixel, S synthesized, where rfactor=2, original Image O shifted left (needs right shift when downsize, ideally shifting src O mid way between upsize O and S ie 1/2 pixel shift)

OSOSOSOS


Upsize shifted, O=Original Pixel, S synthesized, where rfactor=4, original Image O shifted left (needs right shift)

OSSSOSSSOSSSOSSS


EDIT: If any of above wrong, then someone please say.

[LouieChuckyMerry]

Thank you very much for the information, StainlessS :) . I'll save some ammunition and use "RFactor=2" instead of 4. Of course, in the process of trying to figure out how to use NNEDI3_RPow2 in a 16-bit chain I encountered NNEDI3_Resize16, so now I have to learn which suits my needs best by reading the NNEDI3 thread (https://forum.doom9.org/showthread.php?t=170083) (the high bit depth bit begins here (https://forum.doom9.org/showthread.php?p=1811629#post1811629)).

[StainlessS]

Thanx LCM, I was totally unaware of that NNEDI3_Resize16, I'll take a gander later (got to leave soon).

[Motenai Yoda]

Thanks for your reply. Would you please be more specific? Maybe I can learn something ;)

into
### Overall Temporal Denoise ###
SMDegrain(Lsb=True,Lsb_Out=True)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

F is smdegrain 16 bit output rounded to 8 bit
S is filtered 8 bit clip
D is the difference between S and F (what the 8bit filter stage added), converted to 16bit
Dither_Add16 re-add that difference into last which is the 16bit smdegrain output.

this way is pretty lame as lsb data isn't related to what is in msb and while it can be useful with filters like linedarker changing only few masked pixels, I'm not sure about a warpsharp one

it's like as you do
round(pi) * 5 + (pi - round(pi)) = 3*5+.14 = 15.14
where
pi*5 = 15,7

If you add a resize in the 8bit stage, D resolution change but last doesn't