16bit material and 8bit filter

[may24]

Hi all,
I'm not quite sure how I could use an 8bit filter correctly in an 16 bit environment.

So here is a snip of my script:
...
dfttest(Sigma=12,dither=1,tbsize=1,lsb=true)

Dither2Pre("MSharpen(strength=120)")
Dither_resize16(1280,722)

...

Is that correct ? I guess not as it produces a clip with the right width but with only half of its requested height (the other half is occupied with the MSB)

But what will happen if I'm not encapsulating MSharpen in Dither2Pre. Will it still filter the whole picture ? only LSB ? MSB ?
Does anybody can explain this to me ?

[cretindesalpes]

Dither2Pre takes an 8-bit clip as input to generate a 16-bit clip. In your example, the input is already in 16 bits (because of the lsb=true parameter in dfttest; btw, the dither parameter will have no effect here) so it will not work correctly. Dither_resize16 is expecting 16-bit input and output.

If you want to use 8-bit filters in a 16-bit workflow, check the “Combining 8- and 16-bit processing” section of the Dither documentation.

[may24]

Thanks for the support :)

So I tried the first example from the Dither doc. You'll notice I've altered the code a little but that wouldn't change its behavior:

...
s16 = last
as=s16.DitherPost(mode=-1).MSharpen(150).Dither_convert_8_to_16()
s16.Dither_limit_dif16 (as, thr=1.0, elast=2.0)
Dither_out()


The problem here: No matter how "strong" MSharpen is set, it has no impact on the resulting picture - at all.
Even if you remove the MSharpen completely, the picture doesn't change :confused:

No next ry with the second example:
...
f = DitherPost (mode=-1)
s = f.MSharpen ()
d = mt_makediff (s, f)
DitherPost()
mt_adddiff (last, d, u=2, v=2)
Dither_resize16 (1280, 720)


Using the lines above will work, but produces a ghastly squeezed picture with only half its (expected) height.
This is obviously caused by the 2nd "DitherPost()" and the "Dither_resize16 (1280, 720)". But removing it will cause avisynth to crash - almost immediately ... sometimes the first frame got processed but no later than the second pic, it will die.
Only a "general exception" is shown ...

Point is: I'll want to keep it to 16bit till the end. Then reduce it to 10bit and "Dither_Out()" so I can send it to x264.

Any suggestions ?

[may24]

What do you think of this sollution ?
dfttest(Sigma=12,tbsize=1,lsb=true)

o=last
msb = o.Dither_get_msb()
lsb = o.Dither_get_lsb()
smsb=msb.MSharpen(strength=100)
StackVertical(smsb,lsb)
Dither_resize16(1280,720)

GradFun3(lsb=true, lsb_in=true)

Dither_quantize(bitdepth=10, mode=7,reducerange=true)
Dither_out()

[cretindesalpes]

I would recommend that you stick with your first proposal, based on Dither_limit_dif16(). It works, the only problem was the parameter in MSharpen.

The second proposal does all the operations in 8 bits, moreover Dither_resize16() is passed a 8-bit clip whereas it expects a 16-bit one.

The third proposal may cause slight artefacts when replacing the MSB while leaving the LSB as is. You could fix the problem using DitherBuildMask() but this method is not completely safe and gives inferior results compared to the first one.