Resize - Downscale - aliasing

[speedyrazor]

Hi, I am using the below script to convert HD (1920x1080) Prores to PAL (720x576) Prores, which most of the time give me extremely good results. Occasionally the quality is a little 'too' good, in that some very detailed images are aliasing on the down-convert (too much detail). What would be the best way to deal with this aliasing? I am using "spline36", would there be a better kernel to use for down-converting?

import vapoursynth as vs
core = vs.get_core(threads=4)
ret = core.lsmas.LibavSMASHSource(source=r"QuicktimeProresSample.mov")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="444", kernel="spline36")
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, css="420")
ret = core.fmtc.bitdepth (clip=ret, bits=10)
retFinal = ret
retFinal.set_output()

Kind regards.

[feisty2]

bicubic?

[Motenai Yoda]

I would rearrange this way
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="420", kernel="spline36")
ret = core.fmtc.bitdepth (clip=ret, bits=10)

[speedyrazor]

I would rearrange this way
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="420", kernel="spline36")
ret = core.fmtc.bitdepth (clip=ret, bits=10)

What would be the advantage of doing it this way?

[Motenai Yoda]

- colorimetry change on full resolution and best color sampling
- avoid to resample chroma planes twice

[sneaker_ger]

Doesn't fmtconv still require 4:4:4 for matrix change? If his source is 4:2:2 - like typical for ProRes - he would run into an error if he tried to do that directly after the source filter.

[Motenai Yoda]

Yep my bad
add on top
ret = core.fmtc.resample (clip=ret, css="444", kernel="spline36")

[speedyrazor]

Yep my bad
add on top
ret = core.fmtc.resample (clip=ret, css="444", kernel="spline36")

So, just to be clear, you are suggesting:

ret = core.fmtc.resample (clip=ret, css="444", kernel="spline36")
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="420", kernel="spline36")
ret = core.fmtc.bitdepth (clip=ret, bits=10)

So going back to my original question, would this help with the aliasing, and be a better conversion?

[AzraelNewtype]

It would be a better conversion, though you'll have to tell us whether it helps with the aliasing, since we haven't seen your source. If it doesn't, using a less sharp kernel, like feisty2 suggested, certainly will. You can start with spline16 before stepping down to cubic I guess. Testing and using your own eyes is always going to be faster than asking a forum for a priori judgments of output either way.

[speedyrazor]

I just wanted to check I am making the best use of the script, although now I have two "spline36", is that correct?


ret = core.fmtc.resample (clip=ret, css="444", kernel="spline36")
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="420", kernel="spline36")
ret = core.fmtc.bitdepth (clip=ret, bits=10)

Still doing a "resample chroma planes twice"?

[jackoneill]

Still doing a "resample chroma planes twice"?

Yes. You mostly can't avoid that if 1) your input is subsampled and 2) your output needs to be subsampled and 3) you want to use fmtc.matrix.

Protip: if you use resize.Spline36 or any of the other built-in resizers, you won't see these details and so maybe you won't worry about them. Out of sight, out of mind.

[speedyrazor]

Protip: if you use resize.Spline36 or any of the other built-in resizers, you won't see these details and so maybe you won't worry about them. Out of sight, out of mind.

I'm not sure I understand what you mean. Could you elaborate please?

[jackoneill]

I'm not sure I understand what you mean. Could you elaborate please?

You don't see the order of operations. zimg picks a suitable order:

clip = core.resize.Spline36(clip, width=720, height=576, format=vs.YUV420P10, matrix_in_s="709", matrix_s="470bg")

Untested, but it should work.

[speedyrazor]

Untested, but it should work.

So instead of:

import vapoursynth as vs
core = vs.get_core(threads=4)
ret = core.lsmas.LibavSMASHSource(source=r"QuicktimeProresSample.mov")
ret = core.fmtc.resample (clip=ret, w=720, h=576, css="444", kernel="spline36")
ret = core.fmtc.matrix (clip=ret, mats="709", matd="601")
ret = core.fmtc.resample (clip=ret, css="420")
ret = core.fmtc.bitdepth (clip=ret, bits=10)
retFinal = ret
retFinal.set_output()

Do this?:

import vapoursynth as vs
core = vs.get_core(threads=4)
clip = core.lsmas.LibavSMASHSource(source=r"QuicktimeProresSample.mov")
ret = core.resize.Spline36(clip, width=720, height=576, format=vs.YUV420P10, matrix_in_s="709", matrix_s="470bg")
ret.set_output()

I will do some tests, but in your opinion, would this yield a 'better' conversion potentially?

Regards.

[Motenai Yoda]

Still doing a "resample chroma planes twice"?
yes if your input isn't 444 yet, but this is upsampling->downsampling not downsampling->downsampling

also jackoneill's line should be the same internally as it has to change colorimetry as well.

for the aliasing... a less sharp resize like spline64 spline16 or bicubic should avoid it a bit more, or prefilter with a light blur.