Fix distorted aliasing from bad upscales

[simple_simon]

Is there any way to reverse this?: https://ibb.co/G5kXwBz
It looks like the video was badly upsized from 480p to 1080p without correcting the existing line aliasing so now the aliasing is stretched and impossible to fix with normal dealiasing filters. I came up with this filter chain:

propSet("_FieldBased",0).deep_resize(720,240,grain=0,qual=2,gpuid=-1,show=false)
maa2()
deep_resize(720,480,grain=0,qual=2,gpuid=-1,show=false)
maa2()
lsfplus(strength=40,preset="slow",preblur="neo_fft3d(sigma=4,u=2,v=2)")
deep_resize(960,720,grain=0,qual=2,gpuid=-1,show=false)

which works decently but figured I'd crowdsource and see if there a better solution I'm unaware of.

[poisondeathray]

Downscale, perhaps using an inverse kernel / desampler, then use 1x_AnimeUndeint_Compact_130k_net_g, then upscale. Adjust to taste.

I fiddled a bit with a height below 480 which seemed helped better with some aliasing artifacts, you can play with that and other kernels as well. FMTC has inverse kernels too

https://imgsli.com/MjU2Mjcx

ImageSource("snapshot-02-39-411.png")
propSet("_FieldBased",0)
DeBicubicResizeMT(720,400, accuracy=2)
ConvertToPlanarRGB()
z_convertformat(pixel_type="RGBPS", use_props=0)
mlrt_ncnn(network_path="PATH\1x_AnimeUndeint_Compact_130k_net_g.onnx", builtin=false)
mlrt_ncnn(network_path="PATH\2X_DigitalFilmV5_Lite.onnx", builtin=false)
z_convertformat(width=1440, height=1080, use_props=0)

The pavement is slightly less red from one of the models. When you convert it back to YUV, smoothtweak(hue2=1) will make it more like the original

[simple_simon]

Those are a bunch of tools I'm not familiar with so it'll take me some time to play around with that (I don't even have a need to convert colorspace enough to know offhand how to convert back to YUV). But the results are damn impressive. Thanks for the suggestions.

[Emulgator]

Wow PDR, that is impressive.

[poisondeathray]

Originally Posted by simple_simon

Those are a bunch of tools I'm not familiar with so it'll take me some time to play around with that (I don't even have a need to convert colorspace enough to know offhand how to convert back to YUV). But the results are damn impressive. Thanks for the suggestions.

What I mean is the models work in 32bit RGB float (the mlrt_ncnn steps). But usually people encode to 8 or 10bit 4:2:0 YUV (but some people use 4:4:4 when models are used because some of them sharpen the chroma significantly)

The output of the script as-is before the last line is 1440x800 because the downscale was 720x400, then a 2x model was applied

The last line resamples to 1440x1080 (to compare to same as input dimensions) . If you wanted 8bit 4:2:0 , you can add pixel_type="YV12" or whatever format you're encoding to .

Models are slow - you you could use deep_resize instead like you did before instead of 2X_DigitalFilmV5_Lite - whatever you feel like . Most of the problem fixing for the aliasing is from 1x_AnimeUndeint_Compact_130k_net_g on a downscaled input . It's really good model for anime or cartoon type aliasing issues

Before commiting to the Smoothtweak adjustment, I would examine other frames with other colors as well. Look at histogram("levels") and compare the U,V channels

z_convertformat(width=1440, height=1080, pixel_type="YV12", use_props=0)
Smoothtweak(hue2=1)

If you're stuck and can't find the answers just ask and someone will help out

[lollo2]

Originally Posted by poisondeathray

Downscale, perhaps using an inverse kernel / desampler, then use 1x_AnimeUndeint_Compact_130k_net_g, then upscale. Adjust to taste.

I fiddled a bit with a height below 480 which seemed helped better with some aliasing artifacts, you can play with that and other kernels as well. FMTC has inverse kernels too

https://imgsli.com/MjU2Mjcx

Excellent result, master!

[tormento]

Originally Posted by poisondeathray

ImageSource("snapshot-02-39-411.png")
propSet("_FieldBased",0)
DeBicubicResizeMT(720,400, accuracy=2)
ConvertToPlanarRGB()
z_convertformat(pixel_type="RGBPS", use_props=0)
mlrt_ncnn(network_path="PATH\1x_AnimeUndeint_Compact_130k_net_g.onnx", builtin=false)
mlrt_ncnn(network_path="PATH\2X_DigitalFilmV5_Lite.onnx", builtin=false)
z_convertformat(width=1440, height=1080, use_props=0)

Can AVSLibPlacebo or FMTC replace the 2 z_convertformat instructions?

[DTL]

You can even try internal and eveready ConvertBits(32) to go into 32bits float RGB.

[tormento]

Originally Posted by dtl

you can even try internal and eveready convertbits(32) to go into 32bits float rgb.

rgb=rgbps?

[DTL]

RGB with 32bit (float) per sample is RGBPS. AVS do not support integer 32bit samples so ConvertBits(32) can only convert to float.

So you first convert to Planar RGB and next convert any sample format to float32. Typically RGB float32 is the only common format accepted by 'neural-networks' libraries as most fail safe and most dynamic range capable. It is not optimized for performance but easy to implement by current very poor residuals of opensource developers.

The last z_convertformat(width=1440, height=1080, use_props=0) do not change sample format and only resize so you again can use any internal resize (or any other installed).

To go back to some usable by encoders you anyway need to add something like ConvertToYV12() or if it can not make direct convert from float32 - ConvertBits(8).ConvertToYV12()

See example how to feed that NN-plugins with RGBPS format with internal conversions and back - https://forum.doom9.org/showthread.p...55#post1983255