Good resizer from 2160p to 1080p [Archive]

Boulder

19th March 2021, 18:01

Boulder, that sounds intriguing. Would you share your Zopti script?

This is the base I use with Vapoursynth Zopti. The AVI file is created by using SelectEvery to get a range of frames over the whole clip to make sure it's like an average representative, encoded as FFV1 so it's lossless. I usually use a 200-frame clip.
So it will basically compare the original clip with one that has been downsized and upsized back to the original size.

The metric GMSD will favour a slightly softer result. With HDR sources, set hdr=True.

from vapoursynth import core
import resamplehq as rhq
from zoptilib import Zopti

orig = core.ffms2.Source(source=r"c:\zopti\doctorwho_s02e00.avi")

zopti = Zopti(r'results.txt', metrics=['mdsi', 'time'], matrix='709')

b = -40/100.0 # optimize b = _n_/100.0 | -150..50 | b
c = 40/100.0 # optimize c = _n_/100.0 | -100..100 | c
downscaled_width = 1248
downscaled_height = 720
upscaled_width = orig.width
upscaled_height = orig.height

alternate = rhq.resamplehq(orig, width=downscaled_width, height=downscaled_height, kernel="bicubic", filter_param_a=b, filter_param_b=c, hdr=False)
alternate = core.resize.Lanczos(alternate, width=upscaled_width, height=upscaled_height)
zopti.run(orig, alternate)

The analysis command:
zopti compare.vpy -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script

Boulder

19th March 2021, 18:05

I've also done a little helper function to recreate with AVSResize what ResampleHQ does.

function Resizer(clip clp, int "width", int "height", float "b", float "c", bool "hdr", bool "i", bool "z", bool "spline", int "color", bool "debug") {

b = default(b, 0)
c = default(c, 0)
hdr = default(hdr, clp.width() > 1920 ? true : false)
i = default(i, false)
z = default(z, true)
spline = default(spline, b == 0 && c == 0 ? true : false)
color = default(color, hdr ? 0 : clp.width() > 720 ? 2 : clp.framerate == 25 || clp.framerate == 50 ? 4 : 3)
debug = default(debug, false)

result = clp.ConvertYUVtoLinearRGB(threads=1, prefetch=8, color=color, outputmode=2)
result = z && !spline ? result.z_ConvertFormat(width, height, chromaloc_op="auto=>same", colorspace_op="auto=>same", resample_filter="Bicubic", filter_param_a=b, filter_param_b=c, pixel_type="RGBPS") : !z && !spline ? result.BicubicResize(width, height, b, c) : z && spline ? result.z_ConvertFormat(width, height, chromaloc_op="auto=>same", colorspace_op="auto=>same", resample_filter="Spline36", pixel_type="RGBPS") : result.Spline36Resize(width, height)
result = result.ConvertLinearRGBtoYUV(threads=1, prefetch=8, color=color, outputmode=2)

result = i ? interleave(clp, result.LanczosResize(clp.width(),clp.height())) : result
result = debug ? result.subtitle("hdr: " + string(hdr), x=0, y=0).subtitle("color: " + string(color), x=0, y=20) : result

return result
}

wonkey_monkey

19th March 2021, 18:36

I've also done a little helper function to recreate with AVSResize what ResampleHQ does.

That seems to introduce a lot of ringing around edges (x4 zoom):

https://i.imgur.com/JsWCbeR.png

Boulder

19th March 2021, 19:22

Of course, you can change the resampler to for example a user-defined value, in that one it's fixed to Bicubic. I use that function with the value the Zopti run gives me for Bicubic.

tormento

20th March 2021, 22:54

This is the base I use with Vapoursynth Zopti.
Can you create something for AVS+? :)

Boulder

21st March 2021, 11:06

Can you create something for AVS+? :)

Sorry, don't have the time for that since the VS version suits my needs perfectly. This thread has the whole thing explained, you just need to build the script based on the instructions spread over a couple of posts.

https://forum.doom9.org/showthread.php?t=175723

I'm not sure if Avisynth has the plugins to measure using MDSI or GMSD though. Setting VS up is easy and if you don't need it for anything else, there's not much to learn as you can just adapt that sample script and run Zopti. zopti -mode evaluate -log "*.log" will then output the optimal values from the log file(s).

StvG

21st March 2021, 11:16

Sorry, don't have the time for that since the VS version suits my needs perfectly. This thread has the whole thing explained, you just need to build the script based on the instructions spread over a couple of posts.

https://forum.doom9.org/showthread.php?t=175723

I'm not sure if Avisynth has the plugins to measure using MDSI or GMSD though. Setting VS up is easy and if you don't need it for anything else, there's not much to learn as you can just adapt that sample script and run Zopti. zopti -mode evaluate -log "*.log" will then output the optimal values from the log file(s).

One can check this post (https://forum.doom9.org/showthread.php?p=1926987#post1926987) for avs+ MDSI/GMSD...

Selur

22nd March 2021, 16:55

Try using ResampleHQ (http://int64.org/projects/resamplehq/)
Unfortunately 32-bit dll only.
there is also a 64bit dll: https://forum.doom9.org/showthread.php?p=1722300#post1722300

FranceBB

23rd March 2021, 12:29

Hi Tormento,
we already spoke on Telegram, but I'm gonna write it here as well just so that other can see it as well. :)

So... for your particular case, which is taking 4:2:0 Type 2 HDR PQ UHD contents from BDs and downscale them to FULL HD, I would do the following:

#Indexing
FFMpegSource2("file.m2ts", atrack=-1)

#Change chroma location from 4:2:0 Type 2 to the old one
z_ConvertFormat(chromaloc_op="top_left=>mpeg2")

#Reverse upscale the luma and shift it to keep it aligned
Y = ConvertToY(matrix="Rec2020").DeBilinearResizeMT(1920, 1080, src_left=-0.50, src_top=0)

#Keep the chroma as it is
U = UToY()
V = VToY()

#Merge the downscaled luma with the original chroma to get a 4:4:4 10bit HDR PQ clip
YToUV(U, V, Y)

#Bring everything to 16bit planar
ConvertBits(16)

#Bring everything from YUV to XYZ
ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)

#Tonemap HDR PQ to Linear BT709 SDR 100 nits
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.5, contrast_X=0.9)

#Bring everything back from XYZ to YUV
ConvertXYZtoYUV(pColor=0)

and at this point I would be piping this 16bit planar to x264 ad encode in FULL HD 4:4:4 BT709 10bit planar.

This comes from the fact that UHD 4:2:0 sources have the luma channel (Y) as 3840x2160, however the chroma channels U and V are at half resolution, namely 3840:2 and 2160:2, so 1920x1080.
In other words, you can effectively take the luma from the source, downscale it and merge it with the original chroma of the source to get a real 4:4:4.

The second thing I want to add is that some of the very first "old" 4K contents are upscaled with either a dumb bilinear or bicubic and if they are you can easily invert the kernel to get back the original 2K picture. As to the chroma, that is NOT upscaled as 2K masters are generally 4:4:4, so you would get back something pretty good in the end. On the other hand, nowadays we're getting closer and closer to 8K and many studios are effectively recording in 6K and downscaling in 4K, so you can't use the inverse upscale method if the content is truly 4K which leaves you with a number of choices to make about which downscaler to pick. All the above suggestions are good, but as to me particularly, I can say that one of the more promising one is the Sin Squared Kernel which has been integrated in Plugins_JPSDR as you can see from here: Link (https://forum.doom9.org/showthread.php?t=181612)

As always, I hope it helps other people, that's why I generally like to post things on Doom9.

Cheers,
Frank

tormento

23rd March 2021, 14:44

I'm gonna write it here as well just so that other can see it as well. :)
Very kind of you :)

StvG

23rd March 2021, 18:46

...
#Change chroma location from 4:2:0 Type 2 to the old one
z_ConvertFormat(chromaloc_op="left=>top_left")
...

From top left to left location you want it:
z_ConvertFormat(chromaloc_op="top_left=>left")

Boulder

23rd March 2021, 20:46

I use z_ConvertFormat(chromaloc_op="top_left=>mpeg2") to make sure I don't mess anything up by mixing the items :D

FranceBB

23rd March 2021, 22:02

I use z_ConvertFormat(chromaloc_op="top_left=>mpeg2") to make sure I don't mess anything up by mixing the items :D

And you're absolutely right to do it 'cause I just messed it up xD
I edited it in the other post with your option, so now it's not only correct but also much more clear. Thanks ;)

tormento

24th March 2021, 08:55

ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)
Please explain me the advantages to use XYZ space.

frank

24th March 2021, 13:37

XYZ is the physically basic color space. Colors of real world.
It is used by the following tonemapping function, and primaries are adapted.

But why changing colorspace? The question was only for resizing.
Stay on HDR or convert to HLG.

Downsizing from UHD to FHD is a special case:

2/1 3840x2160 -> 1920x1080
Ringing and artefacts are mostly negligible.

Use Spline36Resize()

If you sit 2.5 x screen height in front of TV (recommended for 2k) you will get a nice picture.

FranceBB

24th March 2021, 14:41

But why changing colorspace?
Stay on HDR or convert to HLG.

That's what I proposed initially via message but he said that he's using a box that doesn't support tonemapping on the fly so PQ looks awful due to the logarithmic nature of the curve on his FULL HD TV and HLG still looks awful due to the BT2020 being interpreted as BT709 by his TV... So he doesn't have much choice...

The question was only for resizing.

True, I just happened to know his setup as we became friends so I brought it a bit further to a more complete answer for *his* issue, that's all. ;)

Bye Frank... from... me, Frank! ;)
(ok, I should really stop doing that xD)

frank

24th March 2021, 14:52

OK, understand. :)

WorBry

24th March 2021, 19:34

One can check this post (https://forum.doom9.org/showthread.php?p=1926987#post1926987) for avs+ MDSI/GMSD...

Excellent. vsSSIM also. Love the color-enhanced maps.

WorBry

25th March 2021, 02:54

DTL

26th March 2021, 15:18

I have read many posts to upsample sources from SD to HD but I miss something good to downsample from 2160p to 1080p.

What can I use, avoiding moiré and artifacts?

jpsdr have added UserDefined2ResizeMT to ResampleMT plugin pack - https://github.com/jpsdr/ResampleMT/releases/tag/2.3.2

You can adjust 'local peaking' by adjusting b anc c control params for the required 'sharpness looking' while keeping ringing low enough.

It uses b and c control params same as Bicubic. Starting values for 1/2 downsample is about b=102 c=2. The final values depend on:
1. Current 'sharpness looking' of source.
2. Target 'sharpness looking' of result.

So there is no auto-calculation way of defining coefficients based on downsample ratio.

Other possible built-it Avisynth resizer is only GaussResize, because all other do not put enough efforts for dowsampling content-production task. Though Gauss have less control on combination 'sharpness/ringing/aliasing'. And typically produces less sharper result in case of fixed ringing level.

All resizer's from that pack process in 'input domain' so to process in 'linear domain' it is required to convert to linear (perfferably 16bit or float) before calling resize function and after resizing and all other processing done convert to target rec.709 OETF.

BicubicResize(x, y, b=-0.6, c=0.3). I use Zopti to determine more optimal values for b and c using the MDSI metric.

That is interesting. I think about some tool for adjusting same 2 result-defining control params for UserDefined2ResizeMT() for exact source spectrum properties and downsample ratio. Because it is not easy for average user to understand relationship between more 1 control params and output result. It uses same control params syntax as BicubicResize but different control params range (like about -50..200 or larger). If some software can search for best 'b' and 'c' minimizing some metric with lowest ringing and user-prefferable sharpness it will be great. If we have auto-adjustment of params software the number of control params may be extended to 3 or 4 for better anti-ringing work. But for average user with hand-setup of params even 2 is hard. So the first (older) downsizer was SinPowResizeMT() with only one control param. Though more limited in the quality of output result.

But it currently only avisynth plugin.

#Bring everything from YUV to XYZ

Do it actually converts Y'Cr'Cb' (source system OETF'ed) to linear XYZ domain ?

As latest researches shows the scaling processing should be done with linear data.
This possibly also followed from by ITU Rec.709 4.1 : Coded signal - R, G, B or Y, CB, CR. The signals are listed as RGB, not OETF-corrected R'G'B' or Y' CR' CB'.

If we try to make some scaling with TF-processed data (old named 'gamma-corrected') we got more distortions because TF-processing damages initially prepared and conditioned spectrum (in the case the source was actually prepared this way). So these distortions cause additional ringing/aliasing. And to remove that distortions we need to EOTF data back to linear. Though it may be also source-dependent to it is recommended to make test with both linear and non-linear data domains and compare results. Unfortunately I still can not found any relevant industry standard to put any light directly on this question :( .

real.finder

27th March 2021, 01:26

Anyone up for implementing that ? Asd-g doesn't appear to be a member on the forum.

you can open issue here https://github.com/Asd-g/AviSynthPlus-Scripts/issues

WorBry

27th March 2021, 05:22

OK thanks.

DTL

27th March 2021, 14:33

Create some info-graphic for better understanding different moving pictures data sources properties and development of spectrum-shaper low-pass filter for downscaling work from 2160 to 1080 (or any other downscale work for digitally coded moving pictures data).
https://i4.imageban.ru/thumbs/2021.03.27/5b146cfc56c4956f33bab5c074c6671f.png (https://imageban.ru/show/2021/03/27/5b146cfc56c4956f33bab5c074c6671f/png)
full size image - https://i4.imageban.ru/out/2021/03/27/5b146cfc56c4956f33bab5c074c6671f.png

https://i1.imageban.ru/thumbs/2021.03.27/e49f70960007ca538d98ac8ae80bc4f4.png (https://imageban.ru/show/2021/03/27/e49f70960007ca538d98ac8ae80bc4f4/png)
full size image - https://i1.imageban.ru/out/2021/03/27/e49f70960007ca538d98ac8ae80bc4f4.png

The curves is not exact ofcourse - just to show relationship between different data spectrums.

So for UserDefined(N)Resize() the control params are actually defined in typical 16-235 range encoded (i.e. 0=16, 1.0=235, sub16 is negative etc) coefficients of impulse response of spectrum-shaper low-pass filter. And the exact values differ at least for different source spectrum properties and user-prefferable output sharpness/crispness looking.

That work with BicubicResize b and c params is some approximation of the same process I think. Though Bicubic is general image-processing resizer - not specially designed for moving pictures (video) data. So the max possible quality of simple BicubicResize is limited.

zorr

27th March 2021, 22:42

I asked in that Zopti thread how it might be possible to print a log file that compiles the per-frame metric test results. Zorr's response:

https://forum.doom9.org/showthread.php?p=1938965#post1938965

Anyone up for implementing that ? Asd-g doesn't appear to be a member on the forum.

Sorry, I should have read the implementation more carefully. It's easy to read the value from the frame properties so there's no need to add another way (unless you want something that also works with classic Avisynth).

I will do some tests and post an example script later.

zorr

28th March 2021, 01:20

I posted an example script using GMSD here (https://forum.doom9.org/showthread.php?p=1939226#post1939226).

Boulder

30th March 2021, 15:23

I launched my test script to check out UserDefined2ResizeMT ten hours ago and it's still calculating the optimal b and c :D

Is there some snappy Avisynth trickster like SSS or Gavino who could take a look at the function if it could be make threadsafe with Prefetch? This is the part in which it trips over itself.

global total = 0.0
global mdsi_total = 0.0
FrameEvaluate(last, """
global mdsi = propGetFloat("_FrameMDSI")
global mdsi = (mdsi == 0.0 ? 1.0 : mdsi)
global mdsi_total = mdsi_total + mdsi
""")
# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (mdsi, time)
delimiter = "; "
resultFile = "perFrameResults.txt" # output out1="mdsi: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "mdsi", "delimiter", "avstimer")

# write "stop" at the last frame to tell the optimizer that the script has finished
frame_count = FrameCount()
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "mdsi_total", append=true)

real.finder

30th March 2021, 15:43

if it could be make threadsafe with Prefetch? This is the part in which it trips over itself.

last avs+ should work with runtime scripts like this with no problem, but maybe it need using last update of grunt https://github.com/pinterf/GRunT/releases with/without clean it to not use use "global" things

DTL

30th March 2021, 16:58

I launched my test script to check out UserDefined2ResizeMT ten hours ago and it's still calculating the optimal b and c :D

Hmm - I think it may be simple all-combinations search with some reasonable step for each param.

If brute-force (exhaustive ?) search with all b and c pairs with 1 step in range -50..250 it probably takes 300x300=90000 runs. And for faster speed may be only a small range of frames and small cropped part of frame (may be prefferably center of frame as most important generally) to be used.

Avisynth's resampler V+H is fast enough and if task size fits CPU cache at least L2/L3 it have to run fast enough. With about 100..1000+ fps.

Can it be run with Avisynth or it Vapoursynth only ?

Boulder

30th March 2021, 18:07

Hmm - I think it may be simple all-combinations search with some reasonable step for each param.

If brute-force (exhaustive ?) search with all b and c pairs with 1 step in range -50..250 it probably takes 300x300=90000 runs. And for faster speed may be only a small range of frames and small cropped part of frame (may be prefferably center of frame as most important generally) to be used.

Avisynth's resampler V+H is fast enough and if task size fits CPU cache at least L2/L3 it have to run fast enough. With about 100..1000+ fps.

Can it be run with Avisynth or it Vapoursynth only ?
Zopti does a heuristic search (though it can be set to brute force), now it's on the 355th step. Usually the optimal result is found in 100-150 steps but probably my initial values were too far from them.

https://i.ibb.co/71QXmVN/zopti.png (https://ibb.co/71QXmVN)

The slow thing is the MDSI calculation. My sample clip is 3840x1608 and contains 400 frames so there's a lot of stuff involved.

For these jobs, I've used Avisynth. I didn't test this script as I combined my helper resizer etc. here but the idea can be seen if anyone would like to try it. I run the job with command line zopti testscript.avs -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script

orig = FFVideoSource("c:\zopti\hobbit_auj.avi")

b = 80/1.0 # optimize b = _n_/1.0 | -50..250 | b
c = -20/1.0 # optimize c = _n_/1.0 | -50..250 | c

downscaled_width = 1920
downscaled_height = 808

alternate = orig.ConvertYUVtoLinearRGB(threads=24, prefetch=1, color=1, outputmode=2)
alternate = alternate.UserDefined2ResizeMT(downscaled_width, downscaled_height, b=b, c=c)
alternate = alternate.ConvertLinearRGBtoYUV(threads=24, prefetch=1, color=1, outputmode=2)
alternate = alternate.LanczosResizeMT(orig.width(), orig.height())
alternate = alternate.z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f", cpu_type="avx2")

orig = z_ConvertFormat(orig, pixel_type="RGBP16", colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f", cpu_type="avx2")

MDSI(alternate, orig, show=false)

global total = 0.0
global mdsi_total = 0.0
FrameEvaluate(last, """
global mdsi = propGetFloat("_FrameMDSI")
global mdsi = (mdsi == 0.0 ? 1.0 : mdsi)
global mdsi_total = mdsi_total + mdsi
""")
# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (mdsi, time)
delimiter = "; "
resultFile = "perFrameResults.txt" # output out1="mdsi: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "mdsi", "delimiter", "avstimer")

# write "stop" at the last frame to tell the optimizer that the script has finished
frame_count = FrameCount()
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "mdsi_total", append=true)

DTL

30th March 2021, 19:06

" sample clip is 3840x1608"

Downsampling task do not depends on frame size. So for fast close to optimal results may be found with something like 380x160. But because the most of operation we want to check performed with high and highest frequencies - the frame must contain lots of small details and sharp transitions.

So it is possible to get quickly initial values and later try to check if there any convergence to some exact values if we increase number of frames and frame size.

But I still do not understand how optimizer for downsampler works because there must be some metric algorithm to compare 2 frames with different sizes. And mathematically they have different amount of data. For transient optimizing the task looks simplier enough - we have 'ideal' source transition with over/under shoots (video-look) or without (film-look) with ringing suppressed. And after we made downsampled output we check it for minimum of integral of difference between expected transition (also video or film look) and the current output result. And optimizing target is minimum of integral value.

When downsampling we push highest frequencies to the possible limits (in the lower sample count output domain) so the quality of HF restoration is important and it is better to use as much taps as possible. The upper possible 100 value looks like too much so I think something like 20 is more acceptable for better speed.

As I see from script the compared for MDSI metric clips are the same size and upsampler is

alternate = alternate.LanczosResizeMT(orig.width(), orig.height())

The default LanczosResize taps value is small enough like 3. I think for research better to use taps > 10 at least. Current Avisynth+ looks like relaxed upper limit to 100. So to get more full-strike sinc lobes and with Lanczos weighting it is good to have about N*2..N*3+ Lanczos taps value to have N lobes of sinc do not faded significally to help sinc to made work of restoring highest valid frequencies as best as possible.

Also the more severe error: if downsampling is performed in linear domain - the upsampling also must be performed in linear domain because OETF performes harmonic distortions and you get worse result any way.

In current script you try to downsample in linear domain and upsampling for metric in system-TF domain after ConvertLinearRGBtoYUV(threads=24, prefetch=1, color=1, outputmode=2) as I think.

The only practic task for this script may be if you optimize ONLY for target display system with LanczosResize(taps=3) scaler in TF-domain. It may be process of optimizing for specific hardware display system using this way of data processing.

Though because target display workflow (reference scaler and TF or linear scaling) for moving pictures digital data looks like still is not put to standard it is really a question to study. Current ideas close to upscale in linear domain at the display system point.

Boulder

30th March 2021, 19:39

The whole idea is to find optimal values for b and c when I want to downsample from the original resolution but playback occurs at 4K or 1080p. That's why I downsample with a high-quality method but then use a simple LanczosResize to upsize back to the original size as that's what I expect my media player to be able to do at max. Then I compare the downsampled-upsampled clip with the original, trying to find b and c which cause the least difference in the downsampling.

The test process finished now and I compared the results of Bicubic (b=-1.16, c=-0.2, MDSI score total 71.17249) and UserDefined2ResizeMT (b=2, c=58, MDSI 73.05806). The latter one is blurrier and shows more haloing in this particular case which probably explains the higher score (less is better). Upon playback, I would not be able to distinguish them and it is very, very hard to tell the result from the original to be honest. (The Hobbit - An Unexpected Journey is not a very high quality 4K release IMO, that is why 1080p will do. It's not native 4K no matter what they say.)

DTL

30th March 2021, 19:47

" That's why I downsample with a high-quality method but then use a simple LanczosResize to upsize back to the original size as that's what I expect my media player to be able to do at max."

It still do not covers the second question: Do media player perform upscale with input-domain (system-TFed) or linear after converting to linear ? It possibly may be tested with test pattern like zone plate being feed to media player in linear and OETFed form and to look at the upscaling result.

"(b=2, c=58, "

That is strange because I think typically b > c. If b < c it possibly result to ringing.

Boulder

30th March 2021, 19:59

I'll try to do another test run tomorrow, this one was HDR so the image is very flat and the Bicubic's values are also typical for that. Many Blu-ray sources have produced values around -40..-60 for b and 20..40 for c.

I'll get a regular Blu-ray sample so we'll see what the difference is there.

DTL

31st March 2021, 09:04

Well - my quick hand-picked test with possibly good 'film-looking' conditioned 55,170 transition between 16,235 levels (in linear domain):

Source image (ms-paint designed as usual):
https://i1.imageban.ru/out/2021/03/31/555488cd644903c6f53cab677009d531.png (https://imageban.ru)
direct link https://i1.imageban.ru/out/2021/03/31/555488cd644903c6f53cab677009d531.png

LoadPlugin("ResampleMT.dll")

ImageReader("55_170.png")

ConvertToYV12(matrix="Rec601")

a=last
b=Invert()
StackHorizontal(a,b)

#UserDefined2ResizeMT(width/2, height/2, b=132, c=27) # softer
#UserDefined2ResizeMT(width/2, height/2, b=120, c=18) # softer
#UserDefined2ResizeMT(width/2, height/2, b=102, c=2) # softer
UserDefined2ResizeMT(width/2, height/2, b=90, c=-7) # 1/2 film->film no-overshoot/halo ?
#UserDefined2ResizeMT(width/2, height/2, b=80, c=-15) # sharper, low halo
#UserDefined2ResizeMT(width/2, height/2, b=70, c=-21) # sharper, mid halo
#UserDefined2ResizeMT(width/2, height/2, b=60, c=-32) # sharper, 1/2 film->video
#UserDefined2ResizeMT(width/2, height/2, b=50, c=-42) # start to ring ?

SincLin2ResizeMT(width*4, height*4, taps=32)

So I think typical b and c combinations are around above shown.

The 'no change in sharpness' 1/2 resize with b=90, c=-7 gives this image:
https://i1.imageban.ru/thumbs/2021.03.31/0c2a78e3ea5040427f2e522ff82ea947.png (https://imageban.ru/show/2021/03/31/0c2a78e3ea5040427f2e522ff82ea947/png)
direct link https://i1.imageban.ru/out/2021/03/31/0c2a78e3ea5040427f2e522ff82ea947.png

It is for 'resize in linear domain'. So for content being designed in linear domain it must be ConvertToLinear before and back after resizing.

Do not understand why optimizer start to give b < c results. They distorts significally like b=-50 c=40
https://i3.imageban.ru/thumbs/2021.03.31/e2e69aa19175109ed4934a56c7df76f2.png (https://imageban.ru/show/2021/03/31/e2e69aa19175109ed4934a56c7df76f2/png)
direct link
https://i3.imageban.ru/out/2021/03/31/e2e69aa19175109ed4934a56c7df76f2.png

It looks optimizer looks for sharpest possible result but do not looks for ringing distortion.

Boulder

31st March 2021, 09:57

It looks optimizer looks for sharpest possible result but do not looks for ringing distortion.

It depends on the metric you use.

I've usually used MDSI since it has kept the downscaled-reupscaled result closest to the source. For example this document explains how it works: https://ieeexplore.ieee.org/document/7556976

GMSD is another one, I have used it if the source is very noisy and a slight blurring is not a problem. It seems to cause a little softer results.

VMAF doesn't work properly with the tool, the results are not reliable because the errors are so small.

(vs)SSIM is something I've not tested.

Testing on an average quality Blu-ray source also got b < c for Userdefined2Resample. It is also (very) slightly blurrier than Bicubic and has a bit more haloing. I'll post some comparison screenshots later.

Boulder

31st March 2021, 16:17

Here's two frames as a sample. The differences are very, very small but from the filesize you can see that the other one is slightly blurrier. Check the rim of the hat against the sky to see the very slight difference in ringing.

BicubicResize scored 43.30757 and UD2R 44.781597 with MDSI. I also shared the test clip if you like to try something out. It consists of 200 frames throughout the whole movie, selected automatically by a SelectEvery function.

Anyway, upon playback you really cannot tell which downscaled one is which, and if you are watching a 720p or 1080p version unless the encoder artifacts blow your cover. You'd be amazed to know how few BD or UHD releases actually have details that matter.

1 - Original (https://drive.google.com/file/d/16pywukd5107022RacylAznGHMHUL50_Z/view?usp=sharing)
1 - BicubicResize(1280,536,b=-0.57,c=0.48).LanczosResize(1920,800) (https://drive.google.com/file/d/1ZkKmFaYabaS1SKueEyfH-7RGT_o8cKkL/view?usp=sharing)
1 - UserDefined2ResizeMT(1280,536,b=19,c=32).LanczosResize(1920,800) (https://drive.google.com/file/d/1NfKtLjD0iUd-iCC9d4AkES6wt8VK1F3X/view?usp=sharing)
2 - Original (https://drive.google.com/file/d/1ujQmg6-6dUUYykM3Hi8G9ht5aNTFL_t4/view?usp=sharing)
2 - BicubicResize(1280,536,b=-0.57,c=0.48).LanczosResize(1920,800) (https://drive.google.com/file/d/1ZVFz6VahM8PP8jBrwhybSG4WoQQTHsAC/view?usp=sharing)
2 - UserDefined2ResizeMT(1280,536,b=19,c=32).LanczosResize(1920,800) (https://drive.google.com/file/d/1OmfIeQI-eSFjEisuSykVL8q_1RbfNS-Y/view?usp=sharing)
Testclip (~500 MB) (https://drive.google.com/file/d/1fuVEx4bNDA6UFISaDFxAX7u0f_I0tQ2U/view?usp=sharing)

DTL

31st March 2021, 20:13

"upon playback you really cannot tell which downscaled one is which,"

It looks the source have too low high frequencies to work with when you downsize like 1920->1280. The more difference may be if downsize like 1920->720. (I.e. FullHD->SD).
The difference between downsizers may be better visible if source have much more high frequencies to throw away when we do div2 downsize.

It looks like simply upscaled to about 2x content. So if you downscale /2 it about change nothing. But if downscale /3 and more it starts to show great difference.

Made testscript:

LoadPlugin("ResampleMT.dll")
LoadPlugin("ffms2.dll")

FFMpegSource2("silverado.avi")

sc_ratio=3

#UserDefined2ResizeMT(width/sc_ratio, height/sc_ratio, b=90, c=-7) # 1/2 film->film no-overshoot/halo ?

BicubicResize(width/sc_ratio, height/sc_ratio, b=-0.57, c=0.48)

SincLin2ResizeMT(width*sc_ratio, height*sc_ratio, taps=32)

ConvertBits(8)

It looks Avisynth scaler still do not accepts float target width and height so only integer 'sc_ratio' params accepted (or may be some more math must be added to use float sc_ratio).

If sc_ratio=2 it is very few difference between BicubicResize(b=-0.57, c=0.48) and UserDefined2ResizeMT(b=90, c=-7) in ringing (Bicubic just starts to rings a bit but more sharper). It looks because source just do not contain frequencies higher about half of Nyquist valid.

But if set sc_ratio=3 Bicubic( b=-0.57, c=0.48) produces very ringy result for sinc-based display upscaler.

So this source can not be used for testing /2 downscalers.

Boulder

1st April 2021, 06:57

Sharc

1st April 2021, 08:53

Kind of proves my point of re-encoding 99% of 1080p sources to 720p for storing on the NAS and viewing on the media player :) Most FHD sources just don't have any extra details to lose when downscaling and displaying at 4K.

Agree. The exception is perhaps the reproduction of the grain for grain fetishists.
Off the records: The movie BARAKA was taken at 720p AFAIK and looks better and more detailed than many of the so called 1080 productions.

DTL

1st April 2021, 09:51

Kind of proves my point of re-encoding 99% of 1080p sources to 720p for storing on the NAS and viewing on the media player :) Most FHD sources just don't have any extra details to lose when downscaling and displaying at 4K.

By the way, is there some automated method to check if there are those high frequencies?

I think todays h.264 and later codecs to not wastes more bitrare if it is nothing to encode at high frequencies. So it may be small reason to decrease actual frame resolution for store only purposes.
Also it do not mean there is completely absent anything after for example 1/2 of valid frequencies - it just have very low level in compare to still valid also for anti-Gibbs conditioning.

Some testing possibly may be performed with h.264 encoder: If content is scaled down and with fixed crf and other params the output bitrate is close to be the same - so it looks like nothing significant is thrown away. Though it is good to remove noise (film grain) before testing because downscaling also decreases amount of noise to encode. If noise is present.

So with grainy but not detailed sources it really may be reason to fast downscale instead of longer temporal de-noise and encode with lower bitrate. The source film scanner actually also downscales film grain to target output resolution. Using hardware-ADC with optical and other ways to remove too high but not informative frequencies (caused by film grain for example). Though good shot 35mm films may have at least 1920x1080 useful resolution. So for best possible detailing it is recommended to use temporal-based processing of gathering data from many film's frames.

Tenet as I read was wide-film (like 60..70 mm) shot so it must have > fullHD and also > 4K resolution. Depends on film scanning workflow used.

"The movie BARAKA was taken at 720p AFAIK and looks better and more detailed than many of the so called 1080 productions."

Heh - it is 2021 number of year now. Video systems industry selects between 2160 and 4320 lines and it is not 2000-year to read of 720vs1080 content.

Sharc

1st April 2021, 11:03

"The movie BARAKA was taken at 720p AFAIK and looks better and more detailed than many of the so called 1080 productions."

Heh - it is 2021 number of year now. Video systems industry selects between 2160 and 4320 lines and it is not 2000-year to read of 720vs1080 content.
Right. But even 2021 does not protect us from snake oil :D

Boulder

1st April 2021, 13:38

I think todays h.264 and later codecs to not wastes more bitrare if it is nothing to encode at high frequencies. So it may be small reason to decrease actual frame resolution for store only purposes.
The difference is actually quite big. I didn't do any tests right now but 1920x1080 has over two times as many pixels compared to 1280x720 and the resulting bitrate for the exact same parameters in x265 is easily 30-50% bigger using a Zopti-mized value for Bicubic.

It makes kind of sense because the low and mid frequencies are there in all/most CTUs and the encoder needs to code quite a lot more of them with 1080p. That is also the reason why I go for 1440p for native 4K sources, 1080p for upscaled 4K and 720p for 1080p sources. HD space is cheap but it's still not very convenient to try stuffing the drives into the cupboard with the NAS :D

EDIT: Tenet's results for the 150-frame test clip just came out, didn't compare yet. Based on the score they look pretty much identical.
Bicubic - b=-0.95, c=0.23 - MDSI score 28.545593
UD2R - b=0, c=58 - MDSI score 28.60576

poisondeathray

1st April 2021, 16:10

Off the records: The movie BARAKA was taken at 720p AFAIK and looks better and more detailed than many of the so called 1080 productions.

You can look at imdb , the technical specs section

https://www.imdb.com/title/tt0103767/technical?ref_=tt_dt_spec

Runtime 1 hr 36 min (96 min)

Sound Mix 70 mm 6-Track (70 mm prints) | Dolby SR (35 mm prints)

Color Color

Aspect Ratio 2.20 : 1
2.39 : 1 (35 mm prints)

Camera Mitchell AP-65, Hasselblad, Mamiya, Pentax and Schneider Lenses

Laboratory DeLuxe, Hollywood (CA), USA

Film Length 3,292 m

Negative Format 65 mm (Eastman EXR 100T 5248, EXR 500T 5296)

Cinematographic Process Digital Intermediate (8K) (master format)

Todd-AO (source format)

Printed Film Format 35 mm (Eastman 5384, EXR 5386)
70 mm (Eastman 5384, EXR 5386)

DTL

1st April 2021, 18:09

The difference is actually quite big. I didn't do any tests right now but 1920x1080 has over two times as many pixels compared to 1280x720 and the resulting bitrate for the exact same parameters in x265 is easily 30-50% bigger using a Zopti-mized value for Bicubic.

I think most of natural film movies takes lots ob bitrate on encoding noise/grain instead of small details. So it is good before lowering resolution as last way try to perform noise-reducing before with mvtools - MDegrainN with large enough tr param (like 10 or even 20). And adjust thresholds thSAD to detect and process noise blocks.
It possible you will save same 30..50% bitrate without downscale work.

Sometime it is possible with poor quality grainy BD remux of 20..30 GB size to make 10- GB rip with reduced noise and even raised sharpness keeping full BD resolution unchanged.

Reel.Deel

1st April 2021, 21:52

Kind of proves my point of re-encoding 99% of 1080p sources to 720p for storing on the NAS and viewing on the media player :) ...

Anyway, for a rather typical downscaling scenario, namely resizing 1080p content to 720p, I've found bicubic with ~about~ (-0.5,0.25) fully sufficient. Using FineSharp to re-scale that 720p back to 1080p, the result in many cases is almost indistinguishable from the original source. With carefully chosen settings, I've fooled some friends into believing that the original 1080p vid would be the downscaled-upscaled version, and the (downscaled-upscaled)+FineSharp one would be the original 1080p.

I too have fooled a few people with Didée's suggested method :)