Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th April 2022, 13:38   #1  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Downscaling without aliasing test

I think I found a way to downscale with a bearable amount of aliasing while retaining a lot of high frequencies.
Code:
A = BicubicResizeMT(720,480,0,1)
B = BicubicResizeMT(720,480,1,0)
mt_makediff(A,B,u=3,v=3)
Blur(0,0.333).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
Blur(0.333,0).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
mt_adddiff(B,last,u=3,v=3)
If you want less aliasing, just increase blur strength.
What are your thoughts?
Ceppo is offline   Reply With Quote
Old 4th April 2022, 17:59   #2  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Can you add all required plugins to run this test ?
DTL is offline   Reply With Quote
Old 4th April 2022, 18:37   #3  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
https://forum.doom9.org/showthread.php?t=174333
https://forum.doom9.org/showthread.php?t=174248
But you probably already have them.
Ceppo is offline   Reply With Quote
Old 5th April 2022, 16:35   #4  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
You do not define the target downscale ratio. So I try to downscale 1920x1080 source.

With default 0.333 blur it leave too much of aliasing at near to cut-off frequencies and some at the highest. Even increasing blur value to 1.333 do not saves from highest frequencies aliasing (not great though but may be visible at some scenes). The Bicubic resize have too low 'support/kernel size' value to operate as sharp cut-off low pass filter. And also the used operations may pass some full source high frequencies to output that cause some aliasing patterns at the places of fine sharp input textures.

My test script is:
Code:
LoadPlugin("ResampleMT.dll")
Loadplugin("masktools2.dll")
LoadPlugin("RawSourcePlus.dll")

RawSourcePlus("out8.raw", 288,288, "Y8")

ConvertToYV12(matrix="PC.709")

AddBorders(10,10,10,10,color=$101010)

cl_0d25 = SincResize(last.width, last.height, src_left=0.25, taps=20)
cl_0d5 = SincResize(last.width, last.height, src_left=0.5, taps=20)
cl_0d75 = SincResize(last.width, last.height, src_left=0.75, taps=20)

last + cl_0d25 + cl_0d5 + cl_0d75 + cl_0d5 + cl_0d25

Loop(50)

AddBorders(0,0, 1612, 772)

A = BicubicResizeMT(720,480,0,1)
B = BicubicResizeMT(720,480,1,0)
mt_makediff(A,B,u=3,v=3)
#Blur(0,0.333).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
#Blur(0.333,0).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
Blur(0,1.333).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
Blur(1.333,0).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
mt_adddiff(B,last,u=3,v=3)

SincResize(width * 3, height * 3, taps=20)
It uses raw zoneplate file generated by https://github.com/DTL2020/hpzp and adds some animation to make moving aliasing more visible.

Even simple BicubicResizeMT(720,480) do not leave aliasing at highest input frequencies.
That HF aliasing pass via A = BicubicResizeMT(720,480,0,1) clip. So 'default' BicubicResize params looks like balanced to not allow it.

If replace A clip to A = BicubicResizeMT(720,480) the total script works very close to simple BicubicResizeMT(720,480) with all typical medium frequencies aliasing because too low 'support or kernel size' of bicubic to form more sharp cut-off filtering at new output size.

Last edited by DTL; 5th April 2022 at 17:17.
DTL is offline   Reply With Quote
Old 5th April 2022, 17:44   #5  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Thanks for your answer, I tried the tool you provided and it's very useful. I did only empirical tests because I didn't know the above tool. High frequency gets indeed wiped out and in my anime clip, those ones were not there, so I didn't notice.

Now my problem is, how to differentiate an aliased high frequency from a not aliased one? If I do a reside and like half width/height we have 4 pixels and from those, we need to get 1. So if they are not much different from the average, they are a low/middle frequency and we can use a sharp algorithm without caring about aliasing, yet if it is a high frequency, how do I know if it is a frequency that *looks* aliased to the eye or not? That's the core of my problem. This simple example is given by the fact that I know nothing about resizing algorithms, I just want to get started.
Ceppo is offline   Reply With Quote
Old 5th April 2022, 20:31   #6  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"Now my problem is, how to differentiate an aliased high frequency from a not aliased one? "

If you see aliasing distortion (new, typically low frequency structures) - it is aliasing . When source moves - the aliasing new comb low frequencies also start to move (and may be typically) in back direction. So to see the smallest amount of aliasing it is better to use some moving test patterns with sub-sample shift each frame.
The (hyperbolic) zone plate have continious frequency changing so when you downscale the Nyquist limit moves from outer border to some inner point. So to make non-aliased result the pre-filter (or any other 'magic' scaler's algoriphm) need to filter-out all (most of) frequencies outside Nyquist limit. It is really very simple operation with low-pass filter. But if you use filter with very sharp cut-off you start to get Gibbs distortion (in the restored result). So to fix both aliasing and Gibbs ringing we need also to fade some frequencies in the allowed (Nyquist legal) range. This cause image blurring. To make blurring smaller it is possible to add 'video-makeup' with adding some over/undershoots to levels transients still not starting to ring more.

Note: All above math is valid only if you use optical (continious) image restoration using some sinc-based interpolation (or other large enough kernel). You can use any downscaler and any upscaler that works nice for you in some 'closed' digital imaging system. Also it is sad but currently even 'global' digital moving pictures industry still do not have standard for continious image restoration from sampled (digital) form. So it is still some more or less common practice to use something sinc-based or simple bicubic-based or any propietary scaler at display device. So that Gibbs-distortion may or may not be visible at some display device. The aliasing and dymanic (moving) aliasing typically more visible at most displays because it have low enough frequency.

" I didn't know the above tool. "

Hyperbolic zoneplate is included in the old (ver 1.0) of EBU Test Pattern 7 for testing video monitors and I hope will be included in the version 2.0 (still not officially released). In TP7 it is static but it is good shifted with AVS resample engine and sub-sample 'crop' values for SincResize() with large enough taps value to make animation.

"how do I know if it is a frequency that *looks* aliased to the eye or not?"

The natural moving image data because of the suppression of Gibbs phenomenon typically do not have as powerful high frequencies as zoneplate have (equal power at most of frequencies). So some moderate amount of aliasing may be acceptable because keeping more high frequencies increase sharpness and makes looking of small details better. But if shooting some regular structures - the aliasing may be very visible. So design of downscaler (that is really the core video camera or any image data forming algoriphm like rendering/rasterizing transform) it is balance between gibbs/aliasing/sharpness.

" I just want to get started."

It is good to start from great amount of digital signal processing books (for 1D signals first). Keywords are sampling, Nyquist , Gibbs, aliasing. Though it shows only 'classic' digital signals processing math.

"If I do a reside and like half width/height we have 4 pixels and from those, we need to get 1."

It is not high quality approach typically. The sample-based continious image is described in theory with all samples in any point. But in practice it it enough to process several surrounding samples (depending on the scale factor). For /2 downscale it looks like 4 input converts into 1 output, but practically good resampler operates with more input samples for each output (it probably 'support' param in AVS resampler). The more samples used (the larger kernel size) the more sharp cut-off filter may be designed so better separation of illegal frequencies in new smaller output size.

Also remember it best works in linear data (not system transfer converted) and in 1D. With 2D it not perfectly works even with '2D' resamplers like JincResize if trying to use max Nyquist legal frequiencies of high amplitude. At least at my tests.

Last edited by DTL; 5th April 2022 at 20:48.
DTL is offline   Reply With Quote
Old 6th April 2022, 11:55   #7  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Thanks for your time. That was a lot to track for someone who did literature study lol but now I have a better "picture" of the resizing problem When I'm done I will post other sample filters in C++. Hope for your help again.
Ceppo is offline   Reply With Quote
Old 6th April 2022, 12:24   #8  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Here is images with comments:
Default params Blur(0.333) script running from 1920x1080 source:
https://i1.imageban.ru/out/2022/04/0...3879a07501.png

Examples of default BicubicResizeMT and UserDefined2ResizeMT (default params) processing result:
https://i1.imageban.ru/out/2022/04/0...0c416e09db.png

The UserDefined2Resize as example of mostly suppressed both aliasing at non-valid Nyquist frequencies and Gibbs ringing. Sort of 'film-look makeup' example. But it is much smoother in compare with default Bicubic. So the default Bicubic is a some balance between more possible sharpness with some aliasing and ringing permitted (at some scenes).

If images not load - please suggest another international accessible free hosting if possible.

If you interesting in the downscaling problem currently we have an open problem for development of non-linear software downscaler for subsampled colour 4:2:0 typically used video data representation. The source of the problem is more or less described in https://forum.doom9.org/showthread.php?t=183224 . Currently it have only manual non-linear (trial and error) solution for very limited cases. May be it is possible to develop 'closed loop adaptive' downscaler based on some attempt to bet for transient data (like random generator output ?) and using some metric-based engine to compare restored result with source to select best fitting solution. It may workaround some built-it bugs in current digital video subsampled chroma data systems and make colour images with sharp saturated transients a bit better.

Last edited by DTL; 6th April 2022 at 13:01.
DTL is offline   Reply With Quote
Old 6th April 2022, 14:12   #9  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Thanks for the example. I will look at the suggested thread when I get back from work this evening Do you think this university material is good enough to get started since I can't spend 200 bucks for a book this month due to my dog, and on Amazon they don't look cheap
Ceppo is offline   Reply With Quote
Old 6th April 2022, 14:47   #10  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
" this university material is good enough"

Yes - I think it is good. The math of DSP may be not changed for about half of century so any book of the last decades is mostly good. For image processing even much more simple book is enough. The sad truth is that linear math of 1D signal processing not work perfectly with 2D sampled natural data arrays. Only partially.
DTL is offline   Reply With Quote
Old 6th April 2022, 14:51   #11  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Thank you. I'm on it
Ceppo is offline   Reply With Quote
Old 12th April 2022, 17:21   #12  |  Link
Reclusive Eagle
Registered User
 
Join Date: Oct 2021
Posts: 83
Quote:
Originally Posted by Ceppo View Post
I think I found a way to downscale with a bearable amount of aliasing while retaining a lot of high frequencies.
Code:
A = BicubicResizeMT(720,480,0,1)
B = BicubicResizeMT(720,480,1,0)
mt_makediff(A,B,u=3,v=3)
Blur(0,0.333).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
Blur(0.333,0).mt_lutxy(last,"x y - abs 2 < y x ?",u=3,v=3)
mt_adddiff(B,last,u=3,v=3)
If you want less aliasing, just increase blur strength.
What are your thoughts?
There are actually many ways to downscale without any alising at all.
The problem? Most of this documentation is reserved for the medical industry and mathematicians.

There is next to zero information for the general public when it comes to video processing besides "uSe tHeSe pRemADe fILTers".

Here are 2 papers dealing specifically with downscaling and proper subsampling:

Paper 1: https://www.cs.cornell.edu/courses/cs4670/2018sp/lec06-resampling.pdf

Paper 2: https://www.cs.cornell.edu/courses/c...esampling3.pdf

Hopefully they point anyone looking for more info in the right direction. (Paper 2 is an updated version of paper 1)

Last edited by Reclusive Eagle; 12th April 2022 at 17:29.
Reclusive Eagle is offline   Reply With Quote
Old 13th April 2022, 11:55   #13  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
Thanks for the documentation, I will read them once I'm done with DSP book for better understanding
Ceppo is offline   Reply With Quote
Old 16th April 2022, 09:48   #14  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
All those ways to say: Don't violate Nyquist.

And since finishing my studies in cybernetics back in 1988 I would like to add:
You got to keep a damn good distance from that ugly point because our real-world signals are non-stationary.

One has to make sure that there are no more above-Nyquist frequencies in the source to be sampled,
so one has to abandon these willingly and remove them before sampling.

Or just ADC sample quicker until he gets his desired fidelity.
From fs > 3x fmax on I would feel a bit safer about the reconstruction.

---------

Adding my favourite corner case towards my former cybernetics professor, himself basing on Nyquist alone without margin:

"Professor Göldner, I just imagine a sine wave just satisfying the Nyquist rule, but unfortunately sampled at its nodes.
This gets the result 0 after resampling. Are we nuts using that as a safe rule ?"

Well IIRC he later added the important footnote: "for stationary signals",
but even that may yield zero (or any other DC value if sampled at that particular elongation)...

Nevertheless I decided to implement a 4th order Tshebysheff Low-pass filter before the ADC in question then,
ugly in phase response, but steep enough to yield no false positives in aliasing area.

The trained FFT algos following would listen into the cutting noise of a CNC lathe, guess on a tool's cutting edge breaking, stop the machine, give alarm, mark the workpiece void and change the tool.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."

Last edited by Emulgator; 16th April 2022 at 10:05.
Emulgator is offline   Reply With Quote
Old 16th April 2022, 11:00   #15  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"From fs > 3x fmax on I would feel a bit safer about the reconstruction."

The digital representation of real world images quickly enough become very good when we increase sampling frequency above Nyquist limit. But in still very limited frame size in samples in real world it cause smooth or very smooth images. If trying to view in 'standard' samples per arc degree density like about 60 samples per arc degree. So to have 'perfect digital imaging' we need simply increase 'system' samples density to 150..200+ samples per arc degree. That mean we need to use 4K and more frame size to store FullHD1080 and smaller 'real resolution' images.

The task for downsizers (also in ripping-community) is typically make frame size in samples lower to keep compressed file size lower. But if we perform very nice spectrum shaping with fs >> fmax we got very smooth low sized image. Yes - it will be free from most of 'digital' distortions like aliasing and ringing and may be some more of 'higer order' but very smooth. So it create the very old problem of creating low sample sized but as sharp as possible image data array.

The digital audio solve these issues when progress allow to easy use 96 kHz and more sampling. The digital video still do not move to easy to use 8K and 16K+ frame sizes to handle 'perfect quality' 1080-equivalent analog lines images. So the task of creating balanced between sharpness and distortions images still actual.

Last edited by DTL; 16th April 2022 at 11:03.
DTL is offline   Reply With Quote
Old 16th April 2022, 16:08   #16  |  Link
Ceppo
Registered User
 
Join Date: Feb 2016
Location: Nonsense land
Posts: 339
@DTL
I didn't yet finish the book, but I have an idea and I was wondering if it is already used or if it would not work.

Let's say we have to resize 1920x1080->720x480

We have 1920 samples, by interpolation through a weighted averaging we bright 1920 to 1920*720 samples, then we do decimation to 720 by some kind of 1xn matrix based on the difference between the adjacent original samples. After we have done with all the lines we repeat the process for all vertical lines.

Would something similar to this work?
Ceppo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:47.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.