SD NTSC to PAL conversion - which resizer?

[Lariel22]

For hours now I have been looking through the various resizer threads but can't find a specific answer to my problem, which actually is a standard one.

I have an NTSC DVD (progressive material, 23.98 fps restored with forced film) that I would like to convert to PAL. So I need to resize 720x480 to 720x576. Which resizer is a good match for this? I'm looking for highest quality and don't care how time-consuming it will be.

So it's upscaling, vertical only and by a rather small amount. I read that sharp resizers like Lanczos or Lanczos4 are often used and recommended for upscaling.

But what about aliasing, ringing and blocking effects? I guess Lanczos rings to much. Question 1: Is there a resizer that can avoid all of these effects?

Question 2: Would it make sense to use a sophisticated resizer like JincResize, nnedi3_rpow2 or nnedi3_resize16? (Like I said, I'm looking for highest quality.)

[kedautinh12]

Try nnedi3_rpow2(rfactor=2, cshift="spline64resizemt", fwidth=720, fheight=576)

Load 2 plugins: nnedi3.dll and resampleMT.dll

[johnmeyer]

I've always simply used Spline36Resize for NTSC<-->PAL conversions. When I also have to change the frame rate, which doesn't apply for your conversion, I do the vertical re-size before the frame rate conversion and then do the horizontal resize after changing the frame rate.

[real.finder]

there are thread for opposite case, you may want to convert the color primaries too https://forum.doom9.org/showthread.php?p=1936307#post1936307 (it need to inverse the conversion as "470bg:601:170m:limited=>470bg:601:470bg:limited")

[wonkey_monkey]

nnedi3_rpow2

It'd be nice if nnedi3 had a mode that only doubled height.

I would do the framerate conversion first because it seems to me that you want the cleanest (i.e. not upscaled) data going in to that conversion.

[Sharc]

Question1:
See here for a comparison of ringing, sharpness etc.
https://web.archive.org/web/20170327075350/http:/svn.int64.org/viewvc/int64/resamplehq/doc/kernels.html
(Be patient until the page has fully loaded)

Question 2:
A simple bilinear filter, eventually followed by a slight sharpening, can produce very good results as well. Depends on source.

[Lariel22]

So much input. You are amazing, Doomies. :D

I'm still undecided on what resizer to use. I will put Spline36 here for now. My script looks like this right now:
LoadPlugin("C:\programs\DGMPEGDec\x64\DGDecode.dll")
MPEG2Source("C:\production\video.d2v", idct=5)
Spline36Resize(720,576)
AssumeFPS(25, 1, false) # false because I'm doing pitch correction in another program
z_ConvertFormat(colorspace_op="470bg:601:170m:limited=>470bg:601:470bg:limited")


I'm only changing the frame rate, not the frame count. So I guess it doesn't matter whether I resize first or change the frame rate first. Does it matter whether I resize first or convert the color primaries first?

Edit: Thanks for the link to the kernel visualisations. But the only thing I can see there is that there is no perfect resizer. The sharper it gets, the more ringing it has too. Maybe Robidoux is a good compromise?
Wonkey_monkey, so because nnedi3 doubles height and width, you would advise against it?

Edit2: Thanks, Frank62, my bad. Corrected.

[Frank62]

...
Spline36Resize(720,480)
...


Better to 576. :D
If you just speedup to 25 it doesn't matter when you resize.
When to change color primaries - well - seems to be a rather philosophical question.

[Lariel22]

Now, to find a suitable resizer I did some tests using SSIM as a quality metric. I upscaled to 720x576 and then downscaled to 720x480 again and compared. This can't really tell which resizer is best for upscaling, only which combination is good for upscaling-downscaling. But it's closer to what I intend to do than downscaling and then upscaling to original size. And since I test various combinations, it tells us something about the upscaling qualities.

For the combination of, for example, Catmull-Rom/Spline36 I did it like this:
source=MPEG2Source("C:\production\video.d2v", idct=5)
video=MPEG2Source("C:\production\video.d2v", idct=5).BicubicResize(b=0.0, c=0.5, 720, 576).Spline36Resize(720, 480)
return SSIM(source,video,"catmullrom-s36.csv","catmullrom-s36.txt",lumimask=2)


First, I tested many of the classic resizers (Bilinear, Bicubic, Lanczos, Lanczos4, Spline16, Spline36, Spline64). Here are the averages I got. By the way, it's a 22 min video of film footage, no animation.

The worst I got:
Bilinear/Bilinear: 97.40424380

In between (upper third):
Spline36/Spline36: 99.38529084
Lanczos/Spline36: 99.41915025

The top 3 I got:
Lanczos4/Spline64: 99.49197410
Lanczos4/Lanczos: 99.50300090
Lanczos4/Lanczos4: 99.52909226

So the sharper the better? Maybe that's because of my source. But all the Lanczos and/or Spline variants combinations resulted in pretty close values.

Next, I tested Nnedi3, Jinc, Catmull-Rom, though not in every possible combination. Here is what I got:

Catmull-Rom/Spline36: 99.26893428
Jinc(36)/Spline36: 98.86460999
Jinc64/Spline36: 98.83664241
Jinc144/Spline36: 98.80032054
Nnedi3 rpow2(rfactor=2) with cshift Lanczos4/Lanczos4: 92.55218887
Nnedi3 rpow2(rfactor=2) with cshift Spline36/Lanczos4: 92.46321013

So Catmull-Rom/Spline36 looks acceptable to me. The Jinc/Spline36 combinations are outperformed by any of the Lanczos and/or Spline variants combinations. And Nnedi3 looks really bad. Is it because I downsized two times? Doesn't look like it:

Nnedi3 rpow2(rfactor=2) with cshift Spline36 (so straight to 720x480): 92.51475721

So leaning to the conservative, I think I will go with Spline36 for upscaling in this particular case. Good alternatives appear to be Lanczos, Lanczos4, Spline64 and Catmull-Rom.

Possibly Nnedi3 and Jinc fall off because they are working in 2d space and this case needs a 1d resize only? (If I understood it correctly.) I don't know.

[johnmeyer]

I like your tests. Very scientific.

You implied that you also simply looked at the results subjectively, but you didn't provide that result. I too do scientific tests when trying to "perfect" a workflow, but when I also do the subjective tests, run at full speed on a big screen, but without magnifying or otherwise changing the normal viewing experience, I find that I either get different results, or more frequently, I find that I can't tell the difference, in which case I go with the simplest and/or fastest method.

[Lariel22]

Actually, I did not imply but simply assume. :D So far, I have almost exclusively looked at the bare numbers. It's rather hard for me to spot a difference when looking at it. But you are right, I should also check what subjectively looking at it tells me. I guess I will do that next. I would not necessarily choose the simplest/fastest method if I don't spot a difference as someone else may have a keener eye than me.

[johnmeyer]

At the risk of getting into a discussion that borders on esoteric philosophy, "if you can't see a difference then there is no difference." Or, to paraphrase Obi Wan Kenobi, "Don't trust your eyes, trust your feelings." If it feels good when you watch it, then go with it.

I wasted a LOT of time the past twenty years comparing results of various scripts with the original and, when I'd spot some minor flaw or glitch, I'd obsess over it. It wasn't until I'd done hundreds and hundreds of hours of video that I realized that most of the time these microscopic defects did not make one bit of difference to the enjoyment of watching the result. As a result, I now focus now much more on things that are immediately evident like color grading, gamma adjustments, stabilization, dirt and comet removal, and noise removal (but only if it doesn't introduce artifacts), to name a few.

My current mantra: "natural and enjoyable."

[DTL]

But the only thing I can see there is that there is no perfect resizer. The sharper it gets, the more ringing it has too. Maybe Robidoux is a good compromise?
Wonkey_monkey, so because nnedi3 doubles height and width, you would advise against it?


For correct Film transfer it looks the only target resampler is SincResize. With large enough taps. Also Sinc with taps < about 70 have some bugs at edges so SincLin2ResizeMT (from jpsdr's plugin pack) is prefferable.
May be Jinc with large enough taps if time not greatly matter.
But if it starts to ring (too more) it just mean film transfer was badly conditioned (oversharpened) and user is free to use any other resizer that it like most. Or try to re-condition spectrum source with something like Blur() before Sinc/Jinc. Typically users preffer NNEDI for better result but it may produce artifacts at some frames because it is content-dependent resizer.

"So the sharper the better? Maybe that's because of my source. But all the Lanczos and/or Spline variants combinations resulted in pretty close values."

You just test 'reversibility' of resizers. For sinc/jinc-based it is higher in compare with less sinc-based. Lanczos is just more aggressively weighted Sinc (to have less ringing in general case of unknown source). Using well or at least medium good conditioned for sinc-based workflow source like commertial DVD release. Actually there is no 'sharpness' for correct linear upsampler because upsampler do not changes useful data spectrum in good case. So you can play DVD to CRT via 27 or 54 or 108 MHz DAC without needing any 'sharpness' adjustment - just got possibly less visible aliasing (at test equipment mostly).
All above applied only to film-based DVD, for interlaced things are much more harder.

"Would it make sense to use a sophisticated resizer like JincResize,"

Jinc is just a '2D/planar' from close to Sinc (V+H separated) of linear resampler. So it is content-independent (do not introduce content-dependent artifacts) and may or may not produce more pleasing looking result in compare with Sinc (1D twice) depend on source. For very sharp (or other way specially prepared for Jinc upscale) sources may be Jinc looks better, but Film transfers typically do not have extreme sharpness. And at the era of DVD processing resources were too poor to prepare special source to show Jinc benefits. So it is only for personal taste. Also 576/480 upsample ratio looks too small to show significant difference.
NNEDI is completely different class resizer - non-linear content-dependent.

[Lariel22]

@johnmeyer: You are right. If you can hardly see the difference, it is probably not worth the hassle. It's fun the learn about all these resizers and how they differ though.

@DTL: Why do you think for film transfer SincLin2ResizeMT is the only good resizer? And what tap number do you consider large enough (approximately) in this case? I haven't tried it yet, so thanks for the hint. I will.

I guess I really don't need to consider Jinc anymore as I'm resizing 1D anyway (vertical 480 to 576).

One more thing: i read that one should rather use Spline36ResizeMod instead of Spline36Resize because the former "accounts for a slight chroma shift bug in the default Spline36", source here (https://encoding-guide.neocities.org/03-encoding-mux.html). Now the only Spline36ResizeMod I could find was the one by Asd-g here: https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/spline36resizemod_.avsi
Can someone explain the difference between the normal version and this mod, please? Is the mod to be preferred in each and every case?

[DTL]

"Why do you think for film transfer SincLin2ResizeMT is the only good resizer? "

If film was transfered to DVD as progressive frames (I hope) - so it assumed to use same target resamplers for both horizontal and vertical directions. And target analog video horizontal resampler is (99+%) sinc. Because SincResize in original avisynth suffers from some bug (may be not greatly visible on noisy enough data). It is math computational issue for sinc kernel with very few number of taps, so can not be simply fixed as programming bug. So the workaround SincLin2Resize was made (currently works in jpsdr's MT plugin pack, may be someday added to original Avisynth core). It works without this computational issue at the kernel edge with any number of taps and provide at least taps/2 full-strike (heh - full-blood) sinc kernel samples and +some more (linearly weighted to 0 to end of kernel).

It is not only good - it is 'reference linear' resizer (for H direction old SD video and for H and V for true film (may be +progressive scan) SD). If you happy to use any non-linear (content-dependent) resizer with better result on your current data you may use other.

" And what tap number do you consider large enough (approximately) in this case? "

As I see at typical motion picture data there is very small (read - non visible) difference after 6..8 taps. Only at some synthetic tests with non-typical amplitude of the very high (still valid) frequencies we need about 16 and more. So default in SincLin2Resize is about 16 - to have full-strike 8 and a bit more. General rule - use as much as possible at current computational platform.

It all apply to high-quality film to DVD transfers. If it was not correctly conditioned to video - using sinc may cause ringing so can not be used (or need special preparation operation - lowering high frequencies with additional pre-filtering - like convolution() or blur() or other).

"the difference between the normal version and this mod, please?"

My version:
Resizers may cause spatial chroma shift when applied to non-4:4:4 data. If you see some (visible) spatial color shift (relative to luma) after resize you can try that workaround with separated planes resize and manual aligning chroma and luma by adjusting (float) params in the lines:

u_plane = ExtractU(c).Spline36ResizeMT(target_width/width_divide,target_height/height_divide,(-0.5*(width_divide-1)*((src_width/target_width)-1)+src_left)/width_divide,src_top/height_divide,src_width/width_divide,src_height/height_divide)
v_plane = ExtractV(c).Spline36ResizeMT(target_width/width_divide,target_height/height_divide,(-0.5*(width_divide-1)*((src_width/target_width)-1)+src_left)/width_divide,src_top/height_divide,src_width/width_divide,src_height/height_divide)

-0.5 values. I think it may also depends on your source data. I think on old film data it is hardly to be visible if even exist. It also mostly for perfectionists stuff.

[real.finder]

One more thing: i read that one should rather use Spline36ResizeMod instead of Spline36Resize because the former "accounts for a slight chroma shift bug in the default Spline36", source here (https://encoding-guide.neocities.org/03-encoding-mux.html). Now the only Spline36ResizeMod I could find was the one by Asd-g here: https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/spline36resizemod_.avsi
Can someone explain the difference between the normal version and this mod, please? Is the mod to be preferred in each and every case?

there are also resizex()

[DTL]

Now, to find a suitable resizer I did some tests using SSIM as a quality metric. I upscaled to 720x576 and then downscaled to 720x480 again and compared.

For correctly conditioned motion picture data it is actually "do nothing" operation. Using enough precision processing.

It can be tested with SincResizeMT from jpsdr's plugin pack - it have significally relaxed upper limit of taps like to 150. To work with 'pure blood sinc' and avoid that issue with sub_70 taps computations with at least 8bit integer.

And avisynth's resampler can process with large enough 'resizer's support' if frame size large enough.

So to test 'reversability' of signal processing math we now in 2021 with high performance computer can use something like

sc_ratio = 1.234567 # any number > 1
ConvertBits(32)
SincResizeMT(width*sc_ratio, height*sc_ratio, taps=100)
SincResizeMT(width/sc_ratio, height/sc_ratio, taps=100)
ConvertBits(8)


For good conditioned content the metric like PSNR will be close to infinity (or SSIM to 100 etc). It may be also some form of content's testing for good conditioning to be motion picture data for sinc-based workflow.

Actually upsizing to infinity is DAC and downsizing to original size is ADC and good spectrum-shaped motion picture data can be in theory any time DAC/ADC transformed without distortions. And DVD data in good case is prepared to feed via DVD-player DAC to analog CRT display without distortions.

"Lanczos4/Lanczos4: 99.52909226"

It is hardly weighted sinc and very limited in number of taps. With newer Avisynth(+) Lanczos 'size/support' may be controlleed with 'taps' param also up to 100 as I see from sources https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/resample_functions.cpp . Typically Lanczos(taps=N) is close to SincResize(taps = N/1.5) i.e. Lanczos weighting gives about taps/2..taps/3 close to full-strike sinc kernel. But due to the kernel's edge weighting it also do not suffer's from that issue of simple SincResize with taps < about 70.

"The top 3 I got:
Lanczos4/Spline64: 99.49197410
Lanczos4/Lanczos: 99.50300090
Lanczos4/Lanczos4: 99.52909226"

Sinc-based internal Avisynth's resizers are Blackman, Lanczos and Sinc itself (all are controlled with taps-param but as I see sinc is 20 max only, others already relaxed to 100). So it is generally natural enough for moving picture data to be processed best with sinc-based resizer. With large enough taps num (may be > 6..8 for typical content) they are close to pure sinc. Only internal SincResize with any taps param from 1 to 20 will suffer from that issue so is not recommended. And Blackman and Lanczos with taps > 6..8 will be slower depending on taps param.

Addition: Also there is some hard enough question sadly do not clearly covered in neither rec.601 nor any newer digital moving picture data systems - In which domain spectrum data is conditioned (for resampling and other processing): linear coded (RGB) or system-TF'ed (R'G'B'). It is important because system-TF damages spectrum by non-linear distortion (reversable). So for even better result may be tested resampling in linear data using conversion from https://forum.doom9.org/showthread.php?t=175488

So for NTSC DVD I think some like

sc_ratio = 1.234567 # any number > 1
ConvertBits(16)
ConvertYUVtoLinearRGB(Color=3)
SincResizeMT(width*sc_ratio, height*sc_ratio, taps=100)
SincResizeMT(width/sc_ratio, height/sc_ratio, taps=100)
ConvertLinearRGBtoYUV(Color=3)
ConvertBits(8)


Because I think it is not covered by standard so each mastered DVD may or may not be better scaled in linear domain. At least I not sure if DVD-player's DAC operates in linear domain and performes backward and forward system-TF for this way.

[Lariel22]

Thanks a lot for all the input. So I tried

SincLin2ResizeMT(720, 576, taps=16).SincLin2ResizeMT(720, 480, taps=16)
Average SSIM= 99.63346501

SincResizeMT(720, 576, taps=100).SincResizeMT(720, 480, taps=100)
Average SSIM= 99.34692488

ConvertBits(16).ConvertYUVtoLinearRGB(Color=3).SincResizeMT(720, 576, taps=16).SincResizeMT(720, 480, taps=16).ConvertLinearRGBtoYUV(Color=3,OutputMode=2).ConvertBits(8)
Average SSIM= 98.93411155

ConvertBits(16).ConvertYUVtoLinearRGB(Color=3).SincResizeMT(720, 576, taps=100).SincResizeMT(720, 480, taps=100).ConvertLinearRGBtoYUV(Color=3,OutputMode=2).ConvertBits(8)
Average SSIM= 99.39711914

So converting to RGB 16 bit before resizing doesn't seem to improve the results. And SincLin2ResizeMT with 16 taps does better than SincResizeMT with 100 taps. A little strange.

Anway, When looking at the picture, some edges seem a little pronounced after the resize. So this may be a little bit of ringing. It's subtle but noticeable. So I guess I will go with Spline36Resize using ResizeX.

Spline36Resize(720, 576).Spline36Resize(720, 480)
got me Average SSIM= 99.38529084

ResizeX(720, 576, kernel="Spline36").ResizeX(720, 480, kernel="Spline36")
got me Average SSIM= 99.38529084

So the value is identical. I don't expected any major improvement because, if I understand it correctly, chroma shift correction is supposed to be a subtle effect. However, is it correct that it doesn't influence the SSIM value at all? Am I using ResizeX correctly here?

[SeeMoreDigital]

Out of interest... What video format are you encoding to @Lariel22?

[Lariel22]

I'm encoding straight to MPEG2 for DVD. Wait ... Is the chroma shift correction senseless in this case? :scared: Though I might do an x264 encode later on, too.

[SeeMoreDigital]

I'm encoding straight to MPEG2 for DVD...Which MPEG-2 encoding software are you using?

[Lariel22]

I'm not sure yet. I guess either HCenc or CCE.

[scharfis_brain]

It'd be nice if nnedi3 had a mode that only doubled height.

Nnedi3(dh=true)