Why does sharpening or upscaling an SD625i50 with AviSynth have so many artifacts?

[Mitra]

I have a SD625i50 video pattern sample source and want to upscale it to 1080i50 (to test) with AviSynth, but it has many artifacts.

I even changed the source to progressive and tried again, but it still had artifacts.

my script:


FFMS2("E:\Progressive25.mp4")
QTGMC()
aWarpSharp2(depth=5)
Sharpen(0.5)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1920, fheight=1080)
aWarpSharp2(depth=5)
Sharpen(0.5)



Please try it and tell me how can I do this without artifacts?

If someone can do this by tomorrow, I will give him a cup of tea.( 120$ )


Progressive25.mp4 (https://drive.google.com/file/d/1Qesv5pfTDIJBvpfb-OICKL-AxXv4e0J2/view?usp=share_link)

SD625i50.mov (https://drive.google.com/file/d/1xk1pejUDnnWclfWlNQYNVUiXvedLCHwh/view?usp=share_link)

[poisondeathray]

Test patterns are supposed to exhibit various artifacts with scaling - their purpose is to demonstrate problems with the scaling algorithms and various manipulations


This might be a bit better in terms of artifacts, but 1920x1080 is not a good choice for that source, because you distort the zone plate source which is 4:3 . You can use 1440x1080 and pillarbox it

LSmashVideoSource("SD625i50.mov")
AssumeTFF().SeparateFields()
BilinearResize(1440,540).AddBorders(240,0,240,0)
AssumeFieldBased().AssumeTFF().Weave()

[DTL]

Zoneplate looks like progressive so you can simply use any long kernel sincresize. Example:


LanczosResize(1920, 1080, taps=64)


The yellow rectangle looks like not any conditioned for sinc-based workflows so need to be skipped or redesigned for interlaced sinc-based workflows. After redesign and if interlaced processing is required:

QTGMC() # default double rate mode
LanczosResize(1920, 1080, taps=64)
AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave() #reinterlace to 1080i


*
Some furaffinity cosplay:
Accepting comissions on digital test patterns: YES*. SB/MAB/AB $300/50/1000. Example: UHD HDR https://qc.ebu.io/testmaterial/221/ (some part of this pattern also designed by Simon Thompson from BBC Research Lab).
*

"aWarpSharp2(depth=5)
Sharpen(0.5)"

It is typically assumed the digital test patten is perfectly finetuned in its original digital domain so require no any corrections. Also AVS processing mostly not great or even poor in performance for high-quality designed test patterns so any AVS processing should be used with great care and check each step for quality. Unfortunately for interlaced the high-quality deinterlacer is required so the complex QTGMC script loaded with many AVS processing is close to unavoidable. If it start to show some non-acceptable artifacts - the diffferent may be not as nice but more 'linear' deinterlacer required. The yadif/yadifmod may be next to try after QTGMC.

Also non-linear sharpeners absolutely not-acceptable for quality demandable linear workflows. It may easily damage the total idea of zoneplate (also for may be modern environments all conditioning in linear scene light is required - not in transform-domain).

[Mitra]

@poisondeathray , Thanks for your reply, but as you can see the output of your script also has artifacts and things that are not in the source.

[Mitra]

LanczosResize(1920, 1080, taps=64)


@DTL,Thanks, I know about normal resizing like Lanczos does(I can do it with ffmpeg) , But I want to upscale many source of a TV channel with high quality like what AI does (They gave me this source to use as an example.), I tried many AI software without success (all of them have artifacts), now I want to try AviSynth.

[poisondeathray]

But I want to upscale many source of a TV channel with high quality like what AI does (They gave me this source to use as an example.), I tried many AI software without success (all of them have artifacts), now I want to try AviSynth.

UPscaling ALWAYS has artifacts (including all types "AI" scaling), unless source is vectorized

"AI" that is trained on upscaling test patterns (and performs well on test patterns) will generally fail on normal programming content . You currently can't have both

[Mitra]

Zoneplate looks like progressive so you can
It is typically assumed the digital test patten is perfectly finetuned in its original digital domain so require no any corrections. Also AVS processing mostly not great or even poor in performance for high-quality designed test patterns so any AVS processing should be used with great care and check each step for quality. Unfortunately for interlaced the high-quality deinterlacer is required so the complex QTGMC script loaded with many AVS processing is close to unavoidable. It it start to show some non-acceptable artifacts - the diffferent may be not as nice but more 'linear' deinterlacer required. The yadif/yadifmod may be next to try after QTGMC.

Also non-linear sharpeners absolutely not-acceptable for quality demandable linear workflows. It may easily damage the total idea of zoneplate (also for may be modern environments all conditioning in linear scene light is required - not in transform-domain).

Thank you for the link and explanation.

[Mitra]

@poisondeathray and @DTL ,
There is a strange problem because even with simple sharpening, the output has artifacts.!!

Please try this:

FFMS2("SD625i50.mov")
Sharpen(0.5)

Artifacts are created without changing the size.!! (I changed the title because this problem)

[poisondeathray]

@poisondeathray and @DTL ,
There is a strange problem because even with simple sharpening, the output has artifacts.!!

Please try this:

FFMS2("SD625i50.mov")
Sharpen(0.5)

Artifacts are created without changing the size.!!

Normal and expected. Many test patterns (especially zone plates) are supposed to detect sharpening, resampling and other manipulations

[Mitra]

Normal and expected. Many test patterns (especially zone plates) are supposed to detect sharpening, resampling and other manipulations

Aha, this is very disappointing to me because those who gave me test pattern are judging the quality by that.

[anton_foy]

@poisondeathray and @DTL ,
There is a strange problem because even with simple sharpening, the output has artifacts.!!

Please try this:

FFMS2("SD625i50.mov")
Sharpen(0.5)

Artifacts are created without changing the size.!! (I changed the title because this problem)

Have you tried another sharpener? I have had alot of problems with artifacts on real life footage because of internal sharpen.

[Mitra]

Have you tried another sharpener? I have had alot of problems with artifacts on real life footage because of internal sharpen.

Thank you, I applied unsharp filter by ffmpeg on the source and it works! ( without artifacts)

Which another sharpener can I use with AviSynth?

[DTL]

I tried many AI software without success (all of them have artifacts), now I want to try AviSynth.

AVS do not have any AI magic for upscale inside and also most of currently used non-linear upscalers are poor attempt to use static image upscalers for motion pictures. So the more advanced non-linear upscaler the less stable multi-frame result per total cutscene it may produce and it will cause temporal image non-stability. The most advanced motion pictures non-linear upscalers are multi-frame analysis based to increase temporal prediction stability (may be Topaz Video ?). And currently it looks AVS do not have any multi-frame AI scene analysers for upscaling.

The attempt to use nnedi3_rpow for upscale typically very poor because it is special case NN design to fight the vertical aliasing artifacts in case of separated fields upscale with motion between fields and poor V-direction prefiltering in camera settings (so vertical aliasing at moving areas is high but it helps to keep better resolution on static areas).

Also typically static imaging AI-upscalers take no care about requirements for moving pictures digital systems for conditioning to some target rendering/displaying environment (for example 1D+1D sinc-based or close) so produce additional artifacts (like aliasing/moire/ringing) or require re-conditioning again.

Example of possible 2x target upscale with re-conditioning for target output size:

MyAI_Upscaler(width*8, height*8)
UserDefined2Resize(width/4, height/4, b=.., c=.. ,(s=..))

[DTL]

@poisondeathray and @DTL ,
There is a strange problem because even with simple sharpening, the output has artifacts.!!

Please try this:
FFMS2("SD625i50.mov")
Sharpen(0.5)

Artifacts are created without changing the size.!! (I changed the title because this problem)

It may be not artifacts of sharpen. It is also because you can not control digitally encoded image in its digital sampled domain at poor (pixel-based) display using some random scaler or not use any scaler at all. You need to feed it to video-control display having enough pixel number for oversampling (starting from 2x or better 4x and more) and correct scaler. Also your image digital domain encoding type and display's scaler operation must match.

LanczosResize(width*4, height*4, taps=50) can simulate good quality sinc-based scaler to use poor direct samples to pixels mapping RGB display for quality test. But also you need to know if image data is conditioned in transfer-domain or other (linear scene light for example). And if it is conditioned in other domain you need first convert to conditioning domain before scaling.

Also if you apply frequency-corr operation to high-amplitude encoded data it may easily reach clipping and your carefully designed zoneplate will be destroyed with non-linear transform and new frequencies components. Also the AVS Sharpen() filter not guaranteed to be free from non-linear distortions. AVS is sort of amateurs software for home DVD-backup. Not for creating and operating with industry precise designed digital data. It only may have some non-distortive operations.

"Which another sharpener can I use with AviSynth?"

You can try GeneralConvolution() it may be more linear operation. Also you need to check first the peak-to-peak amplitude of your zoneplate at the high frequencies - if is soft of the current system limit (like 0 to 255 in full encoding) so any attempt to make it 'sharper' will damage it (or you need to work in float domain without clipping of any samples values). If your source is in 16..235 levels narrow range - it can accept very small sharp before clipping and damaging again.

There is some pro-like tools for waveform monitoring for AVS - https://forum.doom9.org/showthread.php?t=175249 . You may try it to control levels range before and after processing. If your levels start to go outside nominal range it already not-safe. AVS do not have any warnings about not-safe or direct damaging operations.

[Mitra]

@DTL,Thank you so much for your explanation.I'll try them.

[FranceBB]

Which another sharpener can I use with AviSynth?

LSFMod() works well, otherwise CAS() if you want something new. If you want to go for an old classic, you can use SeeSaw().

When it comes to Avisynth filtering, this is a list of what I generally use and the relative parameters to fight any kind of issue. I generally use those multiple time a day at work 'cause I'm lazy and copy-pasting is easier and faster than remembering all the options all the time: Link (https://github.com/FranceBB/FranceBB_Sample-Avisynth-Script)

About the sample, I haven't downloaded it yet but I'm at home and I won't be sitting at my desk 'till Tuesday (Easter break, you know) unless something very bad happens, so I won't give it a shot.

What I can tell you is that those kind of patterns are meant to detect any kind of "smart" filtering, so the changes you're seeing don't probably come only from the upscale but also from the motion adaptive deinterlacing. Try something simple and very crappy first to see if there are no "artifacts" (there will be but in this case I mean "artifacts" in the same way you mean it, namely interruption of the pattern):


#Indexing 25i source
LWLibavVideoSource("yoursource.mov")

#Bob to 50p
Bob()

#Resizing while keeping AR
PointResize(1440, 1080)
AddBorders(240, 0, 240, 0)

#Reinterlacing to 25i
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()


Although it will look poor, it should just copy values from the closest sample and duplicate 'em ('cause it's nearest neighbor, you know), resulting in plenty of aliasing, but at least it shouldn't screw up the pattern.

Please NEVER EVER use it with real life footage.

[anton_foy]

Thank you, I applied unsharp filter by ffmpeg on the source and it works! ( without artifacts)

Which another sharpener can I use with AviSynth?

Dogways unshap_HBD or ex_unsharp, xsharpen etc. there are lots.

[DTL]

Some additional note: The provided source test pattern is not interlaced in zoneplate design. And the 'simple' interlacing AVS op of classic

AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

is also not completely correct if we even try to simply convert this animated zoneplate from N p to N/2 i format. It is only about good if the original content was 'linearly' deinterlaced and re-interlacing again. For initially progressive high-sharpness content it produce lots V-aliasing. As I remember with ffmpeg they add some optional V-AA filter to p to i conversion (non-controllable ? and not motion-adaptive ? https://ffmpeg.org/ffmpeg-filters.html#toc-interlace ).

The simple AVS better form may be
UserDefined2Resize(src_top=0.0001, width, height, b=.. , c=.. ,(s=..))
AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

where first line is some AA/conditioning prefilter before vertical halfsampling. In more strict form it can be even
UserDefined2Resize(width, height/2, b=.. , c=.. ,(s=..))
SincLin2Resize(width, height*2, taps=16)
AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

But it will degrade overall frame V-sharpness even more. Some more 'adaptive' form may apply method 2 only to moving areas and close to nothing to static (in some script with motion mask and overlay).

The next sad truth is because we do not have global industry standards on digitally encoded moving images spatial displaying - the 'perfect' filtering for the source test pattern can not be defined too. Only for some closed digital imaging with its own defined spatial display image reconstruction sort of 'perfect' interlaced (and progressive too) test pattern can be designed. So all such test patterns may somehow work for total performance of processing or display performance estimation but not for lab-precise check and grading. One may own one sample of digital test pattern but it will not be completely applicable to test different processing workflows or different displays from different vendors of this planet.

Our production still in 1080i so when we try to import some sharp DSC footage in progressive using forced interlacing in ffmpeg without V-AA it looks awfully aliased. May be in latest versions of Adobe Premiere it is somehow fixed better. The optional V-AA filter in ffmpeg helps significantly (also degrades sharpness).

[FranceBB]

Our production still in 1080i so when we try to import some sharp DSC footage in progressive using forced interlacing without V-AA it looks awfully aliased.

Yeah, that's what happens over here too when we downscale UHD 50p to create FULL HD 25i if we don't blur.
I always end up using some variations of:

blur(0, 1.00)

which blurs only vertically according to how sharp the original source is.
It truly breaks my heart to blur vertically, but it's very much needed.
In other words, for UHD 50p to FULL HD 25i my scripts are generally something like:

blur(0, 1.00) #Always blur vertically before interlacing
SinPowResizeMT(1920, 1080)
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()


If people are interested about why blurring vertically is needed when interlacing a very sharp source (like when you downscale UHD to FULL HD), here: https://forum.doom9.org/showthread.php?p=1874689

[Mitra]

@DTL,Thank you so much for your help, but the UserDefined2Resize gets error : "There is no function named 'UserDefined2Resize'"

@FranceBB, Thanks for your reply and a big thanks for the VideoTek tool.

[DTL]

""There is no function"

It is in 3.7.3. test6 and later. Also jpsdr plugins https://forum.doom9.org/showthread.php?t=173772 2.3.8 have UserDefined2ResizeMT with s-param (also it is defaulted to a bit higher 2.0 to make more difference from SinPowResize).

If you use old AVS you can try to play with GaussResize and its p-param. Or GeneralConvolution with vertical LPF.

"It truly breaks my heart to blur vertically,"

You can try to add more complex scripting with motion mask and overlay blurred only where motion is detected. But it may be much slower (like mvtools + MMask ?) in processing. Also the mask must mark already very slow motion to save from visible temporal aliasing. It is really sort of a task for AI/NN processing (to filter only aliasing-sources using feedback of aliasing detection) - but today interlaced is of low interest for designers.