Upscaling/Anti-Aliasing/Line Smoothing

[oddball]

I've followed a few guides on how to upscale DVD's and have found a happy medium of my own in terms of quality. However two things do not work for me well. AviSynth crashes using the various guides and postprocessing causes too many artefacts (little blocks appearing randomly and especially along edges).

I don't mind leaving either of those options off but the issue I have is with lines. Especially in animation. Lines suffer from stepping/aliasing which is kinda annoying. I was wondering if there is any way to smooth edges/lines?

I hope I am posting this in the right forum.

[Guest]

AviSynth crashes using the various guides What guides? What scripts crash?

and postprocessing causes too many artefacts (little blocks appearing randomly and especially along edges). Postprocessing with what?

I don't mind leaving either of those options off What options for what tool?

but the issue I have is with lines. Especially in animation. Lines suffer from stepping/aliasing which is kinda annoying. I was wondering if there is any way to smooth edges/lines? Have you tried things like nnedi?

Your post is sorely lacking in details. Also, wrong forum.

[oddball]

My apologies. I wasn't sure which forum to post this in. Maybe it should have been in general. I will try to provide more details. This is for realtime playback of DVD's. Not reencoding.

The guide I originally tried to follow was this.

http://forum.doom9.org/showthread.php?t=115727&highlight=dvd+upscaling

Unfortunately AviSynth would crash with these settings. I use MPC Home Cinema latest build playing fullscreen to an HDTv as secondary display in DXVA fullscreen mode. The player always remains on the primary monitor but the video output is to the secondary display.

Using ffdshow to resize to either 720 or 1080 I use swscaler for sharpening, denoise (3d HQ) and a few other tweaks I can't remeber off the top of my head. If I enable AviSynth using the settings in that guiide MPC-HC crashes. Using PostProcessing in ffdshow makes some things look better but causes random little blocks/artefects to appear. Especially alone any lines/edges. It really shows up in animation. Strangely the mode most recommended which is mplayer with accurate deblock causes the most artefacts. This is using ffdshow tryouts build 3507 with MPC-HC build 2099.

I also have Windows 7 64 and an Nvidia G210 GPU.

I hope this is more informative.

EDIT: Hmm. I got AviSynth working but that settings in the guide slow it to a complete crawl (E6600 OC'd to 3.4Ghz here). I do not think this is an interlacing issue although I may be wrong. I have tried enabling deinterlace in ffdshow but it makes things look very ugly. I will provide a screencap.

http://a.imageshack.us/img215/7117/aliasing.jpg

See the aliasing along the nose and edges of the face? Also is there a way to smooth out the stuff around the edges like the noise inside the whites of the eyes?

EDIT2: From Googling around I think the term is mosquito noise and haloing. I have yet to find a way to do realtime dehalo and removal of mosquito noise. Does anyone know how to use AviSynth deblock and undot in realtime in ffdshow?

[Guest]

Now your post is very good! And the forum is correct now that you explain you want real-time playback.

I'm not experienced in these matters. I would just let my TV do the job. :)

Hopefully someone will be able to help out.

[namaiki]

Try use Overlay. IMO, nVidia GPUs have a pretty alright scaler (smoother lines than bicubic and less artifacts from sharpening - not sure what it uses though). (does not apply to EVR, VMR, or Haali as those will use a different scaler)

[naoan]

For cartoon source I usually use aaa and dehalo_alpha coupled with madvr with spline upscaling, this usually will oversmooth the edge however and you may opt to use unfilter or lsfmod(defaults="slow") if you have the processing power to spare.

For denoising I recommend fft3dgpu, deblocking with deblock_qed. All of it can be found in http://avisynth.org/mediawiki/External_filters

Edit: Wait, you wanted to do realtime avisynth processing in 1080p content, then the above filter except fft3dgpu most likely won't run in realtime even with your oc'd cpu. I'd stick with madvr.

[roozhou]

Overlay on nVidia has very strange morie artifact.

[namaiki]

Overlay on nVidia has very strange morie artifact.

Have you seen this when using image enhancements in the nVidia control panel (like sharpening, or noise reduction), or even with default settings? I don't think you would see any such effects when upscaling video though.

[oddball]

Thanks for the input. I have been playing a bit and I think it is impossible to get smooth lines and dehalo in realtime without a very powerful CPU and GPU. I would like to stand corrected though if anyone can provide a few sample AviSynth scripts to try. I have most of the plugins. Not understanding all the various scripts etc though means it's hard to troubleshoot when it complains of things like invalid functions.

[roozhou]

Have you seen this when using image enhancements in the nVidia control panel (like sharpening, or noise reduction), or even with default settings? I don't think you would see any such effects when upscaling video though.
I use all default settings in NV control panel. Moire artifact can be seen on both upscaling and downscaling, and even at the original size.

Here is a test video. I have tested on GF 6/7/8/9 series.
http://www.mediafire.com/?dyommw3mvymlnwg

[namaiki]

I use all default settings in NV control panel. Moire artifact can be seen...even at the original size.

You do realize that nVidia Overlay defaults to not expanding TV levels to PC? I think that's what you're saying, if you are looking at it and it's looking grey instead of sharp.

[roozhou]

You do realize that nVidia Overlay defaults to not expanding TV levels to PC? I think that's what you're saying, if you are looking at it and it's looking grey instead of sharp.
I am not a noob and I can tell full range and limited range at the first glance. This problem is unrelated to TV/PC level. I have manually set Dynamic Range to 0~255 in NV control panel.

The sample I uploaded contains only black and white bars, but overlay on NV cards shows yellow or purple moire artifact even at the original resolution.

[namaiki]

As far as I can tell, on my GeForce 9600M GT, EVR, Haali's Renderer, MadVR and Overlay all look exactly the same down to the pixel at the original resolution. (alt-tabbing between two maximized windows with the different video renderers)

If you have range set to 16-235, do you see these artifacts, or do they only appear when set to 0-255?