I'm trying to apply Avisynth for upscaling and antialiasing of a live gameplay video to enhance video quality. Basically, the game runs in a small window on my second monitor, where I capture its video output, process it in Avisynth and then output it to my main monitor in fullscreen.
The reason is that the game is being run on the PS2 emulator and this particular game has very poor hardware mode support, so I have to resort to software, which is limited to console's native resolution - 512x448. So in order to play it in fullscreen without bleeding my eyes out, I decided to give a try at such an exotic Avisynth application.
Considering that software mode uses only CPU, the upscaling and spatial antialiasing are done using the OpenCL version of NNEDI3 that runs on GPU. Before that I do some temporal antialiasing by averaging source video with its motioncompensated version at the edges.
The script as a whole is being run via Avisynth tab of the FFDShow raw video filter.
Here is my script so far:
Code:
#-Settings-------------------
#Spatial anti-aliasing, runs on GPU. 2x for resolutions around 1280x720, 4x for larger screens if you have spare power.
FSAA=2
#Temporal anti-aliasing, runs on CPU. Reduces flickering in motion, possible values 2 or 3. 0 disables it.
TXAA=3
#HDR dynamic contrast. Increases visibility in overly bright or dark areas. Values list: 0, 0.25, 0.5, 0.75, 1(>0.5 is not recommended).
HDR=0.25
#/Settings------------------
last=(HDR>0)?raveragew(last, 1-HDR, coloryuv(autogain=true ), HDR, mode=4, u=3, v=3, lsb_out=true).ditherpost():last
uv=last
super = MSuper(pel=2, chroma=false)
backward_vectors = MAnalyse(super, isb=true, blksize=8, overlap=2, search=4, truemotion=true, chroma=false, temporal=true, badsad=1000)
forward_vectors = MAnalyse(super, isb=false, blksize=16, blksizev=8, overlap=0, overlapv=2, search=4, truemotion=false, chroma=false, temporal=true, badsad=1000)
backward_compensation = MCompensate(super, backward_vectors, thSCD1=600)
forward_compensation = MCompensate(super, forward_vectors, thSCD1=400)
AA=(TXAA==3)?merge(merge(forward_compensation, backward_compensation).sharpen(0.2),0.67):merge(backward_compensation)
me=mt_edge(mode="laplace", thy1=0, thy2=10, u=1, v=1).mt_inflate()
mm=mt_motion(thy1=0, thy2=15, tht=100, u=1, v=1).mt_inflate()
m=mt_logic(me, mm, mode="min", u=1, v=1).mt_expand(mode="both")
last=(TXAA>0)?mt_merge(AA, m, u=1, v=1):last
uv=uv.bicubicresize(1870, 1050)
nnedi3x_rpow2(rfactor=FSAA, u=false, v=false, nns=2)
spline36resize(1870, 1050)
mergechroma(uv)
sharpen(min(0.33*FSAA, 1.0))
I'm using the modified version of MVTools 2 from Dither package.
Since I have NVidia, I'm only aiming at achieving at most 30 fps of speed. This script already runs at ~30fps but this is at the limits of my system's capabilities (Core i5 2550k @ 4.4GHz, GTX 560 @ 950MHz).
Could you please advise, what can be improved in my script to raise quality or maybe spare some processing power?
So far the most troubling part is the temporal antialiasing, it causes chroma ghosting due to it not being processed together with luma to save time. Another thing that worries me is that I have no way of measuring the lag that this processing causes on the game (responsiveness to controls input). As such I will have to rely only on the theoretical knowledge of what kinds of filters can introduce lag during live processing. For example, what will introduce more lag, forward motion estimation and compensation or backward one? What about motion mask or shifting the video 1 frame forwards or backwards (in case I would need to smooth masks in time, for example).
Here is the recorded gameplay video that I was using for tuning my script:
https://dl.dropboxusercontent.com/u/...mgs3motion.mkv.