Infognition Super Resolution plugin for AviSynth

[Mole]

Super Resolution is a method to resize by using information from neighboring frames. This plugin uses quarter-pixel motion compensation and a fusion process to combine motion-compensated previously upsized frames with spatially resampled current frame. The SR process doubles the resolution and if different resolution is specified in the argument, it will first upscale with SR to 200%, then resample to desired resolution.

Usage is:
SR(width, height, numThreads, options)

For example:
LoadPlugin("avssr.dll")
AviSource("video.avi")
SR(1920,1080,12)

This will resize the video to exactly 1920x1080 using 12 threads.

Default setting SR() is to upscale to 200% in high quality mode using all logical cores.

Please check readme for detailed usage.

This plugin is multi-threaded and it's recommended to not use SetMTMode at all, or SetMTMode(5).
Using SetMTMode(2,0) for example will greatly decrease the performance.

Tests on i7 2600K with Hyper-Threading enabled, optimal performance is achieved with 12 threads.

Also works very well with ffdshow AviSynth filter for real time resizing.
For SD content, such as 704x400 or 832x468, I get great results with SR 200% resize, and let the Enhanced Video Renderer resize to 1920x1080.

On my i7 2600K, 704x400 the CPU load is only around 20%.
Real time upscale should be possible on any i3 CPU.

Unregistered version of this plugin will draw some watermark lines in the video.
If you have Infognition Super Resolution plugin for VirtualDub registered, then this AviSynth plugin is free and it's already registered too.

You're also eligible for license if you already have a license for Video Enhancer.

http://www.infognition.com/super_resolution_avisynth/

[feisty2]

I tested it and got tons of aliasing like something caused by upscaling with bicubicresize (b=-1,c=0)

[DmitryPopov]

Would be nice to see some samples of that aliasing problem. Make sure you don't just look at first frame (where no temporal info is accumulated yet), let it process some number of frames first.
I use this plugin to watch some rips on my laptop now using default 2x upsizing and letting the player to resample the result to full screen resolution. Comparing to available resize methods in MPC-HC I usually see sharper and more detailed image than with bicubic resize. Sample:
Bilinear (http://data.infognition.com/avssr/tf12323a-bili.png)
Bicubic (http://data.infognition.com/avssr/tf12323bc.png)
SR (http://data.infognition.com/avssr/tf12323sr.png)
Try opening in neighbor tabs and switch back and forward to see the difference.

[wonkey_monkey]

While there is a visible difference between bicubic and SR, a lot of it disappears if you apply a Photoshop sharpen to the bicubic one.

[feisty2]

here
original
http://thumbnails110.imagebam.com/30421/aa35e0304206246.jpg (http://www.imagebam.com/image/aa35e0304206246)
sr () (look at the boxed areas, I donno what this kind of pointresize styled artifacts is called, I call it "aliasing" for now)
http://thumbnails104.imagebam.com/30421/16c77b304206358.jpg (http://www.imagebam.com/image/16c77b304206358)
dither_resize16nr (1440,960,kernel="spline",taps=32).ditherpost () (I didn't get any artifacts at all!)
http://thumbnails111.imagebam.com/30421/0e345f304207167.jpg (http://www.imagebam.com/image/0e345f304207167)
edi_resize16 (1440,960) (my own moded version of mawen1250 "nnedi3_resize16") (sharp, thin edge, and still no artifacts at all)
http://thumbnails109.imagebam.com/30421/1eda93304207421.jpg (http://www.imagebam.com/image/1eda93304207421)

[Mole]

How will I be able to view the full size images and not just thumbs?

[feisty2]

How will I be able to view the full size images and not just thumbs?

just click the thumbs

[Mole]

I get an image which is 350x234, is that correct??

http://thumbnails109.imagebam.com/30421/1eda93304207421.jpg

[feisty2]

no the image should be 720x480 1440x960 1440x960 and 1440x960 in PNG format
original http://110.imagebam.com/download/qPwhi4gReP5gDUdjYRFt9Q/30421/304206246/original.png
sr http://104.imagebam.com/download/vfKsT9rRZ9AisibvkJUtpQ/30421/304206358/sr.png
dither_resize16nr http://111.imagebam.com/download/JJ3E8-ya-nfuLRdoJMuvRA/30421/304207167/spline.png
edi_resize16 http://109.imagebam.com/download/weQK0PjCPJZxxHFaEyf8Jw/30421/304207421/edi.png
and the artifacts is very similar to manalyse in mvtools2 with wrong "dct" parameter settings

[Mole]

Nevermind, stupid me... I had a look in my options.

For some reason it was disabled. Can't recall that I've ever set this option before though...

[feisty2]

But the links you've posted are all to the thumbnails, not the full image...

Links to full imagebam images are supposed to be like this:

http://www.imagebam.com/image/aa35e0304206246
http://www.imagebam.com/image/16c77b304206358
http://www.imagebam.com/image/0e345f304207167
http://www.imagebam.com/image/1eda93304207421

open the links you posted or the links I gave you should see the original images, I've tested them, maybe you should change a web browser (or use a proxy server) and try again?

[feisty2]

so can anyone tell me what that pointresize styled artifacts is? I'm very curious

[DmitryPopov]

Hmm, I'll look at the pictures on another monitor tomorrow, it's hard to see the difference now on my laptop.
What it looks to me now is like the nnedi & spline results are either in RGB or YUV 4:4:4 i.e. there is no chroma subsampling while SR one is in YV12 (with U and V half the resolution). Am I right? In that case you may try converting to RGB24 or RGB32 before applying SR, this way it will work in full resolution on all channels.

Also, sometimes SR makes some ghosting artifacts when parts of previous frame add some patterns to the current frame.

Note: when the source is blurred due to bad focus, motion blur or just low bitrate, you won't see much difference between different resize methods. A sharper source video will give more interesting results.

[ChiDragon]

I get these grid artifacts.

http://s22.postimg.org/z8hj3zt1d/example1.png
http://s22.postimg.org/midaqwl35/example2.png

[feisty2]

Clearly its not chroma subsample error cuz the artifacts are on plane y and I did all tests on yv12, it seems like bad motion compensation artifacts

[Forensic]

I have had great success with VideoEnhancer, but only when run as the final step of my process and in its EXE form (not the plug-in). I usually need to run an Unsharp or Sharpening tool as the last filter. But then again, all my work is with surveillance videos so my usage may not be that common. Although VideoEnhancer is a bit dated (I believe it is based on MSU's 2012 work), I have yet to find anything that works better for my needs.

[feisty2]

I get these grid artifacts.

http://s22.postimg.org/z8hj3zt1d/example1.png
http://s22.postimg.org/midaqwl35/example2.png

your grid artifacts are actually the same things I'm talking about :)

EDIT: these weird artifacts can also be found in their official demos

[DmitryPopov]

Forensic,
these AviSynth and VirtualDub plugins share the SR engine with Video Enhancer 1.9.9 released just a month ago. If you're using an older version of it, I recommend downloading the latest version, it's significantly faster (and quality is higher on some files).
It's not based on MSU work. VE first appeared in 2006 and later we had one common project with MSU on super resolution for a particular client, but SR engine of Video Enhancer and these plugins is not based on that code.

feisty2 et al.,
thanks for pointing out these artifacts, we'll try to find what exactly causes them.

[feisty2]

thanks for pointing out these artifacts, we'll try to find what exactly causes them.
are you a member of the developers of this sr filter? your filter is really interesting, cuz it did gave me some more exquisite upscaled result on some areas full of dense and small details (like lots of fallen leaves on the ground) than simple lanczos4resize, just the artifacts are too bad, and are you interested in writing a detail enhancing filter using the same sr technology instead of this upscale filter, that could be a lot more useful :D

[feisty2]

I did another test
original
http://thumbnails101.imagebam.com/30442/7c4105304413013.jpg (http://www.imagebam.com/image/7c4105304413013)
lanczos4resize (1440,960)
http://thumbnails109.imagebam.com/30442/d92aa8304413007.jpg (http://www.imagebam.com/image/d92aa8304413007)
sr
http://thumbnails109.imagebam.com/30442/ff0d4b304413049.jpg (http://www.imagebam.com/image/ff0d4b304413049)
eedi3 (dh=true,sclip=nnedi3 (dh=true)).turnleft ().eedi3 (dh=true,sclip=nnedi3 (dh=true)).turnright ()
http://thumbnails112.imagebam.com/30442/9bb611304415857.jpg (http://www.imagebam.com/image/9bb611304415857)
edi_resize16 (1440,960)
http://thumbnails112.imagebam.com/30442/4af431304416003.jpg (http://www.imagebam.com/image/4af431304416003)
sr gives me slightly better result on boxed area

EDIT: added more tests

[DmitryPopov]

are you a member of the developers of this sr filter?
Yes I am.

and are you interested in writing a detail enhancing filter using the same sr technology instead of this upscale filter, that could be a lot more useful :D

That's tricky. What this thing does now is turning each pixel into 2x2 block of pixels so that their average equals to the original. Which means if you downsize it back you get just the source, no new details. Details are only added on that high-frequency range, on the scale which is effectively one pixel of source video. In other words details are only appearing "inside" pixels, so they are only visible when upscaling.

[feisty2]

I succeeded in making a detail enhancing script by this sr filter

a=last
lanczos=a.lanczos4resize (a.width ()*2,a.height ()*2)
sr=a.sr ()
diff=mt_makediff (sr,lanczos).bicubicresize (a.width (),a.height (),b=-1,c=0).mt_lut("x 128 - "+String(5)+" * 128 +") #"5" is the strength#
mt_adddiff (a,diff)

the result is very impressive, the only problem is I got tons of terrible noise
original
http://thumbnails110.imagebam.com/30531/e71db3305308340.jpg (http://www.imagebam.com/image/e71db3305308340)
enhanced
http://thumbnails111.imagebam.com/30531/26cbc8305308326.jpg (http://www.imagebam.com/image/26cbc8305308326)

[KenJ]

Old topic, I know.

Does anyone know what happened to Dmitry Popov, username here DmitryPopov? I last contacted him on his website www.infognition.com to upgrade my copy of his wonderful software 'Video Enhancer' but now that website is unreachable. All links from web searches are unreachable. The website appears to not be dead but unresponsive. Any help is welcome. Thanks.

-=Ken=-

[DmitryPopov]

Hi Ken. The site is up and running but unfortunately not for everyone: our hoster blocks some IP ranges in defense from spammers. If you try accessing it via proxy or using another ISP, like going via mobile phone instead of home internet, you might have more luck accessing it.

Since the thread is up: now this plugin is free, as well as other Infognition products. I've now moved to London and am doing something completely different.

[StainlessS]

Join Date: Feb 2005
Location: Thailand
Posts: 65
Maybe update your details.

[KenJ]

Hi Ken. The site is up and running but unfortunately not for everyone: our hoster blocks some IP ranges in defense from spammers. If you try accessing it via proxy or using another ISP, like going via mobile phone instead of home internet, you might have more luck accessing it.

Since the thread is up: now this plugin is free, as well as other Infognition products. I've now moved to London and am doing something completely different.Hey, Dmitry, glad to see you are alive and well! I wish you well in your new endeavor. Unfortunately, I tried three different VPNs - and my phone - and I still can't bring up infognition. It seems that I need more luck!

-=Ken=-