PDA

View Full Version : Super Resolution using MVTools


Efenstor
11th November 2008, 13:56
It's possible to use Fizick's MVTools to double video resolution and actually improve detail (i.e. bring out the barely visible). This technique is called "super resolution" and can really work miracles on videos shot with photo cameras and cell phones. Of couse, it can be also used to upscale SD to HD.

Updated on 2009-05-15
Required: MVTools 2.4.2, NNEDI 1.3, MT 0.7 (for mult-core CPU's, if yours is not, remove the SetMTMode lines)

SetMTMode(5)
AVISource("test.avi")
ConvertToYUY2()
SetMTMode(2)

MDeNoise(100)
MSR()


function MDeNoise(clip s, int th)
{
blkH = 8
blkV = 8
ovl = 2
dct = 0
srch = 4
pel = 2

super = MSuper(s, pel=pel)
vec1 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=true, delta=2)
vec2 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=true, delta=1)
vec3 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=false, delta=1)
vec4 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=false, delta=2)
MDegrain2(s, super, vec1, vec2, vec3, vec4, thSAD=th)
}



function MSR(clip s)
{
blkH = 16
blkV = 16
ovl = 8
srch = 3
dct=0
pel=2

bc = BilinearResize(s, Width(s)*2, Height(s)*2)
super = MSuper(bc, pel=pel)
vec1 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=true, delta=2)
vec2 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=true, delta=1)
vec3 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=false, delta=1)
vec4 = MAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb=false, delta=2)

s = nnedi(s, field=0, dh=true)
s = TurnLeft(s)
s = nnedi(s, field=0, dh=true)
s = TurnRight(s)

s = Sharpen(s, 1)

super = MSuper(s, pel=pel)
c1 = MCompensate(s, super, vec1)
c2 = MCompensate(s, super, vec2)
c3 = MCompensate(s, super, vec3)
c4 = MCompensate(s, super, vec4)

t1 = Overlay(s, c1, opacity=.5)
t2 = Overlay(s, c2, opacity=.5)
t3 = Overlay(s, c3, opacity=.5)
t4 = Overlay(s, c4, opacity=.5)

f1 = Overlay(t1,t2, opacity=.5)
f2 = Overlay(t3,t4, opacity=.5)
Overlay(f1,f2, opacity=.5)
}

Efenstor
11th November 2008, 14:39
You can also try

vec1 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-2)
vec2 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-1)
vec3 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=1)
vec4 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=2)

instead of

vec1 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb = true, delta=2)
vec2 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb = true, delta=1)
vec3 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb = false, delta=1)
vec4 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, isb = false, delta=2)

I've discovered it by chance. I don't know for what reason but it gives more details! (though a bit more noise too).

Efenstor
11th November 2008, 14:51
Multithreaded version (the second method, with negative deltas):

SetMTMode(5)
AVISource("11.avi")
SetMTMode(2)
ConvertToYUY2()

blkH = 16
blkV = 16
ovl = 4
srch = 3
dct=0
pel=2

bc = BilinearResize(Width()*2, Height()*2)
super = MVSuper(bc, pel=pel)
vec1 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-2)
vec2 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-1)
vec3 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=1)
vec4 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=2)

Lanczos4Resize(Width()*2, Height()*2)
Sharpen(1)
super = MVSuper(pel=pel)

c1 = MVCompensate(super, vec1)
c2 = MVCompensate(super, vec2)
c3 = MVCompensate(super, vec3)
c4 = MVCompensate(super, vec4)

t1 = Overlay(c1, opacity=.5)
t2 = Overlay(c2, opacity=.5)
t3 = Overlay(c3, opacity=.5)
t4 = Overlay(c4, opacity=.5)

f1 = Overlay(t1,t2, opacity=.5)
f2 = Overlay(t3,t4, opacity=.5)
Overlay(f1,f2, opacity=.5)

Terka
11th November 2008, 14:52
can you put some pictures?

Sagekilla
11th November 2008, 15:19
Sample images:

Downsized from 2160p source:
http://img75.imageshack.us/img75/4126/downsizespline16so3.png

Downsized to 512x288 and upsized to 1024x576 with MVTools:
http://img99.imageshack.us/img99/448/downsizemvtoolshl0.png

Downsized to 512x288 and upsized to 1024x576 with only Spline16:
http://img171.imageshack.us/img171/4152/upsizespline16sr7.png

Didée
11th November 2008, 17:18
Erm, that's old hats, no? Put short, what's happening is

sharpen().MCDenoise()

Nothing wrong with that, basically. It's just way too simple. The mere spatial step of introduces halos and ringing, which won't be cancelled out by the temporal averaging.

You need to figure which parts of the input signal need emphasizing (increase signal amplitude), and which [U]don't need emphasizing, but only final refinement. *That's* the clue.

Hint: one piece of the puzzle is named "median". ;)


Consider:

"Downsized to 512*288, upsized 300%, sharpened, sharpened, resized to 1024*576"

http://img46.imageshack.us/img46/7707/onlyspatialnt6.png (http://imageshack.us)

Compare to your examples.

Efenstor
11th November 2008, 17:59
You mean that adjacent frames contain no additional information? Yes the idea is simple but there's more than that you supposed: I take four adjacent frames, motion compensate them, sharpen them and mix them all together with the central one, so that all exaggregated noises disappear and only real details become more visible. Moreover, since I motion compensate an upscaled frame, the edges of objects in the combined frame will be <=4 times more exact to their real-world shape. That's the method of super resolution, I didn't invent one.

I'll post my examples in a while and you'll see it's better. Not too much, but still better than simple sharpen, whichever it is.

Efenstor
11th November 2008, 18:11
Here they go. I used the "Multithreaded version" posted above. It gives the best quality of all.

Your method:
http://www.efenstor.net/external/sh.jpg
My method:
http://www.efenstor.net/external/sr.jpg

Note that the ear shape is much more accurate, more detail around the mouth are visible, you can see faint rims on the windowpane and black dots in the corners of the windowpane are less smeared. This is not achievable using standard methods.

I don't claim it's "hell that good" but it's a real wonder for lo-fi sources, especially those where shapes are slightly distorted from frame to frame due to heavy compression. Try some actual lo-res examples, not downscaled from HD, and you'll see it's better.

*.mp4 guy
11th November 2008, 18:31
.... How can you reproduce Didée's results when he never posted the script.


Dude, all its doing is removing noise its not recovering detail lost during interpolation, its recovering detail lost during compression, if its doing anything at all, which is debateable. I'm not saying temporal super resolution doesn't work on some sources, but mvtools isn't designed to do anything but denoise, it by design will not work on high contrast, low SAD areas of a frame (aliased lines fall well within this generalization).

Efenstor
11th November 2008, 18:33
Okay, then this method sucks and I'll not use it to improve quality of my lo-fi videos. :) Please suggest better.

*.mp4 guy
11th November 2008, 19:03
I have no idea what your trying to do, or what your videos look like, how am i supposed to suggest something else?

look, what your doing right now is interpolating them to make them bigger, sharpening to make them look less blurry, and denoisng to make them look less crappy, there isn't anything wrong with that. I'm just trying to tell you that mvtools isn't going to help with the interpolation step.

Their are plenty of high quality sharpening and interpolation methods available on the forums. If you don't want to use search to find them, you could post a representative sample from one of the videos you are working with, post what you want to do with it (destination resolution, denoising preferences, etc.) and ask for advice.

Efenstor
11th November 2008, 19:21
I'm trying to upscale by 2, at least in general and want super resolution in particular. As far as I know there's nothing better than EEDI2 but it looks too blurry to me and eats faint details. I tried Video Enhancer and Topaz Enhance that are both said to use super resolution but they are far worse even than my "method" (especially Topaz Enhance). Will there ever be a hero who'll make a plugin for TRUE video super resolution?

Terka
11th November 2008, 19:36
put to google IMARE superresolution
if you want, try matlab application (free for non-commercial)
http://zoi.utia.cas.cz/download
email to sroubekf@utia.cas.cz for download, but dont expect miracles.

Efenstor
11th November 2008, 20:34
I'll try and write back the results. Still I don't understand why Fizick won't try to implement real super-resolution using his motion tracking algorithms?

Nightshiver
11th November 2008, 21:58
Because imo, super-resolution is pointless.

Dark Shikari
11th November 2008, 22:06
I'm trying to upscale by 2, at least in general and want super resolution in particular. As far as I know there's nothing better than EEDI2 but it looks too blurry to me and eats faint details. I tried Video Enhancer and Topaz Enhance that are both said to use super resolution but they are far worse even than my "method" (especially Topaz Enhance). Will there ever be a hero who'll make a plugin for TRUE video super resolution?Try NNEDI + LimitedSharpenFaster.

g-force
11th November 2008, 23:56
Efenstor,

I think you are only going to find a lot of cynicism about "super-resolution", which is a misnomer at best, pie-in-the-sky at worst. The reason really lies in the question "what is super-resolution?" The answer is: "good upsizing with detail bolstering of moving detail from previous and subsequent frames". We already have both of those in the form of NNEDI, and MVTools, and when you put them together, nothing magical happens. It's just good upsizing practices. Nothing more. Definitely nothing to warrant the term "super-resolution".
Now these supposed "super-resolution" packages are just snake-oil. Show the potential customer some still frame, show them what it looks like when it is upsized poorly (point-resize), show them what it looks like upsized correctly, and voila, "super-resolution!"

-G

mikeytown2
12th November 2008, 08:12
Here's a good start
http://hdl.handle.net/2014/39112 (http://trs-new.jpl.nasa.gov/dspace/handle/2014/39112)
http://en.wikipedia.org/wiki/Richardson-Lucy_deconvolution
http://en.wikipedia.org/wiki/Spitzer_Space_Telescope

Efenstor
12th November 2008, 12:50
I don't think super resolution is really pointless, I'm using a program called Photo Acute Studio and it really improves resolution. In theory, suppose we have a single point visible in two adjacent frames: in the first frame it's between the pixels and thus it's smeared 50/50 between them, but in the second frame it falls approximately to the pixel center and so it's much sharper. Go further: in the first frame points surrounding that point are sharp but in the second are smeared. Why can we use sharp information from both frames? There are other, more AI-like approaches such as analyzing which pixel of the two smeared closer represents the original point brightness/color or equalizing both smeared pixels depending on their subpixel position so that we can approximately reveal the two original non-interpolated values. We can just think of pixel grid as of another kind of noise, basically it is noise when we look not at single frame but at several where the grid constantly moves over the identical picture details. It's not pointless, but rather interesting I think. And very complex to implement.

Fizick
12th November 2008, 19:15
Still I don't understand why Fizick won't try to implement real super-resolution using his motion tracking algorithms?

me tried :)

i even created supplement plugin to fill holes, and have some script, but i did not release it, and its development is paused.

Efenstor
12th November 2008, 20:24
me tried :)

i even created supplement plugin to fill holes, and have some script, but i did not release it, and its development is paused.

And how prospective is this technique you think? Still better than any sharpens, I think, especially when there are some twists added to the naked theory, such as intelligent auto-contrast (I just noticed in PhotoActue Studio that all those subpixel details actually become visible but very faint) or something else, even more intelligent.

Fizick
12th November 2008, 20:51
temporal sr is good for some types of source only. may be i will return

mikeytown2
12th November 2008, 20:54
To do this correctly, you need to take some hints from Hugin.


Remove lens and other distortions, so the layers can be lined up.
Create the layers and line them up.
...
Profit!


DeBarrel for #1.
MVTools and/or Unfurl for #2
http://en.wikipedia.org/wiki/Super-resolution #3 & #4


Speaking of that, would DeBarrel -> MVTools -> ReBarrel give getter results for all MVTools functions?

Edit: Found this, (Anti-Lamenessing Engine)
http://www.luxal.eu/resources/hdr/hdrprep/

Efenstor
12th November 2008, 21:28
ALE works only on photos but the technology is there. :) (also I can't even try it because avast! rings the bell about some trojan inside the setup!)

Pre-debarrelizing would give only marginal advantage, mostly on straight lines, since MVTools uses block-based motion compensation.

Fizick, I strongly encourage you to work on SR, it's the only technology that would make SD to close-to-HD possible. Of course it should work not like my pseudo-SR suggested in the beginning of this thread, it should use precise positioning of tracked blocks (with rotation) and then interpolating between pixels (covering the holes). And block sizes should be adaptive, I think. It may be also a good idea to pre-split image to contrast layers, process them separately (though not independently, because then it would affect tracking quality) and then combine back together (with overlap, of course): it will help fighting the edge artifacts (fringing). I did so in my program (Cleanerzoomer) and it really helped, though I used other algorithms there but there were also fringes around the contrast edges.

lesd
13th November 2008, 01:49
Take a look at Topaz enhance. It is an SR app, and it works pretty well. I have used it on SD to HD upres, and depending on the grain, it does a decent job.
SR is highly dependent on what type of noise is in the image, as well as how the images were sampled or downsampled.
On some footage I've tried, it did virtually nothing. On other I was able to get near HD quality from SD source materials.

Efenstor
13th November 2008, 11:03
I tried Topaz but it smears out details and brings no real improvement. Image becomes looking like oil-painted which is bad.

lesd
13th November 2008, 11:31
Like I said, depends on the source footage. I used it for an SD to HD upres of some missing film elements for a bluray for a major studio.
On some other stuff, it flops, for sure.
The question is, if topaz's method does not work on a clip, would any other method do any better, or not.

I tried Topaz but it smears out details and brings no real improvement. Image becomes looking like oil-painted which is bad.

2Bdecided
13th November 2008, 20:27
We've had this discussion before. (a search of this forum for super resolution will find several relevant threads). Super Resolution is real, not snake oil - but Didee explained (and I suspect Efenstor knows) why it's not relevant to normal resampling: the original must include aliasing. Then, and only then, is genuine super resolution possible.

NNEDI or EEDI2 and some smart motion compensated smoothing is a poor man's alternative. I use it and like it with 320x240 videos from my still camera - which do include a lot of aliasing.

(IMO it's a lot better than Topaz, and that's not because of the source - I've tried many!)

Cheers,
David.

Efenstor
13th November 2008, 21:58
And returning to our "poor man's" variant, I've made a comparison of the three methods:

http://www.efenstor.net/external/270_comparison.gif

The "nnedi+sharpen+mc" code (sorry for the repetition but there's a couple of changes):

SetMTMode(5)
AVISource("09.avi")
SetMTMode(2)
ConvertToYV12()

blkH = 16
blkV = 16
ovl = 4
srch = 3
dct=0
pel=2

bc = BilinearResize(Width()*2, Height()*2)
super = MVSuper(bc, pel=pel)
vec1 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-2)
vec2 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=-1)
vec3 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=1)
vec4 = MVAnalyse(super, blksize=blkH, blksizeV=blkV, overlap=ovl, search=srch, dct=dct, delta=2)

nnedi(field=0, dh=true)
TurnLeft()
nnedi(field=0, dh=true)
TurnRight()
Sharpen(1)

super = MVSuper(pel=pel)
c1 = MVCompensate(super, vec1)
c2 = MVCompensate(super, vec2)
c3 = MVCompensate(super, vec3)
c4 = MVCompensate(super, vec4)

t1 = Overlay(c1, opacity=.5)
t2 = Overlay(c2, opacity=.5)
t3 = Overlay(c3, opacity=.5)
t4 = Overlay(c4, opacity=.5)

f1 = Overlay(t1,t2, opacity=.5)
f2 = Overlay(t3,t4, opacity=.5)
Overlay(f1,f2, opacity=.5)


May be with a better kind of sharpen it can give even more?

*.mp4 guy
13th November 2008, 22:19
Post a clip from the bloody source if you want any suggestions. There are no one size fits all solutions, so no one can suggest much if they don't know what you are working with

Efenstor
14th November 2008, 09:12
Here it is :)
http://www.efenstor.net/external/09.avi

*.mp4 guy
14th November 2008, 13:29
=/ ffdshow can't decode whatever that is. Try lagarith. or if it needs to be small, try q1 xvid with the mpeg matrix and no qpel or aq.

kemuri-_9
14th November 2008, 13:56
running a fourCC analyzer returns it as LEAD
http://www.fourcc.org/codecs.php#LEAD

@Efenstor
most of the people here will not have this proprietary codec, encode it in something else.

Dark Shikari
14th November 2008, 14:26
=/ ffdshow can't decode whatever that is. Try lagarith.FFDshow can't decode that either :p

Efenstor
14th November 2008, 15:05
Ok, I've recompressed it to XviD: http://www.efenstor.net/external/09_xvid.avi. I still don't understand why do you need my clip, don't you have your own clips to try?

buletti
14th November 2008, 21:07
here's my humble approach on frame 313:
original upsized with lanczos4 (http://img372.imageshack.us/my.php?image=originaleq1.jpg)
denoised & upsized, no super resolution tho (http://img444.imageshack.us/my.php?image=filteredrj1.jpg)

Efenstor
14th November 2008, 21:33
Well, everyone can denoise :), how about something new in double sizing?
I thought, may be there's another, better way to deal with the compensated frames so that only differences (read, new details) can be subtracted, boosted and readded? This could bring such "fake SR" approach even closer to real super resolution. Gotta experiment with it.

cweb
23rd December 2008, 16:04
I think this approach would be interesting for doing semi-HD youtube uploads...

Efenstor
15th May 2009, 09:42
I'm probably late but only today I've discovered that MotionDSP have released their vReveal. And, being into resolution-enhancement software, nearly took out my credit card, but... still I decided to make some comparisons. And amazingly, my good old MSR method (described in this thread) gave me better results than their forensic "state-of-art" technique!

I've downscaled a 640x480 video to 576x288 (it's the maximum size vReveal allows for x2 upscaling) and processed it with the best settings vReveal has. And then with MSR (MCompensate+NNEDI with MDegrain2 preprocessing).

Here is a source frame:

http://www.efenstor.net/external/mvsr/0000.jpg

The result of MSR:

http://www.efenstor.net/external/mvsr/0000_mvsr.jpg

And the result of vReveal!

http://www.efenstor.net/external/mvsr/0000_vreveal.jpg

The rest of the footage looks the same. The only thing vReveal is good with is speed (it utilizes CUDA).

Chiba
15th May 2009, 11:19
vReveal isn't overfiltered ? take a look at the electric cable :O
I think MSR have good results, something like IIP (by didée) can have good results no ?

Efenstor
15th May 2009, 13:25
vReveal isn't overfiltered ? take a look at the electric cable :O
It isn't! That's what wondered me a lot too, why does it smear thin and faint details so much while should had been enhancing them?

About sharpening (IIP is a sharpener, right?), I don't think any sharpener in its pure form can give the same results as MSR, because MSR uses information from adjacent frames. I dunno, the idea is very simple: motion compensate 5 frames, oversharpen them and then blend together, but it really works. :D May be not that cool as those scientific tests (I mean all those science papers on motion-compensation that are abundant on the Internet) but still better than pure sharpening, no matter how elaborate its algorithm is.

And after all, MSR is based on edge-directed interpolation and sharpening (currently it uses NNEDI plus basic AviSynth sharpen), just try replacing them with newer, better interpolator and sharpener and the result should be even better (the speed can be too, because NNEDI is as slow as slug).

Efenstor
15th May 2009, 13:32
Updated the code in the first post to the version I currently use.

Chiba
15th May 2009, 17:43
Watch carefully the electric cable, it's aliased (raw), but MSR de-aliase it, and if you watch vReveal result, the cable is disapparing :x
Yes, IIP is a sharpener ;) but ok, i believe you and i'll test MSR :D

help_me!
23rd June 2009, 20:42
(This is my first post after 4 years? AWESOME!)

Efenstor, your script creates some nasty artifacts during the transition from one scene to another.

Please take a look at the following frames:

http://i41.tinypic.com/nlo26g.jpg
http://i41.tinypic.com/hrmyqf.jpg
http://i40.tinypic.com/2zhoz79.jpg
http://i42.tinypic.com/2zf1q1t.jpg

As you can see your script causes information from the fourth frame to be "leaked" into the second and third.



Here are the two frames from the source:

http://i41.tinypic.com/2ee8kmo.jpg
http://i40.tinypic.com/wrkyth.jpg


The function that causes this is the MSR. Do you know how to fix this? :thanks:

Mug Funky
24th June 2009, 08:56
i love this super resolution stuff. it's as fashionable as cold fusion was in the '80s.

the analogy about a pixel being sharper in 1 frame than it's adjacent friends only holds for an aliased signal (this has been mentioned already), not a real-life video that will have (hopefully) been correctly filtered from acquisition (via the OLPF layer before the imaging sensor) through to resizing and delivery to you. when you have a correctly filtered video, the aforementioned pixel is exactly the same sharpness in every frame, assuming no motion blur or other stuff.

therefore, all you can hope to do with super resolution is "smooth the stair-steps", without being able to actually resolve any finer detail. a wire in shot will not become thinner, just have a smoother edge. this is where the painterly effect tends to come in.

MfA
24th June 2009, 13:21
There's always a bit of aliasing in video, the point spread function of a pixel is generally not a sinc, which is what it would need to be to be free of aliasing. We wouldn't really like truly aliasing free video even if we figured out a way to physically capture it.

*.mp4 guy
24th June 2009, 20:24
Considering that essentially all video is lowpassed several times before you ever do anything with it (yes even "raw" dv) the gains are going to be marginal at best, with most "pro" sources, the lowpassing is overdone, to the extent that there is absolutely no data anywhere close to the nyquist frequency.

The only place where you could get good results from this sort of thing, is with really nice dv cameras, like a panasonic dvx100b or something similar. The lower quality cameras don't have high enough effective resolution, so at best you will just remove aliasing artifacts slighly better then you othwerwise could.

and this is all besides the implementation issues doing this with avisynth and mvtools.

Mug Funky
25th June 2009, 02:25
ooh, it just occured to me that the output of my telecine is vertically aliased (horizontally it's fine) due to the scanning spot being quite a lot smaller than the distance between 2 lines of video.

certainly with sharp footage it'll alias like crazy (think venetians, tweed jackets and car grilles).

i'm skeptical as to whether a temporal super-res will deliver better results than nnedi though, which essentially deals with aliased material. might be worth a play in any case (but my horizontal resolution will still be limited to 720).
so it's not like i'm gaining a HD telecine for free, and besides which at the processing speeds that mvtools will give, it is much faster and better to just use the up-to 6k scanner sitting not 2 metres away from the old telecine...

[edit]
@ MfA: that's true enough, but how much is there to gain in 8 bit land considering most sources have undergone a bit of compression before they hit avisynth? a perfect sinc will become equal to an approximate one, and the difference probably wont be enough to improve quality.

@ *.mp4 guy: most standard def content these days is originated in HD simply because it's sharper when downrezzed. this presents some interesting aliasing opportunities, as i'm not sure final cu*t pro has the best downscaler ever in it (it smacks of neutral bicubic to me). additionally, a lot of stuff is being shot on RED one cameras in 4k, then given a half res decode to 2k (aliased to buggery), and often downscaled using bilinear (if you're in a SCRATCH system playing out without rendering, you seem to get bilinear for some reason). again more aliasing opportunities.

Jenyok
15th November 2012, 08:30
Did some changes in function for best tune...
.

LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\MVTOOLS-V2_5_11_3\mvtools2.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\NNEDI3\nnedi3.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\NNEDI_20070920\nnedi.dll")



#
# http://forum.doom9.org/showthread.php?t=142704
# Author: Efenstor
#
function MSR(clip clp)
{
# Constants
# Change those constants to tune for the best result of interpolation

# MSuper
#
hpad = 8 # 4
vpad = 8
pel = 4 # 2
rfilter = 4 # 4
sharp = 2 # 2
isse = true

# MAnalyse
#
blks = hpad
blksV = vpad
overlap = blks / 2
overlapV = blksV / 2
search = 5
searchparam = 3
dct = 5

# MCompensate
#
tSAD = 10000
tCD1 = 400
tCD2 = 130

# Overlay
#
opacity = 0.5



brc = BilinearResize(clp, Width(clp) * 2, Height(clp) * 2)
# function MSuper(clip, int "hpad", int "vpad", int "pel", int "levels", bool "chroma", \
# int "sharp", int "rfilter", clip "pelclip", bool "isse", bool "planar")
#
super = MSuper(brc, \
hpad = hpad, \
vpad = vpad, \
pel = pel, \
levels = 0, \
chroma = true, \
sharp = sharp, \
rfilter = rfilter, \
isse = isse, \
planar = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec1 = MAnalyse(super, \
isb = true, \
delta = 2, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)


# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec2 = MAnalyse(super, \
isb = true, \
delta = 1, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec3 = MAnalyse(super, \
isb = false, \
delta = 1, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec4 = MAnalyse(super, \
isb = false, \
delta = 2, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# clp = nnedi(clp, field=0, dh=true)
# clp = TurnLeft(clp)
# clp = nnedi(clp, field=0, dh=true)
# clp = TurnRight(clp)
# clp = NNEDI3_rpow2(clp, rfactor=2, cshift="spline64resize", qual=2, nsize=2, nns=2)
clp = NNEDI3_rpow2(clp, rfactor=2, cshift="lanczos4resize", qual=2, nsize=2, nns=2)

clp = Sharpen(clp, 1)

# function MSuper(clip, int "hpad", int "vpad", int "pel", int "levels", bool "chroma", \
# int "sharp", int "rfilter", clip "pelclip", bool "isse", bool "planar")
#
super = MSuper(clp, \
hpad = hpad, \
vpad = vpad, \
pel = pel, \
levels = 0, \
chroma = true, \
sharp = sharp, \
rfilter = rfilter, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c1 = MCompensate(clp, \
super, \
vec1, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD , \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c2 = MCompensate(clp, \
super, \
vec2, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c3 = MCompensate(clp, \
super, \
vec3, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c4 = MCompensate(clp, \
super, \
vec4, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

t1 = Overlay(clp, c1, opacity=opacity)
t2 = Overlay(clp, c2, opacity=opacity)
t3 = Overlay(clp, c3, opacity=opacity)
t4 = Overlay(clp, c4, opacity=opacity)
f1 = Overlay(t1, t2, opacity=opacity)
f2 = Overlay(t3, t4, opacity=opacity)
last = Overlay(f1, f2, opacity=opacity)

return (last)
}

kolak
15th November 2012, 17:09
@ *.mp4 guy: most standard def content these days is originated in HD simply because it's sharper when downrezzed. this presents some interesting aliasing opportunities, as i'm not sure final cu*t pro has the best downscaler ever in it (it smacks of neutral bicubic to me). additionally, a lot of stuff is being shot on RED one cameras in 4k, then given a half res decode to 2k (aliased to buggery), and often downscaled using bilinear (if you're in a SCRATCH system playing out without rendering, you seem to get bilinear for some reason). again more aliasing opportunities.

I've seen loads of RED footage aliasing, because of the way how they were processed- same people have no clue what they are doing :)

lisztfr9
16th November 2012, 09:46
This is a very interesting thread, but i'm wondering if it's not finally an equivalent to temporal denoising ? T denoising will also manage to enhance contrast by keeping pixels if adjacent to others, and flushing more isolated... details are kept across frames. Maybe mix T denoising with with edge detection, or make temporal edge detection... my 2 cents :)

Jenyok
16th November 2012, 17:23
lisztfr9
.
Any suggestions and improvements ?

lisztfr9
16th November 2012, 23:53
@Jenyok

Maybe blend stronger the frames close to the current. 'cause of temporal drifting, a far frame on time axis holds less relevant information.


Edit :

The problem is to discriminate between what is noise and what isn't. But the sharpening stage will sharpen also noise, which will be blend later inside the other frames. So you can't escape the temporal denoising, more efficient than spatial denoising, combined with motion compensation.

bcn_246
23rd November 2012, 15:53
Quite impressed actually, maybe there is a way to use SVPflow instead of MVTools2?

Has anybody had a look at the Dee Mon Video Enhancer?. I've used it a few times and, while far from perfect, it has produced some pretty good results (on the highest quality settings).

Tempter57
26th November 2012, 11:07
@Jenyok

The problem is to discriminate between what is noise and what isn't. But the sharpening stage will sharpen also noise, which will be blend later inside the other frames. So you can't escape the temporal denoising, more efficient than spatial denoising, combined with motion compensation.

# http://forum.doom9.org/showthread.php?t=142704
# ==== plugins ===
#avstp.dll
#RGTools.dll
#nnedi3.dll
#AddGrainC.dll
#mvtools2mod.dll
#masktools2.dll
#dfttest.dll
#SmoothAdjust.dll
#splineresize.dll
#awarpsharp2.dll
#dither.dll
#dither.avsi
#flash3kyuu_deband.dll
#mt_xxpand_multi.avsi
#MSRmod.avs
#LindaSharp.avs
#O16mod.avsi
#HQDeringmod.avsi
#GrainFactoryLite_v1.2.avsi

function MSRmod(clip clp, int "W", int "H", float "thr", float "depth", float "strength", int "g1str", int "g2str", int "g3str", int "smode", int "radius", bool "aWarp4", bool "GF3", bool "lsb")
{
# Change those constants to tune for the best result of interpolation

# Resize
W = Default( W, 1280 )
H = Default( H, 720 )

# MSuper
hpad = 8 # 4
vpad = 8
pel = 2 # 1 or 4
rfilter = 4 # 2
sharp = 2 # 2

# MAnalyse
blks = 16
blksV = 8
overlap = blks / 2
overlapV = blksV / 2
search = 5
searchparam = 3
dct = 0

# MCompensate
thSAD = 321
thSCD1 = 400
thSCD2 = 120

# GradFun3
GF3 = Default( GF3, false )
smode = Default( smode, 0 ) # GradFun3 "smode" parameter
thr = Default( thr, 0.42 ) # GradFun3 "thr" parameter
radius = Default( radius, 16 ) # GradFun3 "radius" parameter

# AddGrain
g1str = Default( g1str, 8 )
g2str = Default( g2str, 6 )
g3str = Default( g3str, 3 )

# Overlay
opacity = 0.5

# Output_depth
lsb = Default( lsb, false )

# Sharpening
aWarp4 = Default( aWarp4, true )
depth = Default( depth, 2 )
strength = Default( strength, 0.85 )


brc = BilinearResize(clp, W, H)
psuper = MSuper(brc.removegrain(11), hpad=hpad, vpad=vpad, pel=pel, sharp=sharp, rfilter=rfilter)
vec1 = MAnalyse(psuper, truemotion=true, isb=true, delta=2, blksize=blks, blksizeV=blksV, overlap=overlap, overlapV=overlapV, search=search, searchparam=searchparam, lambda=1024, dct=dct)
vec2 = MAnalyse(psuper, truemotion=true, isb=true, delta=1, blksize=blks, blksizeV=blksV, overlap=overlap, overlapV=overlapV, search=search, searchparam=searchparam, lambda=1024, dct=dct)
vec3 = MAnalyse(psuper, truemotion=true, isb=false,delta=1, blksize=blks, blksizeV=blksV, overlap=overlap, overlapV=overlapV, search=search, searchparam=searchparam, lambda=1024, dct=dct)
vec4 = MAnalyse(psuper, truemotion=true, isb=false,delta=2, blksize=blks, blksizeV=blksV, overlap=overlap, overlapV=overlapV, search=search, searchparam=searchparam, lambda=1024, dct=dct)

nnedi3_rpow2(clp, rfactor=2, cshift="spline144resize", qual=2, nsize=0, nns=3, fwidth=W, fheight=H)

clp = (aWarp4 == true ) ? aWarp4(Spline36Resize(width*4, height*4, 0.375, 0.375), aSobel().aBlur(), depth=2, chroma=4) : LindaSharp(strength)
ssuper = MSuper(clp, hpad=hpad, vpad=vpad, pel=pel, sharp=sharp, rfilter=rfilter, levels=1)
c1 = MCompensate(clp, ssuper, vec1, thSAD=thSAD , thSCD1=thSCD1, thSCD2=thSCD2)
c2 = MCompensate(clp, ssuper, vec2, thSAD=thSAD , thSCD1=thSCD1, thSCD2=thSCD2)
c3 = MCompensate(clp, ssuper, vec3, thSAD=thSAD , thSCD1=thSCD1, thSCD2=thSCD2)
c4 = MCompensate(clp, ssuper, vec4, thSAD=thSAD , thSCD1=thSCD1, thSCD2=thSCD2)

t1 = Overlay(clp, c1, opacity=opacity)
t2 = Overlay(clp, c2, opacity=opacity)
t3 = Overlay(clp, c3, opacity=opacity)
t4 = Overlay(clp, c4, opacity=opacity)
f1 = Overlay(t1, t2, opacity=opacity)
f2 = Overlay(t3, t4, opacity=opacity)
Overlay(f1, f2, opacity=opacity)

U16() # 8 bit=> 16 bit
HQDeringmod(mrad=2, mthr=60, minp=1, sharp=1, Y=3, U=2, V=2, lsb_in=true, lsb=true)

Dither_resize16nr (W, H, kernel="spline", taps=32, noring=true)

# DEBANDING
(GF3 == true ) ? GradFun3(smode=smode, thr=thr, radius=radius, lsb_in=true, lsb=true): \
f3kdb(16, 64, 64, 64, 0, 0, dynamic_grain=true, dither_algo=3, input_mode=1, output_mode=1)

# AddGrain
GrainFL(g1str=g1str, g2str=g2str, g3str=g3str, g1size=1.3, g2size=1.1, g3size=0.9, g1soft=2, g2soft=2, g3soft=2, th1=24, th2=64, th3=96, th4=160, lsb=true, lsb_in=true)

Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
output = ( lsb == true ) ? Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=true, mode=7, ampn=1, ampo=0.6, output="YV16") : \
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)
return (output)
}

function LindaSharp (clip clp, float "strength")
{
strength = default(strength, 0.85)

and = clp.sbr()
Peter = and.removegrain(4,-1)
go = mt_makediff(and,Peter)
mt_lutxy(clp, go, "x y 128 - abs 1 2 / ^ 2 * "+string(strength)+" * y 128 - y 128 - abs 0.001 + / * +",U=2,V=2)
}

function sbr(clip o) {
rg11=o.removegrain(11)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}

GEfS
20th December 2012, 19:18
## script Motion Super Resolution
##
blah blah blah blah


Could you send me a package include all plugins I need for this? :goodpost:

Tempter57
21st December 2012, 08:15
Could you send me a package include all plugins I need for this? :goodpost:
Used this plugins (http://nmm-hd.org/doc/AviSynth2.5%E6%89%A9%E5%B1%95%E6%BB%A4%E9%95%9C) and a package dither 1.21.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559)

matfra
28th December 2012, 03:22
hi,
I just tried MSR on a 640/480 video. SetMTMode 2 run at 2FPS. I have a I7 950. Is this a slow filter or im doing something wrong ? Does MSRmode run faster ?

matfra
28th December 2012, 15:31
Hi Tempter57,
The website to download the plugins are all in chinesse. I get find where to download it. Can you pack this into a zip file and share it for us please.

bcn_246
4th January 2013, 04:24
hi,
I just tried MSR on a 640/480 video. SetMTMode 2 run at 2FPS. I have a I7 950. Is this a slow filter or im doing something wrong ? Does MSRmode run faster ?
I dont think so, I tried MSRmod with a 720p sample on my i7@4.2GHz and barely got 1FPS...