Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th September 2004, 15:03   #1  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
temporal & spatial super resolution experiments

I have a lot of low-resolution digicam videos flying around here.

they are mostly 160x120pixels and 15 fps

so I searched for ways improving them.

I found a very good solution for panning-only shots.

It kills all noise and enhances resolution a LOT:



this is the script, I've used:

Code:
loadplugin("D:\x\depan.dll")

avisource("Mvi_0073.avi")

# for comparision
i=lanczos4resize(480,360).converttoyv12()

# for rendering
j=pointresize(480,360).converttoyv12()

data=i.depanestimate(pixaspect=1.0,zoommax=1,improve=false,trust=0)
r=4
j.depaninterleave(data,pixaspect=1.0,prev=r,next=r,mirror=15)
temporalsoften(r,255,255)
selectevery(2*r+1,r)

stackhorizontal(last.subtitle("temporal super resolution"),i.subtitle("original (160x120)"))
but there are downsides:

- differential motion will completely mess up with that script
- if depan doesn't catch the motion, you'll get a super-motion-blurrer
- it is slow.
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.

Last edited by scharfis_brain; 4th September 2004 at 15:26.
scharfis_brain is offline   Reply With Quote
Old 4th September 2004, 16:03   #2  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
It is interesting, yes. I will try.
BTW, i almost finished coding more fast version of depanestimate.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 4th September 2004, 18:29   #3  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Perfect, this is just what I need for those digicam videos I posted about a while ago. I have a suggestion though: since we're talking about a 900% upsize, wouldn't it be a better idea to use somethinglike SharpResize instead of Lanczos?
Chainmax is offline   Reply With Quote
Old 4th September 2004, 19:50   #4  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Hmm, didn't work as well for me. Here's a comparison screenshot:



Also, here's the result of the script sh0dan recommended me to use on my thread:



and the script itself:
loadplugin("wherever\EDIUpsizer.dll")
loadplugin("wherever\MVTools.dll")
loadplugin("wherever\AddGrain.dll")
DirectShowSource(something)
converttoyv12()
selectevery(3,0)
mvback=mvanalyse(isb=true, fth=60, pel=2,sx=4, sy=4)
mvforw=mvanalyse(isb=false, fth=60, pel=2, sx=4, sy=4)
mvconvertfps(last, mvback, mvforw, fps=25.0, fbw=2, thSCD2=255, thSCD1=500)
EDIUpsizer(method=7, window=8)
addgrain(10,0,0)
EDIUpsizer(method=7, window=8)
addgrain(10,0,0)
lanczosresize(480,336)



Maybe the blocking has something to do with the fact that this is the very first frame of the clip?

Last edited by Chainmax; 4th September 2004 at 19:53.
Chainmax is offline   Reply With Quote
Old 4th September 2004, 20:41   #5  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
chainmax: the first & last frames aren't improved with my method, also I said:

- differential motion will completely mess up with that script
- if depan doesn't catch the motion, you'll get a super-motion-blurrer


so, it won't help much with your kind of clip.

it only helps with clips that
-are not that shaky (your is shaking heavily)
-have a very short shutter speed (your does)
-have no differential motion (yours has a huge amount of diff. motion)
-had been point-downscaled by the cam (yours is better scaled -> unfortunately not that suited for my algo....)

handling your clip with my method will result in an ultra-mess: 9 frames blended into another...

also:

wouldn't it be a better idea to use somethinglike SharpResize instead of Lanczos?

I only have used lanczos for the comparision.
my method internally uses pointresize as working clip
and lanczos as estimation clip.
It was the best combination


EDIT:

the result of the video, I used to build this script:
http://home.arcor.de/scharfis_brain/samples/SR1.avi


hint hint hint....

feed it with fieldseparated true interlaced video...

hint hint hint....
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.

Last edited by scharfis_brain; 4th September 2004 at 21:58.
scharfis_brain is offline   Reply With Quote
Old 4th September 2004, 23:04   #6  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Well, I didn't know what differential motion was . While we're at it, would you want to give my clip a go?
Chainmax is offline   Reply With Quote
Old 5th September 2004, 10:30   #7  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
the only thingI could think of improving your clip a little bit, would be using depan to denoise it

you can youe my superresolution-script, if you set f=1 and r=3 and temporalsoften(r,12,20) or any other values for 12 and 20 this should remove the blockyness....

but I cannot think of more improvement

maybe a scientist with fundamental mathematical and programming knowledge could help further with this....
(I am everytime astonished by the universitary papers showing extreme improvements but not publicating binaries for testing ourselves...)
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 5th September 2004, 14:46   #8  |  Link
JanBing
Registered User
 
Join Date: Mar 2004
Posts: 31
So, technically this should make it possible to take a standard PAL/NTSC clip, and blow it up to full HD-resolution, as long as it's an panning-only shot, right?
JanBing is offline   Reply With Quote
Old 5th September 2004, 14:50   #9  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
@Janbing: No.
a normal PAL or NTSC video is (beside the interlacing) soft scaled.
this means the moving shot has no flickering pixels (like my AVI-sample shows on its right) inside.
the pixel-flickering (due to nearest neighbor downscaling inside the cam) iscondition for using the super resolution script.

but your may use it for GMC-deinterlacing of interlaced video...
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 5th September 2004, 15:38   #10  |  Link
JanBing
Registered User
 
Join Date: Mar 2004
Posts: 31
what exactly is GMC? Apart from General Motors, of course...
JanBing is offline   Reply With Quote
Old 5th September 2004, 19:12   #11  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
GMC = Global Motion Compensation:

depan searches for the larges moving object within the video
then depan moves the prev. and next frames, so the object of the current frame is overlaid exactly.

'normal' motion compensation is not moving thewhole frame, it either moves blocks of defined size to synthetisize a new frame or it is warping the frame as if it would be as flexible as a rubber...
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 6th September 2004, 09:14   #12  |  Link
morsa
the dumbest
 
Join Date: Oct 2002
Location: Malvinas
Posts: 494
Amazing results!!!!!
Can't something "simillar" be made using MVTOOLS ???
morsa is offline   Reply With Quote
Old 6th September 2004, 09:39   #13  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Sure it can. Wanted to give it a go today evening. However, for the reasons scharfis_brain mentioned, for general content the result will be more about noise reduction and artefact removal, than about "resolution enhancing".
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 6th September 2004, 23:15   #14  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
morsa, the script that sh0dan cooked up for me uses MVTools among other stuff. If you have any digicam videos lying around, give it a go and report back.

Didée, will your MVTools-based filter be similar to this one then?
Chainmax is offline   Reply With Quote
Old 7th September 2004, 07:46   #15  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally posted by Chainmax
Didée, will your MVTools-based filter be similar to this one then?
Hah, that was an optimist speaking.

I ran into the same problem as before, I just had "forgotten" a little about it:

I don't get clean enough temporal interpolations from the MVTools, whatever ME and other parameters I set. Problems with revealing areas, problems with vector coherence - lambda>1500 misses on contradicting motion, lambda<2000 may give vector salad, sometimes, problems with ... lots of things.

Anyone with good parameter suggestions for mvanalyse, mvconvertfps and mvcompensate, aiming for "average natural content"?
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 7th September 2004, 13:54   #16  |  Link
morsa
the dumbest
 
Join Date: Oct 2002
Location: Malvinas
Posts: 494
@Didee

Yeah, I have the same problems all the time, that's why I posted this:

http://forum.doom9.org/showthread.php?s=&threadid=82015

This ugly mess gives me improved results (it is my one-track-mind way of doing an image segmentation)

vec_back = mvanalyse(last.VagueDenoiser(threshold=4,method=1,nsteps=6,chromaT=0.8,percent=100).convertToYUY2._2DCleanYUY2(0,6,3,3,0,4,4).sharpen(1.0,1.0)._2DCleanYUY2(0,6,3,3,0,4,4).convertToYV12, isb=true, pel=1)

vec_forw = mvanalyse(last.VagueDenoiser(threshold=4,method=1,nsteps=6,chromaT=0.8,percent=100).convertToYUY2._2DCleanYUY2(0,6,3,3,0,4,4).sharpen(1.0,1.0)._2DCleanYUY2(0,6,3,3,0,4,4).convertToYV12, isb=false, pel=1)

MVConvertFPS(last, vec_back, vec_forw, nb=1,fps=50.0, fbw=2,bvt=35000, thSCD1=50)

Last edited by morsa; 7th September 2004 at 15:42.
morsa is offline   Reply With Quote
Old 7th September 2004, 14:00   #17  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Didée, the script I posted uses mvanalyse and mvconvertfps, why don't you try those values?
Chainmax is offline   Reply With Quote
Old 7th September 2004, 16:14   #18  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Chainmax:
Amongst others, I tried them ...
BTW, those settings are a little obsolete. "fth" has no effect in latest version(s) (and so hasn't "sadt"). These were replaced by the "lambda" parameter (vector field coherence).

morsa:
Right method. The more artefacts the source has, the more important cleaning & enhancing major image features becomes, to give the motion engine a reasonable clean playground.

Perhaps that was the reason why I got so bad results yesterday - I was working with an overcompressed mpeg-2 source of a music concert (DVB ... )
I remember to have made some 50fps test clips of some chapters of LOTR-TTT some time ago, with quite nice results.

However, having to set up dedicated cleaning for each single source individually can't be the way eiter.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 7th September 2004, 16:23   #19  |  Link
morsa
the dumbest
 
Join Date: Oct 2002
Location: Malvinas
Posts: 494
I know....
It's just I'm a sucker with this scripting things...
Could you test this method and give an improved version?
May be some other filter does the job easier....
Anyway I hope manao read this and implements some kind of internal preprocessing (may be a bilateral filter with strong settings could do the job)

That thing I've posted above is giving me perfect results, not even a block!!!!!!!
morsa is offline   Reply With Quote
Old 7th September 2004, 16:52   #20  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
A little bird flew along and sang the song of "Manao is overbusy with other important things these days"

(And I'm sure we all will love ... it )
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.