14.999fps to DVD: which way to go?

[Chainmax]

Having experienced the goodness of recent block removal utilities, I decided to dust off my Gabriel Knight 1 CDs (I have two) and try to make a miniDVD out of the "making of". Problem is, the source is exactly 14.999fps, so I figure these are the options:

1) Alter framerate to 23.976fps, encode and then pulldown.

2) Alter framerate to 25fps.


I'd like to hear your opinions on which should be better and why. Also, in order to change the framerate, should I use interpolated/dupe frames or speed up the video?

[AI]

Original Progressive?
NTSC
abc=>aabbccInterleave(Last,Last)
AssumeFPS(60000, 1001,True)
ResampleAudio(48000)PAL
abc-\aabbc
abc-/abbccAssumeFPS(15,True)
DoubleWeave
SelectEvery(6,0,1,2,3,4)
ResampleAudio(48000)PAV v2
abc-\abbcc
abc-/abbcaInterleave(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,1),SelectEvery(3,2),DoubleWeave.SelectEvery(6,5))
AssumeFPS(25,True)
ResampleAudio(48000)

[scharfis_brain]

if you want to force progressive encoding without introducing an unsmooth pattern of duplicated frames for PAL, you may speedup the 14.999fps to 16.667 fps (16 2/3) and then pulldown to 25fps.

16.6667 fps ist the lowest framerate achievable with PAL and pulldown.

[foxyshadis]

Motion-interpolation! :p Okay, with a cinepak source you may as well pray for sierra to send you their master copy, but at least it might help. (The GK videos have been on my "list of stuff to process later" for a while now.)

[Chainmax]

Everyone: thanks for the advices :).

AI: yes, the original is progressive.


if you want to force progressive encoding without introducing an unsmooth pattern of duplicated frames for PAL, you may speedup the 14.999fps to 16.667 fps (16 2/3) and then pulldown to 25fps.

16.6667 fps ist the lowest framerate achievable with PAL and pulldown.
I read once that any other pulldown than 23.976-->29.97 using DGPulldown will cause some jittering (wobbling, right?). Does that not apply to 14.999--->16.667? If not, how would you go about speeding up to 16.667?


foxyshadis: it's not Cinepak, it's Microsoft Video 1 and it's so crappy that I couldn't use AVISource right away since the decoder couldn't handle YUV or RGB. I ended up loading the video itself, making an uncompressed RGB encode and loading that into AVISource.


Everyone, this is the current filterchain:
Levels(0,1,245,16,235)
ConvertToYV12()
FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
hqdn3d(0,0,3,3)
AddBorders(0,4,0,4)
Deblock_QED(quant1=30,quant2=51)
Crop(0,4,160,120,align=true)
GaussResize(192,144)
dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,LMode=3,Strength=1000,wide=true,soft=75)
Soothe(sharp,dull,30)
EEDI2().TurnRight().EEDI2().TurnLeft()
FFT3DFilter(sigma=1,bw=32,bh=32,bt=3,ow=16,oh=16,sharpen=0.7)
GaussResize(3xx,2xx,p=100) <--- depends on wether I go PAL or NTSC
Tweak(sat=1.2)
AddGrain(10,0,0)
AddBorders(16,16,16,16)
ConvertToYUY2() <--- for CCE

(by the way foxyshadis, we should include a check within BlockTerminator that detects when source width and height are not MOD16 so that it adds suitable borders before deblocking and removes them after, like I did here)

and this is a comparison with both source and result blown up to 640x480 for a better view:
http://img82.imageshack.us/img82/8593/ba7oh.png (http://imageshack.us)

Any kind of feedback on how to make it look better is welcome.

[AI]

Chainmax
What you think about my pulldown with out chenge speed?

Note: "a","b","c" is fields as manual built-in pulldoun.

[foxyshadis]

Oh, I knew it was one of those codecs that made me install the stupid "legacy codec pack", that turn every 2x2 pixel block into a single huge pixel. Since I dealt with some similar cinepak videos a few months ago I was going to show you what I did, which was similar to an immature BlockAhnold+immature mcnr_simple, but that already looks better anyway. Dang, I guess I need to redo Jesse Cook now. EEDI2 is so effective at repairing those old junkers!

[IanB]

16.6667 fps is the lowest framerate achievable with PAL and pulldown.Why does it have to be a 3:3 pulldown from an 11% speed up. What is wrong with a 4:3:3 pulldown pattern from 15 fps. The 5Hz jitter won't be detectable over the 15hz judder.

[scharfis_brain]

because then interlace encoding is required.
with an 3:3 pulldown you can stick with progressive encoding.

[IanB]

Ahah, I see your point, if you do it in AviSynth, yes.

But if you use DGPulldown this is not a problem.

So I guess I would have to recomend PAL....
AssumeFPS(15, True)
ResampleAudio(48000)Encode as 15FPS progressive
Use DGPulldown to set to 25 FPS

or NTSC....
AssumeFPS(15000, 1001, True) # 14.985
ResampleAudio(48000)Encode as 14.985FPS progressive
Use DGPulldown to set to 29.97FPS

[scharfis_brain]

Ianb: you didn't read my previous posts carefully enough.

1) I am not suggesting to do anything in AVISynth
2) DGPulldown only allows 3:3 pulldowns at max.
this means:
with PAL (25fps native) you can only go down to 16 2/3 fps. not lower
with NTSC (29.97fps naive) you can only go down to 19.98 fps

this effectively means: for PAL one needs to speedup 15fps to 16.667 fps in order to be ABLE to do the 3:3 Pulldown.

3) framerates like 15 or 14.985 aren't supported by mpeg2.
you need to abuse (speed change) one of the valid framerates like
23.976, 24, or 25.
then use dgpulldown and the desired ratio of 16.6667 to 25.0 .
dgpulldown will overwrite the previously assumed framerate of the encoded mpeg2.

[Chainmax]

foxyshadis: 2x2 pixel block? Is that a real "feature" of those codecs or just a sarcastic coment on the crapyness of these old codecs? By the way, the thing that impresses me the most about this filterchain is that the source doesn't even look like real life footage but hand drawn since the textures (or whatever they are called) are so screwed up, and even though the result is much too blurry, at least it looks like live footage again :).

AI: your methods sound great, but I'm waiting for more opinions on wether I should do everything in Avisynth or use DGPulldown at some point.

scharfis_brain: why does 15fps---25fps needs interlace encoding?

[scharfis_brain]

25p -> 25 points on the time axis per second possible
25i -> 50 point on the time axis per second possible

this brings us to the following patterns (Large Letter: Top field on TV, small letter: Bottom Field on TV):

15p to 25p: AaAaBbBbCcDdDdEeEeFf, which is an 2:2:1 pattern. This means it has a VERY noticeable jerkyness

15p to 25i: AaAbBbCcCcDdDeEeFfFf, which is an 3:3:4 pattern, which only has a slight mostly unnoticeable remaining jerkyness.

My method applies the speedup, to 16,667 fps and thus realizes a 3:3:3 pattern, which is completely jerk-free (exept of the 15fps stutter) and allows progressive encoding!
So it is the best of both methods: progressive encoding AND jerkyness suppression.

Chainmax, can you provide a sample video?
I am just curious, how motion compensated temporal super resolution approaches might help here...

[IanB]

@schafis,

Yes you are correct. I should learn to check these things before I open my big mouth.

So the question remains how do you Force/Trick/Deceive whatever into doing a 4:3:3 pulldown pattern of a progressive mpeg2 encode without an 11% speedup.

i.e. I want AI's pattern with a progessive encode
abc-\aabbc
abc-/abbcc

[scharfis_brain]

it is possible, but it could be VERY hard to set up!

1st step: creat a progressive mpeg file with this pattern:
AaBbCcCcDdEeFfFf (2:2:4 Pattern)

then Pulldown this MPEG to
AaAbBbCcCcDdDeEeFfFf (3:3:4 Pattern)

but it is quite difficult, because the frames containing Aa, Bb, Dd, and Ee needs to become pulldown flagged, while the already doubled Frames containing Cc and Ff must remain untouched.

I have no clue, how to do this with dgpulldown!

maybe a completely new pulldown application could be useful here. I think about an application that adaptively insert Null-P-frames into the video stream, where more than 3 is needed in the pulldown pattern...

[Chainmax]

Here's (http://rapidshare.de/files/16946446/Sample.avi.html) a sample.

[scharfis_brain]

processed file: http://rapidshare.de/files/16951339/sample-25p.avi.html

there are still some blotchy artifacts, for whose I was too lazy to compensate them.
It is just a test of my approach.
also the video has been interpolated up to 25 fps. So no need for any pulldown...

[Chainmax]

The motion seems quite fluid for me, but I'd like to see what others think about it. Is this much different than using 16.667 speedup+pulldown? About the filtering, it doesn't convince me: it's far too smooth and there is some kind of weird residual motion on the backgrounds.

[scharfis_brain]

made another sample: http://rapidshare.de/files/16986312/sample-25p1.avi.html

btw.: please show me the results of your approach as a
video from the same snipplet I got from you.
IMHO the result of your approach according to the image
above has too much staistepping left and thus is only
virtually sharper. stairstepping is a kind of sharpness,
but not one that pleases my eyes.
That's why I prefer some kind of smoothness in that case.

my 2nd sample is a little bit sharper and has more stable backgrounds.
http://home.arcor.de/scharfis_brain/samples/schock.png

of course the approach is also very different from the pulldown
approach, because no pulldown is needed anymore.
It is true 25fps now (upwards interpolated the framrate from 14.999 to 25fps).


EDIT: my script just uses two external plugins: mvtools and eedi2.
the rest is a 3x redundant chain of pointresize, temporalsoften,
eedi2 and mvflow calls to merge 5 frames temporally while
effectively raising resolution and image detail.

[Mug Funky]

@ scharfi:

that's an amazing result right there.

btw, any reason not to bump fps to 30000/1001 and encode progressive? there's redundant frames but that's the least of this sample's problems :)

[Chainmax]

That looks freaking awesome, although I'd use something like AddGrain(2,0,0) at the end to give an illusion of detail. The reason there's stairstepping is probaby because of the p=100 switch I used in GaussResize. I'll encode the sample with my filterchain to ffdshow's huffyuv YV12 and upload it for you soon.

[Chainmax]

You can download my encode here ( http://rapidshare.de/files/17113602/Sample.avi.html). I think we should stick to 352x240 by the way, 704x576 amplifies the errors too much.


[edit]Could you try your filterchain on this (http://www.31012.com/~azulftp/MOV01825.MPG)? I'd like to see what results it yields.

[scharfis_brain]

You should decide.

either 352x240@29.97 fps
or 352x288@25fps

but since the source is able to give the amount of informataion, I'd stick with 640x480 in a letterboxed PAL or in Full NTSC format.

this means: 584x480 with added borders into 704x576

btw.: I know your second clip. It is pretty old right now ;)
1) it doesn't seem to violate the nquist theorem in spatial domian (no staistepping)
2) it has lots of shaky and differential (concurring) movements in it
3) it is block like hell

these conditions make it hard to upsample.

the other clip (the one which is this thread about) is os 'easy' to upsample, because it violates the nyquist theoreme (it shows HEAVY aliasing!): http://en.wikipedia.org/wiki/Super-resolution

[Chainmax]

I see. Please upload an encode made with your filterchain at 704x576 so that I can see how it looks in motion.

[scharfis_brain]

here http://forum.doom9.org/showpost.php?p=808013&postcount=19 I already posted a link to a processed 640x480@25fps video.

just scale that to:
-either 704x576 and recompress as progressive PAL MPEG2
- or 584x480 and addborders (60,48,60,48) afterwards an compress to progressive PAL MPEG2 as well

I'll post my script later.

[Chainmax]

Oh, I didn't see it. It looks pretty good, but I'd still use AddGrain(10,0,0) as a final filter because it looks too "empty", plus the grain should help concealing the slight remaining background instability.
By the way, I've been thinking: since you chose to use interpolation to the final framerate, wouldn't it be better to interpolate to 23.976fps? That way we can use NTSC FILM framerate and framesize for encoding, which would mean a higher bits/pixel*frame ratio and therefore a better overall picture, right?

[scharfis_brain]

By the way, I've been thinking: since you chose to use interpolation to the final framerate, wouldn't it be better to interpolate to 23.976fps? That way we can use NTSC FILM framerate and framesize for encoding, which would mean a higher bits/pixel*frame ratio and therefore a better overall picture, right?
No. The best would be an interpolation to 29.97 fps, because then every even frame is an original and every odd frame is an interpolated image.
Also 23.976 is - at least on TV Sets - much more jerky than 29.97fps. And only every 5th frame is an orignial frame. the other four are interpolated ones.

I won't mind about compressibility and compression artifacts here, because the video itself has not much complicated information in it (except if you are adding grain).
But anyways with 4mbps at 704x480@29.97fps progressive you should end with a very decent compression result.

[Chainmax]

29.97fps it is then.

[Chainmax]

By the way, here's your result + FFT3DFilter on chroma + Tweak + AddGrain:

http://img214.imageshack.us/img214/4618/schfilt0pi.png (http://imageshack.us)

[scharfis_brain]

I'll cleanup my script and post it during the evening.

[2Bdecided]

Any chance of seeing that script please scharfis_brain?

I have some video footage shot on a digital still camera that might respond well to this.

Cheers,
David.

[scharfis_brain]

Sorry. Totally forgot that:

f=4

avisource("Sample1.avi")
a=last
a.converttoyuy2()
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)
vf=last.mvanalyse(isb=false,blksize=16, truemotion=true)
vb=last.mvanalyse(isb=true,blksize=16, truemotion=true)
vf1=last.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=last.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=last.mvflow(vf1)
xf=last.mvflow(vf)
xb=last.mvflow(vb)
xb1=last.mvflow(vb1)

interleave(xf1,xf,last,xb,xb1)
temporalsoften(2,40,50,mode=2)
x=selectevery(5,2)

x.converttorgb()
xa=pointresize(a.width*2,a.height*2).converttoyuy2().eedi2().turnleft().eedi2().turnright()
xb=crop(1,1,0,0).addborders(0,0,1,1).converttoyuy2().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()
x=merge(xa,xb)


xvf=x.mvanalyse(isb=false,blksize=16, truemotion=true)
xvb=x.mvanalyse(isb=true,blksize=16, truemotion=true)
xvf1=x.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=x.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=x.mvflow(xvf1)
xxf=x.mvflow(xvf)
xxb=X.mvflow(xvb)
xxb1=x.mvflow(xvb1)

interleave(xf1,xxf,x,xxb,xxb1)
temporalsoften(2,255,255,mode=2)
selectevery(5,2)




#last.mvflowfps(vb,vf,num=25,den=1)
#bicubicresize(704,576)

[Chainmax]

So, it needs MVTools (is v1.1.1 recent enough?) and EEDI2, right?

[scharfis_brain]

yeah.

[Chainmax]

It has MVFlowFPS set to 25 but the line is commented out, would the final result be 29.97fps? If not, what should I change to make it so?

[scharfis_brain]

uncomment it and set:
num=30000, den=1001

and then resize to 704x480

afterwards you may add grain etc.

[2Bdecided]

Quite a different task from the problem originally posed, but a useful result so I'll report it here...

I tried this script on the file I'm currently working with: a 30fps 320x240 M-JPEG from a 6M-pixel digital still camera. The original is highly aliased (the 320x240 seems to be just spot sampled from the 6M sensor with no aliasing or filtering) and then lots of mosquito noise is added on top.

The results from the script were near miraculous. The amount of detail recovered from the (quite poor) original was great. Not always perfect, but so much better than just bicubic resizing the original! I resampled down to 768x576 at the end to watch on the PC, and didn't change the frame rate.

There were some "interesting" artefacts - fingers or even hands turning into alien like appendages when they waved or moved - but looking at the original clip, it seems there just isn't enough information at these moment to do anything better. I wonder if a fall-back mode which avoided this with fast motion of small objects (i.e. ones where there was nothing to be recovered from this technique) would be possible? But it may not help much.

However, over all I am very impressed with the result, and will be using this.

I might tack a frame rate change to 50fps on the end and interlace for a DVD, and see what happens. I wonder if there's any way of doing the frame rate conversion within the original process to save time and quality? I'll try and get my head around the script.

Only one (obvious) request: what can I do for a speed up? Parsing through AVS2AVI (no recompression) managed 0.05fps. Going straight to TMPG was even worse, and both started eating memory - I think TMPG even hit virtual memory at some point, which may explain the speed problem. Any suggestions?

Many thanks for sharing this scharfis_brain (and everyone who wrote the functions within) - it's really appreciated!

Cheers,
David.

[Chainmax]

Great, thanks :).

[scharfis_brain]

THis script is just a proof of concept. I WILL fail terribly with lots of concurring motion of small objects in it.

It is working like a motion compensated denoiser. But it is merging everything together without any regard to possible artifacts.

It can be improved for sure but I dunno how.
Maybe Didee can tell something about it.

[Chainmax]

I get an error at line 19:

vf=last.mvanalyse(isb=false,blksize=16, truemotion=true)

saying it needs YV12 data. I suspect it comes from line 16:

a.converttoyuy2()

Can I change that to ConvertToYV12() or will it have a negative effect on the script's functionality?

[Chainmax]

Changing all converttoyuy2() calls to converttoyv12() made it work for me. Hopefully I didn't break anything by doing so :).

[Chainmax]

After checking on different spots in the source, I can confirm that this SuperResolution script works only on sources with very little movement, like a person talking. When there is motion, it suffers from the "oil painting" effect. Here's a comparison:

Source + Resize + Borders:
http://img138.imageshack.us/img138/9015/origresizeborders2zz.png (http://imageshack.us)

SuperResolution:
http://img231.imageshack.us/img231/8879/schsr1kb.png (http://imageshack.us)

BlockTerminator:
http://img138.imageshack.us/img138/4860/blockterminator9eg.png (http://imageshack.us)

Now, the BlockTerminator script doesn't have framerate conversion, and all the times I used something other than assume/changefps for framerate conversion I also got oil paiting/motion issues, so I'll try to include the fps conversion from the SuperResolution script (only mvflowfps(num=30000,den=1001), right?) and post the corresponding image.

[scharfis_brain]

something is going wrong there.

please post that scene.

[Chainmax]

Here are two scenes that show the problem:

http://rapidshare.de/files/17857860/Problem_Scenes.avi.html

[Chainmax]

Were you able to download it?

[Chainmax]

Well?

[Chainmax]

bUmP.

[2Bdecided]

I can't make rapid share work, so can't access you file - but either you're very unlucky with that clip, or something is very wrong.

On pans, for example, it's working fine for me.

The only weird effects are where things move against a background; the mo comp fails, and the averaging gives very strange results.

If there's lots of random movement in your clip, I suppose it would look like the stills you posted, even if the software was working "correctly".

Sorry I can't be any more help, but I thought I'd offer this information since no one else stepped in!

Cheers,
David.

[scharfis_brain]

Sorry Chainmax, I was away for 10 days.
I'll be away next week, too.

I won't have time to look at your samples until next week.

[Chainmax]

No problem at all. After all you can't rush a masterpiece :). By the way, YGPM.

[Chainmax]

What about going from 24fps to 25fps? Should I use one of these methods or just speed up the video?

[Mug Funky]

if you're already interpolating up to 24 from 14.999, then there'd be no problem just going to 25 instead of 24. but speeding up is better otherwise.

[Chainmax]

Oh, I'm now asking about a different clip that is 24fps and I want to convert to 25fps.

[Chainmax]

I'd better make a separate thread for this question.

[Chainmax]

When turning a progressive PAL video to NTSC, is using mvflowfps(num=30000,den=1001) a good idea or are there better solutions?

[scharfis_brain]

to your former problem: add thSCD1=9999 to all mvflow calls. that should help the superresolution.

in order to convert 25p to NTSC do a resize to NTSC resolution and encode as 720x480 with 25fps progressive MPEg2. then let DGPulldown do the rest.

[Chainmax]

I'll try your suggestions and post back a comparison for that scene soon, thanks :).
Are you sure that DGPulldown is the best bet? I remember reading comments that its 25-29.97 pulldown yielded more judder than usual.

[IanB]

@Chainmax,

What do you call "more judder than usual".

More judder than a field blended approach, yes it will, but then you have blended fields, which may or may not be acceptable for you clip.

More judder than manually doing the field replication in Avisynth, highly unlikely, the resultant field pattern is pretty much going to be the same. And you would have to encode as interlaced and waste a lot of bits in the process.

More judder than slowing down 4% to 23.976 fps and doing a more orthodox 3:2 pulldown, possibly. You are comparing a 3:2 pattern with a 3:2:3:2:2 pattern. The judder will have a lower frequency. Is that more judder?

[Chainmax]

I meant more judder than a standard 3:2 pulldown and basically wanting to know if there were ways to avoid that. I'd have to consider which is the lesser of two evils: judder or slowdown. What do you think?

[scharfis_brain]

I'd have to consider which is the lesser of two evils: judder or slowdown. What do you think?That depends on your taste.

[IanB]

I meant more judder than a standard 3:2 pulldown and basically wanting to know if there were ways to avoid that. I'd have to consider which is the lesser of two evils: judder or slowdown. What do you think?As Scharfis say it's a lot about personal preference and content.

E.g. A talking head won't matter what you do. A clip that is horizontally panning could be really annoying.

If it is really critical you can try dabbling with the world of motion vectors.

[Chainmax]

Here is the same frame with the new SuperResolution script:

http://img161.imageshack.us/img161/2415/superresolutiontry2sourceframe.png (http://imageshack.us)


which is this:

f=4

AVISource("X:\wherever\GKUncompressedRGB.avi",audio=false)
a=last
a.converttoyv12()
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)
vf=last.mvanalyse(isb=false,blksize=16, truemotion=true)
vb=last.mvanalyse(isb=true,blksize=16, truemotion=true)
vf1=last.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=last.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=last.mvflow(vf1,thSCD1=9999)
xf=last.mvflow(vf,thSCD1=9999)
xb=last.mvflow(vb,thSCD1=9999)
xb1=last.mvflow(vb1,thSCD1=9999)

interleave(xf1,xf,last,xb,xb1)
temporalsoften(2,40,50,mode=2)
x=selectevery(5,2)

x.converttorgb()
xa=pointresize(a.width*2,a.height*2).converttoyv12().eedi2().turnleft().eedi2().turnright()
xb=crop(1,1,0,0).addborders(0,0,1,1).converttoyv12().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()
x=merge(xa,xb)


xvf=x.mvanalyse(isb=false,blksize=16, truemotion=true)
xvb=x.mvanalyse(isb=true,blksize=16, truemotion=true)
xvf1=x.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=x.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=x.mvflow(xvf1,thSCD1=9999)
xxf=x.mvflow(xvf,thSCD1=9999)
xxb=X.mvflow(xvb,thSCD1=9999)
xxb1=x.mvflow(xvb1,thSCD1=9999)

interleave(xf1,xxf,x,xxb,xxb1)
temporalsoften(2,255,255,mode=2)
selectevery(5,2)




last.mvflowfps(vb,vf,num=30000,den=1001)
BicubicResize(656,448)

Levels(0,1,245,16,235)

FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)

Tweak(sat=1.2)

c=last

GrainU=c.UtoY().AddGrain(4,0,0)
GrainV=c.VtoY().AddGrain(4,0,0)
Return ytouv(GrainU,GrainV).MergeLuma(c).AddGrain(10,0,0).AddBorders(24,16,24,16)

I tried to compare to the BlockTerminator script adding mvflowfps(num=30000,den=1001) right before its final resize but I kept getting an access violation error. I assume that the vf and vb switches have to be set but I have no idea how to do so.

[satmonk]

Hi, does anybody still have the video "sample-25p1.avi"? It's the video posted by Scharfis Brain, but it is no more on rapidshare :-(

[Chainmax]

This is an old thread, so the RapidShare probably just expired. Being that the clip was meant to illustrate the effects of the function it's very likely that it's long gone from scharfis_brain's HDD. In any case, I think PMing him will be a better way to know for sure. Welcome to the Doom9 forums, by the way :).

[satmonk]

Thanks for your quick answer.
He told me to ask in this forum, because he do not have this video anymore.

I am sitting at your superresolution script, but can't get it to work (error at line13, unrecognised exception)

The script starts with:

f=4
LoadPlugin("eedi2.dll")
LoadPlugin("mvtools.dll")
LoadPlugin("fft3dfilter.dll")
avisource("klkl.avi")
a=last

all after is the same like your last script

System:
MVTools v.1.6.2
VirtualDUB 1.5.10 build18160
AVISynth 2.5

[Chainmax]

Unfortunately, I deleted that video many months ago. By the way,this function was created by scharfis_brain, not me :).

[satmonk]

Yes, that's right, sorry, was my mistake
But probably you can help me out (you got this script running)
Pleeease.

edit: I put in "mvanalyse()" (after pointresize)
now program says that I need YUY2 or YV12 File. I have an DV-Camcorder file, I thought this is YUY2?

[Chainmax]

If I'm not mistaken, DV is YV12, so you have to either put ConvertToYUY2(interlaced=true) right after loading the file or ConvertToYUY2() after deinterlacing.
As for the rest, I know nothing about unrecognized exception errors, but empyrical evidence tells me that those can usually be solved by reordering the filters (for example, doing SuperResolution-->Filtering instead of Filtering-->SuperResolution), putting SetMemoryMax(x) (with x=your ram/2) at the very beginning of the script (before loading the filters and the source) or fiddling with resizes (some filters need MOD16 resolutions). I hope hat helps.

[2Bdecided]

I think this script (if we're still talking about the same script) is inappropriate for most DV/Camcorder output.

It's most useful for resizing content with horrible aliasing, and even then it's not really usable in its present form, except on very "easy" content.

I could be wrong, but as a new user satmonk maybe you've dropped onto the wrong script. What is it you're trying to do?

Cheers,
David.

[2Bdecided]

To drag this back up from the dead...

This script crashed for me with the latest version of mvtools, but it just needed pel=1 adding to all the mvanalyse lines to make it work - the (new-ish) default pel=2 made it crash on my machine.

However, it still crashes with larger source videos. The error is always just "unrecognised exception" (from within virtualdub). The larger the video, the earlier the line on which the exception ocurrs. It's always an mvtools line (mvanalyse or mvflow).

When working, the process uses about 1GB of memory. When it's not working, it doesn't grab any more than VirtualDub without a video file. It makes no attempt to hit virtual memory.

Cheers,
David.

[Chainmax]

Now that NNEDI and SalFPS3 have been out for a while, it would be nice to see an updated, optimized version of this converted into an AVS function :).

[Leak]

When working, the process uses about 1GB of memory. When it's not working, it doesn't grab any more than VirtualDub without a video file. It makes no attempt to hit virtual memory.
Well, if it grabs all of that memory in one piece it'll probably just bomb out when it tries to allocate >2GB of RAM (that's the usual address space limit for Win32 applications) because the memory allocation call itself will fail and thus not allocate any memory.

np: Savath & Savalas - El Solitario (Golden Pollen)

[Fizick]

mvtools scripts must use idx to decrease memory eating

[2Bdecided]

Fizick,

I've added idx=1 to the first pass and idx=2 to the second pass, and now it works fine with SD video (720x576). Thank you for your help Fizick.


Leak,

It looks like AVIsynth 2.57 would only grab 1GB on my system...
http://avisynth.org/mediawiki/Internal_functions/Control_functions (setmemorymax)
...but later version might try to grab more. Looks like I should explicitly setmemorymax in future. Thanks for the advice.

Cheers,
David.

[Leak]

It looks like AVIsynth 2.57 would only grab 1GB on my system...
http://avisynth.org/mediawiki/Internal_functions/Control_functions (setmemorymax)
...but later version might try to grab more. Looks like I should explicitly setmemorymax in future.
Errr... SetMemoryMax only tries to control the amount of memory used by AviSynth for the frames it creates, which is mostly done by limiting the size of the frame caches automatically placed between the filters you use - it can't (and won't) prevent any filter from allocating all the memory it wants.

[2Bdecided]

Thanks for clearing up my misunderstanding!

Cheers,
David.

[ingoldie]

When I try to use the following script, I receive the error message (error at line28, Unrecognized exception!)

Here is the source clip:
http://www.mediafire.com/file/jvb9vvv5cq9di1i/Sample1_SD_8mm_18_fps.avi

What is the problem?



f=4

avisource("Sample1.avi")
a=last
a.converttoyuy2()
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)
vf=last.mvanalyse(isb=false,blksize=16, truemotion=true)
vb=last.mvanalyse(isb=true,blksize=16, truemotion=true)
vf1=last.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=last.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=last.mvflow(vf1)
xf=last.mvflow(vf)
xb=last.mvflow(vb)
xb1=last.mvflow(vb1)

interleave(xf1,xf,last,xb,xb1)
temporalsoften(2,40,50,mode=2)
x=selectevery(5,2)

x.converttorgb()
xa=pointresize(a.width*2,a.height*2).converttoyuy2().eedi2().turnleft().eedi2().turnright()
xb=crop(1,1,0,0).addborders(0,0,1,1).converttoyuy2().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()
x=merge(xa,xb)


xvf=x.mvanalyse(isb=false,blksize=16, truemotion=true)
xvb=x.mvanalyse(isb=true,blksize=16, truemotion=true)
xvf1=x.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=x.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=x.mvflow(xvf1)
xxf=x.mvflow(xvf)
xxb=X.mvflow(xvb)
xxb1=x.mvflow(xvb1)

interleave(xf1,xxf,x,xxb,xxb1)
temporalsoften(2,255,255,mode=2)
selectevery(5,2)




#last.mvflowfps(vb,vf,num=25,den=1)
#bicubicresize(704,576)

[StainlessS]

What is the problem? buggy plugin.

ingoldie, not sure I got this correct, but seems to work reasonably well.
(Converted to mvtools 2)


f=4

avisource("D:\v\Starwars.avi").Trim(10000,-1000) # 720x576@25
Org=Last # Might want to comment out
BilinearResize(Width/(4*f)*4,Height/(4*f)*4) # these 3 lines
Return PointResize(Org.Width,Org.Height) # to here (make test clip)

a=last
a.converttoyuy2() # Last is YUY2
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)

srcP=Interleaved2Planar() # Last is Planar YUY2

super=srcP.MSuper(Planar=True)

vf =super.manalyse(isb=false,blksize=16, truemotion=true)
vb =super.manalyse(isb=true,blksize=16, truemotion=true)
vf1=super.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=super.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=srcP.mflow(Super,vf1,Planar=true)
xf =srcP.mflow(Super,vf,Planar=true)
xb =srcP.mflow(Super,vb,Planar=true)
xb1=srcP.mflow(Super,vb1,Planar=true)

interleave(xf1,xf,srcP,xb,xb1)
srcI=Planar2Interleaved()
srcI.temporalsoften(2,40,50,mode=2)
xI=selectevery(5,2)


xRGB=xI.converttorgb()
xaI=xRGB.pointresize(a.width*2,a.height*2).converttoyuy2().eedi2().turnleft().eedi2().turnright()

xbI=xRGB.crop(1,1,0,0).addborders(0,0,1,1).converttoyuy2().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()

xI=merge(xaI,xbI)

xP=xI.Interleaved2Planar() # Planar YUY2

xSuper=xP.MSuper(Planar=True)

xvf =xSuper.manalyse(isb=false,blksize=16, truemotion=true)
xvb =xSuper.manalyse(isb=true,blksize=16, truemotion=true)
xvf1=xSuper.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=xSuper.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=xP.mflow(xSuper,xvf1,Planar=True)
xxf =xP.mflow(xSuper,xvf,Planar=True)
xxb =XP.mflow(xSuper,xvb,Planar=True)
xxb1=xP.mflow(xSuper,xvb1,Planar=True)

interleave(xf1,xxf,xP,xxb,xxb1)
xI=Planar2Interleaved()
xI.temporalsoften(2,255,255,mode=2)
selectevery(5,2) # YUY2

Return Last # .bicubicresize(704,576) # Added


FPS=50 # This was just for test (mod to whatever)
last.Interleaved2Planar.mflowfps(Super,vb,vf,num=FPS,den=1,Planar=True).Planar2Interleaved()
Return bicubicresize(704,576)

about as good as I can do, perhaps others can improve.

EDIT: Mega.nz that hosts Ingoldie sample offer to download with browser (I picked), or use their download program,
on download, offers to play clip or save, if chose play then will delete after playing. (stores file in your TEMP folder,
grab it before it gets deleted.)

EDIT: I hope that you are not in a hurry, tis a bit slooooow.
The dupes and jerkiness of your source clip (just downloaded and viewed) are bound to play havoc with the motion vector stuff.

EDIT: I notice that your clip has 18FPS in the name, as already converted to 29.97 fps, will be a problem for motion vectors,
better to start with original 18fps (I think).

[StainlessS]

I believe that there is a bug in original script (and also my conversion of it)
correction in BLUE

f=4

avisource("Sample_SD_8mm_18_fps.avi")

###### Make Test Clip
#avisource("D:\v\Starwars.avi").Trim(10000,-10000) # 720x576@25
#BilinearResize(Width/(4*f)*4,Height/(4*f)*4) # Might want to comment out these lines (Lo rez)
#SelectEven() # Half frame rate
#Return PointResize(Width*f,Height*f) #
#######

a=last
a.converttoyuy2() # Last is YUY2
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)

srcP=Interleaved2Planar() # Last is Planar YUY2

super=srcP.MSuper(Planar=True)

vf =super.manalyse(isb=false,blksize=16, truemotion=true)
vb =super.manalyse(isb=true,blksize=16, truemotion=true)
vf1=super.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=super.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=srcP.mflow(Super,vf1,Planar=true)
xf =srcP.mflow(Super,vf, Planar=true)
xb =srcP.mflow(Super,vb, Planar=true)
xb1=srcP.mflow(Super,vb1,Planar=true)

interleave(xf1,xf,srcP,xb,xb1)
srcI=Planar2Interleaved()
srcI.temporalsoften(2,40,50,mode=2)
xI=selectevery(5,2)

xRGB=xI.converttorgb()

xaI=xRGB.pointresize(a.width*2,a.height*2).converttoyuy2().eedi2().turnleft().eedi2().turnright()

xbI=xRGB.crop(1,1,0,0).addborders(0,0,1,1).converttoyuy2().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()


xmI=merge(xaI,xbI)

xP=xmI.Interleaved2Planar() # Planar YUY2

xSuper=xP.MSuper(Planar=True)

xvf =xSuper.manalyse(isb=false,blksize=16, truemotion=true)
xvb =xSuper.manalyse(isb=true,blksize=16, truemotion=true)
xvf1=xSuper.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=xSuper.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=xP.mflow(xSuper,xvf1,Planar=True)
xxf =xP.mflow(xSuper,xvf,Planar=True)
xxb =XP.mflow(xSuper,xvb,Planar=True)
xxb1=xP.mflow(xSuper,xvb1,Planar=True)

interleave(xxf1,xxf,xP,xxb,xxb1) # BUG HERE used xf1 instead of xxf1
zI=Planar2Interleaved()
zI.temporalsoften(2,255,255,mode=2)
selectevery(5,2) # YUY2

FPS=25
#Interleaved2Planar.mflowfps(super,vb,vf,num=FPS,den=1,Planar=True).Planar2Interleaved()

Return bicubicresize(704,576)


It dont seem to make much difference to result.

EDIT: Also think maybe should have mod to this (or maybe not, not sure if 'f' is intended to be 'tuneable')


xaI=xRGB.pointresize(a.width*(f/2),a.height*(f/2)).converttoyuy2().eedi2().turnleft().eedi2().turnright()

xbI=xRGB.crop(1,1,0,0).addborders(0,0,1,1).converttoyuy2().pointresize(a.width*(f/2)),a.height*(f/2))).eedi2().turnleft().eedi2().turnright()


otherwise size mismatch if f = 2 (EDIT: at framerate conversion, 2nd last line)
Although maybe would be better to just not do the 2nd batch of motion vector stuff, from ConvertToRGB onwards (if f==2),
(EDIT: or do another Super() & MAnalyse()'s on result clip prior to framerate conversion).

EDIT: Damndest thing is, it renders (EDIT:quite a bit) faster WITH the framerate conversion step than without it.

EDIT: your clip looks to be 18fps with dupes. I removed dupes with this

avisource("Sample_SD_8mm_18_fps.avi")
Assumepfs(30.0)
MDec2(rate=18.0,Create=false,dir="")
Return Last

Above is 2nd pass, 1st pass Create=True, is MultiDecimate alternative.

[StainlessS]

I removed the YUY2 stuff, was just so awkward, converting to YV12.

This is what I got, added Blur for SuperPrefilt, analyse blurred, used plain super for MFlow.


f=4

#avisource("Sample_SD_8mm_18_fps.avi")
avisource("18.avi")
ConvertToYV12

a=last
eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)

superPrefilt=Last.Blur(0.6).MSuper() # All Levels for MAnalyse
super=Last.MSuper(Levels=1) # MFlow needs only 1 level

vf =superPrefilt.manalyse(isb=false,blksize=16, truemotion=true)
vb =superPrefilt.manalyse(isb=true,blksize=16, truemotion=true)
vf1=superPrefilt.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=superPrefilt.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=Last.mflow(Super,vf1)
xf =Last.mflow(Super,vf)
xb =Last.mflow(Super,vb)
xb1=Last.mflow(Super,vb1)

interleave(xf1,xf,Last,xb,xb1)
temporalsoften(2,40,50,mode=2)
selectevery(5,2)

RGB=converttorgb()

xaI=RGB.pointresize(a.width*2,a.height*2).converttoYV12().eedi2().turnleft().eedi2().turnright()
xbI=RGB.crop(1,1,0,0).addborders(0,0,1,1).converttoYV12().pointresize(a.width*2,a.height*2).eedi2().turnleft().eedi2().turnright()

xP=merge(xaI,xbI)

xSuperPrefilt=xP.Blur(0.6).MSuper() # All Levels for MAnalyse
xSuper=xP.MSuper(Levels=1) # MFlow needs only 1 level

xvf =xSuperPrefilt.manalyse(isb=false,blksize=16, truemotion=true)
xvb =xSuperPrefilt.manalyse(isb=true,blksize=16, truemotion=true)
xvf1=xSuperPrefilt.manalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=xSuperPrefilt.manalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=xP.mflow(xSuper,xvf1)
xxf =xP.mflow(xSuper,xvf)
xxb =XP.mflow(xSuper,xvb)
xxb1=xP.mflow(xSuper,xvb1)

interleave(xxf1,xxf,xP,xxb,xxb1)
temporalsoften(2,255,255,mode=2)
selectevery(5,2)

NUM=30000 DEN=1001
mflowfps(super,vb,vf,num=NUM,den=DEN)

bicubicresize(704,576)

TWriteAVI("Out.AVI",fourCC="ULY0",Overwrite=True) # Write to AVI as play clip.


Is not very good result especially due to jerkiness (although the latter part of clip is not so bad).
Intention of script is to increase rez of low rez clip, your clip aint that lo in rez and so final result is not
going to be that much different to source (final result not much different to source rez).

EDIT: And I bet that the camera for Scharfis Brain demo clip was fixed ie not jumping about.

EDIT: Changed to Levels=1 for Super clip [MFlow needs only 1 Level].

[ingoldie]

I don't think that script increases the resolution and image detail.
I'm asking for a SuperResolution script to increase image detail.

To do this:
http://i65.tinypic.com/2mfnsbd.jpg

Super-resolution imaging
https://en.wikipedia.org/wiki/Super-resolution_imaging

[StainlessS]

Scharfis Brain demo clip looked quite incredible [EDIT: or rather the still images of it], but rapid moving objects and moving camera are not a good combination and unlikely to produce many miracles (consistently).

EDIT: I would not personally hold my breath waiting for that el dorado to materialise.

[StainlessS]

InGoldie,

Try this out, has some amount of configuration, framerate conversion working quite well and reduces jitteryness of the clip.


f=4 # 4 or 2 or 1 (PointResize OFF)
#
VAGUE=2.5 # PreFiltering (Spatial Only), -1.0 = OFF, 0.0=AUTO, maybe about 0.0 to 3.0
BLUR=1.58 # up to 1.58 only, Prefilter
BS = 16 # 16 or 8
OVERLAP = BS/2 # BS / 2 or 0
PRECISE=True # MRecalculate
MRECALC_ON_PREFILT_SUPER = True # Cant decide which one should be used, you pick one :)
#
MOCOMP=True
MOCOMPBLOCK=True
FFT3D=True
FFT_SIGMA=2.4
FFT_SHARPEN=0.5
TEMPSFT_RADIUS = 1 # 2 or 1 (When FFT3D is false)
#
CHANGEFPS=True
BLOCKFPS=True
NUM=30000 DEN=1001 BLENDFPS=False
########
avisource("18.avi") # MDEC2() Decimated 18FPS
ConvertToYV12

a=last

eedi2().turnleft().eedi2().turnright()
pointresize(a.width*f,a.height*f)

PostPoint = (f>1) ? Last : a

Prefilt=(VAGUE>=0.0)?PostPoint.VagueDenoiser(threshold=VAGUE,method=1,nsteps=6,chromaT=0,interlaced=false):PostPoint

superPrefilt=Prefilt.Blur(BLUR).MSuper() # All Levels for MAnalyse
super=PostPoint.MSuper(Levels=1) # MFlow/MCompensate/ and FPS stuff requires only 1 Level.

vf1=superPrefilt.manalyse(isb=false,delta=2,blksize=bs, overlap=OVERLAP, truemotion=true)
vf =superPrefilt.manalyse(isb=false,blksize=bs, overlap=(CHANGEFPS&&BLOCKFPS&&!PRECISE)?0:OVERLAP, truemotion=true)
vb =superPrefilt.manalyse(isb=true,blksize=bs, overlap=(CHANGEFPS&&BLOCKFPS&&!PRECISE)?0:OVERLAP, truemotion=true)
vb1=superPrefilt.manalyse(isb=true,delta=2,blksize=bs, overlap=OVERLAP, truemotion=true)

recalcSuper = (MRECALC_ON_PREFILT_SUPER) ? superPrefilt : super
v2f1 = (PRECISE) ? MRecalculate(recalcSuper, vf1 ,blksize=bs/2, overlap=OVERLAP/2,thSAD=100) : vf1
v2f = (PRECISE) ? MRecalculate(recalcSuper, vf , blksize=bs/2, overlap=(CHANGEFPS&&BLOCKFPS)?0:OVERLAP/2,thSAD=100) : vf
v2b = (PRECISE) ? MRecalculate(recalcSuper, vb, blksize=bs/2, overlap=(CHANGEFPS&&BLOCKFPS)?0:OVERLAP/2,thSAD=100) : vb
v2b1 = (PRECISE) ? MRecalculate(recalcSuper, vb1, blksize=bs/2, overlap=OVERLAP/2,thSAD=100) : vb1


xf1=(MOCOMPBLOCK)?PostPoint.MCompensate(super,v2f1):PostPoint.mflow(Super,v2f1)
xf =(MOCOMPBLOCK)?PostPoint.MCompensate(super,v2f):PostPoint.mflow(Super,v2f)
xb =(MOCOMPBLOCK)?PostPoint.MCompensate (super,v2b):PostPoint.mflow(Super,v2b)
xb1=(MOCOMPBLOCK)?PostPoint.MCompensate (super,v2b1):PostPoint.mflow(Super,v2b1)

interleave(xf1,xf,PostPoint,xb,xb1) # TEMPSFT_RADIUS==1, and FFT3D only uses middle 3

(FFT3D) ? FFT3DFilter(Plane=4,Sigma=FFT_SIGMA,Sharpen=FFT_SHARPEN) : temporalsoften(TEMPSFT_RADIUS,40,50,mode=2)

(MOCOMP) ? selectevery(5,2) : PostPoint

# MBlockFPS Requires OVERLAP=0
(CHANGEFPS) ? ((BLOCKFPS)?MBlockFPS(super,v2b,v2f,num=NUM,den=DEN,Blend=BLENDFPS):mflowfps(super,v2b,v2f,num=NUM,den=DEN,Blend=BLENDFPS)) : NOP

bicubicresize(720,480) # Whatever resizer you like

(F==1 && !MOCOMP && !CHANGEFPS) ? Subtitle("NO Change",align=5) : NOP

return TWriteAVI("Out7.AVI",fourCC="ULY0",Overwrite=True) # Write to AVI as play clip.


I've taken out the second RGB/upscaling stuff, too unstable.

EDIT: Added PRECISE.

EDIT: Added MRECALC_ON_PREFILT_SUPER

EDIT: Some more changes.

[ingoldie]

I'm getting "TWriteAVI: Couldn't open a compressor with Fourcc code ULY0" error.
How can I fix it?

[StainlessS]

Install UT_Video codec or use another Lossless one eg Lagarith, HuffYUV, Magic. [EDIT: Change FourCC to whatever one it uses].

EDIT: In VDub, open Compression and see what codec fourCC's you have available, and pick a lossless one, just as you would in VD.

EDIT: Your Trees in background @ beginning of sample are always going to be a problem for Mocomp framerate conversion,
lots of black vertical 'things' spread across a whitish sky, jumping in real ground space about 20 meters each (camera movement)
and the mv stuff cannot tell if jumping left or right (ie which is it, right tree jumping left or left tree jumping right).
But the rest of the clip is pretty good.