Having problems deinterlacing / deblocking a source

[EuropeanMan]

First let me present my case.

I've got a DVD9, R0...
title of movie: Veer Zaara 2-disc set, and I am working with Disc 1, the movie only.

It is highly interlaced. Bottom Field First.

I am having a particularly difficult problem solving the deinterlacing for this source...particularly in the action sequences such as the one I am giving below.

My main problem is with the blocking. Let me show you what I mean. These 2 screenshots were taken w/ 2 separate programs...so that's why you will see a difference...

Regular Scene, no action...just dialogues...This particular SS does NOT come from the DVD9 sample given below...this is here just for a frame of reference.

http://maxupload.com/img/36E0D29D.png

Action scene screenshot:

http://maxupload.com/img/8B558A0A.png

Script used to attain above SS's:

http://forum.doom9.org/showpost.php?p=1050818&postcount=4

DVD9 sample (http://maxupload.com/A27B3A67)

Output with the script shown (http://maxupload.com/48C1DCD3)

Output w/ Integrated Imaging (http://maxupload.com/78F76383)

I neglected to put AssumeBFF here...sorry. Didn't know if that would make a difference.

Also this was attempted without success as well:
assumebff().mcbob().selecteven()

I was asked to get this problem to sharfis_brain? i think his ID is here...apparently he is a whiz at deinterlacing. Would love any input/suggestions as to the deinterlacing & deblocking solution. Thanks in advance. I am at wit's end here...

HELP!!!!!!

[Didée]

The primary problem is neither the blocking, nor the interlacing as such. Main problem is that this source has been converted from PAL 25p to NTSC 30i by fieldblending. Simple deinterlacing will not work out with such sources.

To educate yourself on the matter, use the forum search on "Restore24", "MRestore", "normconversion", "blended fields"/"fieldblending", ...

Quick example of a possible basic treatment:

LoadPlugin("DGDecode.dll")
LoadPlugin("mt_masktools.dll")
LoadPlugin("reduceflicker.dll")
LoadPlugin("fft3dfilter.dll")

Import("MRestore.avs")

mpeg2source("test vz_(EuropeanMan).d2v")
crop(0,48,-0,-48,true)

bob(-.2,.6).reduceflicker(strength=1)
MRestore()
FFT3DFilter(sigma=8,sigma2=3,bw=16,bh=16,ow=8,oh=8,bt=3,plane=4)
spline36resize(704,304, 4,4,-4,-4)

Result: Sample (http://maxupload.com/A2090610) (MaxUpload, ~18MB)

[EuropeanMan]

I have attempted to use Mrestore in the past. I'm on vista...using 2.5.7 w/ MT. Let's get that out in the open.

With past rips where I wanted to use mrestore...1st pass in megui...lets say i got 15fps...half-way thru 1st pass i noticed the fps going down...10, 9 8...slowly going down. I was shocked. Apparently mrestore doesn't work on my computer (with or without vista) or perhaps I might have had some bad version. Therefore always gave up on mrestore, and used smartdeci instead w/ other filters. I will look into your suggestions...

In the meantime, Didee (thanks for help)...do you know if there is some fixed version of mrestore that does NOT do the slowdown ? Tx...Will go eat some breakfast & ask you some more pertinent questions...IF you're still around. Only 2pm where you are :) 5am here...looking forward to hearing from you.

[Didée]

Frankly, I never used MRestore on any "full length" content, but only on samples a few minutes in length.

Now that you mention it, I remember there have been reports about some memory leak showing up when using MRestore. The issue is unsolved, afaik.

[EuropeanMan]

A buddy of mine also attempted a CCE on this movie using Mrestore and gave up. I'm also trying my hand at it as well. I know he did a 3CD rip of it...and used some sort of 4 part .avs script. The first one took the other 3 parts and put them together. Each individual part took basically 1hr each of it, and I guess it worked out for him. However, I didn't really like his quality due to cropping and A/R errors.

I also used your above example and did the xvid rip myself on the sample...blocking issues remain...and I will believe that this is due to heavy action, and the fact that it was a 25 to 30i conversion...do you think it was done badly, or just a poor choice on part of the dvd manufacturers?

Yes, it's a 3hr5min film. I still haven't had a chance to look at your output yet...and will d/l now. I purposely set a bitrate of 2840 (for the cce of course) as a setting in my xvid rips to emulate the same output.

[Didée]

Those interlaced AND fieldblended normconversions are giving a hard time for all kinds mpeg encoders. It requires an extra plus of bitrate, only to make up for the problems the encoder is having with the fieldblending.
But alas, instead of "higher-than-should-be", those sources usually have "lower-than-should-be" bitrates, so ... the poor quality is perfectly normal, indeed.

Apart from that, IMO those FFT3DFilter settings deal reasonably well with the blocking. Sure one could try other means to filter the blocking out even more, but I fear it would do more harm than good. Better some small quirks left over in the fast moving parts where the eye can't follow so easy, than removing the issue too aggressively and on the way also harming the low-motion parts where overfiltering becomes blataneously obvious...

[EuropeanMan]

A buddy of mine suggested using SeeSaw on this source...but I hate that filter, no offence. I prefer LSF myself. Even better is iip. Do you think iip along with the above will make any noticable benefit...or is that going to be overkill. I myself will probably NOT prefer doing it as it will take 2 weeks just to encode at 0.4fps (predicting that)...

Also, I'd like to bring a touch of more detail from the source.
I've come up with using smode=1 with a bigger radius, followed possibly with smode=1 smaller radius followed with smode=4...will this in your opinion hurt what we are doing above w/ respect to deblocking/deinterlacing? The reason why I ask...is that if you look at the girl's face in the above screenshot...it's a bit too smooth and I know this actress' face...and she does have detail and is NOT prone to wearing huge amounts of makeup. Also I've seen other closeups of other actors in this...and in my opinion, it's a bit too smooth.

--------

On a separate note...I went through your whole thread on integrated imaging...and towards the end you mentioned that one day you might merge everything using seesaw into a successor for iip. Has this ever been done or are you still testing or is it still out there? I've also looked into the 0.6iip and not too terribly thrilled about it. Although I will be using iip on a particular source after this project of mine completes. Will show you the output, because I was really impressed...so give me a few wks on that. That encode I know will be going @ 1fps or so...haha, but personally I feel it's worth it.

--------
Now going back to this project. With respect to Mrestore, and if I can successfully break the movie in 3 parts...the output I would like is to lagarith lossless...Is this possible? I've never done a 3 part .avs file and using some 'conditional reader'? or something to bring those together. Will that yield 3 separate files which I can combine into one...or will I have to use a 4th script to bring it together and use THAT .avs to feed into VD?

[EuropeanMan]

Another example of the 'smoothing' i feel is present...

http://maxupload.com/img/8F37DF7C.png

http://maxupload.com/img/8E0297BA.png

bottom is dvd9. of course my script corrects the A/R error I feel is present in the DVD9. and you see that work in my script. I'd like to get a bit more detail out of that face...but not make it OVERLY sharp...

[Didée]

A buddy of mine suggested using SeeSaw on this source...but I hate that filter, no offence.
If you tried SeeSaw on sources of similar quality than this one here, then I'm not surprised. SeeSaw deals best in enhancing clean sources. When fed with crappy sources, it enhance mostly the crap.

I don't see what I should recommend here. With such a problematic source, everything is a matter of personal taste. What looks good to some will look poor to others.

However, one thing is sure: if you were to run a script that really gets the most out of this source, your electricity bill would go so high (due to v.e.r.y long processing time) that it would be cheaper to just order a PAL version of that movie, which should have a much better PQ to begin with. (Most probably, since the true rate is 25fps.)

[EuropeanMan]

If you tried SeeSaw on sources of similar quality than this one here, then I'm not surprised. SeeSaw deals best in enhancing clean sources. When fed with crappy sources, it enhance mostly the crap.

I don't see what I should recommend here. With such a problematic source, everything is a matter of personal taste. What looks good to some will look poor to others.

However, one thing is sure: if you were to run a script that really gets the most out of this source, your electricity bill would go so high (due to v.e.r.y long processing time) that it would be cheaper to just order a PAL version of that movie, which should have a much better PQ to begin with. (Most probably, since the true rate is 25fps.)

I don't mind the long processing time :) What do you suggest. It's better for me to do it this way...so at least I can learn what I am doing. Like what YOU have suggested above...that taught me much in itself. :)

As far as finding the true 25PAL source...terribly difficult...will need to ask people in India to get it since the movie IS indian...and I live near los angeles. :( And I'd rather work from this source...it's always a learning experience.

SO PLEASE PLEASE...I don't care if the encode will run for a long time...I am ALL ears & eyes...

[EuropeanMan]

^ Didee...I saw your sample...but why is it at 768 x 304...I don't see how you achieved that with your script here.

Also, what is the purpose of 'true' in the Spline statement? tx.

[foxyshadis]

Now going back to this project. With respect to Mrestore, and if I can successfully break the movie in 3 parts...the output I would like is to lagarith lossless...Is this possible? I've never done a 3 part .avs file and using some 'conditional reader'? or something to bring those together. Will that yield 3 separate files which I can combine into one...or will I have to use a 4th script to bring it together and use THAT .avs to feed into VD?

Something like the first script being trim(0,60000), the second trim(60001,120000), etc? That'll work just fine, aside from the possibility of a blended frame or three along the seams. You can combine the output in virtualdub or an avisynth script with +, both work the same.

If you mean the true in the crop, that's something that creates a new frame instead of just messing with the frame's dimensions in memory. Sometimes it speeds things up, sometimes it slows them down.

[EuropeanMan]

Something like the first script being trim(0,60000), the second trim(60001,120000), etc? That'll work just fine, aside from the possibility of a blended frame or three along the seams. You can combine the output in virtualdub or an avisynth script with +, both work the same.

If you mean the true in the crop, that's something that creates a new frame instead of just messing with the frame's dimensions in memory. Sometimes it speeds things up, sometimes it slows them down.

IF this is the route I take...with the 3 files. And piece them together, what effect will that have on the audio synching, after CCE encodes my .avi file? Because you said there would be the possibility of a blended frame or 3 along the seams....

(of course mpv output which then will be in dgpulldown from 25--> 29.97ntsc)

In addition, do you know of a good script for bringing those 3 .avi files together to feed thru CCE? or should i just piece them together in VD...what's your recommendation? I know the trim command, but don't know of HOW to piece them in one separate .avs file which I would want to put thru in CCE...thanks.

[EuropeanMan]

@ Didee here is another test output...it's a bit smooth so I still need to work out some details somehow :)

http://maxupload.com/CE09C8FB

Let me know if you see any other issues I might have overseen.

[foxyshadis]

The avs is simple:

Avisource("file1.avi")+Avisource("file2.avi")+...etc.

This vs. appending them in virtualdub makes absolutely no difference, aside from needing double the disk space temporarily. Your choice.

Actually you shouldn't even get blends/stuttering at the sync points if your trim statements are at the end, rather than the beginning.

Audio will be in sync as long as you extract it from the DVD and mux it in at the end. Don't even bother trying to sync it in with the file unless you're going to do other editing. Adding pulldown won't create any sync issues. Once you're done you may need to add padding/delay or chop a bit to sync it, but no more than it should have taken to sync it to the original source.

[EuropeanMan]

^ Thanks foxyshadis...my discussion for this particular movie was in 2 parts...here is 2nd thread for the CCE part: http://forum.doom9.org/showthread.php?t=130402

If you have suggestions or thoughts there, PLEASE do let me know, because I've seen your posts all over and highly respect your opinions/suggestions. Thanks again!

[foxyshadis]

I don't know the first thing about CCE, except that its rate control is completely broken and needs a hilarious number of passes, sorry. :p I use HC or quenc, aften, and dvdsubedit (or sometimes just FAVC) for all that.

[EuropeanMan]

Could someone please help me with this little problem.

If I wanted to join 3 .avi's together via another .avs file...

I know how to put them together with "+"
But what if I want to cut off the very very last frame of each segment?

[foxyshadis]

Avisource("...").trim2(0,end=-1) + ...

See http://avisynth.org/stickboy/

You can do it with trim and temporary variables as well, but it's neater this way.

[eagle21]

Dont see any thing to click on to start burnning,how do I start?

[EuropeanMan]

Avisource("...").trim2(0,end=-1) + ...

See http://avisynth.org/stickboy/

You can do it with trim and temporary variables as well, but it's neater this way.

Love you dood :) Thanks!

[EuropeanMan]

Is it a correct assumption that anytime I make an Xvid .avi file that a blank frame is added at the end?

[manono]

I don't quite understand why you're having to jump through hoops to get this encoded. Did you try the RePAL filter yet? Seems to work fine on your sample:

Colormatrix (D2V="I:\Test\Test\Test.d2v",Clamp=False,Interlaced=True)
Yadif(Order=0,Mode=1)
RePAL()

I also sharpened it up a bit, saturated it a bit, and cropped and resized. The max video bitrate on the DVD is only 5500, and when the Indians start dancing, that's not even close to being enough, especially for an interlaced DVD, and you get the mosquito noise, the smeared colors, and the blocks. I didn't bother doing anything about those things:

http://www.mediafire.com/?9ltsyypmadd

And if you really like the film (I thought it only OK), you don't have to go all the way to India for the PAL version. I've ordered from Amazon.com in the UK before with no problems:

http://www.amazon.co.uk/Veer-Zaara-Shah-Rukh-Khan/dp/B0007LYDO6/ref=sr_1_1/203-1403350-9375116?ie=UTF8&s=dvd&qid=1191661979&sr=1-1

Expensive, though, what with the dollar being the way it is these days.