Can't figure out deinterlacing solution

[EuropeanMan]

I am having a HUGE problem trying to deinterlace this DVD9 that I have.

I've used a bunch of different things, decimate, tgmc, mrestore, srestore...but can't figure out WHY it still won't deinterlace.

sample: http://www.sendspace.com/file/d6el63

Please help...and thanks in advance.

[scharfis_brain]

I really don't know what you're complaining about.

If you tried out the solutions you mentioned, you've should noticed that srestore() does a wonderful job:

MPEG2Source("sample dmg.d2v")
bob()
srestore()

If you want to have less flicker and aliasing you might toss in a more advanced deinterlacer:

mpeg2source("E:\forum\EuropeanMan\sample dmg.d2v",cpu=6)
crop(0,56,0,-56,align=true)
tdeint(mode=1, edeint=nnedi2(field=-2), full=false, tryweave=true)
srestore()

[Guest]

He just wants somebody to do his work. That's why he included no details of his efforts and the actual results.

"It still won't deinterlace." Give us a break.

[Amrit]

LOL, Yeah it would be nice if you provide what you did, if you did anyhting.

[EuropeanMan]

He just wants somebody to do his work. That's why he included no details of his efforts and the actual results.

"It still won't deinterlace." Give us a break.

i'll give you everything - no problem.

don't accuse ME of anything...understood? i've been here a long time, and i am NOT shy of providing any proof. i do my work extensively before coming here for help.


Solutions tried


o = crop( 2, 58, -2, -58)
tgmc = o.tempgaussmc_beta1u(1,1,1,edimode="nnedi2",blocksize=16,sharpness=1.33,SVthin=1.25,truemotion=true)
bob1 = o.bob(0,0.5)
bob2 = bob1.clense(reduceflicker=false).merge(bob1,0.5)
bob3 = bob2.mt_adddiff(mt_makediff(bob2,bob2.clense(reduceflicker=false),U=3,V=3),U=3,V=3)

tgmc.Srestore(frate=25.0,dclip=bob3)


tfm(order=1)
tdecimate()



yadif(mode=3) #mode 1 also tried
srestore() # srestore(frate=23.976) also tried

[EuropeanMan]

mpeg2source("E:\forum\EuropeanMan\sample dmg.d2v",cpu=6)
crop(0,56,0,-56,align=true)
tdeint(mode=1, edeint=nnedi2(field=-2), full=false, tryweave=true)
srestore()

this solution results in the following problem:

http://i48.tinypic.com/e1411z.jpg

checked with your OTHER solution that you gave, and still the same as above screenshot.

THIS is why I came here...no matter what solution I get to 25fps, I still get those frames.

[scharfis_brain]

this frame isn't in your sample.

also you never can expect a blend-free output.

garbage in -> garbage out.

you may tweak srestore's parameters to your liking.

maybe you'll find better values then the default ones.

[EuropeanMan]

^ yeah, trying the parameters...well i know about the garbage in / garbage out deal... :(

that type of screenshot i gave you appears everywhere in the whole movie...
well, i'll keep trying...hopefully someone knows something that I don't that might give me a better solution than ones we've been trying so far.

Thanks again for your suggestions...much appreciated.

[manono]

I don't see the problem, not with anything in the sample anyway. It unblends fine. Except for the slo-mo part where the dog gets out of the car, and that's intentional. There's no remaining interlacing.

Much bigger problems, in my opinion, are the horribly crushed blacks and the generally dull appearance of the video.

[scharfis_brain]

this helps a bit against the crushed blacks and dullness:
coloryuv(off_y=15).tweak(sat=1.3)

[Didée]

Just in case - we don't know how you got to that screenshot, but remember that there will be problems if you jump-around in the timeline. When manually forcing a "seek" to another position in the video, then SRestore needs to adapt itself to a new pattern. This takes some frames, hence the first few frames after such a seek usually are erroneous. But will be fine when doing a straight from-start-to-end process.

[EuropeanMan]

^ Thanks Didee...I'll re-run the script, instead of previewing through VDub.

[Dogway]

Didee: wow, thats a reliefing explanation! I was suspecting something alike, but its better to finally know it.

[EuropeanMan]

Just in case - we don't know how you got to that screenshot, but remember that there will be problems if you jump-around in the timeline. When manually forcing a "seek" to another position in the video, then SRestore needs to adapt itself to a new pattern. This takes some frames, hence the first few frames after such a seek usually are erroneous. But will be fine when doing a straight from-start-to-end process.

i ran a yadif + srestore all the way through the movie

problems seen in lag file:

every 4th/5th (out of 5 frames) is almost a dupe . if not that, then there is still ghosting...such as the frame I showed above. :(

[scharfis_brain]

is this true even for the sample video you posted above?

also post your full script.

[Didée]

Check the framerate of your result video. Does it happen to have a 28.742 framerate? :)

Some versions of Srestore don't automatically put the correct output framerate for NTSC input.
Your input is 29.97 NTSC, and the original framerate was 25fps PAL.
So, to be on the safe side, set the framerate explicitely:

mpeg2source(Bollywood)
bob() # or whatever bob filter you wanna use
Srestore(frate=25.000)

[EuropeanMan]

@ scharfis:


LoadPlugin("C:\Program Files\megui\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("x:\VTS_02_1.d2v", info=3, cpu=6)

Import("C:\Program Files\AviSynth 2.5\plugins\srestore.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mvtools2.dll")
Load_Stdcall_Plugin("C:\program files\avisynth 2.5\plugins\yadif.dll")
crop(0, 56, 0, -56, align=true)

yadif(mode=3)
srestore(frate=25)

degrainmedian(mode=5).limitedsharpenfaster(strength=50, smode=4)
Lanczos4Resize(720,304)


yes to the sample given...now running your 2nd 'adv' script on 8000 frames from the movie once again...

@ didee - as u can see - it was set specifically...i don't take chances :)

[EuropeanMan]

An example of the 4th/5th frame in 5 frame sequence
top frame is #4 / bottom frame is # 5

http://comparescreenshots.slicx.com/comparison/37432

would you call this a chroma shift?

[EuropeanMan]

a second sample from dvd9 for you to look at:

http://www.sendspace.com/file/6na9au

[scharfis_brain]

please knock out everything except the deinterlacer and srestore.
then try.

have you tried the unaltered script I showed you?

also you should use a better deinterlacer yadif ist not that good.
tdeint with nnedi2 wasn't without purpose!

[EuropeanMan]

^ the script i'm running now is ur 2nd one...still have 20mins left - will post results in a bit. i know yadif isn't that great, but wanted to try it JUST to make sure.

i would have done all this yesterday, but i cut up one of my fingers very badly with a kitchen knife...

[EuropeanMan]

LoadPlugin("C:\Program Files\megui\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("x:\VTS_02_1.d2v", info=3, cpu=6)

Import("C:\Program Files\AviSynth 2.5\plugins\srestore.avs")

crop(0, 56, 0, -56, align=true)
trim(20000,30000)

tdeint(mode=1, edeint=nnedi2(field=-2), full=false, tryweave=true)
srestore(frate=25)

#coloryuv(off_y=15).tweak(sat=1.15)

LanczosResize(720,304)


yields the same 4/5 sequence
http://comparescreenshots.slicx.com/comparison/37442

[Didée]

Looking closer at the source, I see a few things that make the job more difficult for Srestore.

a1) there are parts where they use dup's to achieve the effect of "half framerate". This makes things more difficult.

a2) this dup'ing was done AFTER the normconversion, which results in forward-backward-forward-backward effect. (Seems we're talking about idiot's work.) Poor Srestore - you can't expect it to handle this situation like you would like it to.

b) There has been used some sort of "temporal sharpening" after the normconversion. See in the posted sample: very obvious e.g. at bob frames 1258+1260. (Surely we're talking about idiot's work.)


Don't blame our tools that they aren't perfect. Blame Bollywood that they produce such a crap. I repeat: crap.

[manono]

Nothing wrong with the samples here. What you describe (yields the same 4/5 sequence) is the kind of thing you get when the field order is set incorrectly. My suggestion is to set a field order and then step through it. If you get those dupes/blends every 5th frame then change the field order and try again.

[scharfis_brain]

also move the import line above the mpeg2source.

[EuropeanMan]

^ thanks for the input Didee, manono & scharfis - it is MUCH appreciated. will work on it later this afternoon...for now, need a few hours break. will post results tonight when i get back. thanks again.

[Didée]

You might try the following tweak. It won't remove the mentioned "additional nasties" from the clip, but it should dimish their negative impact on Srestore's inner working. In the posted sample, I see a clear improvement where the car's door opens and the dog jumps out.

Adapting the posted script:

LoadPlugin("where\you\have\removegrain.dll") # In case of doubt, use v1.0 pre-1 (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar) ...
LoadPlugin("C:\Program Files\megui\tools\dgindex\DGDecode.dll")
Import("C:\Program Files\AviSynth 2.5\plugins\srestore.avs")

DGDecode_mpeg2source("x:\VTS_02_1.d2v", info=3, cpu=6)

crop(0, 56, 0, -56, align=true)
trim(20000,30000)

tdeint(mode=1, edeint=nnedi2(field=-2), full=false, tryweave=true)
knockout=clense(reduceflicker=false).merge(last,0.5).clense(reduceflicker=false)
srestore(frate=25.000,dclip=knockout)

#coloryuv(off_y=15).tweak(sat=1.15)

LanczosResize(720,304)

[EuropeanMan]

^ running that right now, and have set the order to TFF - will post results in a bit, before i leave for the day. thanks Didee :)

[EuropeanMan]

Didee, with your script above, and setting order=1 & order=0 (top field (#1) vs botom field (#2) first) we have this

http://comparescreenshots.slicx.com/comparison/37525

[scharfis_brain]

never ever use the order-parameter unless you really know what you're doing.
use assumetff() / assumebff() directly after the source instead.

[Didée]

Hmh. Can't tell what's going on. Both screens look bad - one shows fieldblending, the other shows a combed frame that has been resized.

It should not be necessary to set any fieldorder at all. The stream is flagged TFF, and with their default settings, both TDeint and NNEDI2 will automatically use the correct fieldorder. In particular, "order=0" is the wrong choice, since the stream *is* TFF. Well, at least the sample is.
In case that the VOBs are also plagued by broken fieldorder, that's another possible problem.
If you just do a plain bob() or Yadif(mode=1) on the source, do you find any places with forward-backward-forward motion? Did DGIndex report something during indexing? Or, when opening the d2v with DGindex->Tools->Parse d2v, is there any other number than "2" at the right side of the list?

Note that I'm wildly poking in the blue ... not having the source, it's pure guesswork if there is a technical error, or if it's just a glitch in the processing. All I can say for sure is that the suggest settings work OK for the posted sample. If you're still in the mood, upload a sample of a section where the suggested settings do *not* work out. (Though it's bedtime for me now, g'night...)

[EuropeanMan]

As far as the parsing goes, you perhaps guessed right. First part of the movie is BFF....most is TFF... http://www.sendspace.com/file/jdgxc6

@Scharfis - thanks; was dumb of me...duh!!! Will try this again.
@Didee, i'll try the plain bob/mode1yadif in just a bit. And thanks :)

[EuropeanMan]

How would I worry about scene transitions - such as this?

http://i46.tinypic.com/2mq6r6s.jpg

[scharfis_brain]

ignore it. it is fieldblended crap with temporal sharpening applied.
fieldblended stuff has no clean scene transitions.

[EuropeanMan]

^ thanks - was going to look at TempGaus again...

I give up on this...garbage

Arghhhhhhhhhh - thanks so MUCH for your guys' help. I learned a little bit, which was the idea anyway. Truly appreciate the feedback.

[Stephen R. Savage]

Out of curiosity, what is "temporal sharpening"?

[Guest]

Consider a pixel making a transition 50->100:

50 50 50 50 100 100 100 100 ... # successive samples

A temporally sharpened version could be:

50 50 50 25 125 100 100 100

It's just using kernels in time rather than space.

[Stephen R. Savage]

This is interesting, and the first time I've ever heard of it. Why would this be done? I thought the purpose of sharpening was to increase edge contrast, an entirely spatial process.

[EuropeanMan]

What I'd like to learn about is the half-rate that was mentioned here in this thread...how is that possible and why?

[manono]

It's just a slow motion effect, like the dog getting out of the car in the first sample. It may or may not have been intentional. At first I thought they did it on purpose, but now I'm not so sure, since earlier, with the van driving along in the very first scene, they did a speed-up effect at the scene change. To create the slo-mo effect you just add in a bunch of duplicate frames, keeping the framerate the same. In AviSynth speak it might go (after the unblending):

ChangeFPS(50)
AssumeFPS(25)

It's probably just more evidence (if more is even needed) of Indian DVD production company incompetence.

[Didée]

Clarification: I assume that the "temporal sharpen" thingy is a feature of the normconversion box. Scharfis_brain had shown me a similar, even more nasty case some years before, an NTSC->PAL converted music video.
What (probably) happens is this: when the converter is getting [a b] and creates [a ab b], in fact it creates [a' ab b], with a' = a + f*(a-ab). In words, a fraction of the difference between blended field and original field is added to the first of the two original fields. I could imagine that the procedure is meant to compensate the blending effect a bit, making it a little less obvious for the naked eye. Sort of a temporal kill-out-effect: If something is darker in the blended field, make it brighter in the original field.

In the sample given here, the effect would be reversable with a strength of 15%, like:
bob
merge(selectevery(1,1),0.15)

... but should be applied only to those frames directly before a blended frame. Technically, it should be possible to integrate something like that into Srestore.

[StainlessS]

#coloryuv(off_y=15).tweak(sat=1.15)

I think you are not the originator of this quoted snippet, but would it not be more efficient with similar effect to use:-

coloryuv(off_y=15,cont_u=20,cont_v=20)

(replace whatever for 20 in chroma contrast)
:)

[scharfis_brain]

I was too lazy to look up the correct parameter names of coloryuv. So I used tweak.
Also within this whole chain, an additional tweak won't have a real impact on rendering speed.

[StainlessS]

Thankyou for your reply, I thought I might be missing something. :)

[Didée]

@ StainlessS: you're by all means right. But still, no cigar.

Surely a single ColorYUV is faster than a ColorYUV+Tweak. Made two test functions:
function f1(clip c) {c.coloryuv(off_y=15).tweak(sat=1.15) }
function f2(clip c) {c.coloryuv(off_y=15,cont_u=20,cont_v=20) }
then compared render time when calling either function 10fold-chained on a PAL test clip.

f1: 67 seconds

f2: 44 seconds


Still you don't get the cigar, because
I think you are not the originator of this quoted snippet,
if you come along like *that* ....... *then* I would've used mt_lut from MaskTools!

function f3(clip c) {c.mt_lut(yexpr="x 15 +",uexpr="x 128 - 1.15 * 128 +",vexpr="x 128 - 1.15 * 128 +",Y=3,U=3,V=3) }

f3: 35 seconds

:p

[StainlessS]

@Didée,

I had read all of this thread and noticed the Tweak() thing, when I went to ask about it,
I just back tracked to the last post to mention it, yours. As your knowledge of all things
Avisynth is legendary (that cigar smoke gets everywhere), I figured that there
were worse people I could ask about it, and so I asked. I shall have to have a play with MT_Lut, never
used it before. Have to wonder if chroma gamma correction is implemented in MT_Lut(), I've just done
a fix (in Avisynth Development section) for chroma gamma in ColorYUV(), it has never worked at all.
Been up there a couple of weeks and still not a single comment. Maybe everyone uses MT_Lut()
instead. (I know I should look up MT_lut to check it myself)

Anyway, you can keep your cigar, I'm gonna try and give up smoking one day very soon. :)

[EuropeanMan]

Question from the d2v - parse file. Apparently the first part of the movie is BFF, and it switches over to TFF.
IF I add a line AssumeTFF before bobbing/decimating or restoring, will THAT have an adverse affect?

[Didée]

According to the log, it's just the very 1st GOP which is BFF, from the 2nd GOP up through the end it's all TFF. I'd say that assumeTFF should be okay. Just to be sure, make a quick visual check that a simple AssumeTFF().Bob() does indeed run strictly forward, not forward-backward-forward.


@ StainlessS: It's not my cigar ... I'm just using code that Manao has written. Though, I hope that Manao doesn't smoke (like you & me), because

function f4(clip c) { c.mt_inflate(Y=3,U=3,V=3) }
function f5(clip c) { mt_logic(c,c.removegrain(19),"max",Y=3,U=3,V=3) }

f4: 92 seconds :eek:
f5: 28 seconds :cool:


I generally vote for giving the cigar to IanB. His sporadic code snipplets showing how to do something even-simpler or even-faster are absolutely priceless.

[EuropeanMan]

According to the log, it's just the very 1st GOP which is BFF, from the 2nd GOP up through the end it's all TFF. I'd say that assumeTFF should be okay. Just to be sure, make a quick visual check that a simple AssumeTFF().Bob() does indeed run strictly forward, not forward-backward-forward.



unfortunately, i've already started the whole lag again...I've got about 8-9 hours left. we'll see how that first portion is.

re: GOP's. How do I split this up? As I 'assume' one could theoretically apply the script with AssumeBFF for the first GOP, and then TFF for the 2nd...right?

[zilog jones]

I'd say you can cut into two clips in AviSynth but you will have to deinterlace them individually, as I assume it can't handle one clip with two different field orders. So you could do like:

BFFclip = Trim(blah).AssumeBFF.tdeint #or whatever deinterlacer
TFFclip = Trim(blah).AssumeTFF.tdeint

BFFclip ++ TFFclip

srestore, etc.