Strange source, any advice? Sample included.

[grumpygamer]

Hey everyone!
I've been experimenting with some basic avisynth and everything seems to be working out pretty fine with just this simple line of code:

source.colormatrix(hints=true).telecide(post=0).decimate(cycle=5).crop(4,2,-4,-2)

results have been perfect up to now.

I have encountered a clip I seem to be unable to deinterlace properly no matter what: tried various methods TDeint, Bob and decimate, etc... but none seem to give me a good result.

The problem: most (if not all) the horizontal edges seem to wobble up and down continuously with obvious degradation.

I then took a look at the original clip and played it AS IT IS, and it has the same problem.

Here is a small sample of it:
http://www.mediafire.com/?hyq6bs7n7q39xvh

Could anyone point me in the right direction to solve this problem?

Thanks in advance

- gg

[wonkey_monkey]

I believe you may need to alter how you load your source. It has flags set to induce telecine (sorry, this is probably the wrong jargon!). Loading the file in VirtualDub-MPEG2 without the 'Honor "repeat first field" flags' results in a fully progressive 23.976fps clip.

I used to love that show :)

David

[Didée]

(@david) ... and when displayed in its native 23.976 progessive form, then it wobbles every two frames. ;)

It looks like some telecined footage has been IVTC'ed not by fieldmatching, but rather by a process similar to

bob().selectevery(...)

which would explain why there is bob-shimmer in 24p footage. :rolleyes::)


Something quick & dirty - just to show one possibe strategy.
o = last
e = o.selecteven()
f = o.selectodd()
g = interleave(e.temporalsoften(1,255,255,26,2).merge(e,0.25),
\ f.temporalsoften(1,255,255,26,2).merge(f,0.25) )
min = g.mt_logic(g.selectevery(1,-1),"min").mt_logic(g.selectevery(1,1),"min")
max = g.mt_logic(g.selectevery(1,-1),"max").mt_logic(g.selectevery(1,1),"max")
mm = mt_lutxy(max,min,"x y - 4 - 40 *")
g.mt_merge(o,mm,luma=true,U=3,V=3)
Note that it's not good enough to do a final job with it. It's just to show how to lock-up static areas ... but the static-detection is too poor, there will be artifacts in areas with motion.

Consider it a starting point, which you can refine to your own liking. ;)

[grumpygamer]

Thanks!
I don't understand much, but I'll see if I can fiddle about with it; I am pretty noob at working with avisynth and have no idea whatsoever what the code you wrote does .)
Anyway if there is a simpler solution I'd really appreciate it!
Cheers
- gg

[grumpygamer]

Would Deshaker be a quicker alternative?

[Guest]

Where did you get the source file?

[grumpygamer]

Thats from the original discs, but what's that got to do with it?

[grumpygamer]

Dideè's script works wonders for the wobbleiness, but as expected it introduces numerous artifacts in other zones...
Unfortunately I am not skilled enough to understand what's going on behind the hood, so if some kind soul coul help me with a script that can handle this kind of problems, it would be really great!
- gg

[Dracaena]

Unfortunately there is no perfect solution to bad sources like this, but you could try this.

Index your file with DGIndex, using the force film option.

Then in your script:
SeparateFields()

# You may have to fiddle with the 3rd paramater of SelectRangeEvery,
# the offset, to get the best result. It will be either 0, 1, 2, or 3.
# For the sample you provided, it was 1.
SelectRangeEvery(4, 2, 1)
Weave()

Load_Stdcall_Plugin("C:\Program Files\Video\AviSynth 2.5\plugins\yadif\yadif.dll")
QTGMC(Preset="slower") #or your bob-deinterlacer of choice

AssumeFPS(24000,1001)

It's not perfect but it eliminates the shimmer. This approach will fall apart if the pattern of frames from your sample is not consistent throughout the rest of your source.

The reasoning for this approach is that as Didée said earlier, the source appears to have been created with a bob deinterlacer - you can see the "shimmer" occurring every two frames. The script above takes one field from each frame, which are hopefully one from the "top" and "bottom" of the original source (that your source was created from).
With a bit of luck then the bob-deinterlacer can recontruct (to some extent) the original frame.

I got this result with the above script:
http://www.mediafire.com/file/qe9bpzg9bbm0a99/clip.mkv

[grumpygamer]

Thanks a real bunch for going through all this!
I would have never been able to come up with anything similar!

As I see from the resulting clip, the result is squashed, should I just resize it?

Sorry for my noobiness... I just about understand the basics of all this!

Edit:
Wow man can't thank you enough... this is great... speechless. Only downside is the enormous time it needs to process now, but we cant win'em all!
If you ever come over send me an e-mail I'll buy you a beer!

ReEdit:
the video seems a bit wobbly though now.

ReReEdit:
hmmm, that jerkiness has gone, but the video has somehow lost "fluidity" and moving objects leave "trails" a bit like ghosting. Is this normal?

ReReReEdit:
heh, been fiddling around a lot, global impression is that it looks kind of sluggish (for example when he holds the jacket.. it would never "flap" so slowly)

ReReReReEdit:
forget previous edits.... forgot to FORCE FILM in DGIndex, done that now, results are astonishing.
Thanks again
- gg

[Dracaena]

Sorry for my noobiness... I just about understand the basics of all this!
Think nothing of it, after 4 years of epic lurking (only 6 posts LOL) I still consider myself profoundly noobish. Most of the discussions here make my brain hurt.

As I see from the resulting clip, the result is squashed, should I just resize it?
No need to worry about that squashing, out of habit I changed the aspect ratio when I muxed it - 15/11 for NTSC 4:3.
It wasn't a result of the script.

[grumpygamer]

Hey
yep, found out about the squashiness, tried to encode some samples from the clip, they seemed ok so now I'm encoding the whole VOB.

Question: how do I understand which value goes as third value of the Selectevery statement?
I understand (partially) what selectevery does, but during my fiddling around I didnt notice any particular difference when varying the value from 0 to 3...

Also I tried using bob() and yadif(mode=1) since you wrote "#a bob deninterlacer of your choice" but both the result was horrible!
The only script that does the job (and makes it look really awesome) is your QM thing!

Other question: is there another bobber that could do the job in a faster way?
Other question: now the ending frames look much sharper and really good, even better than the VOBS that didnt have this problem, is there any code to make them look as good?

Cheers again!

- gg

[grumpygamer]

Hmm, I seem to have stumbled upon an episode that would need
SelectRangeEvery(4, 2, 3) for the first 15 mins, SelectRangeEvery(4, 2, 1) for the next 25 mins and once again SelectRangeEvery(4, 2, 3) for the remaining time.

Is there some way to do this?

I read about ApplyRange, is this a good idea?

What I was thinking was a framecounter variable that increases, something like:

mpeg2source("video.d2v",info=3, cpu=6)
colormatrix(hints=true).SeparateFields()

fCount = 0

if (fCount <= 22537) then SelectRangeEvery(4,2,3)
if (fCount > 22537 and fCount <= 56770) then SelectRangeEvery(4,2,1)
if (fCount > 56770) then SelectRangeEvery(4,2,3)
Weave()
QTGMC(Preset="Slower").crop(4,4,-4,-2)
AssumeFPS(24000,1001)

fCount ++

Would this be correct?

[manono]

A=Trim(0,22537).SelectEvery(4, 2, 3)
B=Trim(22538,56770).SelectEvery(4, 2, 1)
C=Trim(56771,0).SelectEvery(4, 2, 3)
A+B+C

You don't really mean SelectRangeEvery, do you? Anyway, if I understand correctly what you're trying to do, that's one way to do it. Check the results carefully before encoding it.

[grumpygamer]

Thanks!
Argh! You are right! What a noob I am!
It is SelectRangeEvery! (Edited my previous post)
Would that change the script above?

Do I have to weave and do the rest afterwards? Like
mpeg2source("video.d2v",info=3, cpu=6)
colormatrix(hints=true).SeparateFields()

A=Trim(0,22537).SelectRangeEvery(4, 2, 3)
B=Trim(22538,56770).SelectRangeEvery(4, 2, 1)
C=Trim(56771,0).SelectRangeEvery(4, 2, 3)
A+B+C

Weave()
QTGMC(Preset="Slower").crop(4,4,-4,-2)
AssumeFPS(24000,1001)

Edit:
No value seems to be correct for the second chunk; tried 1,2 and 3, all give a jittery frame rate. Don't really know what to do now...

[Dracaena]

All you can hope to do is learn about interlaced video, and hope you can identify a pattern in the source material, and know how to deal with it.

Putting SeparateFields() in an empty script will allow you to examine each field individually, which can help once you know what you're looking for.

Given the nature of the sample you provided, I would hazard a guess that the second part of the video is actual interlaced material (rather than progressive material with pulldown).
In that case you would need to index the file again, this time using the "Honor Pulldown Flags" option.

You then would have to reverse the pulldown process for the first and third sections of the video in your script (eg using Pulldown (http://avisynth.org/mediawiki/Pulldown)

Then you'd have to deinterlace the middle section. Maybe something like this, keeping in mind that the frame numbers will have changed since you haven't forced film in DGIndex

mpeg2source("video_Honor_Pulldown.d2v",info=3, cpu=6)
colormatrix(hints=true)
Source1=Last
Source2=Pulldown(x,y)

A=SeparateFields(Source2).Trim(0,22537).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)
C=SeparateFields(Source2).Trim(56771,0).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)

B=Trim(Source1, xxxx, xxxx).QTGMC().ChangeFPS(24000,1001)

A+B+C
crop(4,4,-4,-2)

Of course, all of that assumes that the middle section is normal interlaced video. Which might not be the case.

[grumpygamer]

thanks a lot Dracaena for spending some time of yours to solve my problem.
It seems as though it is actually interlaced, since I tried to deinterlace it and it works
So I'll try your method and tell you how it goes.

Avisynth throws an exception asking what x and y are in Pulldown; where do I get those values?

Thanks!

[Dracaena]

Avisynth throws an exception asking what x and y are in Pulldown; where do I get those values?

Thanks!
You have to figure them out for yourself. Read AVISynth wiki article that I linked, and read this too: http://avisynth.org/mediawiki/Interlacing_and_deinterlacing.

This will help you to find the correct values - choose the two panels that show no combing and have different pictures.
DGDecode_mpeg2source("clip.d2v", cpu=0, info=3)

w4=width/4
h4=height/4

Crop(w4,h4,-w4,-h4)

a=SelectEvery(5,0).Subtitle("0")
b=SelectEvery(5,1).Subtitle("1")
c=SelectEvery(5,2).Subtitle("2")
d=SelectEvery(5,3).Subtitle("3")
e=SelectEvery(5,4).Subtitle("4")

f=StackHorizontal(a,b,c)
g=StackHorizontal(d,e).AddBorders(w4,0,w4,0)

StackVertical(f,g)

[grumpygamer]

Hey Dracaena,
I am trying your script today since I have some time off work, I must say when I saw the VDub screen split up in 5 sections and saw what you meant, I was blown away!
You are yet another genius on these forums!

Back soon with the results

Edit: here are some screenshots. I am posting them because curiously the only chunk where the pattern stays constant is the first.

CHUNK 1 -> CONSTANT PATTERN

Sample A
http://img409.imageshack.us/img409/6942/chunk1samplea.jpg (http://img409.imageshack.us/i/chunk1samplea.jpg/)

Sample B
http://img36.imageshack.us/img36/5552/chunk1sampleb.jpg (http://img36.imageshack.us/i/chunk1sampleb.jpg/)

CHUNK 2 -> VARIABLE PATTERN

Sample A
http://img214.imageshack.us/img214/4026/chunk2samplea.jpg (http://img214.imageshack.us/i/chunk2samplea.jpg/)

Sample B
Suddenly after a scene change the pattern changes... ARGH
http://img97.imageshack.us/img97/3429/chunk2sampleb.jpg (http://img97.imageshack.us/i/chunk2sampleb.jpg/)

CHUNK 3 -> VARIABLE PATTERN

Sample A
http://img704.imageshack.us/img704/4144/chunk3samplea.jpg (http://img704.imageshack.us/i/chunk3samplea.jpg/)

Sample B
http://img97.imageshack.us/img97/3707/chunk3sampleb.jpg (http://img97.imageshack.us/i/chunk3sampleb.jpg/)

Ok so I am unsure on how to proceed now, any suggestions?

I was thinking of

mpeg2source("video.d2v",info=3, cpu=6)
colormatrix(hints=true)
Source1=Last
Source2=Pulldown(0,4) # entire first chunk

Source3=Pulldown(0,2) #first part of chunk 2
Source4=Pulldown(2,3) #second part of chunk 2

Source5=Pulldown(0,3) #first part of chunk 3
Source6=Pulldown(0,1) #second part of chunk 3 (not sure about this cos the image seems to change in all the 3 top frames)

#Not sure on how to go on from here

This is all with DGINDEX on honor pulldown though

Cheers

- gg

[grumpygamer]

I am bumping this thread since I have given my own intuition a shot...
this is the script:

mpeg2source("F:\GREATEST_AMERICAN_HERO_S2_D1\VIDEO_TS\pulldown.d2v",info=3, cpu=6)
colormatrix(hints=true)
Source1=Pulldown(0,4) #chunk 1
Source2=Pulldown(0,2) #chunk 2 (A)
Source3=Pulldown(2,3) #chunk 2 (B)
Source4=Pulldown(0,3) #chunk 3 (A)
Source5=Pulldown(0,1) #chunk 3 (B)


A=SeparateFields(Source1).Trim(0,22552).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)
B=SeparateFields(Source2).Trim(22553,38357).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)
C=SeparateFields(Source3).Trim(38358,56769).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)
D=SeparateFields(Source4).Trim(56770,65757).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)
E=SeparateFields(Source5).Trim(65758,70758).SelectRangeEvery(4, 2, 3).Weave().QTGMC(Preset="Slower").AssumeFPS(24000,1001)

A+B+C+D+E

crop(4,4,-4,-2)

Something must be wrong since the duration is halved... now only 24:35 mins against the original 49:11.
What am I doing wrong?

[grumpygamer]

* BUMP * I am still stuck here after nearly 4 months...