KernelDeint and identical (dropped) frames in a video

[iparout]

Hi.

I captured a clip off VHS and I'm now trying to de-interlace it. I found that Kerneldeint(mode=1) yields the best results IMO, so I decided to use it.

However, since the clip has dropped frames, VDub handles these frames by repeating the previous frame and as a result, the video has many areas where two consequent frames are identical.

Since Kerneldeint uses motion detection to deinterlace, it cannot deinterlace those areas and the result given is this :

http://users.panafonet.gr/iparout/frame.png

I know that this only concerns the areas where dropped frames used to be (which are not TOO many), however it is quite disturbing to see those kind of artifacts in the final video. The only way to make those artifacts dissapear, is to use threshold=0 on the Kerneldeint filter, however this produces a more grainy picture which I do not want.

What can I do to get rid of those artifacts ?

The only thing I thought of is to use the KernelDeint(mode=1) filter twice. It does get rid of those artifacts and doesn't produce grainy picture, however the encoding takes twice as much, which is much more than I'm willing to sacrifice.

Thanks in advance for any help.

[neuron2]

This doesn't sound right. If your thesis were true, then KD could not deinterlace transitions to or from static scenes. Please post a small unprocessed source clip that can be used to duplicate the issue.

[iparout]

There you go.

http://rapidshare.de/files/6482218/test.avi.html

Frame 2 (in VDub), is broken in the raw data clip. This clip is the huffy converted one, so frame 2 is a dublicate of frame 1.

Now use the following script :

# PLUGINS
LoadPlugin("c:\PixieDust\LoadPluginEx.dll")
LoadPlugin("E:\GORDIA~1\Msharpen.dll")
LoadPlugin("c:\PixieDust\dustv5.dll")
LoadPlugin("E:\GORDIA~1\mpeg2dec3.dll")
LoadPlugin("E:\GORDIA~1\Tdeint\tdeint.dll")
LoadPlugin("E:\GORDIA~1\undot.dll")
LoadPlugin("E:\GORDIA~1\decomb.dll")
LoadPlugin("E:\GORDIA~1\KernelDeInt.dll")
#LoadPlugin("E:\GORDIA~1\dgbob.dll")
#LoadPlugin("E:\GORDIA~1\Convolution3d.dll")
#LoadPlugin("E:\GORDIA~1\FluxSmooth.dll")
#LoadPlugin("E:\GORDIA~1\TomsMoComp.dll")
#LoadPlugin("E:\GORDIA~1\VSFilter.dll")
#LoadPlugin("E:\GORDIA~1\SimpleResize.dll")

Avisource("f:\test.avi")

# DEINTERLACING (2)
KernelDeint(order=1)

#CROPPING
Crop(18,12,-14,-12)

#DENOISE
PixieDust(5)

#SHARPEN
MSharpen()

#RESIZING
LanczosResize(640,480)

Open the avs through VDub and see that frame 2 is still interlaced.

Use KernelDeint(order=1,threshold=0)
and the frame is deinterlaced, however the clip becomes less smooth (this is not visible in the clip I posted, but you can see it in the following screenshots - look at the grass in the lower left corner).

http://users.panafonet.gr/iparout/threshold-0.png
http://users.panafonet.gr/iparout/threshold-default.png


Now use KernelDeint(order=1)
KernelDeint(order=1) and the frame is deinterlaced again, not grainy, but takes twice the time to process.

[neuron2]

If you do AssumeTFF().SeparateFields() you'll see that the fields are sheared. That is the cause of the residual combing.

[iparout]

Hmmm... What does this mean and how can I fix it ? (sorry for being such a noob)... :o

[foxyshadis]

It means that it was a bad capture (or remote possibility a bad broadcast), where it couldn't read and capture the full frame in time, so (depending on the capture software) it fills it in with a previous field or the one it's capturing right then. In your case the former, and not only that, but the second frame looks like a drop that was replaced by the third, which is sheared, so it's a big mess. The only way to fix it is to recapture the scene or cut the frames (and corresponding audio segment) out.

[scharfis_brain]

@neuron2: framedrops are a real pain, when it comes to interlac-processing:

try to simultae a framedrop like this:

loadplugin("D:\x\leakkerneldeint.dll")
avisource("doom9.avi")
assumetff()

#drop frame 10!
deleteframe(10)
duplicateframe(10)

# try deinterlacing it!
leakkerneldeint(order=1)

now you can see, that the deinterlacer fails on frame 10!

somwhere on my HDD there is a script that is able to replace dropped frames automatically by two motion-interpolated fields for interlaced stuff and one motion interpolated frame for progressive stuff. I hope, I can find it again.
Of course, it only works with orphaned drops. Two ore more chained dropped frames won't be fixed with it.

[IanB]

In true interlaced video (yours) each field occurs 1/60th (1/50th pal) of a second apart. When a frame is dropped you loose 2 fields, with the duplicated frame the fields do a 2 step shuffle timewise, thus you get a transient apparent reversal of motion which confuses the deinterlacer.Field count :- 0 1 2 3 2 3 6 7 8 9
Frame count :- 0 1 1 3 4
/a\ /b\At (a) the motion steps back 1/60th second.
At (b) the motion jumps forwards 3/60th second.

[scharfis_brain]

Yep. exactly.

Luckily, dropframes are 100% bit identical to their predecessor.
so with contitionalfilter, a YDifferencetonext() < 0.00001 will identify all drops without false positives.

so one can replace them before applying deinterlace.

[neuron2]

You can fix it manually using FreezeFrame() after KernelDeint has been applied. The jerk may be only slightly less objectionable than the comb.

Or you can use a deinterlacer that is not sensitive to the field order, such as FieldDeinterlace.

>so one can replace them before applying deinterlace

Haven't you already shown that it won't help to do it before deinterlacing?

[scharfis_brain]

I meant, replacing the drops with blends of their neighbers or with motion interpolated frames/fields....

This will avoid deinterlacing problems, will make motion smoother, will be completely automated. No need for manual setting up freezeframe...

[IanB]

I get quite satisfactory results smart bobbing the 2 fields adjacent to the drop frame and using each bobbed field to complete the missing frame. Thus timewise you end up withField count :- 0 1 2 3 3 6 6 7 8 9

[iparout]

Wow, thanks everyone for their help on this matter.

As far as I can understand, the solution scharfis posted would be better than the FreezeFrame() option, so, scharfis, if you can find the script, could you please paste it here ? I would greatly appreciate it. (BTW, there are no "dual dropped frames" on the clip. There are always a few "normal" frames between 2 dropped frames .)

neuron, on the test clip I posted, do you mean, something like :

LeakKernelDeint(order=1)

freezeframe(1,2,1)

#CROPPING
Crop(18,12,-14,-12)

#DENOISE
PixieDust(5)

#RESIZING
LanczosResize(640,480)

Cause I just tried that, it does work (although I don't know how a larger clip with more dropped frames will look), however, the dropped frames are many and I cannot manually write them down and replace them one by one. Perhaps it can be combined with the YDifferencetonext() < 0.00001 idea... It's a start though...

Anxiously awaiting some more feedback...

Thanks again... ;)

EDIT : Ian, just saw you post. What you are doing is actually what scharfis was suggesting, right ? Could you please copy the script here ?

[neuron2]

I would fix the dropped frame problem. :)

Failing that, try FieldDeinterlace().

[iparout]

I would fix the dropped frame problem. :)

Failing that, try FieldDeinterlace().

Well, of course preventing dropped frames from happening would be the ideal solution here, however there's nothing I can do about it (at least I haven't been doing anything wrong for the dropped frames to occur) and I don't mind a few dropped frames in my video since you can't tell the difference, so I've been trying to cope with the problem rather than prevent it... ;)

And I know that FieldDeinterlace fixes the problem, however the image gets blurry as hell, nothing even close to the quality KernelDeinterlace produces.

[scharfis_brain]

I found the script, but it is 16 months old, bloaty and thus won't work anymore.
It will need to be rewritten completely.
But I will be away from my computer the next seven days, so you will have to wait.

[iparout]

Yes, I understand. Well, I believe Ian found a way to do what you are suggesting, so hopefully he will be kind anough to paste his script here.

Otherwise, of course I'll wait.

Thanks a lot.

BTW, by searching, I created the following script, which is supposed to replace each dropped frame with the previous one :

Avisource("f:\test.avi")

last=Avisource("f:\test.avi")
ConvertToYV12
ConditionalFilter(last, last, last.trim(1,0), "YDifferenceToNext()", "<", "0.00001", true)
converttoyuy2

# DEINTERLACING (2)
#TDeint(order=1)

#CROPPING
Crop(18,12,-14,-12)

#DENOISE
PixieDust(5)

#RESIZING
LanczosResize(640,480)

When loading the avs into VDubthough, I get an "Evaluate result=true" for the dropped frames, however it doesn't change them :confused: Now, I know I am doing something wrong since it's the first time I'm using Conditionalfilter, but at least this means that I have found a way to spot the dropped frames, although I don't know what to do about them.

[neuron2]

And I know that FieldDeinterlace fixes the problem, however the image gets blurry as hell, nothing even close to the quality KernelDeinterlace produces. That's a gross exaggeration, IMHO.

[iparout]

That's a gross exaggeration, IMHO.

Well, to each his own I guess. IMO, KernelDeinterlace is much much better than fielddeinterlace. I can post screenshots if you want, but I don't think it'll change anyone's opinion... ;)

BTW, a solution I found to my problem is this :


last=Avisource("f:\test.avi")

ConvertToYV12
ConditionalFilter(last.LeakKernelDeint(order=1), last, "YDifferenceToNext()", "<", "0.00001")
converttoyuy2

# DEINTERLACING (2)
#TDeint(order=1)
LeakKernelDeint(order=1)

#CROPPING
Crop(18,12,-14,-12)

#DENOISE
PixieDust(5)

#RESIZING
LanczosResize(640,480)

1) This way, the parts which have the problem get a second pass of KernelDeinterlace, fixing them, without having to apply the filter to the whole movie for the second time. At least that's what I think... Am I right ?

2) BTW, if anyone has any suggestions to make my script better, feel free to post away. For instance, I would like to change the colorspace only once (or not at all if possible), but ConditionalFilter only works in YV12, whereas PixieDust crashes VirtualDub when working in YV12, that's why I change from YUY2 to YV12 and then back to YUY2, letting XVid do the final colorspace change to YV12 during encoding. Any ideas would be much appreciated.

3) Does anyone know how to modify the script in order for the "faulty frame" to be replaced by its previous one, and not use KernelDeinterlace for the second time on it ?

Thanks...

[scharfis_brain]

I would like to change the colorspace only once (or not at all if possible), but ConditionalFilter only works in YV12

only convert the testclip to yv12. leave the other clips untouched!

[iparout]

I would like to change the colorspace only once (or not at all if possible), but ConditionalFilter only works in YV12

only convert the testclip to yv12. leave the other clips untouched!

Huh ? I haven't configured the testclip. I left it as default. "last.LeakKernelDeint(order=1)" is source 1 and "last" is source 2 (in the Conditional Filter command syntax).

I messed up didn't I ? It seems to work fine though...

Please post the modified script if you want, so that I can understand what you are saying.

[IanB]

# Replace dropped frame F with fields adjacent to drop
Function MaskInterlacedDropFrame(clip Clip, int F) {
Clip
# Generate a full frame per field
# Choose favourite smart bobber here!
# Set parameters appropriate to source material
LeakKernelBob(order=0, threshold=10, sharp=false, twoway=false)
# DGBob(order=0, mode=1, thresh=12, ap=true)

# Split fields and generated anti-fields
SeparateFields()
# Build a frame from the anti-fields
DoubleWeave().SelectEvery(4, 3)

# Paste in replacement frame
Clip.Trim(0, F-1)+Trim(F, -1)+Clip.Trim(F+1, 0)
# Leave audio unmolested
Return AudioDubEx(Clip)
}

[iparout]

Thanks for your input.

However, how is it possible to make avisynth automatically identify frame F ?

And furthermore, I believe that in your script Clip=test.avi, right ?

So I modified it accordingly and nothing happens. I'm sure I'm doing something wrong

Avisource("f:\test.avi")
Clip=Avisource("f:\test.avi")

# Replace dropped frame F with fields adjacent to drop
Function MaskInterlacedDropFrame(clip Clip, int F) {
Clip
# Generate a full frame per field
# Choose favourite smart bobber here!
# Set parameters appropriate to source material
LeakKernelBob(order=1, threshold=10, sharp=false, twoway=false)
# DGBob(order=0, mode=1, thresh=12, ap=true)

# Split fields and generated anti-fields
SeparateFields()
# Build a frame from the anti-fields
DoubleWeave().SelectEvery(4, 3)

# Paste in replacement frame
Clip.Trim(0, F-1)+Trim(F, -1)+Clip.Trim(F+1, 0)
# Leave audio unmolested
Return AudioDubEx(Clip)
}

# DEINTERLACING (2)
#TDeint(order=1)
LeakKernelDeint(order=1)

#CROPPING
Crop(18,12,-14,-12)

#DENOISE
PixieDust(5)

#RESIZING
LanczosResize(640,480)

[IanB]

The code I presented is a callable function. So you would directly use it as :-AviSource("...")
MaskInterlacedDropFrame(123)
MaskInterlacedDropFrame(456)
....For automatic detection you could mix the various ideas in this thread and build something like from scharfi's cond... and my funcAviSource("...")
Vid=last
LeakKernelBob(order=0, threshold=10, sharp=false, twoway=false)
DoubleWeave().SelectEvery(4, 3)
ConditionalFilter(Vid, Vid, Last, "YDifferenceFromPrevious()", ">", "0.001")
...The logic is not quite right at the boundaries but I am sure you can progress the idea as required...