Fighting against fieldblends (constant pattern)

[scharfis_brain]

Worried about fieldblends on old or new movies?
There could be a possible solution, if there is a
CONSTANT blending pattern present in your video.

use this function for testing:

function unblendpattern(clip clip, int offset, float fps, bool refitfps, bool debug)
{
a = (debug == true) ? clip.ShowFrameNumber() : clip
b = changefps(trim(a,0,-1),framerate(a)*100).assumefps(framerate(a))
out = (offset == 0) ? changefps(a, fps) : changefps((b.trim(0, -offset) + a), fps)
out = out.trim(int(offset / (framerate(a)/fps)) , 0)
out = (refitfps == true) ? out.assumefps(framerate(a)/2, true).resampleaudio(audiorate(a)) : out
return out
}

Okay, let me explain, how to use this function.

At first, you have to deinterlace your Video with a
Fullframreate-deinterlacer, like dgbob().
xxxsource("yourblendedvideo.xx")
dgbob(order=?)

then, load your avs to virtualdub and scroll to a
position where movement and blends are very good visible.

Now go frame by frame through your video and write down the
Framenumbers of the UNBLENDED (normal) frames

then do a count, how many good frames (if two identical frames
are not blended, do only count one) you have per second of
Video (50 frames). This will be the orignial Framerate of the
source.

your sequence may look like this:

blend : 0,2,4,6,8,9 ,11,13,15,17,19,21,23,25,27,29,31,33,34,36,38,40,42,44,46,48,50,,,,,,,,,,,
normal : 1,3,5,7, ,10,12,14,16,18,20,22,24,26,28,30,32, ,35,37,39,41,43,45,47,49,,,,,,,,,,,


now add my function to your script (I assume 24fps from now on)
xxxsource("yourblendedvideo.xx")
dgbob(order=?)
unblendpattern(0,24,false,true)

now, load your video to VDub again and change the offset
until unblendpattern only returns the normal frames.

If this is done, examine your whole video. if the blending
pattern slowly slides away, just change the fps value a little
bit (it's float).

now, change the parameters for unblend, to remove the framenumbers
debug=false.
If you want, unblendpattern can correct the unblended video itself
to 25fps (half framrate of inputclip). Just set refitfps=true

xxxsource("yourblendedvideo.xx")
dgbob(order=?)
unblendpattern(22,24,true,false)

Finally, to achieve a better deinterlaing result, use kernelbob
instead of dgbob() for deinterlacing.
(You'll get lesser stairstepping)

function kernelbob(clip a, int th)
{
f=a.kerneldeint(order=1, sharp=true, twoway=true, threshold=th)
e=a.separatefields.trim(1,0).weave.kerneldeint(order=0, sharp=true, twoway=true, threshold=th)
interleave(f,e)
}

xxxsource("yourblendedvideo.xx")
#assumetff #use it, if video jerks back and forth
kernelbob(4)
unblendpattern(22,24,true,false)

samples can be found here: http://home.arcor.de/scharfis_brain/blending/

btw. I invented this Idea, because BaronVlad had problems with an
interlaced PAL movie.
First I thought it has been directly converted from 24p to 50i using
blending. But that wasn't the truth!
The broadcaster took the already preogressive 25fps Film and speeded
it up to unbelievable 26fps using blending. I think this has been done
to be able to broadcast 3.5 minutes more advertisement.
German TV sucks!

[Colyptic]

kernelbob is a great filter...thanks. it was the only one that could handle the 2 telecined PAL DVD's I have been working with for almost a week now.

Telecide(order=1, mode=3)
Decimate(6)
-
dgbob()
repal()

both failed miserably on them.

kernelbob(4)
repal()

worked great

just wanted to add a little bit more to importing these avs files and using them in a script for those who dont know (I was recently one of them)

first of all, you have to create an avs with just the function code in it. Then you have to load that avs into your regular script by:

import("%filename%.avs")

also you need to have kerneldeint.dll on your comp and load it in.

then usage of the kernelbob is i believe

kernelbob(4)

feel free to correct me if i'm wrong, its my first experience with this filter, but definitely not my last.

Also would be nice to get it into a dll if its possible, may speed it up a bit.

[crOOk]

@scharfis_brain
Thank you for another great function! It's amazing how much work you put into providing some of the best deinterlacing techniques.

[r0cket]

...just change the fps value a little...

maybe a tip on how to guess this fps value faster?

[scharfis_brain]

just take 100 (or more) fields (or deinterlaced after dgbob / kernelbob) and count the good (non-blended) fields.
If the same picture appears twice or more, just count it once.
If you have, let's say 30 good out of 100 fields, you have a ratio of 1.0 : 0.3 .

this means a framerate (PAL) of 50 : x = 1.0 : 0.3
where x = 15
with NTSC: 59.94 : x = 1.0 : 0.3 -> x = 17.98 (approx. 18 fps)

but this gives only a raw idea of the fps-value. the exact value can only be found through testing the source from 0 to end.
If there is a swimming or jumping offset in your video, you have to separate your video into segments, unblend them each with their fitting offset and re-join then back together after that.

PS: After testing unblendpattern with NTSC-samples, I found out (with help of manono!), that AVIsynth handles the floating-point fps not very well.
This means for you applying a assumefps(29.97) directly after opening the video.
Else unblendpattern will return a error claiming about non-matching framerates.

[Mug Funky]

hmm. i'm at a loss as to what to do with hybrid 24/30 clips that have become interlaced through PAL conversion.

best i can get is about 70% of good frames which isn't much better than the original content.

good function nonetheless.

[scharfis_brain]

est i can get is about 70% of good frames which isn't much better than the original content.
This heavily depends on the type (or settings) of the standards converter (NTSC -> PAL)
some converters are blending ALL the time, meaning that a slight blending is everytime visible.

ANd others are only doing blending if the new temporal PAL-image position lays roughly between two NTSC-images. Else the nearest neighbor is choosen without blending two NTSC-images together.
When converted with such a converter, you'll be able to select 30 frames out of 50 with success!

[encod3r]

Can this be used to sort of go from PAL back to NTSC? I have NTSC material that was converted and put on a PAL DVD. So when I do my code it looks like this:


bad: 00, 03, 05, 08, 10, 13, 15, 18, 20, 23, 25, 28, 30, 33, 35, 38, 40, 43, 45, 48, 50
good: 01,02, 04, 06,07, 09, 11,12, 14, 16,17, 19, 21,22, 24, 26,27, 29, 31,32, 34, 36,37, 39, 41,42, 44, 46,47, 49


Total good frames is 30, no big surprise there I guess, now what do I do?

[scharfis_brain]

are frames 01&02 indentical or do they represent different frames?

[encod3r]

all good frames are unique

[scharfis_brain]

then try using
kernelbob(4)
unblendpattern(x,29.97,false,true)

x is the Offset, which you have to change until unblendpattern chooses only the good frames.

[encod3r]

sample: s.m2v (http://198.78.80.55/lans/s.m2v)

here's a sample of the material, I have used offsets of 0-15 so far and its still picking bad frames.

[scharfis_brain]

for me, this one works:

mpeg2source("s.d2v")
kernelbob(5,0)
unblendpattern(1,30,false,false)

[encod3r]

did you update kernelbob function? two arguments?

[scharfis_brain]

just remove ,0 [ kernelbob(5) ]

and then you can use the kernelbob, I've posted here

its just my personal super-duper-hyper-useless switch, hehe

[encod3r]

import("F:\dvd\unblender.avs")
mpeg2source("C:\Documents and Settings\Administrator\Desktop\s.d2v")
kernelbob(5)
unblendpattern(1,30,false,false)
#DeRainbow(8)
#crop(8,8,704,564)
#BicubicResize(512,384,0,0.5)


I tried this out on the same sample I posted and I can still clearly see ghosted frames in the final output?

[scharfis_brain]

this function cannot remove blends from existing frames. it only can select the "best" matching frames, meaning that blending on some frames is still visible.

The great advantage is after using this function, that you have a video with its original framrate and smoothness.

Please don't expect wonders from it. It is just a stupid frame-selector. Nothing else!

[r0cket]

today there is at least one dvd that have this problem. "THE WALL" DVD PAL. Just give me the best settings to make it "unblended" and i'll be verry happy. and you'll be very appreciated. ;)

[scharfis_brain]

What tha hell?!? Just give me the best settings[...]

Am I a magican?

How shall I do this kind of remote-diagnosis without something to have a look on?

Sample = 5 to 10 secs long non-reencoded part cutted out of the VOB's
use ChopperXP

[r0cket]

Got problems with chopperxp (bug?). Is there any other program to cut vob's?
Anyway, my idea is to create some database with proper settings for such a DVDs

[scharfis_brain]

please have a look to the doom9-downloads section.
http://www.doom9.org/software2.htm#vobtools

there are some tools

[r0cket]

sorry, still trying to split that vob :(
never did this before, so...

also right now i have some problems with my xp sp1, i'm under 98se now.

[r0cket]

once again, sorry for the delay

here it is the_wall_5mb.vob (http://rocket.web1000.com/the_wall_5mb.vob)

P.S. actually, it took me so long because i ripped DVD to one file and chopperxp has some limit on file size (now i know this), so i had to cut the VOB file two times

[scharfis_brain]

*hrmpf* the board has deleted my post.

Ok, again:

Who the hell did this conversion?!?
It is (again) an ugly FILM -> NSTC -> PAL conversion.

I've got it successfully recovered to 24fps with Didée's RESTORE24 :

setmemorymax(256)
import("C:\x\avs-scripts\restore24.avs")
import("C:\x\avs-scripts\tomsbob.avs")

loadplugin("C:\x\tomsmocomp.dll")
loadplugin("C:\x\masktools141.dll")
loadplugin("C:\x\avisynth_c.dll")
loadplugin("C:\x\smartdecimate.dll")
loadplugin("C:\x\mpeg2dec3.dll")

mpeg2source("wall.d2v")
crop(8,76,-8,-76)
assumebff()
restore24()
bicubicresize(704,288) # <-optional

[r0cket]

ok, thanks a lot, scharfis_brain. it works flawless with this 5mb clip, but in the full movie i get frames like this (http://rocket.web1000.com/wall.jpg). how can i avoid them (maybe edit some script)? btw, i use masktools 1.4.9 as i have not found any other version (and this one seems to be the latest).
P.S. i really don't know who did this conversion. not me, honestly :)

[scharfis_brain]

I am currently playing with restore24, too.

But you cannot do anyting against those blendframes at this time.

Just live with them or encode this movie as interlaced MPEG-2

[encod3r]

source: ff.m2v (http://217.8.140.7/~%63r/ff.m2v)

Can you take a look at this blended source and help me with what I can do?

[Inc]

If you provide to sharfis_Brain samples you should generate an avs script which does import the d2v via mpeg2source. This Script you load into Vdub (If you got an AVI you shure can load that one directly in Vdub). In Vdub you choose "full processing mode" in the video tab and apply the following filter using the following settings:

- Filter: "Resize" to half width but keeping the FULL height, for example in NTSC 352x480 or in PAL 352x576 (choose interlaced resizing!!).
- Filter: "Nulltransform". In here you choose cropping. Do crop the black Borders.
Do cut the stream to an area where also a scene change will be included. The whole cutted area should not take more than 20-30 secs.

Now you got in Vdub a stream which only contains the really needed Picture Information Sharfis_Brain does need to analyse.

Do choose in Vdub "Compression" where you choose for example XVID mpeg4 in XVID Settings you choose "interlaced" encoding! (NO matter the interlaced 4:2:0 Issue of Xvid as Scharfis_Brain only needs to see the Fields behavior (blended/not-blended) ... so color doesn't matter)

Choose NO audio as audio is not needed to analyse the Fields but it takes much more MBs and Sharfis_Brain only gots a modem so you can assume what I mean ;-)

Safe that interlaced XVID Avi to disk and do upload that Video Only stream on your web space where it will be offered to Sharfis_Brain.

I hope I did explain it clear. ;-)

Inc.

[r0cket]

ok, but...

Sample = 5 to 10 secs long non-reencoded part cutted out of the VOB's
is this outdated?

[scharfis_brain]

is this outdated?
yes, a little bit...
I've stopped using DSL due to some personal reasons.

but it's okay, I've DLed this m2v completely within the next minutes while browsing this forum , hehe :)

It is just the BEST way to compress pieces of interlaced content for providing to others, incredible described above.

I think, someone should cut out and make his post sticky.

I really hate getting interlaced samples reencoded as progressive.
SO all the DLing was wasted time (& money)...

[scharfis_brain]

Okay, two ways to go on this sample:

1) using repal:

a)speedup from 24,975 to 25 fps
(stutters slightly, because repal drops one frame in 1000 of the PAL-Squence, sound will be slightly 'off')

converttoyuy2(interlaced=true)
kernelbob(5)
repal()
assumefps(25,true)
resampleaudio(48000)

b)inserting 1 frame every 1000 to achieve 25fps
(stutters slightly more, because one dupe is additional created by changefps)
converttoyuy2(interlaced=true)
kernelbob(5)
repal()
changefps(25)

2) using unblendpattern (my preferred choice)
(none of the problems mentioned above, faster, but offset may be hard to find)
converttoyuy2(interlaced=true)
kernelbob(5)
unblendpattern(youroffset,25,false,false)

if the offset is changin, you have to separate the video using trim(x,y) into parts, then applying unblendpattern with different offsets.
after doing this merge them together again

[encod3r]

thanks for the advice, ill try these solutions and choose one that suits me best

[Inc]

At Sharfi

We now should now use SSRC(48000) instead of ResampleAudio(48000) as I have read that this one ends up in a preciser audio rate resampling ?
;)