Urgent! Deinterlacing "Enterprise Season 2" in Germany (SAT1)

[Didée]

Hey folks,

my blood pressure is beyond any measurement!
Season 1 of (the new) Enterprise series was put on air perfecly deinterlaced by SAT1. Well, almost.

Now, on Sat Sep 06, they started season 2 ...
... blend-decimated from telecined NTSC, as it seems!
I just returned on Monday to have a look on Saturday's remote capture, and then I saw - that! Oh Lord!

Has anyone already managed to get this one de'combed in a useful way???
My experience in deinterlacing is far from perfect - PAL user, see...
Sadly, my time is limited, and I can not park too much stuff on my capture HDs. Thus, I need and search help.

What I tried so far:

telecide(order=1,triple=true,post=2,vthresh=45) # decomb 5 - last beta

Most Blends are gone, but the result is unbearable jerky.

telecide(post=false).telecide(reverse=true,post=true) # good old decomb 4

Little better, but still unbearable.

Tried to find a pattern. In panning scenes I found
(perhaps I miscounted a little, but you get the idea)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
A B C D E eF G gH hI iJ jK kL lM M nO O P Q S T U V W X Y
a bC c dE e f g h i j k l m n o p q rS t tU uV vW wX x yZ
(This counting seems somehow off-set, yes. I just started to count somewhere).
However, the phase/offset (?) of this pattern changes very quick - if I get a scene right with separatefields+selectevery, just a few seconds later everything goes noodles again :(

Then, I remembered "deblend", but couldn't find the readme...
Nevertheless, the following

TomsBob().deblend(2,2).decimate(2)

is getting close. BUT, one time every second, there is a big jerk, almost like a dup frame followed by one frame missing. Probably the fields R-r on frames 16-17 in above pattern.

Please, experts! Has anyone already had more success? My abilities in this area are simply not good enough :(
I converted Season 1 with quite satisfacting quality - now I don't want to make some "avarage(ing) crap" out of Season 2.
And my HDs are crying "Too much, cant take no more, too much ..."

Everything appreciated

- Didée

[Si]

Have you tried my PFR filter?

Could you post a clip to test?

regards
Simon

[Didée]

Well, after having come over the first anger, I calmed down a little (but still I'd like to throw a bomb at some company [joke!])

siwalters: no, didn't try PFR yet - isn't it dealing with phase shifting only? Here, we need to work a way around lots of blended fields, because ...

... it simply is blend-decimated by a converter box, it seems. I looked closer at the pattern, and it is clearly "12 blends in top, 12 blends in bottom", with a hickup in the middle, to blend 60 telecined fields down to 50 fields for PAL.

BUT, this I don't understand: the pattern indeed shifts around very quickly, and not even at scene cuts only!
E.g., there's one scene of about 20 or 30 seconds without any cut - the camera is constantly travelling through a room. No break in-between. If I lock to the pattern at the start of the scene, I'm out of pattern already after ~ 4 or 5 seconds. Locking again, again loosing pattern after some seconds. It's quasi impossible to select the good fields manually, unless investing a whole week for editing one episode (and SAT1 is airing *two* episodes per week).
How can it be that the pattern resulting from a converter box can change over time? Even if someone is braindead enough to edit the stream after *raping* it to PAL, I'd expect this to happen at scene cuts only, if at all!

Furthermore, with deblend or unblend, it seems I can only catch at best 2/3 of the blended fields, no matter how I set the thresholds.
I'm condsidering to simply bob it to 50 fps, and let it be ...

I'd happily provide samples from the capture - but where to? The capture is 720x432 VBLE, thus quite big filesizes. Anyone a little FTP space to upload some hundred MBs? I've not even a homepage or anything ... And I'm on broadband only Wednesday(today), Thursday, and then again on Monday, Wednesday etc - the other days are 56k analog :o

- Didée

[mf]

Just encode some samples encoded in interlaced XviD at constant quant 2-3-2/150/100/10 (meaning constant quality 100, with I-frame quant capped at 2-2, p-frame quant capped at 3-3, and b-frame settings at count 2, ratio 150, offset 100, threshold 10). That should provide adequate quality for just trying some deinterlacing on.

[Didée]

Will do.
Would it be clever to encode after 'SeparateFields()'? Should compress better, and could be simply restored by AviSource().weave(). Pity that 'interlaced' fieldbased encoding currently doesn't work correctly with XviD's Bframes...

- Didée

[mf]

Hm. Yeah. :D

[Si]

Sorry didn't read your post fully.

I was just wondering how you know when a blended field comes from a mixture of it and an adjacent but spatially differnent one

0 1 2
A B C
a bC c


why do you think C and not c ?

regards
Simon

[Didée]

That's maybe a fault, and should read B\bc/C instead of B\bC/C. But perhaps it's not all wrong, because some of the source fields that should already be blend-free look a little strange.

However, this was not so important to me when looking for the pattern, because I want to throw out all blended fields anyway ... no matter what exactly was blended.

One thing I didn't think of earlier:
Is the pattern supposed to shift every few seconds, because of 29.97 !== 30 [fps] ? (I'm really not too familiar with all this NTSC<->PAL conversion stuff)
If so, in what interval should the shift occur?

The good part is, since SAT1 airs the show with "overlap" before and after the comercial breaks, it is easily possible to cut out the commercials frame-accurate.

Ah, upload finished.

sample1 (5.2 MB) (http://www.spselectronic.de/ftp/temp/test1.zip)
sample2 (6.6 MB) (http://www.spselectronic.de/ftp/temp/test2.zip)
sample3 (4.6 MB) (http://www.spselectronic.de/ftp/temp/test3.zip)
ridiculous (1.2 MB) (http://www.spselectronic.de/ftp/temp/no_test.zip)

Sample1, 2 & 3 should be adequate to test deinterlacing. I mostly looked in samle2. I did them exacly like mf proposed. The naming is according to the used "trim()" numbers - so looking for long-distance pattern shift is possible.
"Ridiculous" is just to show how discouraging capturing in German television sometimes may be :devil:

- Didée

[Si]

Suggestion but not tested or tried :(

How about using some of the Conditional filtering options.

Make 2 clips - one with bottom field synthesised from the top field and one with top synthesised from the bottom.

Then try comparing them with the original and each other to see which is the "cleanest".

If the 1st clip matches best with the whole frame then I'd think it was more likely to contain the blends and vice versa.

Might be worth trying somthing like this anyway :)

regards

Simon

[Didée]

Ha!

How about using some of the Conditional filtering options.
That's where I am the last few days :)

Actually, I am ScriptClip'ing my own deblend-function, since I'm not too lucky with the behaviour of the two de- & unblend plugins. While my idea is working not so bad, it is pretty difficult (for me) to get it done correctly with the conditional functions (brain is knotting).

Simon, your approach is so much simpler :o

The really genious ideas are the simple ones ... I'm always thinking too complicated.

Now I'd like to know why the following line from within a ScriptClip command:(condition1 && condition2) ? condition3 ? out3 : out2 : out1returns out1 correctly, but when I expect either out2 or out3, it's always out3, never out2 ??

ScriptClip'ing :rolleyes:

- Didée

[Didée]

Guys, I'm on the way to something!

:D

First, I need LOTS of sleep now ... forgot that completely.
and the mess of what I've hacked together must be cleaned, too.

Stay tuned!

- Didée [/dancing around his chair like crazy]

[Wilbert]

(condition1 && condition2) ? (condition3 ? out3 : out2) : out1
In other words, if cond1 and cond2 are both true, cond3 is false, it doesn't return out2?

[Didée]

Yes, you got it right. Yes, the PARANTHESES :rolleyes:
That was clear to me in the meantime ;) - sorry.

But I run into serious problems if it's like that:
global #*all variables*

function out1 (clip c) {
(conditionX && conditionY) ? (conditionZ ? outX : outY) : outZ
global var1 = something
return last }

# ^ similar functions out2 & out3

scriptclip(output, "(condition1 && condition2) ? (condition3 ? out3 : out2) : out1")something like above: when entering the third or fourth nested level, AviSynth borks out with missing "}" or expected "," or ":", and the error is spotted in completely unrelated sections of the script.
I only want to count and reset variables while scriptclip is travelling, but I simply can't do it ?! :confused:
And that's what I need to make the stream 99.99% clean, not only 99% as for now.

- Didée

[Wilbert]

Could you provide the full script?

[Didée]

The script is here. (http://forum.doom9.org/showthread.php?s=&threadid=61792)

Slowly, I'm getting into the syntax ;)