Hard-telecined 29.97fps DVD of NTSC TV with field blending? (+ interlaced resizing?)

[ChiDragon]

This is a US TV show that was never blended in TV airings or any other NTSC release. None of the footage was from a PAL source. The DVD is slightly cropped and resized in both directions, so my theory is that the problem was somehow introduced by doing interlaced resizing.

http://s11.postimg.org/c2b8qwis3/mmpr_154.gif

I have a clean source for this particular episode from another's company's release, but they've only put out 4 episodes so I'm hoping to repair the other 151 in this flawed box set.

Samples:
MMPR-54_LG.m2v (http://chidragon.thedessie.com/Doom9/MMPR-samples/MMPR-54_LG.m2v) (normal pulldown)
MMPR-54_SF.m2v (http://chidragon.thedessie.com/Doom9/MMPR-samples/MMPR-54_SF.m2v) (pulldown + blending)

Here is the script I used to view them side-by-side:
A = MPEG2Source("MMPR-54_LG.d2v").Trim(3,0)
B = MPEG2Source("MMPR-54_SF.d2v")

StackHorizontal(A,B)

Bob()

The pattern is 4 blended fields out of every 10:

LG: AA BB BC CD DD
SF: AA BB [B/C][C/B] [C/D][D/C] DD


So stepping through it in weaved form, the only frames that show crosstalk are those that were interlaced in the original source.

In the pattern notation above, "C" is the only frame that I need to unblend. Ghosts of it exist across 4 fields. Any way?

[johnmeyer]

So stepping through it in weaved form, the only frames that show crosstalk are those that were interlaced in the original source.

In the pattern notation above, "C" is the only frame that I need to unblend. Ghosts of it exist across 4 fields. Any way?I see blended fields not only for the pulldown field, but on some of the fields in the frame that should not have any temporal differences between fields. SRestore does remove the blended pulldown fields, but I don't have a solution for the blends in the remaining fields. Here is the script I used, and it gets rid of over 80% of the problems:

loadPlugin("C:\Program Files\AviSynth 2.5\plugins\Srestore\mt_masktools-25.dll")
Import("C:\Program Files\AviSynth 2.5\plugins\Srestore\srestore.avs")

source=mpeg2source("E:\Documents\Dnload\UNPACK\MMPR-54_SF.d2v")
bob(source,-0.2,0.6)
srestore(frate=23.976, omode=6)

[ChiDragon]

Thanks, but that's just picking one of the four blended fields and trying/failing to use the clean frames for the rest. Is there no way to actually deblend them to get something resembling the original C frame?

Frames 7,8,11,15,1619,20 still exhibit dark halos around her arms and bloomed contrast on them. Probably thanks to the onscreen text confusing things. After it goes away, the problem frames are 1 in 4 as expected: 79,83,87... And then at the angle change it messes up and starts duplicating frames.

IVTC followed by deinterlacing to pick one of the bad fields produces a better result, but again does nothing to address the ghosted aspects:
MPEG2Source("MMPR-54_SF.d2v")

A = Bob(-0.2,0.6).SRestore(frate=23.976, omode=6)
B = DoubleWeave().SelectEvery(10, 0,2,4,7).Trim(0,115)+DoubleWeave().SelectEvery(10, 0,3,6,8).Trim(116,0)
B = B.TDeint(full=false)

StackVertical(A,B)

[ChiDragon]

I suppose it would help to more accurately identify the problem. The artifacts actually look like "burn/dodge" in an image editing application like Photoshop rather than actual blending. Does anyone have an idea for how to reverse that?

[manono]

What's the trick to downloading this thing? All I can do is to play it in my browser.

[ChiDragon]

Shouldn't be any trick. Right-click and Save As or disable whichever browser plugin is providing the ability to play M2V files. (Assuming Chrome or something doesn't have playback support built-in.)

[manono]

Thanks. That did it. I expected clicking on the link would take me to the download.

In the pattern notation above, "C" is the only frame that I need to unblend. Ghosts of it exist across 4 fields. Any way?
Again I don't understand. Are you saying you want it field-matched when possible, and not use a bobbed field for the parts that aren't blended, no matter how good the bobber (QTGMC)? If so, then try this:

Interleave(TFM(field=1), TFM(field=0))
SRestore(omode=4, speed=3)
TDecimate(Mode=2,rate=23.976)

It's partially from a script jagabo came up with on videohelp.com to solve a different problem. I haven't a clue how to get rid of that dark crap around her body on different frames but if, as you say, it exists across four fields, then maybe it's not possible. Maybe the script above improves the problem, maybe it just shifts the dark 'halos' to different frames than the ones you listed.

[ChiDragon]

Thanks for your suggestion. I do want it field-matched for the clean fields yes, but that part isn't giving me problems. It's just the ghosts/burns/halos that I want to find a solution for. I'm not even sure how to simulate the issue to begin thinking about inverting the process.

[ChiDragon]

From Wikipedia:
The Color Dodge blend mode divides the bottom layer by the inverted top layer. ... The operation is not invertible due to possible clipping of highlights. (The clipping happens in the same area as for the Linear Dodge.)
The Color Burn mode divides the inverted bottom layer by the top layer, and then inverts the result. ... The operation is not invertible due to possible clipping of shadows. (The clipping happens in the same area as for the Linear Burn.)

I do wonder whether these operations are invertible in images where the brights and darks aren't clipped...? For now I'm proceeding under the assumption that I have to accept the damage and just try to reduce its visibility.

I've found that leaving the bad frames interlaced and applying Vinverse generally looks better than deinterlacing, as the intensity of the halos are reduced to 50%.

http://109.imagebam.com/download/SLB4opL81te3W7IzmY-4qA/29783/297823546/MMPR-54_stack.jpg

This can still be improved upon, because sometimes the bottom field looks notably better (when the "next" film frame is particularly similar to the "current" film frame and the "previous" film frame particularly differs) and sometimes the top field looks notably better (vice versa).

I thought maybe these can be picked out and swapped in by using YDifferenceFromPrevious and YDifferenceToNext. Do you have any idea how that could be scripted? The numbers don't seem to agree with my eyes.

Bottom field better than Vinverse (http://110.imagebam.com/download/90yK3P6kw8aVwV1P2VQ2Gg/29783/297823539/MMPR-54_bottombetter.jpg)
Top field better than Vinverse (http://110.imagebam.com/download/d9VjTV3RkmffRX8lCmbnqQ/29783/297823554/MMPR-54_topbetter.jpg)
(the numbers for "X" in the screenshots are wrong due to a misplaced command in my initial script)


MPEG2Source("MMPR-54_SF.d2v")

X = DoubleWeave().SelectEvery(10, 0,2,4,7).Trim(0,115)+DoubleWeave().SelectEvery(10, 0,3,6,8).Trim(116,0)

A = X.Vinverse()
B = X.TDeint(full=false)
C = X.TDeint(order=1,full=false)

L = StackVertical(X.Subtitle("X: Residual interlacing").SubDiff(),B.Subtitle("B: Top field (dark halos from prev)").SubDiff())
R = StackVertical(A.Subtitle("A: Vinverse").SubDiff(),C.Subtitle("C: Bottom field (dark halos from next)").SubDiff())
StackHorizontal(L,R)

function SubDiff(clip c)
{
c.ScriptClip("Subtitle(String(YDifferenceFromPrevious),y=20)").ScriptClip("Subtitle(String(YDifferenceToNext),y=40)")
}