Srestore inconsistent on bouncy video

[johnmeyer]

I'm helping a collector restore an entire 1960s Kinescope. I don't have access to the original Kinescope film, but the transfer of this film introduced all sorts of up/down movement. I suspect the sprocket holes were damaged or the film was shrinking.

When I use "separatefields()" on the original video, it looks like the cadence should be "easy" to recover:

two good matching fields;
two good matching fields;
one lone blended field;
... repeat ...

I simply want to decimate the one blended field.

Here is a short clip, cut with Womble from the original MPEG-2 TFF transfer:

Test Clip.mpg (http://dl.dropbox.com/u/1561578/test%20clip.mpg)

Srestore seems like the function to use, but it fails sometimes. I'm thinking that perhaps I need to somehow feed it the output of a clip stabilized with Depan, but then recover the original clip, not the stabilized clip (i.e., after recovering the 24p cadence, I then want to stabilize the film.) However, without some sort of stabilization before Srestore, I think it is having problems consistently recognizing and matching fields.

Does this make sense?

Here's the simple script I'm using in step one to recover the original frames (i.e., this is the script that is failing):

my_frame_rate = 24
bob(source,-.2,.6)
output = SRestore(frate=my_frame_rate)

What happens with this script is that sometimes it combines the blended field with one of the two adjacent matching fields, and instead decimates a matching field.

If I can make this step work reliably, and always remove that one blended field, I can do everything else, including reducing the hot spot, denoising, sharpening, and of course removing the up/down jumping.

Thanks!

[Selur]

What's wrong with normal IVTC (TFM().TDecimate()) ?

[johnmeyer]

What's wrong with normal IVTC (TFM().TDecimate()) ?It doesn't work in this case.

[AzraelNewtype]

Try: nnedi3(field=-2).selectevery(10,0,2,4,7). You may want to tweak which four fields you take (these looked good against the test), but if you can be certain that the blends always fall in the same spot you can be sure to avoid them with this method. Using a better bobber like qtgmc might be beneficial for preserving the detail of the matching fields rather than faking it, though it will propagate some of the blend before/after it, but if it's only one blended field in five you can probably find four clean frames to cherrypick.

If the pattern changes though, you'll have to segment and switch which frames of the bob you're taking, and that can get tedious fast. This is kind of a stupid solution, but it should do the trick.

[johnmeyer]

Try: nnedi3(field=-2).selectevery(10,0,2,4,7)
...

If the pattern changes though, you'll have to segment and switch which frames of the bob you're taking, and that can get tedious fast. This is kind of a stupid solution, but it should do the trick.Thanks for the idea. I tried this, and it didn't work for the reason you state at the end of the post: while the pattern is relatively consistent, it is not perfectly consistent, and once the pattern changes in the slightest, nnedi3 starts choosing the wrong fields.

I'm going to go back to TFM again (as was suggested) and see if I can't get it to look at a long string (maybe four or five cycles) of fields and figure out how to get it to decimate those fields which have the worst matches with the two adjacent same parity fields.

There has got to be a way to do this ...

Thanks!

[manono]

How about:

Yadif(Mode=1)#or your favorite bobber
CDeblend()#part of the SRestore package
BlendBob()
TDecimate()

[johnmeyer]

How about:

Yadif(Mode=1)#or your favorite bobber
CDeblend()#part of the SRestore package
BlendBob()
TDecimate()Thanks for the suggestion. I hadn't looked at CDeblend().

Unfortunately, this actually makes everything much worse.

First, I had to use Bob() because Yadif refuses to load into my 2.60 version of AVISynth. However, I don't think this is the reason for it not working.

To debug, I commented out the BlendBob() and TDecimate, and compared the output of CDeblend() to the bobbed source. It missed every single chance to replace the field (or "frame," once it is bobbed) that doesn't match either of its neighbors. By comparison, this code:

bob(source,-0.2,0.6)
output=srestore(frate=24, omode=4)


works correctly about 80% of the time. I've done enough of this to know that nothing works perfectly on 100% of all frames, but I was hoping for something that would catch 95-99% of the frames that don't match their neighbors (i.e., recover the progressive cadence).

I've spent several hours now, tweaking all the SRestore parameters, as well as trying something similar using TFM, but I'm not making much progress.

One thing I did since my initial post was to first stabilize the footage with Deshaker, to remove most of the up/down motion, and then feed that into SRestore. This helps quite a bit, but doesn't quite get the job done.

If I can get this to work, my idea is to feed this into the conditional script we discussed last spring:

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?p=1510780#post1510780)

and inserting a MVTools2-synthesized frame between the two frames in a group that have the largest temporal gap), and insert frames in the video that results from the elimination of the bad fields and subsequent decimation, and take it back up to 60i, thus completing the Kinescope film to video conversion.

[Didée]

This statement in post#1 irritates me:
What happens with this script is that sometimes it combines the blended field with one of the two adjacent matching fields, and instead decimates a matching field.
I don't see. bob() is purely spatial. Srestore does not do any temporal processing, Srestore only acts as a decimator.
So, where shall "combines blended field with an adjacent field" come from? Such combination is not possible in that little script.

Also, is the sample suited to show up the problem? I see the footage is problematic, but running the given Srestore script, I don't spot any false decisions made by Srestore. Am I not looking close enough, or does the sample not show the problem?

[manono]

Unfortunately, this actually makes everything much worse.

Using the script I posted, with the exception of a single scene change which was blended, every single frame was clean and unblended. Every one. All I have to go on is the sample. If there are problems, post a sample that shows them.

And what makes you think the output should be 24fps? The best unblending results are for the correct framerate.

Edit: When I suggested going the CDeblend/BlendBob/TDecimate route it was from reading your earlier replies. After reading what Didée had to say I went back and tried the standard:

Yadif(Mode=1)
SRestore(Frate=23.976)

and it works equally well (except for using bobbed fields as frames, rather than field-matching) - the only blended frame is at the scene change. There's something very wrong at your end and not with the filters suggested.

[johnmeyer]

Gees, guys, don't jump on me!! I'm not complaining ... I've posted many times before and have great respect for everyone here.

Sigh ...

I'll go back on work on this some more when I get up tomorrow morning.

[AzraelNewtype]

Thanks for the idea. I tried this, and it didn't work for the reason you state at the end of the post: while the pattern is relatively consistent, it is not perfectly consistent, and once the pattern changes in the slightest, nnedi3 starts choosing the wrong fields.


Impossible. nnedi3 in this mode is turning every field into a frame, all the choosing is the domain of selectevery, and that's much easier to fix, just time consuming. But if everyone else is saying conventional methods are actually working, don't do this either way.

[johnmeyer]

Using the script I posted, with the exception of a single scene change which was blended, every single frame was clean and unblended. Every one. All I have to go on is the sample. If there are problems, post a sample that shows them.

And what makes you think the output should be 24fps? The best unblending results are for the correct framerate.

<snip>

There's something very wrong at your end and not with the filters suggested.Ah yes ... thank you for this. When I woke up this morning and re-read your post, this part of it stood out: "All I have to go on is the sample ... There's something very wrong at your end and not with the filters suggested.".

Something wrong on my end?? No!!

Well, unfortunately, the answer is yes: as everyone surmised, I screwed up. I had done a lot of work in the day since I posted that sample, and I was now working on something to which I applied other processing. When I went all the way back to the original (the full version of the clip I posted), the CDeblend worked exactly as I had hoped.

The SRestore would also probably work. I was using it in the non-decimation mode (omode<=5), followed by TDecimate. I did this to get more insight into what was happening, and to get more control, if needed, over the decimation. This is probably more or less equivalent to CDeblend. However, since the CDeblend way of doing it works, I'm sticking with that.

Thank you!

One final note, in answer to your question about why I think the output should be 24 fps. This is 16mm Kinescope film. As you probably know, Kinescope was the process for preserving 29.97 NTSC video and was done by using a heavily modified 16m 24 fps camera, pointed at a specially-designed TV monitor. The process is too complex to describe in this post, and there were quite a few variations on how it was done. However, the film that resulted from this definitely ran at the standard 24 fps rate (although there were a few attempts at using a 30 fps film camera, for obvious reasons, but those higher-speed Kinescope films are rare).

So, that's why I'm pretty certain that the final rate should be 24 fps (adjusted to 23.976, as needed).

[manono]

I'm glad you got it sorted out
So, that's why I'm pretty certain that the final rate should be 24 fps (adjusted to 23.976, as needed).

I wouldn't think any encoders would be capable of converting from 24fps to 29.97fps, especially ones as poor as the one that created this video with its blended fields. More likely, I think, is that the drop frame flag was set to bring it down from 30fps back to 29.97fps. If true, that would also mean the true 'base' framerate for this is 23.976fps. Not that one frame in 1001 matters much in the larger scheme of things.

Movies (as shown in a movie theater) are also 24fps. With 3:2 pulldown applied they go to 30fps. It's the drop frame flag that brings them back down to 29.97fps

[johnmeyer]

I thought I should post the real reason I was not getting the right results: SetMTMode.

I neglected to mention in my OP that I was using SetMTMode. I am well aware that some processes can become unsynced in time when this method of multi-threading is used, but in my (limited) experience, it is usually really obvious that something is wrong. In this case, however, I kept getting "matches" and field combinations that were off by just one field, but only some times. Sometimes I would run the script and it would work, and then run it again, and it wouldn't.

When I reported earlier that I thought I had it working, I thought it was working, because the script processed one or two test clips without a problem. However, when I processed the entire 90 minute football game, things didn't work.

As soon as I eliminated the multi-threading, everything worked just perfectly.

I just thought I should leave a record behind in case some other person has a similar problem.