Restoring ABBCCDDEEEE frames using srestore

[a1s2d3f4]

I am having trouble restoring a clip of a 1959 European black and white capture (a French studio).
I thought I would be getting an interlaced 25fps footage that contained in it 24 fps film somehow.
Instead, after I loaded this 25 fps clip using DirectShowSource and applied Yadif(1,1), getting a 50fps clip, this is what I got:
# 0151-A; 0152-B; 0153-B; 0154-C; 0155-C
# 0156-D; 0157-E; 0158-E; 0159-E; 0160-E
# 0161-A; 0162-B; 0163-B; 0164-C; 0165-C
# 0166-D; 0167-E; 0168-E; 0169-E; 0170-E

In other words 5 real frames over the span of 10 (deinterlaced) frames.
I don't think I have ever seen this before, but then I haven't dealt with French-derived clips before.
So, clearly, I need to get these original 25 frames out of there somehow.
Usually, I would use srestore(frate=25) and it would do it nicely, but this time I think I need to help it out more, and I don't know how to give it that help. Using srestore(frate=25) by itself causes it to skip over the frames it shouldn't skip.

Would anyone know what I should do here?

Thanks.

[poisondeathray]

#1) Don't use directshowsource . Not frame accurate with non linear seeks. Especially with srestore. Some of the problems and patterns may change just by using it

[hello_hello]

Is it definitely a job for Srestore? Could it be some type of pulldown? You probably should upload a sample.

[Midzuki]

#1) Don't use directshowsource . Not frame accurate with non linear seeks.

DirectShowSource + .GRF files + LAV Filters has been frame-accurate with non-linear seeks.

[johnmeyer]

You need to evaluate the clip before you apply deinterlacing. And, you need to look at the original clip field-by-field, so use separatefields(), and nothing else.

Otherwise, you have no idea what is really going on.

[manono]

As hello_hello suggested, upload an untouched sample. No one should have to guess.

[poisondeathray]

DirectShowSource + .GRF files + LAV Filters has been frame-accurate with non-linear seeks.

"has been" as in maybe once in a while :D

It's just not reliable. Green or grey frames, or missing frames, or added frames, or wrong seek, and/or mixed up frames. Regardless of configured decoder, even SW decoding / HW off.

I can post a video screen capture illustrating the common problems if you like

[Midzuki]

"has been" as in maybe once in a while :D

It's just not reliable. Green or grey frames, or missing frames, or added frames, or wrong seek, and/or mixed up frames.

Regardless of configured decoder, even SW decoding / HW off.

Well, for me at least, it has worked nicely where both FFMS2 and L-SmashWorks have failed "epically" :D

I can post a video screen capture illustrating the common problems if you like

No, thanks, I trust you :)
EXCEPT IF those "common problems" also happen to you when the playback graphs are built manually, instead of relying on the frequently-random "wisdom" of Windows itself.

[poisondeathray]

Well, for me at least, it has worked nicely where both FFMS2 and L-SmashWorks have failed "epically" :D



It's true, there are cases where they can fail too, some types of transport streams for example



No, thanks, I trust you :)
EXCEPT IF those "common problems" also happen to you when the playback graphs are built manually, instead of relying on the frequently-random "wisdom" of Windows itself.


No this DEFINITELY occurs with manual grf. 100% certain.

With manual grf , at least you can specify exactly which decoder, but there are still serious problems directshowsource, manual or automatic, any decoder, or dss2, cache or no cache.

When your framecount is different with a manual grf, this impacts the metrics when you're trying to decimate or srestore, even without seeking! These are fairly common , occurrences , even youtube videos . It's not some rare insignificant event I'm talking about here. In fact I just double checked since you seemed so adamant. Guess what? My very first video had an extra gray frame at the end... Youtube deadpool trailer AVC 720p. Guess what ? My 2nd test, pokemon DVD, 13 extra frames. DVD/MPEG2 are especially bad for DSS derivatives. Granted, the seeking might be slightly better with a grf, but it still doesn't overcome the other problems. (Now these are manually constructed graphs, everything from source, splitter, decoder, decoder settings (and you know that I'm very familiar with graphedit / graphstudio from previous discussions) )

[a1s2d3f4]

Thanks everyone. I am happy to do "the right thing" if I knew what it should be. DirectShowSource has been reliable for me so far.

Lets backtrack and start with the obvious: I am an idiot; I am a musician who programs for fun and I don't know enough to be any kind of expert in video processing.
However, I have learned from playing with AviSynth certain tricks that work to get me the results I am after.

In this problem, I am stuck and can't yet find a solution and would love it if someone who knows how to solve this problem could help me out.

Since there was a request for an example clip, no problem. The original file is AVI. Encoded as XVid at 25fps (I got it from someone else - no control here).
So, I just loaded it into VirtualDub (1.10.3 - 32bit), set everything to Direct stream copy, and made a small clip of just the first 684 frames.
Here it the link to this file:
https://drive.google.com/file/d/1S1xtcvBjfYyIrHivYpp2I3ZufxawboWO/view?usp=sharing

In case it helps, here is my entire AVISynth code. (you can ignore the audio part, since it's a separate issue - Commenting out those lines)

video_file = "<path to AVI>Gilels_Test_for_Doom9.avi"
#audio_file = "<path to AVI>Gilels_Test_for_Doom9.wav"
#audio_delay_value = -0.075 #Schubert

start_time_of_track_in_sec = 10 #Schubert
end_time_of_track_in_sec = 60*11 +25 #end of Schubert

#audio = WavSource(audio_file)

# here is another option to load it - I have tried it, too, no difference with DirectShowSource
v1 = FFMpegSource2(video_file, atrack = -2)
#v1 = DirectShowSource(video_file)

fps = v1.framerate
start_frame = Round(fps * start_time_of_track_in_sec)
end_frame = Round(fps * end_time_of_track_in_sec)

#v1 = AudioDub(v1, audio)
v1 = Trim(v1, start_frame, end_frame).ConvertToYV12().Crop(16, 6, -20, -12).FadeIO0(5)

border_size = 154
#v1 = DelayAudio(v1, audio_delay_value)
v1 = AddBorders(v1, border_size, 0, border_size, 0)

v2 = Yadif(v1,1,1) #last number changes field order
#return v2
# comment on what we are seeing after Yadif call
# I have 50fps
# and here is how the frames look:
# 0151-A; 0152-B; 0153-B; 0154-C; 0155-C
# 0156-D; 0157-E; 0158-E; 0159-E; 0160-E
# 0161-A; 0162-B; 0163-B; 0164-C; 0165-C
# 0166-D; 0167-E; 0168-E; 0169-E; 0170-E
#
# I have 5 real frames over the span of 10. That's "25 real frames". Hmm. Interesting. What a mess.

# And here is the srestore call that isn't the correct call. I need help either at this point
# Or a completely different solution that I can't even imagine at this point.
v2 = Trim(v2,4,999999).srestore(frate=25)
return v2



Here is the above code in an .AVS file:
https://drive.google.com/file/d/1DdjKvVgZ3Z-UZEvq_8UZff5zvo1XID4Z/view?usp=sharing

Please help!
I just want to have a perfect ABCDEABCDEABCDE type of video (at 25 fps) and I don't want to go through all the tens of thousands of frames manually, removing the duplicates, while deciding which duplicate "looks better".

[a1s2d3f4]

You need to evaluate the clip before you apply deinterlacing. And, you need to look at the original clip field-by-field, so use separatefields(), and nothing else.

Otherwise, you have no idea what is really going on.

Isn't "Deinterlacing" and "separting fields" the same thing?

[a1s2d3f4]

#1) Don't use directshowsource . Not frame accurate with non linear seeks. Especially with srestore. Some of the problems and patterns may change just by using it

I tested just now loading one of my clips with DirectShowSource and then loading the same with FFMpegSource2 and do confirm that they were off by one frame.

I then looked at the AVS file that generated this clip I was testing. IT agreed with FFMpegSource2. So perhaps, DSS is not reliable, like you are saying.

[manono]

Next time don't waste 40% of a sample with a useless text screen. It's an XviD so the obvious source filter to use is AVISource. With the only movement being the hands and the head and body sometimes, it's hard to tell with so little movement. Maybe it's supposed to be 24fps. I don't know. Nor can I tell if the following works perfectly or not, but it's a start:

AviSource("Gilels_Test_for_Doom9.avi")
Yadif(Order=1,Mode=1)###or a better bobber like QTGMC
Tdecimate(Mode=2,Rate=25,MaxNDL=4)

As you may know already, TDecimate is a part of the TIVTC (http://avisynth.nl/index.php/TIVTC) set of IVTC filters.

[hello_hello]

I messed around a bit and ended up with the same solution as manono, in a more convoluted form. And yes, it's very hard to tell what's happening when there's so little movement.

I de-interlaced with QTGMC and created an intermediate lossless file, then ran two passes with TFM and TDecimate.

The script for creating the intermediate file:

AviSource("E:\Gilels_Test_for_Doom9.avi")
GreyScale() # which unfortunately seemed to make the banding worse.
AssumeTFF()
QTGMC()

Analysis pass script for TIVTC:

LWLibavVideoSource("Intermediate file")
TFM(Output="D:\TFM.txt", order=1)
TDecimate(mode=4, Output="D:\TDec.txt")

Script for encoding:

LWLibavVideoSource("Intermediate file")
TFM(Input="D:\TFM.txt", order=1)
TDecimate(TFMIn="D:\TFM.txt", mode=2, Input="D:\TDec.txt", rate=22.975)
SomeCroppingAndResizing()

A frame rate of 22.975fps seems to be what's needed to remove all the duplicate frames, which made me think I'd probably just de-interlace to 25fps and be done with it. SRestore couldn't decimate to 22.975fps properly. As there's missing fields it'll never be perfect anyway. Maybe you could remove the duplicate fields and try to replace them with interpolated ones, but that sounds hard, and after bobbing to 50fps and decimating to 22.975fps it looks fairly smooth to me. In my attached sample, you can see there's more movement between frames 335 and 336 than between the surrounding frames (there must have been a missing field there), but you'd probably have to step through the frames one at a time to notice.

Gilels_Test_for_Doom9_.zip (https://ufile.io/n3d15)

[a1s2d3f4]

Next time don't waste 40% of a sample with a useless text screen. It's an XviD so the obvious source filter to use is AVISource. With the only movement being the hands and the head and body sometimes, it's hard to tell with so little movement. Maybe it's supposed to be 24fps. I don't know. Nor can I tell if the following works perfectly or not, but it's a start:

AviSource("Gilels_Test_for_Doom9.avi")
Yadif(Order=1,Mode=1)###or a better bobber like QTGMC
Tdecimate(Mode=2,Rate=25,MaxNDL=4)

As you may know already, TDecimate is a part of the TIVTC (http://avisynth.nl/index.php/TIVTC) set of IVTC filters.

Thank you. It does, in fact, seem to work perfectly! I compared the original clip (returning after Yadif(1,1)) with what you did, and here is the result:

Below are the frame images that match in both clips

My original clip - (50fps):
0721-A; 0722-B; 0723-B; 0724-C; 0725-C;
0726-D; 0727-E; 0728-E; 0729-E; 0730-E;
0731-A; 0732-B; 0733-B; 0734-C; 0735-C;
0736-D; 0737-E; 0738-E; 0739-E; 0730-E;

manono's script (25 fps):
0360-A(0721); 0361-B(0722); 0362-C(0724); 0363-D(0726); 0364-E(0727)
0365-A(0731); 0366-B(0732); 0367-C(0734); 0368-D(0736); 0369-E(0737)

This is an excellent piece of help. Really thank you. I have read about TDecimate before but had never used it, finding srestore to work for me in the past.
I have applied the script to the entire clip (40000 frames) and it seems to be working glitch free.
I have noticed that what it does is select the very first frame of the duplicated set, without looking for the "best" variant. (Again, I thought I saw somewhere that srestore can be called in a way that it will look for the "best" variant, which is why I stuck with it without looking for other AVS functions). For example, above, the "0728 version" of image "E" is less fuzzy than 0727, but at this point I am so happy there is a solution, I willing to ignore this little bit of occasional fuzz.

Now, that, with your help I have a usable clip, I am able to look at it more closely and observe these strange "doubled up" images in a single frame artifacts, which cause me to wonder about whether or not this was a "24fps" shoot that was "PAL-ed" up to "25fps".
See the attached screenshot from one of the frames further on. Noticed the doubled-up cuffs on the right hand.
https://drive.google.com/file/d/1GcNN3dQ10CmBABjkCXNfvtdSgS5d_1Ah/view?usp=sharing

Also, the only reason I didn't cut out the titles at the beginning is so that my frame/field numbers in the OP matched with that clip.

And, yes, somehow I completely forgot to try AVISource() because I am so used to dealing with mp4 clips.

[a1s2d3f4]

I messed around a bit and ended up with the same solution as manono, in a more convoluted form. And yes, it's very hard to tell what's happening when there's so little movement.

I de-interlaced with QTGMC and created an intermediate lossless file, then ran two passes with TFM and TDecimate.

The script for creating the intermediate file:

AviSource("E:\Gilels_Test_for_Doom9.avi")
GreyScale() # which unfortunately seemed to make the banding worse.
AssumeTFF()
QTGMC()

Analysis pass script for TIVTC:

LWLibavVideoSource("Intermediate file")
TFM(Output="D:\TFM.txt", order=1)
TDecimate(mode=4, Output="D:\TDec.txt")

Script for encoding:

LWLibavVideoSource("Intermediate file")
TFM(Input="D:\TFM.txt", order=1)
TDecimate(TFMIn="D:\TFM.txt", mode=2, Input="D:\TDec.txt", rate=22.975)
SomeCroppingAndResizing()

A frame rate of 22.975fps seems to be what's needed to remove all the duplicate frames, which made me think I'd probably just de-interlace to 25fps and be done with it. SRestore couldn't decimate to 22.975fps properly. As there's missing fields it'll never be perfect anyway. Maybe you could remove the duplicate fields and try to replace them with interpolated ones, but that sounds hard, and after bobbing to 50fps and decimating to 22.975fps it looks fairly smooth to me. In my attached sample, you can see there's more movement between frames 335 and 336 than between the surrounding frames (there must have been a missing field there), but you'd probably have to step through the frames one at a time to notice.

Gilels_Test_for_Doom9_.zip (https://ufile.io/n3d15)

@hello_hello
Thank you so much for your effort also. I downloaded your zip file of MKV and loaded it directly into VirtualDub.
I then started going through it carefully.
I noticed that it skipped over one of my ABCDE frames when going from frame 345 to 346. (If you look at it, you'll see that the foot jumps, and in my original clip there is another frame between these two). This is the very problem I ran into when trying to use srestore(frate=25). )
Also, I'll be honest with you, I am not sure how you ended up with 22.866 as, clearly, ABBCCDEEEE (5 real images over the span of 10 video clip frames) results in exactly 25fps....

Now, as you'll notice from my reply to manono, I am suspecting that there is some blending going on, where, possibly there are two original film images superimposed inside one video frame, but it doesn't look like that's what your script is about. (i.e. no srestore, no deblenders)

[manono]

If you notice blending after the decimation, the next step will be to try and figure out if it occurs every 25 frames (or 24-26 frames). That would be a sure sign it's from a 24fps source. If so, you can deal with it. You can use SRestore after the decimation or maybe SelectEvery if it's consistent throughout. TDecimate may or may not work to give you 24fps directly, but I don't think so.

Don't forget to greyscale it. And maybe get rid of the logo.

And to keep from going blind while doing all this concentrated work, isn't there anything in the video with more movement? The guy never stands up to take a bow? I'm glad you're making progress.

[hello_hello]

@hello_hello
Thank you so much for your effort also. I downloaded your zip file of MKV and loaded it directly into VirtualDub.
I then started going through it carefully.
I noticed that it skipped over one of my ABCDE frames when going from frame 345 to 346. (If you look at it, you'll see that the foot jumps, and in my original clip there is another frame between these two). This is the very problem I ran into when trying to use srestore(frate=25). )

Yeah, I mentioned that in my previous post.
There's not another complete frame between those two in your original clip due to the repeated field. There might seem to be, depending how you look at it... whether it's being de-interlaced and/or how it's being de-interlaced.... but due to the repeated field there can't be a complete unique frame between them.

Also, I'll be honest with you, I am not sure how you ended up with 22.866 as, clearly, ABBCCDEEEE (5 real images over the span of 10 video clip frames) results in exactly 25fps....

Clearly, EEEE isn't two different images. It's the same principle as decimating film with pulldown applied to go from 29.97fps to 23.976fps. The duplicate frames are removed after field matching, and the frame rate is reduced. 22.975 was the frame rate required to remove all the duplicates (or near duplicates caused by the repeated fields) and leave nothing but unique frames. I don't know where you got 22.866 from, but naturally there will be the occasional "jitter" in motion because there were fields missing.

Now, as you'll notice from my reply to manono, I am suspecting that there is some blending going on, where, possibly there are two original film images superimposed inside one video frame, but it doesn't look like that's what your script is about. (i.e. no srestore, no deblenders)

I don't think there's any blending going on but the clip you uploaded had so little movement it was fairly useless for trying to work that out. If there's field blending, you should see a pattern of blending when stepping through the individual fields.

It appears jagabo posted a method for replacing the duplicate fields with interpolated ones over at VideoHelp. It might be easier than I thought, assuming it works okay with video of such low quality and with so much noise. If it does, you could just replace the duplicate fields and then de-interlace to 50fps.

I'd try using FrameRateComverter (https://github.com/mysteryx93/FrameRateConverter/releases) for interpolating new fields. It'll probably produce less artefacts than Interframe.