Fixing frame rate problems ...

[frobber]

Dear Fellow AviSynthers,

I'm trying to fix a video that was recorded from TV. It has an odd frame rate (VFR at 33.756 fps, originally 29.97) — which causes jerky playback on my machine with VLC, and massively bloats the MKV file.

Bit rate mode : Variable
Bit rate : 15.1 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 33.756 FPS
Original frame rate : 29.970 (30000/1001) FPS

So, I tried this with AviSynth:

LWLibavVideoSource("Test-clip-1.mkv")

AssumeFPS("ntsc_video")
tfm().TDecimate()

AutoCrop(mode=0, aspect=2.35)

...which gives me the desired 23.976 fps in the output and the letterboxing is cropped away nicely, but there are strange and terrible motion artifacts, like a bunch of frames are out of order when large areas of the image are in motion.

No doubt, I'm doing something wrong but am at the limit of my understanding.

Any ideas about how I can fix this...? Thanks !

[Selur]

Are those artifacts there if you play the file in a media player?

[frobber]

Are those artifacts there if you play the file in a media player?

Yes, I see them in AviSynth's player and also VLC.

[Emulgator]

A sample would be helpful, because guessing is in vain.

[hello_hello]

By default, VFR video is decoded at the average frame rate, and AssumeFPS simply changes the playback speed but it doesn't keep the frames in the original locations so it'll change the duration. You're likely to have A/V sync problems if it's really VFR. The following method keeps the frames in the correct place, while adding duplicates or dropping frames as required to output the specified frame rate and maintain sync.

LWLibavVideoSource("Test-clip-1.mkv", FPSNum=30000, FPSDen=1001)
TFM().TDecimate()
AutoCrop(mode=0, aspect=2.35)

I doubt the artifacts have anything to do with the frame rate. Either the video was encoded that way or it's a decoding issue. Try a different source filter, but it sounds like it was encoded that way. Is the video progressive or interlaced?

FFVideoSource("Test-clip-1.mkv", FPSNum=30000, FPSDen=1001)
TFM().TDecimate()
AutoCrop(mode=0, aspect=2.35)

Try just adding Info() to the script after the source filter to see what it does to the frame count with and without FPSNum/Den. The duration shouldn't change, but if the video is VRF the total frame count should. And check with Info to make sure the video isn't being decoded at 29.97fps before you add FPSNum/Den, because if it is they're probably not needed.

By the way, your player mightn't like the video being variable frame rate, but it shouldn't bloat the size of the MKV as such.

I think when MediaInfo displays two frame rates it means there's one rate written to the video stream (original frame rate) and another to the MKV container. The same thing can also happen for constant frame rate video if the two frame rates are different.

[Otyli]

Hey, I’ve dealt with VFR headaches before, and it’s a pain. Your script looks solid, but the motion artifacts might be from the tfm().TDecimate() combo. Sometimes it struggles with VFR sources, especially if the frame rate is all over the place.
Try this: before AssumeFPS, use ChangeFPS(29.970) to force it back to the original frame rate first. Then apply tfm().TDecimate() to handle the IVTC. This might smooth out the motion issues. Also, double-check if your source has duplicate frames or interlacing artifacts
If that doesn’t work, you could try QTGMC for deinterlacing before IVTC. It’s slower but often handles tricky sources better.

[frobber]

Thanks much for these pointers ! I tried a quick test of each approach and the results are promising. I couldn't get Info() to report more than a single line of general text, tho. No deets.

The speed looks a little fast, but maybe I've been staring at the wonky video for too long. I'm going to try the whole file and check sound sync.

I'll report back with my findings tomorrow.

[RetsimLegin]

The first thing I'd experiment with (using a gui in my case - I have no knowledge of scripting) would be to hard resample the frame rate at 23.976 or 29.97, (i.e. without changing the duration) and see if the output is usable.

[Selur]

Typical candidates I would try on stuff with broken references are BasicVSR++ (<- Vapoursynth), MCTemporalDenoise(veryHigh), SpotLess, RemoveDirtMC,...

[frobber]

Thank you, all, for very helpful suggestions. My first test eliminated the motion artifacts and there was a big win on file size (less than half the original bloated MKV), but audio didn't sync.

FWIW, here's a sample of what the motion artifacts look like: https://www.sendspace.com/file/8h3ukj (33 secs, 22 MB) -- you can see it when the boy in the black uniform stands up and then sits down.

I'm still running the other suggestions through the MeGUI queue, but noticed that I failed to answer one question, above: yes, the source is interlaced. I.e.:

Codec ID/Info : MPEG 1 or 2 Video
Duration : 1 h 45 min
Bit rate mode : Variable
Bit rate : 15.1 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 33.756 FPS
Original frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First

I'll update on my findings and settings when it's done crunching.

P.S. I don't follow the suggestion of trying MCTemporalDenoise(veryHigh), SpotLess, RemoveDirtMC -- could one/all of those help with fixing the frame rate?

[Selur]

Ah, totally different from what I expected. :)
The suggested filters will not help with that.
There are just some frames are out of order.
You might want to try different source filters and try to fix your deinterlacing and maybe add SelectEvery where you change the field order.

[frobber]

You might want to try different source filters and try to fix your deinterlacing and maybe add SelectEvery where you change the field order.

Thanks. I have tried both LWLibavVideoSource() and FFVideoSource(), and the second one seems better. Any others you recommend?

Next, is there a guide somewhere on how to use SelectEvery()? I looked at the wiki page but I can't grok how it would be used other than just sheer guesswork.

[Selur]

You either would need to find a pattern (like: every X frames fame x + 1 is switched with frame x +y), if you can't find a global pattern using you would handle each scene separately.
Maybe the cause of the problem is the 'deinterlacing' method you used,... => analyse the source not some reencode

[frobber]

OK, I tried stepping frame-by-frame through the source using VLC (pressing 'E'), and there seems to be a repeating pattern of 10 frames, e.g.: -N-N-N-N-D where '-' indicates nothing whatsoever changes, 'N' indicates a new frame, and 'D' appears to be a duplicate of the previous frame. It all seems to be forward motion, i.e., no frames are out of chronological order. I don't see anything that looks like an interlacing artifact, but perhaps(?) VLC is taking care of that.

Should I use SelectEvery() to fix this, and if so, how?

[poisondeathray]

Thanks. I have tried both LWLibavVideoSource() and FFVideoSource(), and the second one seems better. Any others you recommend?


If you have a NVidia card, use DGSource. It's probably the most reliable


OK, I tried stepping frame-by-frame through the source using VLC (pressing 'E'), and there seems to be a repeating pattern of 10 frames, e.g.: -N-N-N-N-D where '-' indicates nothing whatsoever changes, 'N' indicates a new frame, and 'D' appears to be a duplicate of the previous frame. It all seems to be forward motion, i.e., no frames are out of chronological order. I don't see anything that looks like an interlacing artifact, but perhaps(?) VLC is taking care of that.

Should I use SelectEvery() to fix this, and if so, how?

For a VFR source, you might want to convert to CFR first, otherwise you might have sync issues

If it's NNNNNNNNND

SelectEvery(10, 0,1,2,3,4,5,6,7,8)


You would change the notation based on where the "D" is . This means for every group of 10 (the 1st number) , keep the following frames. Notice "9" is missing - that represents the last frame, the "D"


But I would use TDecimate, because if the pattern changes, it will adapt. For 1 in 10 decimation
TDecimate(cycleR=1, cycle=10)

But everything you reported might be incorrect, because VLC might be processing it e.g. deinterlacing. It would be better if you uploaded an unprocessed source sample

[frobber]

Thanks. I'm not sure how to obtain an unprocessed sample of the source. I tried this using LosslessCut, but the resulting frame rate is 2278.163 FPS, which seems insane.

[Selur]

Simply share the sample, my guess is one should simply drop all the time codes.

[frobber]

Okay, here's a sample extracted from the source, using LosslessCut: https://www.sendspace.com/file/plyai9 (18 secs, 32 MB)

Again, the frame rate according to mediainfo is a nonsensical value (the source file is 33.756 fps, original 29.97 fps), and apologies but I don't know how to drop the time codes.

Thanks !

EDIT: I have now tried the following three scripts...

# First test
FFVideoSource("Source-1.mkv", FPSNum=30000, FPSDen=1001)
AssumeFPS("ntsc_video")
tfm().TDecimate()
AutoCrop(mode=0, aspect=2.35)

# Second test
FFVideoSource("Source-1.mkv", FPSNum=30000, FPSDen=1001)
ChangeFPS(29.970)
tfm().TDecimate()
AutoCrop(mode=0, aspect=2.35)

# Third test
FFVideoSource("Source-1.mkv", FPSNum=30000, FPSDen=1001)
AssumeFPS("ntsc_video")
QTGMC().SelectEven()
tfm().TDecimate()
AutoCrop(mode=0, aspect=2.35)

...but none of them produce video that syncs with the original audio track (I use mkvtoolnix to join the video output from AVS with the original audio track as a MKA file).

Any suggestions on how I should proceed... ?

[Emulgator]

What a nice mess. Stream/mux is broken, frames 4,5: teacher has blocky face.
Writing library was Lavf60.3.100. Hard to believe that NHK uses LAV for MPEG-2 encoding and screws it up.

LWLibavVideoSource tries 559f@29.97 repeats frames, skips criss-cross and chokes, tries to deliver audio.
FFMpegSource2 delivers 559f@31.028fps, repeats frames, delivers audio.
BSSource 10 does the best it can and delivers 541f@30.610fps, omits audio because stream is faulty.
DGSource decodes 541f@29.97fps.

So for a straight first test you may try demuxing using DGSource and remux.
If that works, since you wanted to work on the source anyway: decode using BSSource or DGSource, IVTC, crop, reencode and remux.

[frobber]

Thanks, @Emulgator, for this great summary !

I can't vouch for the integrity of the sample file that I upped, as LosslessCut doesn't locate any key frames and it outputs a wonky fps value. Also, Lavf60.3.100 is used by LosslessCut for extracting the sample file. The original source MKV was written with libebml v1.4.1 + libmatroska v1.6.2.

I am unfamiliar with the source filters DGSource and BSSource. I don't have a nVidia GPU, so I guess I can't(?) use DGSource. I tried BSSource(), but MeGUI just sat cogitating for well over 30 mins. I thought indexing(?), but I didn't see a file being created. Finally, it seems way too slow, so I interrupted MeGUI. Maybe(?) a problem with the FFmpeg setup.

I'm trying this DLL with 64-bit AviSynth+ invoked via MeGUI: https://github.com/vapoursynth/bestsource/releases/

Is this the correct approach? I must be doing something wrong. -_-

EDIT: I also tried this:

FFVideoSource("Source-1.mkv", FPSNum=30000, FPSDen=1001)
ChangeFPS(29.970)
QTGMC().SelectEven()
tfm().TDecimate(cycleR=1, cycle=10)
AutoCrop(mode=0, aspect=2.35)

...and the output looks okay, no duplicate frames, but it's 26.973 fps. Again, I'm probably doing something wrong.

If BSSource or DGSource don't work, what are my options...?

[hello_hello]

I tried three different source filters and they're all decoding the sample at 23.976 fps (FFVideoSource, LWLibavVideoSource and BSVideoSource).
The problem seems to be out of order frames, but it seems pretty random so I don't know if there's a way to fix it, aside from manually re-ordering the frames.

https://imgur.com/DaTFLrP.png

[frobber]

Thanks, @hello_hello.

I'm a bit confused. I can get video output that appears to have frames in the correct order. i.e., the motion artifacts go away. But the problem is that the original audio then doesn't sync.

That's where I'm stuck... I think.

[hello_hello]

I don't understand either, although I forgot to mention that MediaInfo reports the sample as being progressive CFR.

Width : 1 856 pixels
Height : 790 pixels
Display aspect ratio : 2.35:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive


Edit: It has a different resolution to the MediaInfo example you posted originally, Are you sure it was a lossless cut?
It might also explain why you're not seeing all the information from Info(). Were there any other filters following Info() such as cropping?

[hello_hello]

My bad, it looks like I downloaded the wrong sample. I'll try again.

[Emulgator]

All repaired and IVTC'd, audio in sync
Still the incoherent channel count from the .aac will lead to audio source filters failing,
delivering noise bursts for the right channel, some more serious audio source filters may decide to inhibit decoding.
StaxRip encode directly from .avs script, you may use any encoder/muxer you like, wetransfer link valid for 3 days
https://we.tl/t-6V1UwA959e
#v=DGSource("Sample-1.1080p.HDTV-00.04.04.749-00.04.22.231.dgi")
v=MPEG2Source("Sample-1.1080p.HDTV-00.04.04.749-00.04.22.231.d2v")
a=LWLibavAudioSource("Sample-1.1080p.HDTV-00.04.04.749-00.04.22.231_track2_und_DELAY 7ms.aac")
AudioDub(v,a)
AssumeTFF()#.SeparateFields()
Telecide(guide=1).Decimate()
#portalscope(mode=0)
DGSource can read even from this borked .mkv directly. Good work DG !
You may use the non-nVidia DG decoder (DGIndex) as well, I just don't know how safely it demuxes from borked .mkv.
BTW, forgive the poor jpg, 200KB per attachment are not very sufficient for a video website
(Swede, Wilbert: Can we lift this limitation a bit, BTW?
I have donated a bit, 400..500KB would be improving the situation already.
External links will die sooner or later and render threads incomplete)

[hello_hello]

I tried remuxing the sample as a TS file with TSMuxer so I could open it with DGIndex and it offered to fix the bad field transitions. There's still a few duplicate frames at the spot where the guys sits down, which I think is putting the audio sync out near the end. You might have to manually delete them. For some reason too, cropping before TFM and TDecimate seems to do odd things to the chroma where the bad field transitions are. I don't know why, but I wouldn't crop first.

MPEG2Source("N:\Sample-1.d2v")
TFM()
TDecimate()
CropResize(1920,0, 2,144,-2,-146)
QTGMC(InputType=1, EzDenoise=1.0)
GradFun3()

Sample-2.mkv (https://files.videohelp.com/u/210984/Sample-2.mkv)

[hello_hello]

It looks like you might be better off not letting DGIndex fix the field order transitions. TFM seems to clean them up okay, there's no duplicate frames and the audio stays in sync.

Sample-3.mkv (https://files.videohelp.com/u/210984/Sample-3.mkv)

[frobber]

Wow, you guys rawk ! This is super-helpful, thanks.

I tried with the source MKV but there were problems. Remuxing to TS with ffmpeg helped, so thanks also for that tip!

For reference, here's what finally worked for me:

MPEG2Source("Source-1.d2v")
AssumeTFF()
Telecide(guide=1).Decimate()
AutoCrop(mode=0, aspect=2.35)

I wasn't able to get the audio using LWLibavAudioSource(), but that's probably pilot error on my part. I just added the audio track using mkvtoolnix and it sync'd up fine.