Perfectly synced footage desyncs when doing a PAL speedup

[Adam Cole]

I have no idea where this question belongs so please feel free to move it to the right place @mods.

I made an mkv from a japanese DVD with MakeMKV and I wanted to do a PAL speedup to it. The unaltered mkv has, according to mediainfo, a variable framerate of 23.976fps and an original framerate of 29.970fps (whatever that means). This is the mediainfo:

Video
ID : 1
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Duration : 24 min 39 s
Bit rate mode : Variable
Bit rate : 7 375 kb/s
Maximum bit rate : 9 800 kb/s
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Variable
Frame rate : 23.976 (23976/1000) FPS
Original frame rate : 29.970 (30000/1001) FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.890
Time code of first frame : 01:30:05:05
Time code source : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first fram : Closed
Stream size : 1.27 GiB (90%)
Default : Yes
Forced : No
Color primaries : BT.601 NTSC
Transfer characteristics : BT.601
Matrix coefficients : BT.601

Audio and video of the unaltered mkv are perfetly in sync. Now I wanted to do a PAL speedup (change video and audio to 25fps).

First I increased the tempo of the audio with audacity:

Tempo change 23.976 to 25: +4,271%

Next is the video. As I don't want to transcode, I remuxed the video with MKVToolNix by entering 25p in the 'default duration' field

https://i.ibb.co/Pgrt3qN/untitled.png

Now video and audio should have the same speed and be in sync, but they're not. There is a 300ms difference after the intro.

Here is the mediainfo of the remuxed mkv:

Video
ID : 1
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Duration : 23 min 39 s
Bit rate mode : Variable
Bit rate : 7 688 kb/s
Maximum bit rate : 9 800 kb/s
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 FPS
Original frame rate : 29.970 (30000/1001) FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.890
Time code of first frame : 01:30:05:05
Time code source : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first fram : Closed
Stream size : 1.27 GiB (100%)
Default : Yes
Forced : No
Color primaries : BT.601 NTSC
Transfer characteristics : BT.601
Matrix coefficients : BT.601


This desync issue actually started after 100 or so episodes of the japanese DVD show I am working with. All episodes before don't have this desync issue. Their framerate info in mediafire also is slightly different:

Case in point:

All unaltered episodes without desync issue:

Frame rate mode : Variable
Frame rate : 23.976 FPS

All unaltered episodes with desync issue:

Frame rate mode : Variable
Frame rate : 23.976 (23976/1000) FPS
Original frame rate : 29.970 (30000/1001) FPS

Note that all problematic episodes have an 'Original Framerate' entry that says 29.970 (30000/1001) FPS in mediainfo, which is missing from the episodes without desync issue.

The following are the framerate infos of the problematic and issue-less episodes after applying the PAL speedup:

Episodes without issue:

Frame rate mode : Constant
Frame rate : 25.000 FPS
Original frame rate : 23.976 (24000/1001) FPS

Episodes with desync issue:

Frame rate mode : Constant
Frame rate : 25.000 FPS
Original frame rate : 29.970 (30000/1001) FPS

The difference here is that the 'Original framerate' entries are different between the problematic and issue-less episodes. Could it be that the fine episodes are true 23.976fps (as they lack an original framerate entry) and the problematic episodes not (as they have an original framerate entry)? Does the issue lie within there?

Any information on this subject is greatly appreciated.

[manono]

I made an mkv from a japanese DVD with MakeMKV and I wanted to do a PAL speedup to it.
All Japanese DVDs output CFR 29.97fps, All that VFR nonsense is you chasing a red herring.
Could it be that the fine episodes are true 23.976fps
Not if they're from DVDs, no. They might be 23.976fps after an IVTC, though. Your problem could be related to you making an MKV before working on these.
Now video and audio should have the same speed and be in sync, but they're not. There is a 300ms difference after the intro.
There may have been a delay set in the DVD that disappeared after making an MKV of them. You could add it back in Audacity.

What I wonder is why anyone would purposely change the intended framerate and audio pitch. Or did the pitch stay the same after the Audacity work?

[Cary Knoop]

Start with the 60i source, then perform an IVTC to 24p and then do the speedup to 25p. Then optionally re-interlace to 50i for PAL.

[Adam Cole]

There may have been a delay set in the DVD that disappeared after making an MKV of them.
This wouldn’t make sense because the made, unaltered mkv is perfectly in sync. It only desyncs after remuxing with the 25p flag set.
Or did the pitch stay the same after the Audacity work?

Audacity’s tempo change function doesn’t alter the pitch.

Start with the 60i source, then perform an IVTC to 24p and then do the speedup to 25p. Then optionally re-interlace to 50i for PAL.

The mkvs MakeMKV produces are all progressive. Why re-interlace to 50i? The DVD source is encoded progressively.

Edit: Another question: How do you IVTC the source?

[sneaker_ger]

This wouldn’t make sense because the made, unaltered mkv is perfectly in sync. It only desyncs after remuxing with the 25p flag set.
Because you are manipulating the timestamps from the source where they have variable lengths (variable framerate from mixed content/pulldown or something like that) to be constant 25 fps. Like the others have said that will not work.

[Adam Cole]

Because you are manipulating the timestamps from the source where they have variable lengths (variable framerate from mixed content/pulldown or something like that) to be constant 25 fps. Like the others have said that will not work.

How do you explain that that did work for episode 1 till 165 then? The sync issue only started beginning with 166 and forth. Episode 1 till 165 also have a variable framerate.

[SeeMoreDigital]

In the past I used to remove the pull-down flags from all of my NTSC movie DVD's in order to obtain the pure-progressive frames. And with some of these movies I would change their speed from 23.976fps to 25.000fps. Most were quite straight forward to do but some were a real pain.

The main problem I came across was that some of the pull-down flags can be out of sequence (especially after a layer change) or (for what ever reason) you get a succession of interlaced field frames instead of progressive frames.

From what I remember, I got this with early NTSC DVD versions of Waterworld, Pink Floyd The Wall and Mars Attacks.

[Adam Cole]

In the past I used to remove the pull-down flags from all of my NTSC movie DVD's in order to obtain the pure-progressive frames. And with some of these movies I would change their speed from 23.976fps to 25.000fps. Most were quite straight forward to do but some were a real pain.

The main problem I came across was that some of the pull-down flags can be out of sequence (especially after a layer change) or (for what ever reason) you get a succession of interlaced field frames instead of progressive frames.

From what I remember, I got this with early NTSC DVD versions of Waterworld, Pink Floyd The Wall and Mars Attacks.

Which tool/tools did you use to remove the pulldown flags?

[sneaker_ger]

How do you explain that that did work for episode 1 till 165 then?
Because those episode were authored differently.


The sync issue only started beginning with 166 and forth. Episode 1 till 165 also have a variable framerate.
MediaInfo's VFR detection is not very useful. It can be straight out wrong or even if it isn't it doesn't tell you anything about what exactly is VFR about the file.


A simple PAL speed-up without re-encoding only works if the source is 100% 23.976 fps. If it isn't (e.g. a mix between 23.976 pulldowned to 29.97 fps and "true" 29.97 fps) you are out of luck.

[SeeMoreDigital]

Which tool/tools did you use to remove the pulldown flags?From what I remember I used TSmuxerGUI along with its 'Demux' option. Which is capable of highlighting video and audio stream errors...

A simple PAL speed-up without re-encoding only works if the source is 100% 23.976 fps. If it isn't (e.g. a mix between 23.976 pulldowned to 29.97 fps and "true" 29.97 fps) you are out of luck.Agreed...

[manono]

Go back to the beginning. Use the decrypted VOBs as your source and not the MKV. Make a D2V using DGIndex and then use MPEG2Source followed by the IVTC. The needed tools and docs are included in the DGMPGDec (https://www.videohelp.com/software/DGMPGDec) package. The IVTC I'll assume you already know about. If not, check out the TIVTC page (http://avisynth.nl/index.php/TIVTC). After getting a progressive 23.976fps script, then you can speed up the video using AssumeFPS(25) (http://avisynth.nl/index.php/AssumeFPS), and speed up the audio to match.

Come back if you have further questions.

[hello_hello]

The trouble is, he doesn't want to re-encode, and specifying a frame rate when remuxing will output a constant (25fps) frame rate. If there's film sections at 23.976 and video sections at 29.970 it'll make a mess.

It probably requires a screwdriver rather than a hammer when remuxing.
In the "Stretch By" field just above MKVToolNixGUI's "Default Duration / FPS" field, try adding the following and leaving the frame rate field empty.

960/1001

It's the same as 25/(24000/1001), only as a two integer fraction.

I gave it a test drive using a variable frame rate encode and it didn't effect the video/audio sync. I applied the same stretch to the audio for testing, but that's generally a bad idea as it can make it sound "jittery". I'd re-encode that as you have been. You can stretch subtitles too if need be.

Original MKV timecodes:
# timestamp format v2
0
42
83
125
167
.
.
(first frame of a 29.970 section)
834
868
901
934
968




Stretched:
# timestamp format v2
0
40
80
120
160
.
.
(first frame of what was a 29.970 section)
800
832
864
896
928

[manono]

The trouble is, he doesn't want to re-encode
Better than beating his head against a wall and getting nowhere.

...and specifying a frame rate when remuxing will output a constant (25fps) frame rate. If there's film sections at 23.976 and video sections at 29.970 it'll make a mess.
Exactly my point.

960/1001

It's the same as 25/(24000/1001), only as a two integer fraction.
You sure about that? The result of the former is 0.9590, of the latter 1.0427. I think for one or the other you're supposed to switch the dividend and divisor. And this method removes the interlacing and makes it entirely progressive 25fps? I've already given my opinion of purposely doing a PAL speedup.

[sneaker_ger]

What's the point? The only reason to do PAL-Speed Up is to make it compatible with PAL devices. With stretching you still have variable framerate, not constant 25 fps. I can only imagine this as some kind of temporary preprocessing. E.g. first do the stretching and then do a VFR->CFR conversion using e.g. ffms2 do get the non-25-fps parts to 25 fps.

[StainlessS]

I think for one or the other you're supposed to switch the dividend and divisor.

Yep,

A / (B/C) == A * (1/(B/C)) == A * (C/B) == (A*C) / B

25/(24000/1001) == (25*1001)/24000 == 1.042708333 recurring

[sneaker_ger]

Yes, he mixed it a bit but his first value (960/1001) is the correct one to stretch 23.976 to 25 fps.

[StainlessS]

I have not really followed this thread all through (busy doing mucho other stuff), but perhaps below of some use, :- http://forum.doom9.org/showthread.php?p=1810135#post1810135


Function TimeStretchTempo(clip InC,Float OutNum,Float "OutDen") {
# http://forum.doom9.org/showthread.php?p=1810135#post1810135
# Returns Timestretch() Tempo arg, to change playback tempo when changing Video Framerate, without changing pitch:
Num = InC.FrameRateNumerator
Den = InC.FrameRateDenominator
OutDen = Default(OutDen,1.0).Float
Assert(0.0<OutNum,String(OutNum,"TempoAsPercent: 0.0 < OutNum(%f)"))
Assert(0.0<OutDen,String(OutDen,"TempoAsPercent: 0.0 < OutDen(%f)"))
Return 100.0 * OutNum * Den / (OutDen * Num) # All are Coerced to Float (v2.58 Incl)
}


Does nothing that you could not do by hand, but easier just to let function do the work.

[hello_hello]

You sure about that? The result of the former is 0.9590, of the latter 1.0427.

Yeah that was a silly. I should have said 960/1001 is the same as 24000/1001/25.
I only mentioned it because MKVToolNixGUI only accepts two integers as a fraction for stretching, or a single float value.

Anyway...
PAL Speedup = 960/1001 Same as 24000/1001/25
PAL Slowdown = 1001/960 Same as 25/(24000/1001)

And this method removes the interlacing and makes it entirely progressive 25fps? I've already given my opinion of purposely doing a PAL speedup.

I'm not advocating a PAL speed-up, but if that's what he wants to do, I think it's the way to do it. Naturally the 29.97fps sections are sped up by the same percentage and end up 31 point something frames per second.
I can't remember the last time I used a PAL player/TV that wouldn't do NTSC. Especially these days when "PAL" TVs and players often give the impression they're NTSC at heart. The Bluray players here switch the TVs to 60Hz to display their menus, or more accurately, everything runs at 60Hz until you insert a "PAL" disc.

I'm not sure about the "removes the de-interlacing" thing though. MediaInfo doesn't seem to agree. I'm pretty certain this is a combination of film and video and the spedup version plays normally. The spedup audio is in sync. Constant frame rate for the second one is wrong.

Video
ID : 1
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Format settings, picture structure : Frame
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Duration : 22 min 52 s
Bit rate mode : Variable
Bit rate : 6 492 kb/s
Maximum bit rate : 9 800 kb/s
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 29.970 (30000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.627
Time code of first frame : 09:59:59:00
Time code source : Group of pictures header
Stream size : 1.04 GiB (97%)
Language : English
Default : Yes
Forced : No



Video
ID : 1
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Format settings, picture structure : Frame
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Duration : 21 min 56 s
Bit rate mode : Variable
Bit rate : 6 770 kb/s
Maximum bit rate : 9 800 kb/s
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 31.250 FPS
Original frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.627
Time code of first frame : 09:59:59:00
Time code source : Group of pictures header
Stream size : 1.04 GiB (97%)
Language : English
Default : Yes
Forced : No

The DVD I was playing with sped up into something I still don't understand, causing programs to count frames at one rate while they display them at another. It's sometimes a good thing. If ReClock only sees one frame rate, it doesn't adjust the speed up and down when the frame rate changes. Combinations of 23.976 and 29.97fps are adjusted by the same amount for a 60Hz display, but other frame rate combinations and refresh rates can cause it to adjust the adjustment when the frame rate changes, which is generally ugly.

ffdshow has labelled two sequential frames with the same frame number. It's probably counting at 29.97fps while they display at 31.25fps

https://i.postimg.cc/5jkrJWWW/1.jpg (https://postimg.cc/5jkrJWWW)
https://i.postimg.cc/V5Nh2LTF/2.jpg (https://postimg.cc/V5Nh2LTF)

I'm sure it's related to the MKV timebase, because changing the timebase does that sort of thing. Before I die, I'd really like to understand it properly.

PS manono, I noticed the frames in the screenshots have combing. I don't think it's due to the remuxing or speedup. I checked some other files and nothing is being de-interlaced at the moment. I'm not sure when that started. I rarely watch interlaced video. Maybe I broke something. I probably should try to work it out.

[hello_hello]

I have not really followed this thread all through (busy doing mucho other stuff), but perhaps below of some use, :- http://forum.doom9.org/showthread.php?p=1810135#post1810135

I did one of them a while back too, aimed at making matching audio to video a little easier when you're being tested, or for standard speedups and slowdowns etc. Sometimes it's comforting to to see the numbers. :)

AudioSpeed function:
https://forum.videohelp.com/threads/384967-AudioSpeed-AudioMeter-scripts