Odd mpeg2 + dts sync problems on reencode

[Mai]

Hi @ all

sorry for the title but I cannot think of a really fitting one because I don't know where the actual problem lies (hopefully we can figure this out). Also this might not be the right sub-forum I wasn't sure of that either. I did a search but as I cannot figure out a title for my own so I couldn't guess what others might have called their thread ...

Ok, this is all part of a longer story but I figure the interesting part goes like this:

My source is an NTSC DVD, which I got in elementary streams on my hdd. The two of interest are the video (orig.m2v from here on) and the sound (sound.dts). My processing pipeline for the video is: orig.m2v->AviSynth->TMPGEnc->filtered.m2v. (My avs does not do frame rate conversion, but only a AssumeFPS("ntsc_film") which sould not change anything considering the source.) So I did that and got new.m2v, then I used ffmpeg to mux sound.dts and new.m2v into new.mpg. While playing I noticed, that the sound is dead on at the beginning, and about 1-2 sec. off at the end of new.mpg (~115 min running time).

So I said: ok maybe something went wrong while reading the input in the avs. So I opened new.m2v and orig.m2v, each directly in VDubMod (using it's internal decoder). Then I skipped to some frame in the end and checked if this frame was the same in both: it was -> so as far as I can see no frame was dropped or duplicated, new.m2v and orig.m2v are picture for picture identical.

My next guess was that on obtaining the orig.m2v and sound.dts something went wrong. So I remuxed sound.dts and orig.m2v using ffmpeg into orig.mpg. If I play this the sound is in sync perfectly.

Both, orig.m2v and new.m2v are progressive and are encoded in such a way that the 3:2 pull-down is done on playback. And as far I understand the frame rate for an m2v cannot be chosen randomly, there are only well defined ones available, or am I wrong here? Also where should this 2 sec. error in 115min of film come from? Could this be some wired pull down related thing?

What I did then do was a mpgtx -i on both:
------------------------------------------------
orig.m2v
Mpeg 2 Video File
Estimated Duration: 01:27:18.92s
Aspect ratio 16/9 (large TV)
Interlaced, chroma format: 4:2:0
Size [720 x 480] 29.97 fps 9.80 Mbps

mpgtx -i new.m2v
Mpeg 2 Video File
Estimated Duration: 01:26:06.29s
Aspect ratio 16/9 (large TV)
Interlaced, chroma format: 4:2:0
Video Format: NTSC
Size [720 x 480] 29.97 fps 8.00 Mbps
------------------------------------------------

One can easily see the difference in the duration. Well it's estimated, but a whole minute? -> also this does not fit the 1-2 sec error I have, thus might be unrelated. Then there is this additional line saying "NTSC" for new.m2v?

I did another thing: pipeline same as above, but with x264 at the end. Then I muxed the resulting new_264.mp4 together with sound.dts into new_264.mkv using ffmpeg again. Playing this resulted in the same sync problems.

I'm pretty baffled, because I don't know enough of the internals of mpeg to get my head around this. Does somebody have any idea? The only other thing I can think of is removing like 30 or so frames across the 115 minutes, that doesn't sound right...

Thanks for any suggestions
Cheers

[mp3dom]

NTSC DVD is 30fps (well, 29.97 to be precise). AssumeFPS("ntsc_film") assume your video as 23.98 fps.
More infos needed:
- Have you used DGIndex do index your mpeg file? If so, is your file at 23.98fps? Does it contain soft or hard pulldown? How do you have indexed your file (with Honored Pulldown, forced film or ignored pulldown?)
- Have you IVTCed the file?

The right way to do your job should be:
- Index the m2v with DGIndex/DGTools honoring pulldown
- In case of a true movie (23.98fps) do the IVTC with TFM/TDecimate (you'll end with a same duration movie at different framerate)
- Encode the file with the encoder of choice applying 3:2 pulldown (this will encode only the 23.98fps stream but will mark the repeated frame flags to output a 29.97 file)

[Mai]

Hi,

Have you used DGIndex do index your mpeg file? ... How do you have indexed your file (with Honored Pulldown, forced film or ignored pulldown?)

Ah sorry left that out: The index comes from an mp4.

ffmpeg -i orig.m2v -an -vcodec copy orig.mp4

If so, is your file at 23.98fps?

Inside my avs I use FFMpegSource to load the above mp4. What I get from this is a video stream with 24000/1001 fps which is fully progressive.

Does it contain soft or hard pulldown?

So as far as I understand that, it should be soft pulldown, right?

For curiosity I ran a filter across the untampered mp4 output from my avs, which looks for interlaced frames. It does this by visually checking the content of a frame and ignoring all other properties of an input stream. I looked at the results: it could not find any interlaced frames in there. This stream should be progressive, only containing "real" frames from the movie, all the way through...

Have you IVTCed the file?

I guess that's a >yes< then?

Encode the file with the encoder of choice applying 3:2 pulldown (this will encode only the 23.98fps stream but will mark the repeated frame flags to output a 29.97 file)

So I did using TMPGEnc; the two settings of interest should be "Frame Rate" -> "23.976 (internally 29.97fps)" and "Encode Mode" -> "3:2 pulldown when playback"

Is there a tool maybe, or can ffmpeg somehow show us some deeper info on such an m2v?

Thanks so far
Cheers Mai


Edit:
The longer I think about it ... the problem doesn't seam to lie on the encoding side of my pipe line, because the results were the same with x264. And I've checked, the sync error really is 25 or 26 frames. So roughly 1.1 sec. across the whole movie; that difference in fps is like 0.015 % of 24000/1001 -> 23,9796203796 vs 23,976023976?

[mp3dom]

I don't see the reason to put an mpeg2 into an mp4 container and then index the latter with ffmpegsource.
A 'safe' way would be to index directly the m2v with DGIndex or DGTools honoring pulldown and then fed it to the encoder (at original framerate or inverse telecined in avisynth, if film).
Your original m2v could have mainly soft pulldown but at scene change the original encoder can leave some hard-pulldown. The correct IVTC process (with TFM+TDecimate for example) would 'lock' the framerate to a fixed and synchronized 23.98fps.
Your 'way' of re-encoding the m2v seems to me not completely safe about fixed frame rate. For example how your pipeline would handle an m2v with both soft and hard pulldown?

[Mai]

Goood evening!

I don't see the reason to put an mpeg2 into an mp4 container and then index the latter with ffmpegsource.

Nope didn't index the mp4. I did the whole thing to get an index by building an mp4. But as you pointed out DGIndex of course does a better job at this

For example how your pipeline would handle an m2v with both soft and hard pulldown?

By reporting it as 24000/1001 with some interlaced frames in there, tried that on another occasion.

Your original m2v could have mainly soft pulldown but at scene change the original encoder can leave some hard-pulldown.

That was the piece of information I was missing, thanks! -> Now I did do it with DGIndex and IVTCed the whole thing in avisynth -> exactly 26 frames less than previous attempts. I have to encode it and check the end results, but I guess this is solved

Cheers Mai


Edit:
Did encode and check: 99.9% (or so) looks good. But there are still some interlaces frames in there.

Can the pull down pattern jump just for some frames? Because that would explain that. I'm just running the above mentioned filter to check for interlaced frames: so far the ones it found were all on scene boundaries, and my guess is they will sum up to 26 So it seams that the pull down pattern for the hard pd frames is different from the one for soft pd frames -> did I do something wrong at the indexing stage (I did choose "honor pulldown")? Otherwise I could correct them manually. -> learning some good stuff here -> awesome