FFmpeg & MPEG2 discontinuity [Archive]

View Full Version : FFmpeg & MPEG2 discontinuity

nab622

1st October 2021, 00:11

Hi! I'm new around here, and I was hoping maybe someone might have some insight.

I'm creating an automated system of Python scripts to rip DVDs, and using FFmpeg, reprocess them into a modern format (Including the original image-based subtitles & closed captions), and create a digital library, including all content from the discs. Everything works great, except for one major snag.

Some discs have discontinuity in the VOB files, and apparently MPEG2 discontinuity is not supported by FFmpeg's demuxer, so when I try to run a concat: operation on the video files, FFmpeg gets stuck in an infinite loop. Since I am doing this with commercial DVDs, I cannot send a copy of the data to the FFmpeg team to create a bug report.
Here's a video of what happens (1:15 onward):
https://www.youtube.com/watch?v=H8qFeB-NtHY&t=75s

So I have two questions:

Is there some way to create non-copyrighted files worthy of a bug report to FFmpeg? (Or maybe someone here is an FFmpeg coder willing to take a look at what I have? I'd be happy to work 1-on-1 with someone!)

Is there some Linux, command-line based workaround that can fix the discontinuity before FFmpeg touches the files? Every single program seems to be moving toward using FFmpeg internally, so ultimately FFmpeg needs to be fixed, but I'm still curious. I tried AVIDemux, but if I recall correctly, it discarded the subpictures, so I can't use it.

I am running Manjaro Linux, in case that matters.

Katie Boundary

3rd October 2021, 06:50

Mpeg2 "discontinuity"...?

hello_hello

3rd October 2021, 07:24

Until someone who's familiar with ffmpeg comes along......

If the vob files are split into segments, concat is probably the wrong method for joining them, but I can't find what the correct method might be, if ffmpeg has one.

Try MKVToolNixGUI with a set of vob files. Add the first vob file in the set to the multiplexer. Join the remaining vob files from the right click "add files as additional parts" menu. There's an Append option, but that's not the correct method for segmented vob files. Save the vob files as a single MKV and try giving that to ffmpeg.

If the vob files aren't split into segments that's not the problem, but can you rip a DVD as a single vob file per movie or TV episode rather than splitting them? Normally I'd rip a DVD and re-author it at the same time with DVDShrink, but it's a Windows program and I'm not familiar with much Linux stuff.

Emulgator

3rd October 2021, 08:35

You can just binary append all _1.._9VOBs to one single vob.
It had been described here some times.
copy /b <filename1> + <filename2> + ... does it.

For discontinuities I would just use a windows machine and go from there.
SmartRipper can rip as one vob.
PTS/DTS Discontinuity: VideoReDo
GOP Timestamps: DVDLabPro, ReStream.
Multi-PGC ? Angle change ?
Reauthoring comes into play.
Not yet implemented for Linux at given time I guess.

Richard1485

3rd October 2021, 09:34

Is there some Linux, command-line based workaround that can fix the discontinuity before FFmpeg touches the files?

Yeah, that is what you should be looking for, because this kind of demuxing issue is the sort of thing that's generally not well supported in FFmpeg. How are you ripping the discs in the first place? Dvdbackup (https://wiki.archlinux.org/title/Dvdbackup)'s title-extraction capability might help. Alternatively, MakeMKV can rip titles as individual MKVs. I tend to use the latter because it's the easiest method on Linux (which doesn't really have a PGCDemux equivalent that matches its reliability/versatility).

I realize that your question is generic, and that your use of TNG in the video was just an example, but if you are planning to rip that series, consider getting the Blu-rays instead: re-encoding (if that's what you mean by putting content in a modern format) those particular DVDs will present a whole range of problems, way beyond simple demuxing issues.

richardpl

3rd October 2021, 10:20

nab622

3rd October 2021, 12:35

Wow, first off, thanks for the responses. I didn't expect to see so many. I'll try and address everything I can here...

Mpeg2 "discontinuity"...?
MPEG2 files can have non-sequential timestamps throughout them. Anything non-sequential causes issues in FFmpeg's MPEG2 demuxer because it is unsupported, and if it's bad enough FFmpeg will lock up completely.

If the vob files are split into segments, concat is probably the wrong method for joining them, but I can't find what the correct method might be, if ffmpeg has one.
The correct way with FFmpeg is to use a perl script called dvd2concat along with FFmpeg to stitch the files together. The problem I am having is that it creates stutters at every join it creates (Usually one per DVD chapter). I am attempting to circumvent this by modifying dvd2concat to use FFmpeg's concat protocol in every place it possibly can, but this is where the discontinuity breaks everything.

Try MKVToolNixGUI
I'm really trying to keep this to command-line only. I need to automate everything so I'm not sitting at the computer babysitting the entire operation. I might try MKVToolNix over command line if I have to, but it's certainly not my first option.

If the vob files aren't split into segments that's not the problem, but can you rip a DVD as a single vob file per movie or TV episode rather than splitting them? Normally I'd rip a DVD and re-author it at the same time with DVDShrink, but it's a Windows program and I'm not familiar with much Linux stuff.
Yeah, I've used DVDShrink in the past. It's okay for what it does, but it does have some issues and it also can't read newer discs. That said, I have ditched windows and it is spitefully relegated to my malfunctioning beater machine, which I barely use.

You can just binary append all _1.._9VOBs to one single vob.
It had been described here some times.
I did try that method when I started this project, but abandoned it pretty quickly. If I do that, the titles will have to be extracted manually. I don't want to manually make precision cuts in the file if I can help it.

How are you ripping the discs in the first place?
I'm using dvdbackup to mirror the entire disc to the hard drive and lsdvd to generate metadata for the Python scripts, then using FFmpeg for everything else. In the past I had tried extracting individual titles with dvdbackup, but I ran into problems. I don't remember what was wrong. And if I recall correctly, the discontinuity remained, so it was still unusable for FFmpeg, which is the most important part of this.

Alternatively, MakeMKV can rip titles as individual MKVs
Was not aware of this. Might look into it if no FFmpeg solution/bugfix happens.

I realize that your question is generic, and that your use of TNG in the video was just an example, but if you are planning to rip that series, consider getting the Blu-rays instead: re-encoding (if that's what you mean by putting content in a modern format) those particular DVDs will present a whole range of problems, way beyond simple demuxing issues.
I do plan to rip TNG in it's entirety, in fact I'm planning to rip all Trek shows and movies from TOS through Enterprise. I got them on DVD specifically because I prefer the cheesy practical effects and didn't want the fancy modern CG ships. But now you've made me curious, what other problems are you referring to with the TNG discs? I know there are volume leveling issues, but those are being (Mostly) corrected by my Python script.

When I say re-encoding, here's what is happening from beginning to end:

Input videos are deinterlaced, lightly denoised and converted to AV1.
All audio tracks are normalized to peak and converted to OGG (Opus doesn't support 5.1(side) and 5.1(rear) or I would use that).
All subpictures are copied over with the correct color palette applied.
Closed captions are extracted from the video and converted to .ass.
Everything is assembled into an MKV, complete with a font file for the closed captions to use.
The entire re-encoding process is already automated. At this point the ripping is the holdup. And of course, I need to make sure the re-encoded files work correctly, but I can't do any of that without getting past the discontinuity problem.

Could you post exact errors that ffmpeg outputs?
It starts out as a lot of Non-monotonous DTS errors. When FFmpeg gets stuck in the infinite loop, a mess of buffer underflow errors start happening too. Have a look at the video, the sheer number of errors is staggering. I have let it run for more than 24 hours and it encoded less than 10 MB of video without adding anything to the playback time, so I have no idea what it was doing.
https://www.youtube.com/watch?v=H8qFeB-NtHY&t=75s

Richard1485

3rd October 2021, 13:38

Was not aware of this. Might look into it if no FFmpeg solution/bugfix happens.

Yeah, it's how I rip TV series, although I rarely re-encode afterwards.

But now you've made me curious, what other problems are you referring to with the TNG discs? I know there are volume leveling issues, but those are being (Mostly) corrected by my Python script.

The TNG DVDS, like the DS9 and Voyager DVDs, are hybrid material, which makes them impossible to inverse telecine without compromises and therefore they've been the bane of re-encoders since they were released. Some people find solutions that satisfy them, but I prefer the BDs. Maybe have a look in the DS9 upscale (https://forum.doom9.org/showthread.php?t=181209) thread.

I do plan to rip TNG in it's entirety, in fact I'm planning to rip all Trek shows and movies from TOS through Enterprise. I got them on DVD specifically because I prefer the cheesy practical effects and didn't want the fancy modern CG ships.

Understood. The original effects are available via seamless-branching on TOS BDs.

nab622

3rd October 2021, 20:58

Yeah, it's how I rip TV series, although I rarely re-encode afterwards.
I'm just trying to save on some space. I'd like to be able to use these on my phone so I can take them to work with me over the holidays, amongst other things. Ultimately, though, my plan is to have a web-based library running on my LAN at home that acts as a directory for all my DVDs. Like Hulu/Netflix, but everything's locally hosted and has no ads. Yes, I'm crazy.

The TNG DVDS, like the DS9 and Voyager DVDs, are hybrid material, which makes them impossible to inverse telecine without compromises and therefore they've been the bane of re-encoders since they were released.
Interesting....I don't know anything about telecine, but that might explain why FFmpeg could not process the video correctly into AV1 without converting to a lossless MP4 first. I looked at the upscale page linked in the thread you pointed out, and while I do have some very minor interlacing (Telecine?) artifacts occasionally, I don't have anything nearly that bad. In fact, mine actually looks pretty good by comparison. Maybe I just haven't hit an episode that is problematic yet, or somehow FFmpeg is fixing it in a way that actually looks okay. Do you happen to know of a problematic episode in TNG? (I don't want to spoil DS9 by using a later episode if I can help it, I'm still watching that)
Edit: I will go through the rest of the DS9 upscale thread later on, I don't have time at the moment. Interesting stuff.

Understood. The original effects are available via seamless-branching on TOS BDs.
Did not know that! I was disappointed that the TOS DVDs had a CG Enterprise, although after looking through the special features and seeing how decayed the film was, I understand it to a point. I'll probably get them all on blu-ray eventually anyway, but I wanted to start with DVDs because I know how to handle them. Well, mostly, apparently.

Richard1485

3rd October 2021, 21:28

Yes, I'm crazy.
No, what you are doing is perfectly reasonable, and I understood straight away. I re-encode (progressive) content to save space myself, but when it comes to TV series, especially hybrid material, I tend to leave them as they are because it takes too much time and effort to sort them out "properly" with AviSynth, and I can't be bothered these days.
Do you happen to know of a problematic episode in TNG?
No. I haven't looked at them in a long time; it's a generic problem with the series. The DS9 thread is useful because people post/discuss the merits of various ways of IVTCing/deinterlacing prior to upscale. But most people don't notice much of a difference anyway, so for casual viewing, maybe it doesn't matter. If you're happy, you're happy. :)

nab622

4th October 2021, 12:57

The DS9 thread is useful because people post/discuss the merits of various ways of IVTCing/deinterlacing prior to upscale.
So I just tested a few clips from the Voyager DVDs between VLC, MPV and a physical DVD player...and while VLC and MPV both had very clear telecine artifacts, the official DVD player had a beautiful picture with no apparent interlace or telecine issues (I only compared a few scenes, but the difference was striking). Is it just me, or is it possible that the open-source MPEG2 codec is missing some very important detelecine data?? If this is true, then the output will always be borked no matter what I do, so I won't bother trying to detelecine the files. Instead, I'll probably opt for a method to detect telecine and keep the original MPEG2 video intact, despite the larger size.

Either way, thank you very much for bringing it to my attention, I had no idea what telecine was!

Richard1485

4th October 2021, 13:47

So I just tested a few clips from the Voyager DVDs between VLC, MPV and a physical DVD player...and while VLC and MPV both had very clear telecine artifacts, the official DVD player had a beautiful picture with no apparent interlace or telecine issues (I only compared a few scenes, but the difference was striking). Is it just me, or is it possible that the open-source MPEG2 codec is missing some very important detelecine data??

It's not deletecine data as such. Your DVD player is almost certainly sending the picture to the television (monitor?) in a different way from the computer. This gets horrendously complicated fast because Progressive scan DVD players (https://en.wikipedia.org/wiki/Progressive_scan_DVD_player) work differently from older units, and televisions vary as to the refresh rates that they support and how they handle/display input (CRTs being different from modern panels), but you can experiment with changing the refresh rate (even with interlaced options) under display settings in whatever desktop environment you are running (or in xrandr) to see how your television displays the input. You'll probably find one that approximates what your DVD player is doing. (Many televisions now have frame-interpolation/motion-smoothing modes aimed at sports and the like, so watch out for that.)

Instead, I'll probably opt for a method to detect telecine and keep the original MPEG2 video intact, despite the larger size.
That's generally what I do -- look for a solution on the fly and save myself the re-encoding time at the cost of increased file-size.

nab622

2nd November 2021, 12:32

I FINALLY managed to figure out the discontinuity issue. Because it was so complicated, I'll post a full workaround here for anyone searching it in the future. Buckle up, because this is a mess, but it actually turns out a beautiful product.

Since using dvd2concat with ffmpeg will result in a desync'd video and pops in the audio, it cannot be used for them. However, it will need to be used for the subtitles, because they get all bugged up otherwise.

Here's the full process (Linux only!):

1. Using lsdvd, get all DVD data and save it somewhere.
2. Using dvdbackup, mirror the entire DVD to the hard drive.
3. Using dvdbackup again, extract the desired title from the copy you made in step 2, but save it somewhere else. You should get a number of VOB files.
4. Using ffmpeg, concatenate the VOBs created in step 3 (Do not use dvd2concat, just concat: the files directly!). Save only the video and audio, no subtitles.
5. Using ffmpeg and dvd2concat, extract the desired title from the mirrored DVD from step 2, but save it under a different name, and use -vn and -an to remove the video and audio. All we need in this file is the subtitles.
6. Using ffprobe, find out the exact duration of the file created in step 4.
7. Find the duration of the desired title as reported by lsdvd in step 1.
8. Divide the number from step 6 by the number from step 7. (The durations will not be the same, so we have to scale the subtitles using this number)
9. From the lsdvd data in step 1, look through the subtitle tracks and find the earliest start time.
10. Using ffprobe, find the time of the earliest starting subtitle track from the video file created in step 5.
11. Subtract the number from step 10 from the number in step 9, then multiply the result by the number in step 8.
12. Using ffmpeg, merge the files from steps 4 and 5 together. Use -map to take the video and audio from step 4, and the subtitles from step 5. In the ffmpeg command, just before the input command for the subtitle track, add the following:
-itsoffset <Number from step 11>
-itsscale <Number from step 8>
These two options will adjust the start time and scale of the subtitle tracks to match the output video file (Using dvd2concat messes this up).

Repeat steps 3-12 until all the titles you want have been extracted from the disc.

All ffmpeg operations should use -c copy, so nothing gets re-encoded. As such this is a relatively quick process when automated; it's just a pain to code because of all the workarounds it requires. I'm sure this process isn't perfect, either, but at least it works.

Oh, and when using ffmpeg, make sure to set -analyzeduration and -probesize every time you use it, or you'll have all kinds of stuff go missing from your resulting files.

Hopefully this helps someone!!