FFMS2000 - The experimental future of FFMS2

[Myrsloik]

Originally Posted by Selur

Feature request: Support .mpls parsing. It's really a pain to always load all the separate files a playlist makes up and later join them together,... (FFmpeg itself can handle playlist files fine when build with libbluray)

That sounds complicated... I'd kinda like to avoid it. Maybe someone can make a ffms2 compile with libbluray and see what happens. Maybe it'll work automatically then. I don't think anyone's done that.

[Myrsloik]

Test3 uploaded. Some more fixinations and improvements. Now handles all your VP9 needs!

[shekh]

Is it good idea to merge (some part of) it to VirtualDub source plugin? Basically what I miss from bare ffmpeg is accurate per-frame timestamps.

[Myrsloik]

Originally Posted by shekh

Is it good idea to merge (some part of) it to VirtualDub source plugin? Basically what I miss from bare ffmpeg is accurate per-frame timestamps.

I would definitely make it a separate input plugin. All the tricks used can lead to issues worse than naive ffmpeg usage. I wouldn't mind including vdub support of it's contributed.

[shekh]

I already accumulated a frightening amount of tricks to make ffmpeg work.. Want to add some sort of option, not separate plugin.
Can you briefly describe what ffms does now? My idea is to just run a scan of stream and save pts/dst of each frame (similar to running ffprobe). And which frames are keys.

[Myrsloik]

There's no brief description. It's a long list of horrible ideas. See the source. Your hacks are a puny collection. We parse the bitstream for some formats. Use dts OR pts depending on the time of day. Switch between byte and timestamp seeking. Guess decoding delays (badly). Audio is even nastier.

[TheFluff]

Originally Posted by shekh

I already accumulated a frightening amount of tricks to make ffmpeg work.. Want to add some sort of option, not separate plugin.
Can you briefly describe what ffms does now? My idea is to just run a scan of stream and save pts/dst of each frame (similar to running ffprobe). And which frames are keys.

That'll just get you to where FFMS was like five years ago. FFMS has an API, use that.

I think the main thing we've learned from FFMS is that while ffmpeg has a unified API that is the same for all formats, you can definitely not count on every format behaving the same (or in fact even behave similarly), nor on the documented functionality/behavior to work consistently for all formats, etc etc. One of the best ways to illustrate this is some of the comments in the ffms source - where someone's been forced to add some godawful workaround and explained why. Some of my favorites:

https://github.com/FFMS/ffms2/blob/f...exing.cpp#L471
https://github.com/FFMS/ffms2/blob/f...track.cpp#L218 (really, just read this entire source file, it shows why your idea is painfully inadequate)
https://github.com/FFMS/ffms2/blob/f...ource.cpp#L654
https://github.com/FFMS/ffms2/blob/f...ource.cpp#L160 (again, audiosource.cpp is worth reading in its entirety)

[shekh]

Yes I have experience. My favorite is "block_align" field. The docs should really say "this is meaningless value, we just need room for random stuff"
I wonder if any issue found by FFMS resulted in some fix in FFMPEG to improve it?

[Myrsloik]

Originally Posted by shekh

Yes I have experience. My favorite is "block_align" field. The docs should really say "this is meaningless value, we just need room for random stuff"
I wonder if any issue found by FFMS resulted in some fix in FFMPEG to improve it?

I don't think any meaningful fixes happened. Framr accurate seeking isn't a supported usr case.

[Mystery Keeper]

Test 3 still gives me wrong frames for VP9. Further in time compared to the source video.

[dipje]

I have issues with .MTS files (avchd so they contain h264 and ac3).

The files have some weird timing . The audio starts sooner than the video most of the time. It seems when I press record on my camera it starts capturing audio immediately, but video takes half a second or so to get going. So the timecodes of the videostream don't start at 00:00:00. What ffms2000 seems to do is freeze-frame the first videoframe until a/v sync is achieved.
But halfway through the files the frames seem to jump backward and forward , weird glitches. Glitches that aren't there with ffplay or playing through mpc-hc and lavfilters.

The thing is, I don't think this is a regression from ffms2, I think ffms2 also has the same issue, and the issue comes from the libavformat demuxer for MTS which is 'not recommended' I believe.

In older ffms2 versions (2.1 or earlier even ?) You could disable the libav demuxer during indexing, so it would use another one (don't know which, haali or something from matroska that could read MTS ? ). Or maybe I remuxed the MTS files to mkv to prevent issues.

Anyway, lsmash libav reads them ok. It uses a different MTS parser right ?

All other files I use recently (magicyuv 10bit RGB, dnxhr 444, Cineform RGB ) seem to work just fine. (Only tested through Vapoursynth x64)

[Myrsloik]

Originally Posted by dipje

I have issues with .MTS files (avchd so they contain h264 and ac3).

The files have some weird timing . The audio starts sooner than the video most of the time. It seems when I press record on my camera it starts capturing audio immediately, but video takes half a second or so to get going. So the timecodes of the videostream don't start at 00:00:00. What ffms2000 seems to do is freeze-frame the first videoframe until a/v sync is achieved.
But halfway through the files the frames seem to jump backward and forward , weird glitches. Glitches that aren't there with ffplay or playing through mpc-hc and lavfilters.

The thing is, I don't think this is a regression from ffms2, I think ffms2 also has the same issue, and the issue comes from the libavformat demuxer for MTS which is 'not recommended' I believe.

In older ffms2 versions (2.1 or earlier even ?) You could disable the libav demuxer during indexing, so it would use another one (don't know which, haali or something from matroska that could read MTS ? ). Or maybe I remuxed the MTS files to mkv to prevent issues.

Anyway, lsmash libav reads them ok. It uses a different MTS parser right ?

All other files I use recently (magicyuv 10bit RGB, dnxhr 444, Cineform RGB ) seem to work just fine. (Only tested through Vapoursynth x64)

Do you have a sample MTS file? It could be timestamp discontinuities (or something else) we're trying to fix right now.

I have no idea what l-smash uses. All I know is that it does everything very differently.

Haali splitter support was removed several years ago because haali stopped developing it looooooooong ago (and lavf got better)

[zub35]

Add the ability to convert vfr to cfr using two methods: duplicate or blend.
Only DSS2 does the correct conversion, by duplication.

[Myrsloik]

Originally Posted by zub35

Add the ability to convert vfr to cfr using two methods: duplicate or blend.
Only DSS2 does the correct conversion, by duplication.

Duplication already exists. Also lrn2vfr. It's 2017.

[zub35]

Originally Posted by Myrsloik

Duplication already exists. Also lrn2vfr. It's 2017.

This is not right. ffms averages fps and discards frames that exceed it.
Dss2 works with the top bar of VFR and duplicates frames to it, without discarding.
p.s. Or maybe I do not understand something ...

Also that's when fps jumps to the insane 1000, and then all the other frames are duplicated to it, which is also bad.
It is necessary that ffms find the average maximum value, discarding too large bursts.

[Myrsloik]

Originally Posted by zub35

This is not right. ffms averages fps and discards frames that exceed it.
Dss2 works with the top bar of VFR and duplicates frames to it, without discarding.
p.s. Or maybe I do not understand something ...

No frames are discarded umless you use the cfr mode. You're wrong.

[zub35]

sample: https://cloud.mail.ru/public/BX68/Cxg3rF7iX (sorry, only erotic)
dss2 30fps and 20970 frames [11:38.999]
ffms2000 20.220fps and 14113 frames [11:37.958]

Because of what, the video in ffms is accelerating, then slowing down

[Myrsloik]

Don't even have to look at it. It's because it has dropped frames (duplicates/nvops) and only returns actually coded frames. The file is vfr and you're using the vfr mode. Everything works as expected.

[zub35]

The video is not behaving naturally. Rides with acceleration and deceleration. This not normal
Also, it would still be great ffms2000 worked with timecodes_v2 in this conversion vfr to cfr, loading them from file. This was-would be useful for raw streams.

[Myrsloik]

Originally Posted by zub35

The video is not behaving naturally. Rides with acceleration and deceleration. This not normal

Lrn2vfr. The dropped frames do exactly that.