FFmpegSource - C-plugin

[FranceBB]

I see... Thanks for this new build, but I would still change a thing.
(I know, I know, you are gonna hate me right now xD).
I tried to open a 10bit file and it says "no suitable output format found", as expected.
I tried to output an 8bit dithered video from a 10bit source and it worked, as expected.
I then tried to output a 16bit video via the 10bit hack, but it didn't work.
My question is: how can I output a 16bit video (stacked - Dither Tools - or interleaved - HDRCore) in plain Avisynth via ffms2 C plugin?
I think you should enable the 10bit hack for Avisynth users, otherwise we won't be able to preserve the bitdepth of the source.

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv")

no suitable output format found

---

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv", enable10bithack=true)

FFVideoSource does not have a named argument "enable10bithack"

---

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv", colorspace="YV12")

Works: output is dithered down 8bit 4:2:0 video

Thank you in advance.

[qyot27]

No. The 10bit hack was never in the C plugin (or mainline FFMS2, for that matter), and there's no reason for it to be put in now that we have an option that can preserve and give proper high bit depth input/output. If you want to use high bit depth, use AviSynth+.

[manolito]

When using the expanded script manolito posted, it crashed at 6½ seconds (roughly). I whittled down the problem to the fpsnum=/fpsden= settings. Which is strange, since I don't think I've ever touched anything related to those in the C-plugin; anyone that can test with the main C++ FFMS2, does it happen when fpsnum/fpsden on this clip?

Thanks for looking into it, and thanks for the new build, too... :thanks:

I tried using the very old C++ version 2.17 of ffms2 (the last version which runs on my machine), and it handled it nicely. I think I saw a short hickup pause at 6 1/2 seconds, but the conversion went on smoothly.

I was too lazy to pull out my Win7 laptop from the closet to test it with version 2.23.1 (last version on GitHub), but I think it does not really matter. The problem clearly comes from the broken source file. After transcoding this source to exactly the same format using either FFmpeg or AviSynth -> H.264 and feeding the transcoded file to AVStoDVD using the latest ffms2 C-plugin (making sure that fpsnum and fpsden were in the params list), the conversion proceeded all the way to the end.

My conclusion is that the older ffms2 versions are just more tolerant when they encounter broken source files. No big deal...


Cheers
manolito

[qyot27]

If you're using 2.17 to test, make sure to force the lavf demuxer; that's the only way to make sure it's directly comparable to current builds. The Haali demuxers (which may have been more tolerant of bad streams, I'm not sure) were removed a couple or three years ago, so newer FFMS2 builds only use libavformat.

[manolito]

OK, I did my homework, here are the results:

Not so sure about using Haali demuxers when testing version 2.17, simply because Haali is not installed on my computer. I do have avss_26.dll from DSS2Mod in my plugins folder, but would this be enough to enable Haali demuxers in ffms2?

Anyways, I fired up my Win7-64 laptop and repeated the conversion using the C++ version 2.23.1 from GitHub and the latest FFMS2000 test8 build from Myrsloik. Interesting results:

Version 2.23.1 handled the broken source file just fine, FFMS2000 crashed at the same point where the current C-plugins crash.

Working:
C++ versions 2.17 and 2.23.1
C-plugin version 1140+101

Not working:
C++ version FFMS2000 test8
All C-plugin versions after 1140

But again, this is caused by the broken source file. After transcoding this source file all ffms2 versions handled it correctly.


Cheers
manolito

[qyot27]

Ok, well if it occurs in mainline FFMS2, then the root cause is in the FFMS2 core, not in the plugins. Since it only occurs with fpsnum/fpsden, it probably involves one of the good handful of commits about timestamp or frame rate handling that came in over the course of 2017, although without bisecting it, I won't know for sure which one is the culprit. Likely it's specifically using fpsnum/fpsden on broken files like that, because the timestamps are already messed up.

My memory is really fuzzy involving the Haali stuff (but it doesn't matter since we know it happened in both the C++ and C plugins around the same time). The MPEG and OGG demuxers relied on the Haali DirectShow splitter, though, so I'm not sure if that was the *.dll or the *.ax file.

[Selur]

@qyot27: Encountered a strange problem with this (https://drive.google.com/file/d/1kmhaQxO3Au6B6G7Fuv4BppY43Q4ASxtI/view?usp=sharing) file.
It works fine with the 64bit Vapoursynth filter, but with Avisynth 2.6 MT I get dropped and repeated frames.

Using just FFVideoSource:
FFVideoSource("C:\Users\Selur\Desktop\2SOURC~1.AVI",cachefile="H:\Temp\avi_3a7584798a428c1a525d134a8f55420b_18467_1_0.ffindex",fpsnum=30000,fpsden=1001)
# current resolution: 640x480
return last
Frame with time 14 is shown as frame 14&15.
Frame with time 29 is shown as frame 29 and 30.
Frame with time 31 is missing.
Frame with time 32 is shown as frame 32&34.
Frame with time 33 is missing.
Frame with time 54 is shown as frame 54&55.
Frame with time 55 is missing.
Frame with time 56 is shown as frame 56&57.
Frame with time 59 is shown as frame 59&60.
Using just FFms2000:
same result as when using FFVideoSource.
Using just LWLibavVideoSource:
All frames are properly decoded.
Installing a Xvid vfw decoder and using AviSource:
All frames are properly decoded.

Using Vapoursynth + FFMS2K
All frames are properly decoded.
Using Vapoursynth + FFVideoSource
All frames are properly decoded.
Using Vapoursynth + LibAV
All frames are properly decoded.

Cu Selur

[manolito]

Looks like a problem with Avisynth 2.6 MT to me. Under plain vanilla AviSynth 2.60 (and 2.61 Alpha) I cannot reproduce these issues, I cannot see any dropped or repeated frames.

Have you tried to use MPEG4 Modifier by Moitah to remove the Packed Bitstream from the source file?


Cheers
manolito

[qyot27]

That it does the same thing when using FFMS2000 means that the issue is not C-plugin-specific. That it doesn't occur with VapourSynth means it's probably not in the core.

Here's the thing, though: it only happens when forcing fpsnum= and fpsden=. It doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion. So that would seem to suggest that it's an issue with the AviSynth plugin interface code, but since FFMS2000 and the C-plugin use two entirely different plugin codebases, the fpsnum/den bug when handling AVI would have to either have been implemented separately in the C-plugin, or it arises from some quirk of AVI, AviSynth, libavformat and/or libavcodec, or some combination of the three.

It does seem to be AVI-specific; I remuxed into MKV two different ways, as native MPEG4 and as VFW-compat, and both of them decoded the frames correctly even with fpsnum/den present.

[manolito]

it only happens when forcing fpsnum= and fpsden=.

This is only partly true... :devil:

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

This means that we are dealing with a very stupid bug in one of the libraries which the latest C-plugin uses. Is there any chance to find and fix this bug?


Cheers
manolito

[hello_hello]

Here's the thing, though: it only happens when forcing fpsnum= and fpsden=. It doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion.

FPSNum and FPSDen have been problematic for as long as I've been using them. I think it's due to timecode jitter and as a result, FFmpegSource sometimes drops or duplicates frames unnecessarily so it's best not to use frame rate conversion unless you need to convert the frame rate.

FFmpegSource isn't alone though. LSmash can do the same thing. Mercifully though, they don't tend to misbehave for the same sources, so usually if one doesn't work correctly, the other will.

And there's always the VFRToCFR (https://forum.doom9.org/showthread.php?t=165045) plugin.

[hello_hello]

This is only partly true... :devil:

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

I don't know how timecodes work or what the source file was, but that's probably just some sort of timecode jitter that doesn't upset FFmpegSource when it's using 2997/100.

Sometimes remuxing before indexing can fix the problem, even if you have to use a different container.

I've bumped into this problem when encoding 25fps AVIs, so it's not the NTSC frame rate as such.
After trying to convince Zathor it'd be a good idea for MeGUI to use it by default, I then tried to convince him that it wasn't so clever after-all.
https://sourceforge.net/p/megui/feature-requests/591/

[manolito]

It gets even weirder... :rolleyes:

I reverted back to the older C-plugin version 1140+101 fully expecting that the fpsnum/fpsden issue would be solved (like it did in earlier tests). But not this time. The output showed the same errors as the latest C-plugin version 1315+119.

Then I went back to the ancient Standard-plugin version 2.17, and this version handled the source correctly. Of course this is not really helpful because this old version cannot decode HEVC sources.

What does this mean? Could the issue lie in the C-plugin core after all? So far the only reliable workaround when using the C-plugin seems to be to ditch fpsnum/fpsden completely and follow the ffvideosource call immediately with a ChangeFPS command, like qyot27 recommends (I still do a lot of conversions to DVD, and VFR to CFR conversion is essential for me).


Cheers
manolito

[hello_hello]

It gets even weirder... :rolleyes:

I reverted back to the older C-plugin version 1140+101 fully expecting that the fpsnum/fpsden issue would be solved (like it did in earlier tests). But not this time. The output showed the same errors as the latest C-plugin version 1315+119.

Then I went back to the ancient Standard-plugin version 2.17, and this version handled the source correctly. Of course this is not really helpful because this old version cannot decode HEVC sources.

What does this mean? Could the issue lie in the C-plugin core after all? So far the only reliable workaround when using the C-plugin seems to be to ditch fpsnum/fpsden completely and follow the ffvideosource call immediately with a ChangeFPS command, like qyot27 recommends (I still do a lot of conversions to DVD, and VFR to CFR conversion is essential for me).

If the source was VFR, it'd be decoded at the average frame rate, the audio would no doubt go out of sync, and ChangeFPS() won't do anything to fix that. It has to be decoded at the correct frame rate.
For minor 29.970fps to 30000/1001 corrections I'd use AssumeFPS() instead. It'll probably only change the duration by a few nanoseconds anyway.
I kind of remember TIVTC uses 23.976 instead of 24000/1001 so I generally follow it with AssumeFPS() too, or I could be remembering wrong and that's a waste of time. :)

It might be the source. If you can, try remuxing it while setting the frame rate to 30000/1001 to see if that makes a difference. It might be that the source wasn't exactly 30000/1001 to begin with, and when it is, the problem will go away.

All that's assuming the source really is constant frame rate. Whenever MeGUI adds frame rate conversion to a script and it's not exactly standard NTSC or PAL, I always remove it and refresh the preview to see if the frame count and total duration changes. If the frame count stays the same and the duration only changes by a few milliseconds, then it's AssumeFPS() instead.

If it is a standard NTSC or PAL frame rate I remove it anyway as I've had too many bad experiences with dropped and incorrectly duplicated frames to use the frame rate conversion unless I really need to convert VFR to CFR.
Is AVSToDVD using MediaInfo to determine the frame rate? It's just that when it's VFR MediaInfo can confabulate all sorts of "real" frame rates.

The same MKV muxed with track statistic tags:
Frame rate mode : Variable
Frame rate : 52.594 FPS

Without them:
Frame rate mode : Constant
Frame rate : 25.000 FPS

That may be just what it displays in the GUI rather than what it tells other software, although as a test I indexed both MKVs with MeGUI/FFMS2.
For the first one MeGUI added fpsnum=26297 & fpsden=500 to the script. For the second it added fpsnum=25 & fpsden=1

I think I mentioned to Zathor at one stage that MeGUI should only add standard NTSC or PAL frame rate conversions because I don't think anything else makes sense. For example, if MediaInfo says the frame rate is 28.45fps, then it's a pretty good bet the video started life as 29.970fps, only duplicate frames were removed. The main point of the VFR to CFR conversion would be to put them back. Encoding at 28.45fps would make a mess, dropping frames that don't need dropping etc. The same probably applies to VFR sources from smart phones etc. If they're an odd average frame rate, you'd want to convert them "up" to the next highest standard NTSC or PAL frame rate. And for AVSToDVD, you're limited to NTSC or PAL frame rates anyway (I assume).

[manolito]

The source is Selur's uploaded file from this post:
https://forum.doom9.org/showthread.php?p=1845939#post1845939

It is an XviD AVI, MediaInfo says that it is 29.97 progressive. It is CFR (I believe that XviD does not support VFR).

In ffms2 the fpsnum/fpsden parameters are used to guarantee CFR output (just like "Convertfps=true" in DirectShowSource). DSS2Mod will always output CFR. I do need CFR output because the DVD specs require it.

If the source was VFR, it'd be decoded at the average frame rate, the audio would no doubt go out of sync


I strongly question this. Decoding of VFR sources at a different average frame rate may certainly ruin the video quality, but it will NOT alter the duration. So there should be no audio sync problems whatsoever.


Cheers
manolito

[qyot27]

That's partially why I said that it may even be down to a quirk of AVI. Since AVI has to adhere to a constant frame-rate anyway*, fpsnum/den conversions are probably not a good idea - actually, short of it being a format with no available VFW decoder, like ffvhuff or newer revisions of ffv1, I wouldn't use FFMS2 to handle AVI files at all. MKV and other containers don't have that restriction, or are more or less assumed to be natively VFR, even if the content's framerate doesn't vary.

*because the only real way to force VFR into AVI is by using dummy frames, which still means that the source is CFR - just at a higher framerate with the lower rate parts padded up to the higher rate of the rest of the content (or both/all of the parts padded out to the lowest common denominator).

So the way this might tie back to the previous fpsnum/den situation is that AVI parsing could be a lot more fragile in terms of what framerates it allows. If the fps is already stored as 30000/1001, it may be okay. But if it was created with a float 29.97 or an integer 2997/100 rate like was mentioned as working, forcing 30000/1001 could be all it takes to trip the problem. And that could be in the AviSynth plugin logic (but again, it would have to exist in both the C++ and C plugins, and since they're separate code it's either really old, or the problem is actually in libavformat and can only express itself in a constant framerate environment like AviSynth).

For the record, 3000/101 seems to work too, even though the end result of that is 29.7029703. It makes me wonder if AVI can even store framerates as integers, or whether it's always converted to float beforehand.

[hello_hello]

The source is Selur's uploaded file from this post:
https://forum.doom9.org/showthread.php?p=1845939#post1845939

It is an XviD AVI, MediaInfo says that it is 29.97 progressive. It is CFR (I believe that XviD does not support VFR).

Neither does the AVI container, although AVI supports some sort of "null frames" which means it can vary.

In ffms2 the fpsnum/fpsden parameters are used to guarantee CFR output (just like "Convertfps=true" in DirectShowSource). DSS2Mod will always output CFR. I do need CFR output because the DVD specs require it.

Decoders always output a constant frame rate, but which frame rate....
AVISource always outputs a CFR and it duplicates frames to replace "null frames". DirectShowSource with ConvertFPS=true does the same.
AVISource is clever enough to output the correct frame rate without being told. For a 25fps AVI with a few null frames it'll output 25fps. By default, ffms2 and LSmash would treat it as VFR and output the average frame rate, which might be 24.992 without the frame rate conversion. They spread the frames out over the total duration.

I strongly question this. Decoding of VFR sources at a different average frame rate may certainly ruin the video quality, but it will NOT alter the duration. So there should be no audio sync problems whatsoever.

Edit: Sorry, when I referred to the audio going out of sync I meant when the video is decoded at the average frame rate without any conversion. I did mention the audio sync when converting to the average frame rate though. I guess I wasn't thinking, but I made up for it by doing a nueron2 and editing my post to remove the evidence :). And then I misunderstood you initially, so explanation deleted.... I'm over-tired.

I still think my logic is sound when it comes to converting to a constant frame rate. Converting to a standard frame rate seems logical to me. ie The appropriate peak frame rate for the source and PAL or NTSC. Unless you're happy to drop frames.

These days I've embraced VFR encoding. Not that I do a lot of it as there's not a huge need for it in PAL-Land, but I doubt I'd ever convert a hybrid NTSC DVD to a constant frame rate again. I only encode with x264 though, and obviously for DVDs you don't have the same luxury.

[hello_hello]

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

It seems to be as I thought it might. If you remux the AVI as an MKV the problem goes away. Well... I only checked two of the problem frames, but they were fine. Not that MKVs are immune, but in this case it seems to work.

Mind you I don't know why AVStoDVD was using "fpsnum=2997, fpsden=100" because MeGUI picked "fpsnum=30000, fpsden=1001".
Although as I type, there's a memory returning.... something about some players crashing when 30000/1001 is used....
I guess AVStoDVD is playing it safe.

http://www.avisynth.org.ru/docs/english/corefilters/fps.htm
In v2.57, the behaviour with respect to the framerate is a bit changed. The main issue is that users are allowed to specify the framerate as float, but the NTSC (FILM and Video) and PAL standards require ratios as framerate. Besides this AviSynth exports the framerate as a ratio, so when specifying a float, it will be converted to a ratio. The ratios of the standards are given by 24000/1001 for 23.976 (FILM) and 30000/1001 for 29.97 (Video). When specifying these floats, they are exported by AviSynth as ratios, but not as the standard ratios. One of the reasons for this is, that those floats are approximations (remember that 24000/1001 = 23.9760239760...), so how should AviSynth know how to choose the correct ratio? In order to overcome this issue, the user can use AssumeFPS(24000,1001) or simply AssumeFPS("ntsc_film").
Another problem is that the converted floats were (in v2.56 and older) exported with 64 bit precision, resulting in very large numerators and denominators, making some players crash. To overcome this, a smart float-ratio is added internally, and the framerates are approximated accurately by ratios of small numbers. For example, AssumeFPS(23.976) is converted to AssumeFPS(2997,125) as can be checked with Info.

[manolito]

OK, I see your points, but for me repacking source files depending on their formats is not an option. I mainly use AVStoDVD for DVD conversions, and the whole point of using such a software is to have solutions which work each and every time no matter what the peculiarities of the sources are...

I think that the best workaround when using ffms2 is to ditch all fpsnum/fpsden parameters from the ffvideosource call and instead follow the ffvideosource call with a ChangeFPS command. I just posted a request for the author to change the handling of ffms2 accordingly.


Cheers
manolito

[hello_hello]

I think that the best workaround when using ffms2 is to ditch all fpsnum/fpsden parameters from the ffvideosource call and instead follow the ffvideosource call with a ChangeFPS command. I just posted a request for the author to change the handling of ffms2 accordingly.

But if the source is VFR, ditching the conversion can potentially cause audio sync issues and ChangeFPS can't help.
That was why I requested MeGUI add it to scripts originally, as then it can convert VFR sources, even if not in an ideal way, but it turned out to be a bad idea due to the frame dropping/duplication issue.

If you use ChangeFPS instead you'd have to be converting from one constant frame rate to another, which is probably what happens most of the time anyway. Or maybe a checkbox might be useful. I assume if AVStoDVD is converting the frame rate it'd be converting to a DVD compliant one, but an ability to tell it the source is CFR or VFR would allow AVStoDVD to use fpsnum/fpsden or ChangeFPS as appropriate.

[manolito]

but an ability to tell it the source is CFR or VFR would allow AVStoDVD to use fpsnum/fpsden or ChangeFPS as appropriate.

Sorry but I do not really understand...

Isn't using fpsnum/fpsden exactly the same thing as using ChangeFPS? According to the ffms2 docs using fpsnum/fpsden forces the specified output fps by dropping or repeating frames. ChangeFPS does exactly the same thing. Where is the difference?

The whole point IMO is to make sure that the output is CFR no matter if the source is VFR or CFR. And if the source already is CFR then it would not matter, using fpsnum/fpsden or ChangeFPS would not do anything in this case.


Cheers
manolito

[hello_hello]

Without fpsnum/fpsden, the frames are just output at the average frame rate. There's no frames added or deleted, they're just spread out evenly. When fpsnum/fpsden is specified the video is converted to the specified frame rate, even if it's the average frame rate, adding and/or deleting them as required to maintain audio sync.
Consider a source that's 23.976 for the first half and 29.97 for the second. You could add frames to the first half or delete frames from the second half or some combination of the two, but you can't just spread the existing frames out evenly.

It makes sense because you don't want FFMS to add or remove frames unless it's specifically told to, or at least without a method to stop it from doing so, otherwise it'd be really hard to encode VFR as VFR, where you want the existing frames with no additions or deletions.

That's what I was getting at before. If the video is VFR it has to be decoded to a specified frame rate. If fpsnum/fpsden isn't specified it's not. ChangeFPS has no idea how the frames are spread out in the source.

Of course if you're happy to assume all your sources are CFR, then dropping fpsnum/fpsden is a good idea, although it mostly works fine when it's converting real VFR to CFR, as opposed to converting 29.970 to 30000/1001, which requires an extra frame roughly every ten hours, so AssumeFPS would make more sense for that sort of thing.

[manolito]

ChangeFPS has no idea how the frames are spread out in the source.

Why would it need to know this? And do you really believe that when using fpsnum/fpsden the plugin does a global analysis of the source before decoding in order to retrieve such knowlege about how frames are spread out in the source? I do not think so...

From the DirectShowSource docs:
convertfps = false (in v2.56): When setting it to true, it turns variable framerate video (vfr) into constant framerate video (cfr) by duplicating or skipping frames.

From the ffms2 docs:
This may naturally cause `FFVideoSource` to drop or duplicate frames to achieve the desired frame rate,

From qyot27:
it doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion.

I still believe that avoiding fpsnum/fpsden and instead use ChangeFPS after the ffvideosource call will render the same result (or amost the same). In the end we should leave the answer to folks like qyot27 or Myrsloik who understand the inner workings of the plugin... :p


Generally the AVStoDVD concept for creating DVD output is to always treat the source as VFR (use DSS2Mod, use convertfps=true for DirectShowSource and use fpsnum/fpsden for ffms2). This will ensure that VFR sources are handled correctly, and for CFR sources no harm will be done. Using AssumeFPS is not an option since this will alter the video duration causing audio sync errors.


I am more interested to find out which version of ffms2 was the first to show these issues. After finding that the standard C++ version 2.17 worked fine I just tried the C-plugin version 2.17. And this one also showed no problems.

Unfortunately I was unable to find C-plugin versions between 2.17 and 2.23 (GitHub does not have them, and the MediaFire links from qyot27 were expired). So all I know is that C-plugin version 2.17 works while version 2.23 (r1140+101) does not work. The issue must have been introduced somewhere between these two versions.


Cheers
manolito

[hello_hello]

Why would it need to know this? And do you really believe that when using fpsnum/fpsden the plugin does a global analysis of the source before decoding in order to retrieve such knowlege about how frames are spread out in the source? I do not think so...

If that's not the exact definition of indexing it's got to be a very close relative. DSS would output the specified constant frame rate, no doubt adding or dropping them as required based on each incoming frame duration. DSS would have to do it on the fly. FFMS2 runs an indexing pass first.
All you need is the frame count and duration and some easy maths to output the average frame rate, but I can't imagine how FFMS2 could convert VFR to CFR by duplicating/dropping frames if it doesn't know the duration of each frame in a VFR source, or in other words the speed at which the source frames are zipping by at any moment in time. ChangeFPS would just see the existing frames at "some" constant frame rate because that's what Avisynth needs, so it can't convert VFR that's been made constant simply by averaging out the frame durations, and if you tell FFMS2 to drop or duplicate frames to convert to a constant frame rate there's no need to ConvertFPS again.

It'd be impossible to re-encode VFR sources as VFR unless only the original frames are decoded. No more, no less.
The standard method is to extract the timecodes file, encode without any frame rate conversion, the script frame rate is irrelevant but the average frame rate seems a popular choice as it doesn't change the duration, give x264 the timecodes file so it knows where the frames really belong time-wise, or use the timecodes when muxing to distribute the frames in the same manner as the source.

Here's what the help file I have says (FFMS2 version 2.23.1)

##### int fpsnum = -1, int fpsden = 1
Controls the framerate of the output; used for VFR to CFR conversions.
If `fpsnum` is less than or equal to zero (the default), the output will contain the same frames that the input did, and the frame rate reported to Avisynth will be set based on the input clip's average frame duration.
If `fpsnum` is greater than zero, `FFVideoSource` will force a constant frame rate, expressed as a rational number where `fpsnum` is the numerator and `fpsden` is the denominator.
This may naturally cause `FFVideoSource` to drop or duplicate frames to achieve the desired frame rate, and the output is not guaranteed to have the same number of frames that the input did.

I just checked a VFR MKV with a section at the beginning that's 25fps while the bulk of it is 59.94fps. The 25fps section has 5877 frames and ends at 3min 55 seconds. 5877 frames at a CFR of 25fps does work out to 3min 55sec. Without FFMS converting the frame rate it still had 5877 frames but in MeGUI's preview it finished at 1min 21sec and wouldn't have looked out of place as a closing for The Benny Hill Show.
With FFMS2 converting to some constant frame rate the frame count increases or decreases accordingly but the scene ends at 3min 55sec.

That's part of the issue people seem to have when ripping DVDs with MakeMKV and then indexing them with FFMS2. Unlike DGIndex, FFMS2 ignores repeat flags by default, which means for hybrid NTSC it just outputs the existing frames, and I'm pretty sure it does so at the average frame rate.
The help file says it assumes a CFR when the RFF option is enabled, although I'd imagine that's not living too dangerously given I assume the RF flags ensure the frame rate is constant anyway.

[manolito]

If that's not the exact definition of indexing it's got to be a very close relative. DSS would output the specified constant frame rate, no doubt adding or dropping them as required based on each incoming frame duration. DSS would have to do it on the fly. FFMS2 runs an indexing pass first.

Alright, you score for ffms2, but for DSS2Mod and DSS there should be no difference when using ChangeFPS instead of "convertfps=true". In the couple of tests I did I could not see any problems with this approach.


We are dealing with two different problems here:
1. How to treat real VFR sources when CFR output is required?
2. How to deal with the regression in ffms2 where fpsnum/fpsden is broken?


For the first problem I had a dispute with Zenitram some time ago when he stopped displaying the frame rate and duration of VFR MKV files. This post by Foxyshadis
https://forum.doom9.org/showthread.php?p=1739508#post1739508
saved my life more than once. Easy to use and good quality output.

For the second problem I would really like to see current ffms2 versions fixed. It did work in version 2.17 for both the standard and the C-plugin, so it must be possible to find the error and fix it. If this is not going to happen then I will need to employ the workaround using ChangeFPS instead of fpsnum/fpsden. I am a result oriented person, I will use "whatever works", but the method has to be universal, it has to work for all kinds of sources without deeply analyzing them.


Cheers
manolito

[qyot27]

To be fair, the situation where I said ChangeFPS would be preferable is not VFR-to-CFR, it was AVI files like that sample specifically (I also don't really know how the fps conversions in FFMS2 perform their logic, it was a general statement about framerate).

The process of using AssumeFPS when the framerate is close enough to standard* is the most correct way, but for safety's sake I always go with sync_audio=true and then run SSRC or ResampleAudio so that the audio frequency gets fixed back to a standard 44100 or 48000 Hz, depending on what exactly it was that I was encoding for. But if you're trying to preserve sync whilst absolutely not touching the audio, the only solution is to drop or insert duplicate frames to do it.

*Even though the minor framerate differences won't result in any audible pitch shift; the shift is there, but human ears really can't tell if 44100 was slowed down to, say, 44030 before getting re-normalized to 44100; not unless you do it dozens of times in a row. Thousandths or tens of thousandths of a frame, here: I usually would find myself having to do it to correct whole 30->29.97 or 24->23.976, but we're talking about stuff like 2997/100->30000/1001; 24/30->NTSC Film/Video is a 1.001% shift, and I couldn't hear the difference - simplified 23.976/29.97->NTSC Film/Video is a difference of 0.0001%.


I kind of feel like we're getting a bit into the weeds here, though.

[hello_hello]

When the source frame rate is already DVD complaint, AVStoDVD should be aware of it and not add any ffms2 frame rate conversion to the script. Ideally it should have no effect if it's added, but obviously that's not what happens all the time.

For VFR frame rates that aren't in the DVD compliance ballpark, and when DSS isn't an option, there's really no other way than to convert it to a CFR as it's decoded. Lsmash has the same conversion options as ffms2, so maybe if your source is VFR and there's incorrectly dropped/duplicated frames in the output, an ability to pick the indexer might be enough. I've definitely seen Lsmash go nuts in the past and duplicate/drop frames it shouldn't (ie when the specified frame rate is the same as the source frame rate) so I'm not sure it's an ffms2 specific bug, but fortunately I don't think I've had a source that neither indexers could convert successfully. Maybe it's just luck.

Still feeling obligated to argue about the ability of ChangeFPS to convert a VFR source to CFR correctly.....

When the VFR source is decoded without ffms2 converting it to a constant frame rate, FFMS2 outputs the average frame rate.
The average frame rate is pretty much the VFR equivalent of AssumeFPS, only because it's the average frame rate it can do the assuming without changing the duration. Still, it'll effectively speed up some sections and slow down others, just like AssumeFPS() changes the speed of constant frame rate video.

If you happen to get lucky and you can decode a VFR source at the average frame rate without effecting the audio sync, then of course you can convert that to a different frame rate with ConvertFPS and still not effect the audio sync any further, but it doesn't mean it converted VFR to CFR. When the frame rate varies enough or even early enough that the audio slowly loses sync when decoding at average frame rate, ConvertFPS won't fix it. It'll output the frame rate you tell it to, but the audio sync won't change.

[qyot27]

Also, I don't know why I didn't do this earlier, but there does appear to be something wonky with that sample file. MKVToolNix accepts it as input, but ffmpeg won't remux it into MKV, because it complains about the file's timestamps (it will allow it to mux into another AVI, but still throws a non-fatal warning about the timestamps in the file):
[avi @ 00000226993850c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

And from running a grep on the FFMS2 source code in the avisynth, avisynth_c, and vapoursynth directories, I'm starting to suspect that this is entirely external to FFMS2, and that the issue is partially in libavformat and partially in AviSynth itself. The fps-related hits in all three of those directories were all pretty similar, and nothing seemed to stick out as being really different, which points to this disconnect between AviSynth and VapourSynth probably being more about how those programs themselves interact with FFMS2 than something entirely in FFMS2's hands. I haven't tested VapourSynth, though (mostly because I already installed Python 3.7 for all my Python things, and don't want to also have to 3.6 on here too; I'll wait until the Windows builds are built against 3.7).

[manolito]

First of all this issue applies to the standard plugin as well as to the C-plugin, I'll post it here because I usually use the C-plugin.

I've been thinking for some time that aac_latm is an evil format, but since it has become one of the two audio standards in German DVB-T2 broadcasts I have no choice but dealing with it.

I uploaded a short sample file here:
https://www51.zippyshare.com/v/KY10rZkC/file.html

This format is used by half of the public stations (the other half use E-AC3). It is not this specific file, it's all captured files with this audio format. I also doubt that my receiver has anything to do with it (a Xoro with the latest firmware). The file plays without errors in all my software players, MediaInfo has no problems with it. And using LAVFilters with DSS2Mod also handles it nicely.

Only ffaudiosource has issues, and it applies to all current versions up to the latest 2.23.1 of the standard plugin and r1315 of the C-plugin. Tested on 2 different computers under WinXP and Win7-64bit.

This is the problem:
When using ffaudiosource("filename", track=-1) I get the error "No audio track found". When I use "track=1" or "track=2" the error is "Track has no audio samples". (I did verify that the 2 audio tracks are track #1 and track #2).

First I thought that ffms2 was just unable to decode this audio format, but this is not true. When I use the shortcut "ffmpegsource2" instead of "ffaudiosource" then the audio is decoded properly. This is true for the C-plugin where the shortcut comes from "ffms2.avsi" as well as for the current standard-plugin where the shortcut is integrated into the DLL.

Can anyone explain this weird behavior? I certainly do not have a clue... :devil:


Cheers
manolito

[qyot27]

FFMS2 C-plugin r1327+120 (https://forum.doom9.org/showthread.php?p=1829061#post1829061) (directs to first post in this thread)

The reason it took me so long before posting another build was that I was doing a pretty deep refactoring of the FFmpeg and mpv build guide I maintain, to switch all of the entries to out-of-tree builds (unless that was impossible; I think a grand total of 5 pieces couldn't be, out of 74-ish?). The result is that the 32-bit and 64-bit instructions in the guide can now use a single download of the source code, which saves download time/bandwidth, and it saves storage space.

Then, because I'm apparently a glutton for punishment, I'd gotten so fed up with having to build the 32-bit environment under a VM that I decided to try doing a multilib environment again (thus saving time at the beginning, because only one cross environment needs to be built, as well as being able to avoid all the nasty performance hits of a VM). This was...significantly trickier, as it exposed some pretty deep problems with how some of the pieces in FFmpeg's dependency chain have their build systems configured - and most of it caused headaches for the 32-bit parts. I had to resort to disabling assembly code on some of those 32-bit parts because their build systems won't tell nasm/yasm/gcc to build the asm for i686 in a multilib environment, so it tries building it as amd64 instead, and that causes it to fail to link at the end. Hopefully there's actually a way to force it without just having to wait on those projects to fix their autoconf setups (because it seemed to hit autoconf projects the most).

And then, because the Universe hates me, the computer I had been using as my main setup for the last couple years (a Quantum Byte mini-PC I bought in 2015) ended up getting fried due to a faulty power strip when I was just under halfway done with the move to multilib. So I had to order a new mini-PC (a Byte3), and go through setting up both the OSes *and* then get back to and finish that move to multilib in the guide. At least the new one runs 64-bit Windows 10 and I immediately migrated the Windows install on it to an SSD, so that crap shouldn't happen again (and enough free space to allow for installing MSVC, so I can get back to helping with AviSynth+ development...again, hopefully).


Only ffaudiosource has issues, and it applies to all current versions up to the latest 2.23.1 of the standard plugin and r1315 of the C-plugin. Tested on 2 different computers under WinXP and Win7-64bit.

This is the problem:
When using ffaudiosource("filename", track=-1) I get the error "No audio track found". When I use "track=1" or "track=2" the error is "Track has no audio samples". (I did verify that the 2 audio tracks are track #1 and track #2).

First I thought that ffms2 was just unable to decode this audio format, but this is not true. When I use the shortcut "ffmpegsource2" instead of "ffaudiosource" then the audio is decoded properly. This is true for the C-plugin where the shortcut comes from "ffms2.avsi" as well as for the current standard-plugin where the shortcut is integrated into the DLL.
It works with the build I just posted...sort of. I always index with ffmsindex first, and using the normal ffmsindex -t -1 ARD.mts to index the video and audio fails with an error message about the audio track.

Referring to the documentation, I set the errorhandling value to 2:
ffmsindex -t -1 -s 2 ARD.mts
This allows ffmsindex to complete the indexing. -t -1 indexes all the tracks, but the error specifically occurs on track 2 - setting -t 1 (for just track 1) without -s 2, it can index successfully. The resulting FFAudioSource script with track=1 plays back.

It also - probably incorrectly - will play back the same track if you select track=2 with that first -t -1 -s 2 script.

But -t 2 without -s 2 errors out; -t 2 -s 2 does allow indexing to complete, but FFAudioSource complains about the track not containing samples.

[manolito]

Thanks qyot27 for looking into this issue... :thanks:

This also explains why using the shortcut "ffmpegsource2" works (because indexing will be done before the ffaudiosource call). Of course using this workaround is not really satisfactory if the second track cannot be decoded.


Cheers
manolito

[FranceBB]

Thank you for the new build! :D
Still, adding enable10bithack would be really useful for our poor Avisynth (not Avisynth+) users. :')

[StvG]

Comparing 2.23.1 and 1327+120 in AvsPmod (Avisynth+ r2728), the outputs are different. Is it a bug or because of newer ffmpeg?
Comparing 2.23.1 and 1140+101 in AvsPmod (Avisynth+ r2728), the outputs are identical.
Checked with 8/10bit progressive video (23.976/24.000).
Simple script:FFVideoSource("input.mkv")

[qyot27]

You're going to have to be more specific. Are you sure that's not just the fact that newer versions of the C plugin don't automatically downsample to 8bit? Did you compare 2.23.1 and the latest mainline FFMS2 build (https://forum.doom9.org/showthread.php?t=174469)?

[StvG]

What I mean - https://imgbox.com/g/JWldJPTvIb

Script:FFVideoSource("HEVC.10bit.mkv") #2.23.1
ConvertBits(8)

LoadCPlugin("ffms2_r1140+101-avs+vsp\64-bit\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="r1140.ffindex")
ConvertBits(8)

LoadCPlugin("ffms2_r1327+120-avs+vsp\64bit\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="r1327.ffindex")
ConvertBits(8)

LoadPlugin("ffms2000-test8\64\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="test8.ffindex")
ConvertBits(8)


It happens with AVC 8bit videos too.
Also some frames of the video could be ok, other frames could be off.

[qyot27]

Okay, it looks like there's a frame skew compared to the other builds.

FFMS2000 test8 is from March 3rd, and there are only a couple of extant C-plugin builds from after that point (including r1327+120, which is from July 10th). Try with the 'lastxp' build in the first post (r1315+119). Since r1315+119 is from April 25th, if it doesn't exhibit this, it was introduced *after* April. If it does show the issue, it happened sometime between March 3rd and April 25th.

There were multiple commits that occurred on FFMS2's master branch during mid-March dealing with H.264 PAFF and interlaced HEVC, frame delays, and such. This is stuff that could simply make the frame counts different from then on, and would be visible on a hypothetical FFMS2000 test9 or the next point release. It doesn't necessarily mean it's a bug, just a change in behavior.

[StvG]

r1315+119 has the same behavior as r1327+120. http://imgbox.com/g/DAz5CXMPN4 - AVC 8bit video, some of the frames are ok, other frames are off.

[qyot27]

Then it occurred somewhere in the five frame-related commits in March*. All of those commits relate to how frames are handled in terms of delay, half-frames/interlacing, discarded frames, and so on. Considering that, is the output from newer FFMS2 builds just different or is playback affected negatively?

*https://github.com/FFMS/ffms2/commit/3d5867dc63dc69b98c3940860d2d32ed87d7c9ee
https://github.com/FFMS/ffms2/commit/13d9f01f5af62a30fe162b0068cb3c18d4db401a
https://github.com/FFMS/ffms2/commit/60860f7b3d53884acb79ceef30ea22ed534d01ba
https://github.com/FFMS/ffms2/commit/0562ba3c64a8e36cfe5950b41b0cb6eaf5f5a81a
https://github.com/FFMS/ffms2/commit/c870059ddb2131e6bf20c89b164aa69e5531c0e8

[StvG]

When the frames are off, usually r1327 shows 2-3 frames ahead compared to 2.23.1 and the other versions. So playback seems ok. Both the first and the last frame always are matching.
Seems some I frames decoded by 2.23.1 are decoded as B frames by r1327 and those are places where the mismatch is starting until the next I frame, which could be decoded either as I frame (frames are matching again) or as B frame (the mismatch is continuing).
Also tested some files encoded with x264 and always everything seems ok with these files.

[sneaker_ger]

Can you upload a sample?

[StvG]

Sample (https://a.doko.moe/gpkoqx.mkv)

[sneaker_ger]

The sample is not cut cleanly (open GOP not cut on IDR). There are 3 stray bframes left in the beginning.

[StvG]

The sample is cut with mkvtoolnix (https://gitlab.com/mbunkus/mkvtoolnix/wikis/Splitting-imprecise). 2.23.1 shows 4 I frames in the beginning.

[sneaker_ger]

Yes, but there aren't 4 I frames. 1 I and 3 B frames. It seems to freeze the I frame for 4 frames total. Now, the 2 strategies to deal with this broken file:
1.) freeze first I frame/recovery point for 4 frames replacing the 3 stray b frames or
2.) skipping/deleting the 3 stray b frames
are both valid in itself. You have to do something.

The questions would be:
A.) is av sync maintained
B.) what happens at the end of the clip (both show a length of 501 frames)

[StvG]

The sample is a part of randomly picked trailer just for testing.

Another sample (https://a.doko.moe/ypqmje.mkv) (should be fine now). When you go forward and backward through the video picking random frames, you will see at some moment that frames by 2.23.1 and r1327 are off. Then if you go backward to find the last I frame (let's call X the frame number) showed by 2.23.1, you will see that X frame by r1327 is B. If you refresh the video by r1327, the frame is still B type. Then if you go backward again to a place where frames by 2.23.1 and r1327 are the same and go forward frame by frame, X frame by r1327 will be now I frame and there will no further mismatch until you start picking randomly frames again. I encoded (x264) this sample with both 2.23.1 and r1327, and compared the encoded files - no matter how long I'm randomly picking forward/backward frames, the frames by 2.23.1 and r1327 are always matching.

[sneaker_ger]

Thx. I see both problems with r1327+120.

[StvG]

Btw FFMS2000 test8 (https://forum.doom9.org/showthread.php?t=174469) has similar behavior in a lighter form.

It seems the ffmpeg version also has influence. Github tag 2.23 built with the latest ffmpeg master branch has the same behavior as FFMS2000 test 8 - both have a bit different behavior than 2.23.1 release posted here (https://github.com/FFMS/ffms2/releases).
Tag 2.23 built with the latest release/3.4 ffmpeg branch has the same behavior as 2.23.1 github release.
It seems FFMS2000 test 8 behavior starts from this commit (https://github.com/FFMS/ffms2/commit/f3fb0e0c8910321bfef542836e4391e57ad3c6fb), because version compiled from this tree (https://github.com/FFMS/ffms2/tree/5eb741e45ab333bfb9799c243bf1156d0719a843) with ffmpeg release/3.4 has the same behavior as the github release.

P.S. This problem obviously is not specifically related to the c-plugin, so a staff could move the whole conversion to the appropriate thread?

Edit: Reverting back these commits*, build from master branch with the latest ffmpeg has the same behavior as 2.23.1.

*https://github.com/FFMS/ffms2/commit/f3fb0e0c8910321bfef542836e4391e57ad3c6fb
https://github.com/FFMS/ffms2/commit/6c8bc8cbca0060a454507cc4ac2f582183d3af6d
https://github.com/FFMS/ffms2/commit/13d9f01f5af62a30fe162b0068cb3c18d4db401a

[Selur]

A fresh build to support av1 would be great. :)
Thanks!

[qyot27]

It was about time for a new build anyway.

FFMS2 C-plugin r1327+119 (https://forum.doom9.org/showthread.php?p=1829061#post1829061)
(the backtrack of the C-plugin rev # is nothing to worry about, just a mix-up with the working trees on my local setup - it's been resolved on Github)

I tested it against the AV1 samples on https://www.elecard.com/videos, it seems to work.

[Selur]

Thanks!