FFmpegSource

[rgr]

I don't really have a choice, as I haven't found a version other than 2390 that works properly with ANSI.

[StvG]

Of course are different versions, but your r2390 (https://codeberg.org/StvG/ffms2/releases/tag/r2390) is older (2020) than 5.0 (https://github.com/FFMS/ffms2/releases/tag/5.0) (2024) from Myrsloik.

The question is this file:

saved like ANSI or UTF8 (without BOM) work fine.

But using C:\Test\5.0\ffms2.dll only work saved as UTF8 (without BOM).
There are other differences (see the links and the ChaosKing tests #2617 and #2941) and select your preference.

r2390 is not from 2020 - it's from 2024-03-06 (https://codeberg.org/StvG/ffms2/releases/tag/r2390).

The official ffms2 doesn't handle ANSI and will not handle it until there is change in its code. If you don't see changes in the changelog about it don't expect to magically work.

[tebasuna51]

r2390 is not from 2020 - it's from 2024-03-06 (https://codeberg.org/StvG/ffms2/releases/tag/r2390).

You are right I mistake the folder date (2022-04 image 1) with the file date (2024-03 image 2) but still the 5.0 is newer (2024-05 image 3).

BTW the preference must be by other differences, not by not support ANSI avs files when there are free tools (notepad++ (https://notepad-plus-plus.org/downloads/)) to easy change ANSI <-> UTF8 codification.

I don't know how rgr make the avs files but some tools (like MeGUI) make avs with UTF8 already.

[tebasuna51]

I don't know if is still valid the differences here:

ffms2 r2390 [StvG] StvG publicó esto 2024-03-06 06:18:34 +00:00
Differences compared to 5.0-RC2:
1 In some cases can be seek accurate while 5.0-RC2 is not
2 AviSynth: GetParity and _FieldBased are set per frame
3 AviSynth: rffmode=1 that uses every frame tff/bff
4 AviSynth: set _EncodedFrameTop and _EncodedFrameBottom (rffmode > 0)
5 AviSynth: both ANSI and UTF8 file names are supported
6 _ColorRange is set per frame
7 Supported older AviSynth+ version that doesn't have audio channel mask property
Some comments:
2 and 6) I don't know when is needed GetParity, _FieldBased and _ColorRange set per frame

7) Use Avs+ older than 3.7.3 it is not recommended at all.

5) Problem already commented

3 and 4) rffmode removed in 5.0. To evaluate the impact read the help in RC 4:
##### int rffmode = 0
Controls how RFF flags in the video stream are treated; in other words it's equivalent to the "field operation" mode switch in DVD2AVI/DGIndex.
Valid modes are:

- **0**: Ignore all flags (the default mode).
- **1**: Honor all pulldown flags.
- **2**: Equivalent to DVD2AVI's "force film" mode.

Note that using modes 1 or 2 will make `FFVideoSource` throw an error if the video stream has no RFF flags at all.
When using either of those modes, it will also make the output be assumed as CFR, disallow vertical scaling and disallow setting the output colorspace.
`FFPICT_TYPE` will also not be set as the output is a combination of several frames.
Other subtle behavior changes may also exist.

Also note that "force film" is mostly useless and only here for completeness' sake, since if your source really is safe to force film on, using mode 0 will have the exact same effect while being considerably more efficient.

1) About the seek accurate we have the ChaosKing test (image). Seems we need set seekmode = 0 (slow seek) for mpeg2.VOB (standard DVD), mpeg1.mpg and VC1.mpg. The default seems valid for other files.
The help about this parameter can be interesting:
##### int seekmode = 1
Controls how seeking is done. Mostly useful for getting uncooperative files to work.
Valid modes are:

- **-1**: Linear access without rewind; i.e. will throw an error if each successive requested frame number isn't bigger than the last one.
Only intended for opening images but might work on well with some obscure video format.
- **0**: Linear access (i.e. if you request frame `n` without having requested all frames from 0 to `n-1` in order first, all frames from 0 to `n` will have to be decoded before `n` can be delivered).
The definition of slow, but should make some formats "usable".
- **1**: Safe normal. Bases seeking decisions on the keyframe positions reported by libavformat.
- **2**: Unsafe normal. Same as mode 1, but no error will be thrown if the exact seek destination has to be guessed.
- **3**: Aggressive. Seeks in the forward direction even if no closer keyframe is known to exist. Only useful for testing and containers where libavformat doesn't report keyframes properly.

[StvG]

I don't know if is still valid the differences here:

ffms2 r2390 [StvG] StvG publicó esto 2024-03-06 06:18:34 +00:00
Differences compared to 5.0-RC2:
1 In some cases can be seek accurate while 5.0-RC2 is not
2 AviSynth: GetParity and _FieldBased are set per frame
3 AviSynth: rffmode=1 that uses every frame tff/bff
4 AviSynth: set _EncodedFrameTop and _EncodedFrameBottom (rffmode > 0)
5 AviSynth: both ANSI and UTF8 file names are supported
6 _ColorRange is set per frame
7 Supported older AviSynth+ version that doesn't have audio channel mask property

Some comments:
2 and 6) I don't know when is needed GetParity, _FieldBased and _ColorRange set per frame

7) Use Avs+ older than 3.7.3 it is not recommended at all.

5) Problem already commented

3 and 4) rffmode removed in 5.0. To evaluate the impact read the help in RC 4:

2, 3, 4 - discussed (https://forum.doom9.org/showpost.php?p=1994388&postcount=2820), discussed (https://forum.doom9.org/showpost.php?p=1994660&postcount=2827) and next posts.

7 - keep in mind the official ffms2 will just crash instead to give error message.

The latest official ffms2 5.0 (2024-05-28) have changes that significantly impact the decoding (https://github.com/FFMS/ffms2/pull/437). This is the main reason that I wrote that both variants shouldn't be considered as just newer/older.

[rgr]

I don't know if is still valid the differences here:


Some comments:
2 and 6) I don't know when is needed GetParity, _FieldBased and _ColorRange set per frame

7) Use Avs+ older than 3.7.3 it is not recommended at all.

5) Problem already commented

3 and 4) rffmode removed in 5.0. To evaluate the impact read the help in RC 4:


1) About the seek accurate we have the ChaosKing test (image). Seems we need set seekmode = 0 (slow seek) for mpeg2.VOB (standard DVD), mpeg1.mpg and VC1.mpg. The default seems valid for other files.
The help about this parameter can be interesting:

Other:
- 2390 does not set PAR for each frame separately (but rather sets it, but from the stream property, not from the frame property)
- 2390 for AVI files (captured from a DV camera) recognizes incorrect fps (50.32 or 49.68 or similar instead of 50)
- 2390 does not handle VFR correctly, but I don't think any ffms2 does

[StvG]

Other:
- 2390 does not set PAR for each frame separately (but rather sets it, but from the stream property, not from the frame property)
- 2390 for AVI files (captured from a DV camera) recognizes incorrect fps (50.32 or 49.68 or similar instead of 50)
- 2390 does not handle VFR correctly, but I don't think any ffms2 does

Can you elaborate?

- PAR: can you show example of what you mean?
- AVI fps: it looks like rounding errors. Can you share short video sample with this issue?

[rgr]

- PAR
When I read an AVI file from a DV camera that is partly recorded in 4:3 and partly in 16:9, the 2390 version returns the same PAR for each frame. LWLibavVideoSource changes the PAR to the correct one for each frame, MPC also changes the aspect ratio from where the PAR changes.

- AVI (M2TS)
My mistake -- it's an M2T file. I'll give the link privately, because ffms2 has bigger problems with it anyway :)

- VFR
https://files.fm/u/cemhy7vrmx (upload in progress... 6 files)

[StvG]

The tebasuna51 comment you quoted is about the differences between the ffms2 version not between ffms2 and LSMASHSource/other source filter.
To be clear - the differences you wrote are between ffms2 and LSMASHSource not between the ffms2 versions.

Thanks for the samples.
You can test this version (https://files.catbox.moe/n7x0o5.zip). It sets _SARxxx frames props per frame and uses more robust fps calculation.

Edit: The file you shared (M2T one) is damaged (has video corrupted packages) - https://i.ibb.co/TxCGxwTR/Untitled.png

[FranceBB]

Hey StvG, thank you for picking up ffms2 and maintaining it while Myrsloyk was away, I think everyone here appreciates what you've done during this time.
To me and plenty others your builds are now the de facto standard and main branch of ffms2. :)
After talking to asd-g, he introduced a new parameter in LWLibavAudioSource() which I think FFAudioSource() would also greatly benefit from.
Such a parameter is a boolean called "fill_audio_gaps" which can be set to either true or false.
What it does is basically read the PTS while indexing and - if there's a gap in the audio track - it fills it with silence so that the audio stays synced with the video and ends up having the same duration instead of being shortened (and therefore desynced) from the point of the gap onwards.
Here's the commit: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/7dc8ba9295aafa5e99a59903cdd62c4fffc512a5
I tested it and it seems to be working reliably well and it could help in plenty of occasions, especially with .ts and .mxf containers in which CRC Errors are very common as there might be plenty of issues with incoming signals that are recorded, so my question is: would you be interested in adding a similar parameter/feature in FFAudioSource()?

[StvG]

There is "fill_gaps" parameter (int) that was broken the last time I tested it.

[wonkey_monkey]

How easy would it be for an external program to read an .ffindex file to inform itself which frames are keyframes?

[LigH]

Not so trivial. In the sources (ffms2-git\src\core\indexing.cpp) I read that it is single-file zipped (like in UNIX zip .z or GNU zip .gz or .xz or similar, not PKZIP, I believe, see: ffms2-git\src\core\zipfile.cpp).

If I am not wrong, L-SMASH Source does not compress the index.

[FranceBB]

If I am not wrong, L-SMASH Source does not compress the index.

Correct, unlike the .ffindex files produced by FFVideoSource() and FFAudioSource(), the .lwi created by LWLibavVideoSource() and LWLibavAudioSource() is not compressed, it's human readable and it has an XML based structure.
Truncated example for reference:


<LSMASHWorksIndexVersion=0.0.3.0>
<LibavReaderIndexFile=18>
<InputFilePath>A:\Ingest\MEDIA\temp\HNK02043_MO.mxf</InputFilePath>
<FileSize=110707651>
<FileLastModificationTime=1739477225>
<FileHash=0x7f12b33eb742a48e>
<LibavReaderIndex=0x04000000,0,mxf>
<ActiveVideoStreamIndex>+0000000000</ActiveVideoStreamIndex>
<ActiveAudioStreamIndex>+0000000004</ActiveAudioStreamIndex>
<DefaultAudioStreamIndex>-0000000001</DefaultAudioStreamIndex>
<FillAudioGaps>1</FillAudioGaps>
<StreamInfo=0,0>
Codec=2,TimeBase=1/25,Width=1920,Height=1080,Format=yuv422p,ColorSpace=2
</StreamInfo>
<StreamInfo=1,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=2,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=3,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=4,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
Index=1,POS=182043,PTS=0,DTS=0,EDI=0
Length=1920
Index=2,POS=193583,PTS=0,DTS=0,EDI=0
Length=1920
Index=3,POS=205123,PTS=0,DTS=0,EDI=0
Length=1920
Index=4,POS=216663,PTS=0,DTS=0,EDI=0
Length=1920
Index=0,POS=228203,PTS=0,DTS=-1,EDI=0
Key=1,Pic=1,POC=0,Repeat=1,Field=1
Index=1,POS=419285,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=2,POS=430825,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=3,POS=442365,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=4,POS=453905,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=0,POS=465445,PTS=3,DTS=0,EDI=0
Key=0,Pic=2,POC=3,Repeat=1,Field=1
Index=1,POS=885093,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=2,POS=896633,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=3,POS=908173,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=4,POS=919713,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=0,POS=931253,PTS=1,DTS=1,EDI=0
Key=0,Pic=3,POC=1,Repeat=1,Field=1
Index=1,POS=1025687,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=2,POS=1037227,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=3,POS=1048767,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=4,POS=1060307,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=0,POS=1071847,PTS=2,DTS=2,EDI=0
Key=0,Pic=3,POC=2,Repeat=1,Field=1
Index=1,POS=1422586,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=2,POS=1434126,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=3,POS=1445666,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=4,POS=1457206,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=0,POS=1468746,PTS=6,DTS=3,EDI=0
Key=0,Pic=2,POC=6,Repeat=1,Field=1
Index=1,POS=1859511,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=2,POS=1871051,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=3,POS=1882591,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=4,POS=1894131,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=0,POS=1905671,PTS=4,DTS=4,EDI=0
Key=0,Pic=3,POC=4,Repeat=1,Field=1
Index=1,POS=2281430,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=2,POS=2292970,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=3,POS=2304510,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=4,POS=2316050,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=0,POS=2327590,PTS=5,DTS=5,EDI=0
Key=0,Pic=3,POC=5,Repeat=1,Field=1
Index=1,POS=2704255,PTS=13440,DTS=13440,EDI=0
Length=1920
Index=2,POS=2715795,PTS=13440,DTS=13440,EDI=0
Length=1920

[wonkey_monkey]

Not so trivial. In the sources (ffms2-git\src\core\indexing.cpp) I read that it is single-file zipped (like in UNIX zip .z or GNU zip .gz or .xz or similar, not PKZIP, I believe, see: ffms2-git\src\core\zipfile.cpp).

Ah, zlib, my old nemesis (I tried to get it to work once but it's... less than simple).

Luckily I've found a nice modernised stream buffer wrapper (zstr) for zlib, although it did need a tweak to allow to read "fast compressed" zlib as used by FFMS2 (the author was guided by a Stack Overflow post that omitted that kind of header).

[Edit: zstr doesn't seem to work properly with ffindex files, bugging out after ~2500 bytes]

Hopefully indexing.cpp can guide me through parsing an index.

Thanks!

[qyot27]

How easy would it be for an external program to read an .ffindex file to inform itself which frames are keyframes?
If you're controlling the generation of the .ffindex in the first place through a host application, I would think it easier to just make sure you pass -k to ffmsindex:
FFmpegSource2 indexing app
Usage: ffmsindex [options] inputfile [outputfile]
If no output filename is specified, inputfile.ffindex will be used.

Options:
-f Force overwriting of existing index file, if any (default: no)
-v Set FFmpeg verbosity level. Can be repeated for more verbosity. (default: no messages printed)
-p Disable progress reporting. (default: progress reporting on)
-c Write timecodes for all video tracks to outputfile_track00.tc.txt (default: no)
-k Write keyframes for all video tracks to outputfile_track00.kf.txt (default: no)
-t N Set the audio indexing mask to N (-1 means index all tracks, 0 means index none, default: 0)
-s N Set audio decoding error handling. See the documentation for details. (default: 0)
-u N Set the progress update frequency in seconds. Set to 0 for every percent. (default: 1)
-o string Set demuxer options to be used in the form of 'key=val:key=val'. (default: none)

FFmpeg Demuxer Options:
--enable_drefs
--use_absolute_path

Obviously that's not quite the same as reading it from an existing .ffindex, but if the mechanism is already there in some form...

[wonkey_monkey]

I've managed to get this far decoding the video track in the index (values are little endian):

PTS OriginalPTS Keyframe FilePos MarkedHidden OriginalPos PosInDecodingOrder RepeatPict SecondField
80|02|00|00|00|00|00|00 80|02|00|00|00|00|00|00 01 4c|01|00|00|00|00|00|00 00 ff|ff|ff|ff|ff|ff|ff|ff ff|ff|ff|ff|ff|ff|ff|ff 01|00|00|00 00
80|02|00|00|00|00|00|00 80|02|00|00|00|00|00|00 01 bc|4e|00|00|00|00|00|00 00 00|00|00|00|00|00|00|00 00|00|00|00|00|00|00|00 01|00|00|00 00
80|02|00|00|00|00|00|00 80|02|00|00|00|00|00|00 00 9e|4e|00|00|00|00|00|00 00 00|00|00|00|00|00|00|00 00|00|00|00|00|00|00|00 01|00|00|00 00
80|02|00|00|00|00|00|00 80|02|00|00|00|00|00|00 00 bb|30|00|00|00|00|00|00 00 00|00|00|00|00|00|00|00 00|00|00|00|00|00|00|00 01|00|00|00 00


This seems to indicate that the first two frames are keyframes, but from opening the file directly in VirtualDub I know that keyframes are at frame 0, 25, 50, 75... every 25 frames (presumably maximum GOP length, as later in the index I can see several examples of keyframes separated by 25 frames) until the first out-of-pattern keyframe at frame 557.

So the frames VirtualDub reports as keyframes do all seem to be marked as keyframes in the index, but so are many other apparently non-keyframe frames :confused: E.g. frame 1 (shown above), 4, 7, 11, in fact every third frame up to frame 28. Then there are a few more spurious keyframes before it settles at around frame 50 and matches the video - until frame 540, which is reported as a keyframe in the index but not by VirtualDub.

Not sure what FilePos could mean either, since its value never goes very high (only up to about 300,000) compared to the filesize of 14Gb.

The other fields never seem to change - PTS and OriginalPTS obviously relate to the constant framerate - except for the -1s for OriginalPos and PosInDecodingOrder on frame 0.

Anyone got any idea what I'm missing with regard to keyframes?

[StvG]

...
Hopefully indexing.cpp can guide me through parsing an index.
...

You should look at track.cpp for parsing .ffindex. An example (https://github.com/FFMS/ffms2/blob/master/src/core/track.cpp#L38-L56).

[wonkey_monkey]

I used track.cpp to determine what those fields delimited above are.

Could there be such a thing as a de facto keyframe? I.e. a frame labelled as P or B but which doesn't actually depend on the content of any other frames? If so, maybe ffmpeg is identifying them as keyframes.

[StvG]

P and B frame never can be keyframe.
Key frames are only I type.

The frame info in .ffindex is pretty much the same as what ffmpeg/ffprobe output. You can just do ffprobe -show_frames -select_streams 0 -i input.video | findstr "key_frame pict_type" > frame_info.txt


key_frame=1
pict_type=I
key_frame=0
pict_type=B
key_frame=0
pict_type=B
key_frame=0
pict_type=B
key_frame=0
pict_type=P

[wonkey_monkey]

Ah, now I get it: https://forum.doom9.net/showthread.php?p=1834568

VirtualDub differentiates between IDR frames [K] and regular I-frames [I]. Ideally I need to know which are IDR frames, but that doesn't seem to be specified in the ffindex. I can probably work with that, it just won't be quite as effective.

[StvG]

https://github.com/FFMS/ffms2/blob/master/src/core/indexing.cpp#L531
Here is stored in ffindex https://github.com/FFMS/ffms2/blob/master/src/core/indexing.cpp#L567

[Myrsloik]

I don't know why you need to know keyframes but beware, FFmpeg has a lot of bugs. For example the huffyuv decoder reports no keyframes (lol) and while the keyframe flag should only be set for IDR-frames in h264 due to bugs in FFmpeg it will happily mark I-frames as "keyframes" in remuxed files.

Lots of fun bugs and corner cases even in common containers and formats.

[wonkey_monkey]

I don't know why you need to know keyframes

A cache so I can step backward without it stuttering on every frame (now it only stutters when it moves into the previous GOP)/to make edits and dissolves faster. Works pretty well.

[wonkey_monkey]

...and then I found out ffms2 isn't frame accurate with h264 in .mp4 :eek: Back to the drawing board...

[Myrsloik]

...and then I found out ffms2 isn't frame accurate with h264 in .mp4 :eek: Back to the drawing board...

May I interest you in our lord and savior BestSource?

[wonkey_monkey]

May I interest you in our lord and savior BestSource?

Right now, if there's a finished and working Windows build (for Avisynth+), then sure. If I have to compile it myself, I'm still too traumatised from LSMASHSource.

Edit: Oh there is. Good. Testing now.

More edit: Does over ten minutes (still waiting) to open a one hour 14Gb .mp4 sound right to you? :confused:

Yet another edit: 30 mins total indexing time. It's going to take two solid days to index my sources :D:eek:

More info: Once first load finished in VirtualDub, all the audio channels were present, but they were all silent. Opening the same .avs file in a new VirtuaDub window, the audio was all present, but the framerate was incorrectly determined to be 25.764fps instead of 25fps (same as with original load). SeekTester confirms frame accuracy, so that's nice. But that indexing speed is a killer (but worth it for the frame-accuracy guarantee).