FFmpegSource

[StvG]

There is "fill_gaps" parameter (int) that was broken the last time I tested it.

[wonkey_monkey]

How easy would it be for an external program to read an .ffindex file to inform itself which frames are keyframes?

[LigH]

Not so trivial. In the sources (ffms2-git\src\core\indexing.cpp) I read that it is single-file zipped (like in UNIX zip .z or GNU zip .gz or .xz or similar, not PKZIP, I believe, see: ffms2-git\src\core\zipfile.cpp).

If I am not wrong, L-SMASH Source does not compress the index.

[FranceBB]

Originally Posted by LigH

If I am not wrong, L-SMASH Source does not compress the index.

Correct, unlike the .ffindex files produced by FFVideoSource() and FFAudioSource(), the .lwi created by LWLibavVideoSource() and LWLibavAudioSource() is not compressed, it's human readable and it has an XML based structure.
Truncated example for reference:

<LSMASHWorksIndexVersion=0.0.3.0>
<LibavReaderIndexFile=18>
<InputFilePath>A:\Ingest\MEDIA\temp\HNK02043_MO.mxf</InputFilePath>
<FileSize=110707651>
<FileLastModificationTime=1739477225>
<FileHash=0x7f12b33eb742a48e>
<LibavReaderIndex=0x04000000,0,mxf>
<ActiveVideoStreamIndex>+0000000000</ActiveVideoStreamIndex>
<ActiveAudioStreamIndex>+0000000004</ActiveAudioStreamIndex>
<DefaultAudioStreamIndex>-0000000001</DefaultAudioStreamIndex>
<FillAudioGaps>1</FillAudioGaps>
<StreamInfo=0,0>
Codec=2,TimeBase=1/25,Width=1920,Height=1080,Format=yuv422p,ColorSpace=2
</StreamInfo>
<StreamInfo=1,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=2,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=3,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
<StreamInfo=4,1>
Codec=65548,TimeBase=1/48000,Channels=2:0x3,Rate=48000,Format=s32,BPS=24
</StreamInfo>
Index=1,POS=182043,PTS=0,DTS=0,EDI=0
Length=1920
Index=2,POS=193583,PTS=0,DTS=0,EDI=0
Length=1920
Index=3,POS=205123,PTS=0,DTS=0,EDI=0
Length=1920
Index=4,POS=216663,PTS=0,DTS=0,EDI=0
Length=1920
Index=0,POS=228203,PTS=0,DTS=-1,EDI=0
Key=1,Pic=1,POC=0,Repeat=1,Field=1
Index=1,POS=419285,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=2,POS=430825,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=3,POS=442365,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=4,POS=453905,PTS=1920,DTS=1920,EDI=0
Length=1920
Index=0,POS=465445,PTS=3,DTS=0,EDI=0
Key=0,Pic=2,POC=3,Repeat=1,Field=1
Index=1,POS=885093,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=2,POS=896633,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=3,POS=908173,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=4,POS=919713,PTS=3840,DTS=3840,EDI=0
Length=1920
Index=0,POS=931253,PTS=1,DTS=1,EDI=0
Key=0,Pic=3,POC=1,Repeat=1,Field=1
Index=1,POS=1025687,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=2,POS=1037227,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=3,POS=1048767,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=4,POS=1060307,PTS=5760,DTS=5760,EDI=0
Length=1920
Index=0,POS=1071847,PTS=2,DTS=2,EDI=0
Key=0,Pic=3,POC=2,Repeat=1,Field=1
Index=1,POS=1422586,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=2,POS=1434126,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=3,POS=1445666,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=4,POS=1457206,PTS=7680,DTS=7680,EDI=0
Length=1920
Index=0,POS=1468746,PTS=6,DTS=3,EDI=0
Key=0,Pic=2,POC=6,Repeat=1,Field=1
Index=1,POS=1859511,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=2,POS=1871051,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=3,POS=1882591,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=4,POS=1894131,PTS=9600,DTS=9600,EDI=0
Length=1920
Index=0,POS=1905671,PTS=4,DTS=4,EDI=0
Key=0,Pic=3,POC=4,Repeat=1,Field=1
Index=1,POS=2281430,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=2,POS=2292970,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=3,POS=2304510,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=4,POS=2316050,PTS=11520,DTS=11520,EDI=0
Length=1920
Index=0,POS=2327590,PTS=5,DTS=5,EDI=0
Key=0,Pic=3,POC=5,Repeat=1,Field=1
Index=1,POS=2704255,PTS=13440,DTS=13440,EDI=0
Length=1920
Index=2,POS=2715795,PTS=13440,DTS=13440,EDI=0
Length=1920

[wonkey_monkey]

Originally Posted by LigH

Not so trivial. In the sources (ffms2-git\src\core\indexing.cpp) I read that it is single-file zipped (like in UNIX zip .z or GNU zip .gz or .xz or similar, not PKZIP, I believe, see: ffms2-git\src\core\zipfile.cpp).

Ah, zlib, my old nemesis (I tried to get it to work once but it's... less than simple).

Luckily I've found a nice modernised stream buffer wrapper (zstr) for zlib, although it did need a tweak to allow to read "fast compressed" zlib as used by FFMS2 (the author was guided by a Stack Overflow post that omitted that kind of header).

[Edit: zstr doesn't seem to work properly with ffindex files, bugging out after ~2500 bytes]

Hopefully indexing.cpp can guide me through parsing an index.

Thanks!

[qyot27]

Originally Posted by wonkey_monkey

How easy would it be for an external program to read an .ffindex file to inform itself which frames are keyframes?

If you're controlling the generation of the .ffindex in the first place through a host application, I would think it easier to just make sure you pass -k to ffmsindex:

FFmpegSource2 indexing app
Usage: ffmsindex [options] inputfile [outputfile]
If no output filename is specified, inputfile.ffindex will be used.

Options:
-f        Force overwriting of existing index file, if any (default: no)
-v        Set FFmpeg verbosity level. Can be repeated for more verbosity. (default: no messages printed)
-p        Disable progress reporting. (default: progress reporting on)
-c        Write timecodes for all video tracks to outputfile_track00.tc.txt (default: no)
-k        Write keyframes for all video tracks to outputfile_track00.kf.txt (default: no)
-t N      Set the audio indexing mask to N (-1 means index all tracks, 0 means index none, default: 0)
-s N      Set audio decoding error handling. See the documentation for details. (default: 0)
-u N      Set the progress update frequency in seconds. Set to 0 for every percent. (default: 1)
-o string Set demuxer options to be used in the form of 'key=val:key=val'. (default: none)

FFmpeg Demuxer Options:
--enable_drefs
--use_absolute_path

Obviously that's not quite the same as reading it from an existing .ffindex, but if the mechanism is already there in some form...

[wonkey_monkey]

I've managed to get this far decoding the video track in the index (values are little endian):

         PTS                   OriginalPTS        Keyframe         FilePos           MarkedHidden        OriginalPos           PosInDecodingOrder     RepeatPict  SecondField
80|02|00|00|00|00|00|00  80|02|00|00|00|00|00|00     01     4c|01|00|00|00|00|00|00       00       ff|ff|ff|ff|ff|ff|ff|ff  ff|ff|ff|ff|ff|ff|ff|ff  01|00|00|00       00
80|02|00|00|00|00|00|00  80|02|00|00|00|00|00|00     01     bc|4e|00|00|00|00|00|00       00       00|00|00|00|00|00|00|00  00|00|00|00|00|00|00|00  01|00|00|00       00
80|02|00|00|00|00|00|00  80|02|00|00|00|00|00|00     00     9e|4e|00|00|00|00|00|00       00       00|00|00|00|00|00|00|00  00|00|00|00|00|00|00|00  01|00|00|00       00
80|02|00|00|00|00|00|00  80|02|00|00|00|00|00|00     00     bb|30|00|00|00|00|00|00       00       00|00|00|00|00|00|00|00  00|00|00|00|00|00|00|00  01|00|00|00       00

This seems to indicate that the first two frames are keyframes, but from opening the file directly in VirtualDub I know that keyframes are at frame 0, 25, 50, 75... every 25 frames (presumably maximum GOP length, as later in the index I can see several examples of keyframes separated by 25 frames) until the first out-of-pattern keyframe at frame 557.

So the frames VirtualDub reports as keyframes do all seem to be marked as keyframes in the index, but so are many other apparently non-keyframe frames E.g. frame 1 (shown above), 4, 7, 11, in fact every third frame up to frame 28. Then there are a few more spurious keyframes before it settles at around frame 50 and matches the video - until frame 540, which is reported as a keyframe in the index but not by VirtualDub.

Not sure what FilePos could mean either, since its value never goes very high (only up to about 300,000) compared to the filesize of 14Gb.

The other fields never seem to change - PTS and OriginalPTS obviously relate to the constant framerate - except for the -1s for OriginalPos and PosInDecodingOrder on frame 0.

Anyone got any idea what I'm missing with regard to keyframes?

[StvG]

Originally Posted by wonkey_monkey

...
Hopefully indexing.cpp can guide me through parsing an index.
...

You should look at track.cpp for parsing .ffindex. An example.

[wonkey_monkey]

I used track.cpp to determine what those fields delimited above are.

Could there be such a thing as a de facto keyframe? I.e. a frame labelled as P or B but which doesn't actually depend on the content of any other frames? If so, maybe ffmpeg is identifying them as keyframes.

[StvG]

P and B frame never can be keyframe.
Key frames are only I type.

The frame info in .ffindex is pretty much the same as what ffmpeg/ffprobe output. You can just do ffprobe -show_frames -select_streams 0 -i input.video | findstr "key_frame pict_type" > frame_info.txt

key_frame=1
pict_type=I
key_frame=0
pict_type=B
key_frame=0
pict_type=B
key_frame=0
pict_type=B
key_frame=0
pict_type=P

[wonkey_monkey]

Ah, now I get it: https://forum.doom9.net/showthread.php?p=1834568

VirtualDub differentiates between IDR frames [K] and regular I-frames [I]. Ideally I need to know which are IDR frames, but that doesn't seem to be specified in the ffindex. I can probably work with that, it just won't be quite as effective.

[StvG]

https://github.com/FFMS/ffms2/blob/m...exing.cpp#L531
Here is stored in ffindex https://github.com/FFMS/ffms2/blob/m...exing.cpp#L567

[Myrsloik]

I don't know why you need to know keyframes but beware, FFmpeg has a lot of bugs. For example the huffyuv decoder reports no keyframes (lol) and while the keyframe flag should only be set for IDR-frames in h264 due to bugs in FFmpeg it will happily mark I-frames as "keyframes" in remuxed files.

Lots of fun bugs and corner cases even in common containers and formats.

[wonkey_monkey]

Originally Posted by Myrsloik

I don't know why you need to know keyframes

A cache so I can step backward without it stuttering on every frame (now it only stutters when it moves into the previous GOP)/to make edits and dissolves faster. Works pretty well.

[wonkey_monkey]

...and then I found out ffms2 isn't frame accurate with h264 in .mp4 Back to the drawing board...

[Myrsloik]

Originally Posted by wonkey_monkey

...and then I found out ffms2 isn't frame accurate with h264 in .mp4 Back to the drawing board...

May I interest you in our lord and savior BestSource?

[wonkey_monkey]

Originally Posted by Myrsloik

May I interest you in our lord and savior BestSource?

Right now, if there's a finished and working Windows build (for Avisynth+), then sure. If I have to compile it myself, I'm still too traumatised from LSMASHSource.

Edit: Oh there is. Good. Testing now.

More edit: Does over ten minutes (still waiting) to open a one hour 14Gb .mp4 sound right to you?

Yet another edit: 30 mins total indexing time. It's going to take two solid days to index my sources

More info: Once first load finished in VirtualDub, all the audio channels were present, but they were all silent. Opening the same .avs file in a new VirtuaDub window, the audio was all present, but the framerate was incorrectly determined to be 25.764fps instead of 25fps (same as with original load). SeekTester confirms frame accuracy, so that's nice. But that indexing speed is a killer (but worth it for the frame-accuracy guarantee).