Recent change regarding vfr? (x264)

[cdz]

Has there been any recent change (within the past 10 months) in terms of handling variable framerate?

It seems that x264 is discarding frames when the actual framerate is lower than specified.
This concerns sources captured with Fraps and the like, where fps is variable.

This leaves the video slightly faster than the audio. I suppose this is an upgrade, but can someone confirm this?

[raffriff42]

Fraps video is not variable framerate. It can have null frames though, which are inserted when the system is not able to keep up with the workload. Some decoders and/or demuxers don't handle these null frames well; they seem to simply ignore them. Leaving, as you have seen, video shorter than audio.

Null frames = total frames minus keyframes (as reported in VirtualDub File Info etc).

[cdz]

Right, but I've assumed 'null' would mean 'previous frame', because over the several years I've used fraps, this has never happened. The framerate was kept constant, seemingly, by inserting identical frames where the fps drops.

I could only assume this occurs because of a change in lavf/ffdshow or x264.

Encoding via ffdshow syncs a/v but still drops frames and stats from playing it shows varying fps. Lag intact by proper timecodes(?).
Stats from playing older encodes show a constant fps, even though the footage was generated with the same version of Fraps. Lag intact by identical frames.

When playing any 'lagging' clips with mplayer, it outputs several chunks like this:

37: 00dc 0 00000000043A40BC len: 8 pos:70734224->3773.096 36-> 1.502

These 00dc chunks with 8(bits?) appear in same amount as frames dropped - i.e not encoded - by x264.

Encoding the same clip on a year old virtual image, yields the same mplayer log(as expected), but no frames are dropped.
The resulting file plays back a/v synced and with the 'lag' intact.

Current setup:

Linux 3.9.7-1-ARCH #1 SMP PREEMPT Thu Jun 20 22:45:32 CEST 2013 x86_64 GNU/Linux

x264 0.130.x
(libswscale 2.2.100)
(libavformat 54.63.104)
built on Jul 3 2013, gcc: 4.8.0
configuration: --bit-depth=10 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat license: GPL version 2 or later

ffmpeg version 1.2.1
built on May 10 2013 16:31:05 with gcc 4.8.0 (GCC) 20130502 (prerelease)
configuration: \
--prefix=/usr --disable-debug --disable-static --enable-vresample \
--enable-dxva2 --enable-fontconfig --enable-gpl --enable-libass \
--enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug \
--enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg \
--enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger \
--enable-libspeex --enable-libtheora --enable-libv4l2 --enable-libvorbis \
--enable-libvpx --enable-libx264 --enable-libxvid --enable-postproc \
--enable-runtime-cpudetect --enable-shared --enable-vdpau --enable-version3 --enable-x11grab
libavutil 52. 18.100 / 52. 18.100
libavcodec 54. 92.100 / 54. 92.100
libavformat 54. 63.104 / 54. 63.104
libavdevice 54. 3.103 / 54. 3.103
libavfilter 3. 42.103 / 3. 42.103
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 2.100 / 52. 2.100


Old vmachine:

Linux 3.3.7-1-ARCH #1 SMP PREEMPT Tue May 22 00:26:26 CEST 2012 x86_64 GNU/Linux

x264 0.124.x
(libswscale 2.1.100)
(libavformat 53.32.100)
built on Jul 21 2012, gcc: 4.7.0 20120505 (prerelease)
configuration: --bit-depth=10 --chroma-format=all

ffmpeg version 0.10.3
built on May 9 2012 17:51:07 with gcc 4.7.0 20120505 (prerelease)
configuration: \
--prefix=/usr --enable-libmp3lame --enable-libvorbis --enable-libxvid \
--enable-libx264 --enable-libvpx --enable-libtheora --enable-libgsm \
--enable-libspeex --enable-postproc --enable-shared --enable-x11grab \
--enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libschroedinger --enable-libopenjpeg \
--enable-librtmp --enable-libpulse --enable-gpl --enable-version3 \
--enable-runtime-cpudetect --disable-debug --disable-static
libavutil 51. 35.100 / 51. 35.100
libavcodec 53. 61.100 / 53. 61.100
libavformat 53. 32.100 / 53. 32.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 61.100 / 2. 61.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100
libpostproc 52. 0.100 / 52. 0.100


I've tried compiling ~6 version of x264 (monthly picks from 2013) against the three latest versions of ffdshow.
Compiling x264 against older ffdshow versions gives segfaults.. probably older build tools or whatever.
So I've only been able to test with the latest ffdshow and recent-ish x264 builds. They all yield the same results though.

[raffriff42]

> I've assumed 'null' would mean 'previous frame'
Yes. See ffmpeg source (http://ffmpeg.org/doxygen/trunk/fraps_8c-source.html#l00165)

>These 00dc chunks with 8(bits?) appear in same amount as frames dropped
Those *are* the dropped frames. 8 bytes. Very minimal. Easy to see how some decoders might just ignore them.

[cdz]

I edited it -- basically what I meant was that when playing it back with OSD stats, the duplicate frame version(vfr as cfr) showed constant fps, and the synced ffdshow-version showed varying fps. As expected. But not really.
I can see how not storing these duplicate frames would make the stream more efficient -- actually it has annoyed me a bit that these duplicate frames where encoded.
Would be nice to get some 'official' statement on this though. And an how-to on getting timecodes from avi...

[STaRGaZeR]

Yes, ffmpeg started to ignore those repeated frames quite some time ago when they implemented multithreading in the decoder. Use LAV Filters instead of ffdshow, there is a patch in there that fixes this issue.

[cdz]

But I've been using x264 directly - compiled with lavf. But lavf IS ffmpeg.. or what do you mean?

[raffriff42]

Thank you STaRGaZeR! Will try LAV Filters. Since FFmpegSource is also libav (http://libav.org/)-based, it should be OK too?

>how-to on getting timecodes from avi
There aren't any timecodes/timestamps in an AVI. Just a constant frame rate and chunk IDs.

[cdz]

>There aren't any timecodes/timestamps in an AVI. Just a constant frame rate and chunk IDs.
Yes, but how to process / decode / analyze the file to yield a proper timecode file?
To use mkv2vfr I need to have an mkv available... So I am forced to used ffmpeg.

As I mentioned previously, by using ffmpeg directly, I end up with an mkv that has proper timecodes. But I'm interested in finding a solution that doesn't require to use ffmpeg directly.

So it seem either I need to find another decoder to pipe over to x264, change some code in lavf, or stick with ffmpeg?

>Thank you STaRGaZeR! Will try LAV Filters. Since FFmpegSource is also libav-based, it should be OK too?
libav == ffmpeg, is it not? So you're getting the same package in a different wrapping.

[STaRGaZeR]

But I've been using x264 directly - compiled with lavf. But lavf IS ffmpeg.. or what do you mean?

Note that I said LAV Filters, not libav or lavf :)

LAV Filters (http://forum.doom9.org/showthread.php?t=156191) is a set of DirectShow filters based on ffmpeg just like ffdshow (much better too, but it lacks video processing filters), with some custom patches, including the one that restores the repeated frames in fraps.

I did that patch, and nevcairiel was kind enough to include it. Tried the same with ffmpeg, but the author refused it because he had no problem with ffmpeg outputting VFR from a CFR codec like fraps.

Thank you STaRGaZeR! Will try LAV Filters. Since FFmpegSource is also libav (http://libav.org/)-based, it should be OK too?

Unfortunately you can't do that, the patch is only in nevcairiel's custom ffmpeg (http://git.1f0.de/gitweb?p=ffmpeg.git;a=summary), that is only used for LAV Filters AFAIK.

What you can do is copy the fraps.c file in nevcairiel's ffmpeg repository, paste it in whatever flavor of ffmpeg you want to and compile it. That should do the trick.

[cdz]

I got bored and made this (http://pastebin.com/upr8rUhc) bash script.
It demonstrates how you can take the timecodes from a 'dummy' mkv created with ffdshow, to mux in with a stream made with x264.
If I could increase the speed further, it would be a viable alternative to have when you don't want to encode through, but still not completely without, ffmpeg.
Please try it out and let me know if something doesn't work.

If you don't have fraps, I'm sure the demo version will suffice. You only need a <30sec anyway.
To make the clip laggy, you just have to set the fraps fps higher than the application you are recording -- or vice versa.
Easiest is to set 30 fps @ fraps and the limit the fps to, say 20, in the application you are capturing.

BTW, edi

[cdz]

More confusion:

The duplicate frames are kept when piping from ffmpeg i.e:
ffmpeg -pix_fmt yuv420p -f yuv4mpegpipe - | x264 --demuxer y4m [...]
Isn't ffmpeg dropping the frames immediately? Or are the frames handled differently when using pipes?

x264 output:

encoded 890 frames, 14.57 fps, 26814.40 kb/s (direct)
encoded 948 frames, 21.12 fps, 28248.74 kb/s (piped)

[cdz]

Just adding that using '-vsync drop' with ffmpeg solves the problem. I'm also using the 'dummy' method described above, which runs at max HDD speed, so it doesn't require that much time.

[Snowknight26]

The ffms demuxer does not have this issue.

lavf:
x264.x64.exe --crf 20 --preset veryfast --demuxer lavf -o test.lavf.mkv "Dwm 2013-07-15 16-16-41-61.avi"
[avi @ 00000000011172E0] non-interleaved AVI
[avi @ 00000000011172E0] Stream #1: not enough frames to estimate rate; consider increasing probesize
lavf [info]: 1920x1080p 0:1 @ 120/1 fps (vfr)
resize [warning]: converting from bgr24 to yuv420p
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
x264 [info]: profile High, level 5.1
x264 [info]: frame I:1 Avg QP:21.08 size:111073
x264 [info]: frame P:67 Avg QP:24.04 size: 13136
x264 [info]: frame B:49 Avg QP:25.83 size: 3399
x264 [info]: consecutive B-frames: 42.7% 1.7% 7.7% 47.9%
x264 [info]: mb I I16..4: 75.5% 3.9% 20.6%
x264 [info]: mb P I16..4: 5.7% 0.8% 2.3% P16..4: 1.6% 0.5% 0.2% 0.0% 0.0% skip:88.9%
x264 [info]: mb B I16..4: 1.6% 0.2% 0.5% B16..8: 1.2% 0.2% 0.1% direct: 0.6% skip:95.7% L0:47.7% L1:50.2% BI: 2.1%
x264 [info]: 8x8 transform intra:9.0% inter:22.8%
x264 [info]: coded y,uvDC,uvAC intra: 25.9% 33.5% 21.9% inter: 0.7% 1.0% 0.2%
x264 [info]: i16 v,h,dc,p: 32% 65% 2% 1%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 43% 21% 2% 3% 2% 3% 1% 4%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 40% 15% 2% 3% 3% 4% 2% 4%
x264 [info]: i8c dc,h,v,p: 44% 44% 10% 1%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: kb/s:3913.45

encoded 117 frames, 48.77 fps, 3915.89 kb/s

ffms:
x264.x64.exe --crf 20 --preset veryfast --demuxer ffms -o test.ffms.mkv "Dwm 2013-07-15 16-16-41-61.avi"
ffms [info]: 1920x1080p 0:1 @ 120125/1001 fps (vfr)
resize [warning]: converting from bgr24 to yuv420p
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
x264 [info]: profile High, level 5.1
x264 [info]: frame I:2 Avg QP:23.87 size: 92935
x264 [info]: frame P:149 Avg QP:22.79 size: 6310
x264 [info]: frame B:166 Avg QP:26.50 size: 236
x264 [info]: consecutive B-frames: 30.0% 0.6% 0.0% 69.4%
x264 [info]: mb I I16..4: 76.0% 4.9% 19.1%
x264 [info]: mb P I16..4: 2.8% 0.4% 1.1% P16..4: 1.0% 0.3% 0.1% 0.0% 0.0% skip:94.3%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 0.4% 0.0% 0.0% direct: 0.1% skip:99.3% L0:74.9% L1:22.5% BI: 2.6%
x264 [info]: 8x8 transform intra:8.6% inter:21.0%
x264 [info]: coded y,uvDC,uvAC intra: 23.6% 30.4% 19.7% inter: 0.2% 0.4% 0.1%
x264 [info]: i16 v,h,dc,p: 35% 62% 2% 1%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 42% 21% 2% 3% 2% 3% 2% 4%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 40% 15% 2% 3% 3% 4% 2% 4%
x264 [info]: i8c dc,h,v,p: 48% 40% 10% 1%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: kb/s:3528.86

encoded 317 frames, 33.97 fps, 3531.04 kb/s

The only downside is that the ffms demuxer needs to index the whole video and it gets the frame rate wrong by a very small percent (120125/1001 vs. 120/1).

[cdz]

Why does it even parse it as x/1001?
For normal clips, the added lag is probably negligible, however you will get a noticeable lag after about two hours

-1/(120-120125/1001)*(120*0,33)/60


I'm already using the 'dummy' method as described further back in the thread. It maxes out the the disk bandwidth with plenty of CPU clocks to spare, and doesn't add that much time to the total.

ffmpeg doesn't seem to have any problems parsing the clips:

Opening an input file: /mnt/cifsenc/fraps/.~archx264/Dwm 2013-07-16 23-40-20-18.avi.
[avi @ 0x285f2e0] Format avi probed with size=2048 and score=100
[avi @ 0x285f920] use odml:1
st:0 removing common factor 1000 from timebase
st:1 removing common factor 4 from timebase
[avi @ 0x285f2e0] File position before avformat_find_stream_info() is 32768
[avi @ 0x285f2e0] parser not found for codec fraps, packets or times may be invalid.
[avi @ 0x285f2e0] parser not found for codec pcm_s16le, packets or times may be invalid.
[avi @ 0x285f2e0] parser not found for codec fraps, packets or times may be invalid.
[avi @ 0x285f2e0] parser not found for codec pcm_s16le, packets or times may be invalid.
[avi @ 0x285f2e0] All info found
[avi @ 0x285f2e0] File position after avformat_find_stream_info() is 1001912
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, avi, from 'Dwm 2013-07-16 23-58-20-18.avi':
Duration: 00:00:12.98, start: 0.000000, bitrate: 177616 kb/s
Stream #0:0, 128, 1/120: Video: fraps (FPS1 / 0x31535046), bgr24, 1920x1200, 120 fps, 120 tbr, 120 tbn, 120 tbc
Stream #0:1, 1, 1/48000: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
Successfully opened the file.

[raffriff42]

>Why does it even parse it as x/1001?
ffmbc is broadcast-centric, so it probably defaults to x/1001 for anything near 30, 60,... fps.

Try an explicit frame rate:
"-r rate: set frame rate (Hz value, fraction or abbreviation)"
This works the same whether for ffmpeg or ffmbc, per ff* -help anyway.

[STaRGaZeR]

The ffms demuxer does not have this issue.

The only downside is that the ffms demuxer needs to index the whole video and it gets the frame rate wrong by a very small percent (120125/1001 vs. 120/1).

Looks like the fraps decoder in libav (http://git.libav.org/?p=libav.git;a=blob;f=libavcodec/fraps.c;h=66ecdef419c770405980cbc4dd0638261cdbc1b5;hb=HEAD) is quite an old version, with some recent fixes on top of it. It doesn't have the repeated frames issue, but it's not multithreaded either. Since ffms is based on libav and not ffmpeg it works there too. That's nice, at least one input method to x264 works without much hassle.