HowTo create Variable Framerate MPEG-4 video streams in MP4

[bond]

as most of you might propably already know its possible to create variable framerate video streams with mpeg-4 video
the fact why it is often not used is probably caused by the crappy avi container, which (apart from many other issues) is not able to store real variable framerate streams
now with the upcoming of far more advanced containers (like matroska and mp4) able to handle vfr, i think we should explore the possibilites of vfr far more (and i hope i can encourage some people to help me with my currently ongoing testings):

as avi and vfw (used by xvid and divx5 in vdm) doesnt seem to be able to handle vfr at all there are two possibilities to create vfr mpeg-4 video streams
1) use a directshow encoder, like the one from 3ivx (suxendrol is already working on a xvid dshow enc filter) and dont even think about touching crappy outdated avi and vfw
2) use vfw encoders, with avi workarounds, and do some smart avi->mp4 conversions:

the xvid encoder already offers the so called "frame drop ratio":
with a setting of 0 only frames get dropped which are 100% the same as the frame before, with 100 all frames are dropped
this option's name and description might be a little misleading as the frames arent really totally dropped (leading to vfr or less frames in the output), also a value of 50 doesnt mean that every second frame is affected -> frames get simply not coded, which means so called n-vops are placed in the stream
this value specifies a threshold under which, if the difference of the following frame to the previous frame is below or equal to this threshold, a frame gets not coded (see it could even be that you set a value of 50, but still no frame gets not coded)
on the decoder side, everytime when the decoder comes to such a n-vop, it displays the frame before the n-vop again, therefore its some sort of "fake vfr stream"

now with the 3ivx or MP4Box MP4 muxers these n-vops can be really dropped and real vfr mpeg-4 streams get created inside the mp4 container


small howto
1) open your to-encode source in virtualdubmod (http://sourceforge.net/projects/virtualdubmod/)
2) go to the xvid (http://xvid.gamrdev.com/) codec settings and choose the settings as you wish and set the frame drop ratio you want to use (on my testssample it seemed that i didnt get a dropped frame with a value of 15, with 90 there were (of course) far too much dropped frames) also it seems that the frame drop ratio seems to work better when packed bitstream (b-frames) is disabled

now you can decide whether you want to use the commandline MP4Box or the 3ivx directshow muxer to create a VFR MP4 file:

MP4Box:
3) download mp4box from rarewares (http://www.rarewares.org/mp4.html)
4) run the following commandline:
MP4Box -import myavi.avi mymp4.mp4 3ivx:
3) install 3ivx (http://www.3ivx.com/download/index.html)
4) open the resulting avi in graphedit (http://www.doom9.org/Soft21/Filters/graphedit.rar)
5) remove all filters, except the source and the avi splitter, load the 3ivx mp4 muxer and the file writer (*.mp4) and connect the 3ivx muxer between avi splitter and file writer
6) enable the "compress n-vops" option in the 3ivx muxer and push play

you should now be able to enjoy and play the created variable frame rate mp4 file with every mpeg-4 decoder (xvid, divx5, ffdshow, 3ivx etc...) in any dshow based player without loosing sync with the audio aso...


edit:
note that during my testings i found that the frame drop ratio was broken in the current release candidates (koepis rc3). syskin now fixed some bugs but it seems that there are still "unfixable" problems when using it with b-frames
therefore the frame drop option in xvid will only work without b-frames from now on :(

edit2: added gpac's mp4box, as it offers exactly the same features as 3ivx now too

[bond]

ok some test results on how/if mpeg-4 codecs will create not coded frames (n-vops) when two frames are the same/similar:

1) xvid (tested rc4, till rc3 no n-vops were created with the default settings (framedrop ratio on 0)!!)
- when b-frames are disabled xvid will create n-vops even if the framedrop ratio is set to 0 (which means it only drops frames that are 100% the same) a higher value will lead to that even frames which are not 100% the same can be dropped
- with b-frames enabled it doesnt output n-vops
2) 3ivx (4.5.1) outputs n-vops
3) nero digital (first version), with and without b-frames, outputs n-vops
4) divx5 (5.1.1) doesnt output n-vops
5) ffvfw (20031117) doesnt output n-vops with the default settings (i didnt find an option which could enable n-vops creation)

[SeeMoreDigital]

Sounds interesting.

Have you any short examples you can provide a link to? I guess 2 encodes would be needed. One with variable frame rate and one without!

Cheers

[bond]

Originally posted by SeeMoreDigital
Have you any short examples you can provide a link to? I guess 2 encodes would be needed. One with variable frame rate and one without!nope but its really easy to do yourself
simply encode a still (anime) or black scene with 3ivx or with xvid (without b-frames) and transmux with the 3ivx muxer
you can checkout with mp4ui if the stream is vfr (for example if the framerate shown is not the same as the input one of course)

during playback you will not see a difference

[Selur]

did I get it right now:

The framdrop ratio is a threshold how much a frame from his predecessor is allowed to differ to be dropped (coded as n-vob) ?

So a ratio of e.g. 0.1-1% shouldn't do much harm in normal movie? (is it even possible to enter values like 0.1?)

Cu Selur

[bond]

Originally posted by Selur
The framdrop ratio is a threshold how much a frame from his predecessor is allowed to differ to be dropped (coded as n-vob) ? yep
it also depends on the chosen bitrate i think (therefore i am not sure how xvid/3ivx measures the differences between two frames)

So a ratio of e.g. 0.1-1% shouldn't do much harm in normal movie? (is it even possible to enter values like 0.1?)who knows. this needs more testing (i only played around with a value of 90, which was too high of course)
with a value of 15 i didnt get a frame drop with rc3 (which was buggy tough)

[Selur]

thx for the answer
okay, if I find some time I'll do a littel testing ;)

Cu Selur

[RadicalEd]

The actual internal value is an int, so no 0.1s. I'm still code-diving to figure out exactly how it works :|

[BITS]

this is fine and well but without B-Frames is this even worth the bother?

Wouldnt you prefer to have all frames using B-Frames than missing frames?

in a world where everybody seems to be trying to get the smoothest playback...this almost seems like a step backwards.

im off to give it a shot...who knows, but i dont think it is for ME. (Unless you can still have B-Frames someday)

[BoNz1]

Originally posted by BITS
im off to give it a shot...who knows, but i dont think it is for ME. (Unless you can still have B-Frames someday)

I believe that it will work with b-frames but just that this will not be added till after 1.0. Keep in mind that when this frame dropping was added it was pretty experimental and nobody used it. So, it is probably untested and might not even give great results now but it definitely could :0. BTW doesn't cartoon mode drop frames? From what I can see in vdub status window it does, and this would explain my problems with 3ivx and mp4 because I always use bframes.

[RadicalEd]

The vfw "Cartoon mode" (which I've found is not the same as what xvidcore considers Cartoon Mode) is *supposed* to create more n-blocks, and I guess therefore more n-frames, but it doesn't seem to work (http://forum.doom9.org/showthread.php?s=&threadid=73876). The core has these two functions seperate and considers vop_cartoon exclusively as Cartoon Mode. At least, that's what the comments indicate :|

So I'd like to say that it doesn't drop frames, but I'm really not sure at this point.

[bond]

i added MP4Box to the guide as it now also handles the detection and removal of encoder dropped N-VOPS and the outputting of a VFR MP4 in the same way as the 3ivx muxer :)

[bond]

i now tested divx5, the latest ffdshow (using libav) and 3ivx (with b-frames and without), whether they place n-vops in the stream...

divx5 and ffdshow still dont do this, 3ivx does it even with b-frames

[lunaticmoon]

Does the display interval of a frame differ from RV10 DropDupe?

RV10 DropDupe (I think):

+----+----+----+----+---------+
drop! and a front frame stays on

True VFR:

+-----+-----+-----+-----+-----+
drop! and equally spaced in the section

3ivx:
?????:confused:

I want True VFR... :)

[stephanV]

my 3ivx doesnt have b-frames...

[DeathTheSheep]

Yes, this was a more than a year ago and it still doesn't have B-frames.

There was a test version that had them (look at the latest Doom9 codec shootout), but I have no idea why it hasn't been released yet. Come on, 3ivX5!

[SeeMoreDigital]

Yes....... whatever happened to 3ivx?

Some of the 3ivx guys used to regularly post on Doom9 and their beta looked promising!


Cheers

[DeathTheSheep]

A lot more than just promising... Doom9's codec comparison showed that its performance was equal to DivX Fusion alpha at like 5 times the speed... faster than XviD even. And if that was about a year ago, imagine the quality now.

----------------------------------------------------------------------------
"Interesting indeed, Dr. House." "Oh really? What would you know, Matilda?" :D

[movax]

I've heard stuff like the "3ivx MP4 splitter" and such bandied about...maybe they shifted focus? The early 3ivx's looked very promising indeed though.

[DeathTheSheep]

Yes, don't even get me started on that wonderful filter set... It used to be the standard among many MPEG4 enthusiasts for muxing and splitting content in directshow.

Almost 2 years later, the filter suite is still experiencing heavy usage, despite its lack of updates (particularly in the AVC field).

The 3ivX codec supported VFR, right? So its stands to reason that its filters support it as well. An update to them as well as the codec itself will surely be beneficial to many an-ASP enthusiast (or any user of MPEG-4 in general).

[bond]

3ivx has lots of interesting updates internally working, some of which have been discussed already publically, still they never released it, why?
propably because their interest shifted away from giving the nerds their full stuff for free!?

[SeeMoreDigital]

...propably because their interest shifted away from giving the nerds their full stuff for free!?In that case.... I wonder when they'll start "selling" us their new products?


Cheers

[bond]

In that case.... I wonder when they'll start "selling" us their new products?maybe they dont aim at joe average users anymore that much

[SeeMoreDigital]

maybe they dont aim at joe average users anymore that muchI guess it would not do any harm if somebody here contacted them to find out what they're doing now... Just to set the record straight... hint, hint ;)

[puffpio]

bump...
just wondering if ffdshow will properly playback a VFR mp4 file?
and if x264 encodes VFR video?

final question, when using mp4box to mux video and audio...if the video has VFR, do I set the framerate based on what it would be had I not used VFR?

thanks!

[bond]

just wondering if ffdshow will properly playback a VFR mp4 file?yes

and if x264 encodes VFR video?no

final question, when using mp4box to mux video and audio...if the video has VFR, do I set the framerate based on what it would be had I not used VFR?no need to set the framerate when muxing from .mp4

[Elias]

3ivx has lots of interesting updates internally working, some of which have been discussed already publically, still they never released it, why?
propably because their interest shifted away from giving the nerds their full stuff for free!?What the hell? What kind of attitude is that? Are you serious? Did they actually say that?

[bond]

What the hell? What kind of attitude is that? Are you serious? Did they actually say that?capitalism is about making money, not love ;)

[Elias]

capitalism is about making money, not love ;)I realise this, but that is such an arrogant behaviour. If they don't want to update their codec, they won't be making money out of it anyway.

[bond]

I realise this, but that is such an arrogant behaviour. If they don't want to update their codec, they won't be making money out of it anyway.its arrogant because they dont give their things away for free? ;)

[Elias]

its arrogant because they dont give their things away for free? ;)No... it's arrogant to think that we're nerds for using their products, when they're actually more nerds because they're coding them :)