How to automatically encode VFR with Mencoder (and some bash magic)

[Dinominant]

Lately I've been attempting to convert some Babylon 5 episodes to VFR MKV exclusevly with Mencoder. 2 days ago I found a way to do it:
#!/bin/bash

# Dump video
mencoder -dvd-device ../Babylon5.iso dvd://1 -noskip -mc 0 -vf phase,softskip,harddup -ofps 1 -nosound -ovc raw -of avi -o video.phase.avi -frames 1200

# extract 5 frame sections
for LIMIT in `seq -w 0 239`; do
mencoder video.phase.avi -noskip -mc 0 -ss $(( `printf "$LIMIT" | sed -r "s/0*([0-9])/\\1/"` * 5 )) -ovc raw -of avi -o $LIMIT.avi -frames 5
done

# if exactly 1 frame is a duplicate then drop it and change the framerate
for FILE in [0-9]*.avi; do
if mencoder "$FILE" -vf decimate -ovc raw -o /dev/null 2>&1 >/dev/null | grep -q "1 duplicate frame"; then
# drop duplicate and convert to 24000/1001
mencoder "$FILE" -vf decimate=-4 -noskip -mc 0 -ofps 24000/1001 -ovc raw -o temp.avi >/dev/null 2>/dev/null
mv temp.avi "$FILE"
else
# convert 30000/1001
mencoder "$FILE" -noskip -mc 0 -ofps 30000/1001 -ovc raw -o temp.avi >/dev/null 2>/dev/null
mv temp.avi "$FILE"
fi
done

# join various files
mencoder [0-9]*.avi -forceidx -ofps 120000/1001 -ovc raw -o video.vfr.avi

If you don't feel like reading the code here is how it works:
1. Change the framerate to 1fps so you can randomly seek with perfect precition.
2. Split the video into 5 frame chunks.
3. Determine if each chunk needs to be inverse telecined.
4. Convert each chunk into 120000/1001fps with null frames.
5. Merge the chunks.
6. Mux into MKV with mkvmerge. The null frames will be removed and proper timecodes are created by mkvmerge.

Now, this prototype does have 2 drawbacks (one of which is solved in the next prototype):

Timecode alignment can be off by 1-5 frames. IE the framerate could change 1-5 frames too soon or too late. AV sync is *not* affected in any negative way. IMO this isn't that big of a problem since you would only notice it by counting the NULL frames at a scene change before they are removed by mkvmerge. And this is assuming the framerate changes on a scence change.
This prototype requires many (read hundreds) GB of space to work

I solved the problem of storing hundreds of GB of temporary files with a few thousand FIFO's and some EDL's:
# dump video and convert fps so we can randomly seek it with perfect precision
FRAMES=78901
printf "Dumping video... "
mencoder -dvd-device ../Babylon5.iso dvd://1 -noskip -mc 0 -fps 1 -nosound -ovc copy -o "video.m2v" >/dev/null 2>/dev/null
printf "done.\n"

# generate segment fifo's and EDL's. This is a pain but it'll save hundreds of gigabytes of space
printf "Generating FIFO and EDL segments... "
mkdir "segments"
mkfifo "segments/first.fifo"
printf "6 78901 0\n" > "segments/first.edl"
for SEGMENT in `seq -w 1 $(( $FRAMES / 5 ))`; do # the math is bad. Some frames at the end are not encoded...
INT=$(( `printf "$SEGMENT" | sed -r "s/0*([0-9])/\\1/"` * 5 ))
mkfifo "segments/$SEGMENT.fifo"
printf "0 "$(( $INT ))" 0\n"$(( $INT + 6))" 78901 0\n" > "segments/$SEGMENT.edl"
done
printf "done.\n"

# start the encoder
function end
{
kill -9 $SERVER
kill $ENCODER
exit
}
trap end SIGINT SIGTERM
cat segments/first.fifo segments/[0-9]*.fifo | mencoder - -demuxer rawvideo -rawvideo fps=120000/1001:w=720:h=480:format=yv12 -vf decimate=-0 -ovc x264 -x264encopts qp=10 -of avi -o video.vfr.avi &
ENCODER=$!

# serve the segments to the encoder through the fifo's w00t
for SEGMENT in first `seq -w 1 $(( $FRAMES / 5 ))`; do # the math is bad. Some frames at the end are not encoded...
mencoder "video.m2v" -edl segments/$SEGMENT.edl -hr-edl-seek -ss $(( `printf "$SEGMENT" | sed -r "s/0*([0-9])/\\1/"` * 5 - 100 )) -frames 5 -vf phase -ovc raw -of avi -o "phased.avi" -really-quiet
if mencoder "phased.avi" -vf decimate=-4 -ovc raw -o /dev/null 2>&1 >/dev/null | grep -q "1 duplicate frame"; then
# drop duplicate and convert to 24000/1001
mencoder "phased.avi" -noskip -mc 0 -vf decimate=-4 -ofps 24000/1001 -ovc raw -of avi -o "converted.avi" -really-quiet
else
# convert 30000/1001
mencoder "phased.avi" -noskip -mc 0 -ofps 30000/1001 -ovc raw -of avi -o "converted.avi" -really-quiet
fi
mencoder "converted.avi" -vf harddup -ofps 120000/1001 -ovc raw -of rawvideo -o "segments/$SEGMENT.fifo" -really-quiet &
SERVER=$!
wait $SERVER
done
Here I'm simply using FIFO's and the power of pipes ^_^. Each fifo acts as a single 5 frame chunk that has been converted into 20 frames (5 30000/1001fps frames converted into 20 120000/1001fps progressive frames). Mencoder removes the duplicates, encodes null frames, and you have the AVI ready to mux with mkvmerge.

Here is a sample, before (http://www.nathanshearer.com/modules/core/home/3/temp/Before.avi) and after (http://www.nathanshearer.com/modules/core/home/3/temp/After.mkv), of mixed 30p and 24p Babylon 5. Notice that the mkv is smaller, even with the same QP encode options due to fewer frames and that they are all progressive. Please ignore the audio channel issues, I'm lazy with the downmixing...

Once I find a way to automatically detect VFR material I'll release a new version of Abdvde that can produce VFR encodes ^_^

[JohnAStebbins]

Or you could just use HandBrake which defaults to VFR output.

[b66pak]

@Dinominant thanks for sharing...
_

[Selur]

@Dinominant thanks for sharing...
I second that (and hoping someone finds an easier way)

[Erik_Osterholm]

When you say that it doesn't impact sync negatively, do you mean that there literally is no impact, or that it won't be noticeable? Wouldn't changing the frame rate 5 frames too late result in sync occurring up to 208ms too late for a group of 5, or 416ms too late if you're particularly unlucky?

[nm]

I think the sync difference would be more like 5 * (1 s / 24) - 5 * (1 s / 30) = 0.0417 s, which is not noticeable during such a short period.

[rernst]

I don't see how you could do two-pass encoding. What I do is simply do an -ofps 120000/1001 with pullup and softskip which gives me null frames (and proper error messages about skipped frames). I then mux using mkvmerge which throws away the null frames and I have perfect 'vfr' footage. Sync can be off slightly due to the skipped frames in the beginning (I have always seen 1/3rd second) which you can fix using mkvmerge (there is no drift).

I see no need to switch back and forth.

[Selur]

"and proper error messages about skipped frames"
hmmm,.. just an idea:
may be it would be possible to build a timecode file based on these messages,...

[nm]

If mkvmerge throws null frames away like rernst said, you can read the correct timecodes later with mkvextract. But there's no need for that if you're fine with MKV.

[Dinominant]

When you say that it doesn't impact sync negatively, do you mean that there literally is no impact, or that it won't be noticeable? Wouldn't changing the frame rate 5 frames too late result in sync occurring up to 208ms too late for a group of 5, or 416ms too late if you're particularly unlucky?

I think the sync difference would be more like 5 * (1 s / 24) - 5 * (1 s / 30) = 0.0417 s, which is not noticeable during such a short period.

Close, it's actually 4(1/24) - 4(1/30) = 0.033 seconds

If the framerate changes from 24fps to 30fps, then in the worst case the video could lag behind by a maximum of 4(1/24) - 4(1/30) = 0.033 seconds. The delay only lasts for 4 30fps frames. After that everything is syncronized again. This happens becuse the framerate changes during a 5-frame sample. This sample must be encoded at 24fps or 30fps, so for a maximum of 4 frames (133ms) things could be out of sync by 33ms. I suppose you could notice this if the framerate changed every 5 frames and you watched for it; thank god DVD's aren't mastered that badly...

I don't see how you could do two-pass encoding. What I do is simply do an -ofps 120000/1001 with pullup and softskip which gives me null frames (and proper error messages about skipped frames). I then mux using mkvmerge which throws away the null frames and I have perfect 'vfr' footage. Sync can be off slightly due to the skipped frames in the beginning (I have always seen 1/3rd second) which you can fix using mkvmerge (there is no drift).

I see no need to switch back and forth.
Actually, I tried this approach a while back and I found that pullup simply drops frames during the 30fps sequences, then after dropping the frames mencoder adds the null frames. The result is CFR 24fps with juddering 30fps sequences all packed into a 120fps avi with null frames.

Encoding multiple passes requires slightly more bash magic. Just do everything 2 or 3 times. It's not fast, or ideal, but gets the job done if you don't want to use handbrake. I also briefly tried handbrake and found that the decomb filter didn't do a very good job deinterlacing compared to filmdint. A lot of the video was blured.

Abdvde 1.6.0.0 will have this added as a feature, but you'll need to explicitly set the telecine method to "VFR hard telecined to 30000/1001t" since I don't have any code that can detect a VFR DVD.

I'm thinking of writing my own deinterlacing/IVTC filter for mencoder that outputs VFR friendly video (via null frames), but have yet to make any progress on that front.

[Selur]

I'm thinking of writing my own deinterlacing/IVTC filter for mencoder that outputs VFR friendly video (via null frames), but have yet to make any progress on that front.
that really would be a nice addition to the currently present deinterlacers,..