VFR trouble: Preview with skips in sync, linear encode out of sync

[LigH]

I have some WMV videos which apparently use variable frame rate (probably rather an "unstable frame rate") and I want to convert them to a constant frame rate by transcoding with a high bitrate to reliably trim clips.

The video I'm currently testing plays nicely in sync in media players like MPC-BE.

It is nicely in sync when I import the WMV into VDub2 (v2.2, v2.3, v2.4) and skip through.
But when I convert the whole video linearly, using x264 and ffmpeg-AAC as codecs in VDub2, saving as MP4 +faststart, the result is out of sync.

It is nicely in sync when I import a remultiplex of the WMV to MKV with ffmpeg (-c copy) into VDub2 and skip through.
But when I convert the whole video linearly, using x264 and ffmpeg-AAC as codecs in VDub2, saving as MP4 +faststart, the result is out of sync.

It is very out of sync when I use DirectShowSource without fps conversion parameters to load the WMV into AviSynth+. Even the number of frames differs by dozens.

It is nicely in sync when I use DirectShowSource without fps conversion parameters to load the MKV into AviSynth+ and skip through.
But when I convert the whole video linearly, using x264 and ffmpeg-AAC as codecs in VDub2, saving as MP4 +faststart, the result is out of sync.

It is out of sync when I use FFmpegSource2 without fps conversion parameters to load either the WMV or MKV into AviSynth+. Not as much as DSS(WMV) but still obviously.

It is nicely in sync when I use DirectShowSource with fps and convertfps=true to load either the WMV or MKV into AviSynth+ and skip through.
But when I convert the whole video linearly, using x264 and ffmpeg-AAC as codecs in VDub2, saving as MP4 +faststart, the result is out of sync.

Using FFmpegSource2 with atrack=-1, fpsnum and fpsden, already the preview is out of sync, half as much as when omitting fps conversion parameters.

So ... what shall we do with the Drunken Sailor?

[poisondeathray]

Use DirectShowSource(convertfps=true) with EnsureVBRMP3Sync for variable WMV's
http://avisynth.nl/index.php/EnsureVBRMP3Sync

[LigH]

Aye, captain!

Trying with the remuxed MKV: The audio is a little late in a preview in the VDub2 timeline, similar to FFmpegSource2 with atrack=-1, fpsnum and fpsden.

Trying with the original WMV: VDub2 has some trouble displaying the video until it reached a keyframe; then same as above.

In both cases, the result after a linear encode is just as async as the timeline preview.

[poisondeathray]

Is wmv disabled in lav ? ie. are you using the official MS decoder for wmv3/wvc1 for directshow ? or use Graphstudio to check

[LigH]

[X] Use Microsoft WMV9 MFT decoder for WMV3 and VC-1 — is enabled. Tray icons for LAV video and audio decoder don't appear despite I enabled showing system tray icons, only the splitter icon appears (twice).

As long as preview and encode delay match reliably, I could manually fix it: Tweak target fps in the conversion, assume standardized fps for the video track, dub with audio later.

[poisondeathray]

Why type of async are you getting for DirectShowSource("wmv", fps=x, convertfps=true) + EnsureVBRMP3Sync? Constant delay ? progressively worsening ? etc...

If you test that script in MPCHC is it in sync ?

Can you try another encoder to rule out a vdub2 issue ?

[LigH]

Unfortunately it is hard to check if the asynchrony is progressive, in the first 2/3 that video contains no obvious action in video and audio to match. I would have to find another video where I can find more spots.

I strongly doubt that VDub2 should be blamed. But I can try other ways to encode. MPC-BE with ScriptSource plays the AVS file with the same delay.

[poisondeathray]

I don't know what ScriptSource is, but something weird going on if the avs preview is different in vdub vs. mpc


It is nicely in sync when I use DirectShowSource with fps and convertfps=true to load either the WMV or MKV into AviSynth+ and skip through. But when I convert the whole video linearly, using x264 and ffmpeg-AAC as codecs in VDub2, saving as MP4 +faststart, the result is out of sync.

MPC-BE with ScriptSource plays the AVS file with the same delay.


Similarly, something weird going on if the avs preview is ok "nicely in sync" , but encode is not ok . If avs preview is ok, but export is not....usually the culprit is encoder or muxer

[LigH]

I believe it is more or less probable that the timecodes are not exactly continuous. I know the same movie exists as a set of segments. So it is not impossible that my version is the result of a concatenation.

Therefore I can imagine that the preview is in sync when skipping through the timeline. Each time AviSynth needs to seek, it looks in the index to find a close keyframe; it doesn't need that in a linear encode. And EnsureVBRMP3Sync will probably do something similar?

PS: Maybe I could use a parameter in ffmpeg to let it rebuild timecodes while remuxing the WMV to MKV?

[poisondeathray]

I believe it is more or less probable that the timecodes are not exactly continuous. I know the same movie exists as a set of segments. So it is not impossible that my version is the result of a concatenation.


Even so , if the original plays back ok in a media player, then DirectShowSource("wmv", fps=blah, convertfps=true) + EnsureVBRMP3Sync should work



Therefore I can imagine that the preview is in sync when skipping through the timeline. Each time AviSynth needs to seek, it looks in the index to find a close keyframe; it doesn't need that in a linear encode. And EnsureVBRMP3Sync will probably do something similar?

Not sure about the mechanism, but it works on those variable WMV's . Without EnsureVBRMP3Sync, there are problems in many cases. There are old threads on this topic (and in other forums()

Another trick to improve preview is preroll(x) , like 30 or 60 or some number. This can help with previewing finicky filters like srestore . Linear encode shouldn't require it, but I don't see how adding it will make your case any worse

[LigH]

I tried to remux the WMV to MKV with the option -fflags +genpts or --fps_mode drop; the result was a very different one, about half the delay of DSS(WMV) without fps conversion, but the same for these two parameters. So I believe irregular timestamps are a quite probable reason.

[LigH]

For some videos I noticed that, in addition to the unstable frame rate, they also erroneously report a double frame rate.

I was not sure whether the frames are exactly duplicated or indeed encoded twice, so I experimented a bit trying to blend two frames to reduce compression artefacts in the second case. But I wanted to avoid too obvious ghosting in case the frame fixing selects a differing pair (similar to field shifted material in case of interlaced fields). During my tests I noticed that blending framerate-fixed clips slows down the processing extremely, possibly a side effect of matching audio with EnsureVBRMP3Sync to the resulting video. I wonder if that reveals a culprit of an optimization strategy in AviSynth+?

fn = "filename_without_extension"
vfn = fn+".mkv" # or ".wmv"
afn = fn+".mkv" # or ".wmv"
a = FFAudioSource(afn).EnsureVBRMP3Sync()
#FFVideoSource(vfn, fpsnum=30010, fpsden=1001) # comment out both Merge's
#FFVideoSource(vfn, fpsnum=60020, fpsden=1001) # comment out one of the Merge's
FFVideoSource(vfn, fpsnum=120040, fpsden=1001)
Merge(SelectOdd(), SelectEven())
Merge(SelectOdd(), SelectEven())
Preroll(300)
AssumeFPS(30000, 1001)
AudioDub(a)

The processing speed varies a lot, from a convenient rate in some scenes down below 1 fps in others. Just as if the caching by EnsureVBRMP3Sync and Preroll is not useful, maybe even counterproductive.

Disclaimer: Using double/quadruple target FPS with Merge to blend to half is visually not a great solution, but it exposes the speed penalty most obviously.

[poisondeathray]

You can also try running through asfbin to see if it fixes errors

And explicitly make a directshow graph .grf for using in avisynth, in case you have other directshow filters with higher merit that are actually being used instead