ffms2 and AVCHD

[ganymede]

Hi,

I'm using avisynth to process AVCHD footage from my camcorder (Sony HDR-CX550VE). Under windows I can use DGDecNV to read these files, and it works very well. Nevertheless, I work mostly under linux (video editing with cinelerra, etc.) and DGDecNV won't run with wine. So, in order to get a frame-accurate input for avisynth under linux/wine, I have to demux the AVCHD files (with tsmuxer, most of the time), remux them -at least the video track- into mkv (with mkvmerge) and input these mkv files into avisynth with ffms2 (vanilla stable version 2.14).
This workflow is usually OK for me with progressive or MBAFF H.264 streams (eg. HDTV video recordings), but not with my AVCHD footage. The video streams I want to process are interlaced H.264 1920x1080i @25 fps (50 fields/s, 25 fps). Here's the output of mediainfo for one of these files (see links below) :

General
ID                               : 0 (0x0)
Complete name                    : 00023.MTS
Format                           : BDAV
Format/Info                      : Blu-ray Video
File size                        : 26.2 MiB
Duration                         : 12s 961ms
Overall bit rate                 : 16.9 Mbps
Maximum Overall bit rate         : 18.0 Mbps

Video
ID                               : 4113 (0x1011)
Menu ID                          : 1 (0x1)
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 2 frames
Format settings, GOP             : M=1, N=26
Codec ID                         : 27
Duration                         : 12s 920ms
Bit rate mode                    : Variable
Bit rate                         : 16.0 Mbps
Maximum bit rate                 : 16.0 Mbps
Width                            : 1 920 pixels
Height                           : 1 080 pixels
Display aspect ratio             : 16:9
Frame rate                       : 25.000 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Interlaced
Scan order                       : Top Field First
Bits/(Pixel*Frame)               : 0.309
Stream size                      : 24.6 MiB (94%)

Audio
ID                               : 4352 (0x1100)
Menu ID                          : 1 (0x1)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : 129
Duration                         : 13s 24ms
Bit rate mode                    : Constant
Bit rate                         : 256 Kbps
Channel(s)                       : 2 channels
Channel positions                : Front: L R
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Delay relative to video          : -80ms
Stream size                      : 407 KiB (2%)

Text
ID                               : 4608 (0x1200)
Menu ID                          : 1 (0x1)
Format                           : PGS
Codec ID                         : 144
Duration                         : 12s 415ms
Delay relative to video          : -80ms

A first problem arises with mkvmerge : I have to set FPS value to 50 in mmg (mkvmerge gui), otherwise the video duration is doubled. I tried several splitters to demux the video stream : tsmuxer (native linux version), eac3to (running under wine) and even DGIndexNV (under windows), but it didn't change anything, so I suppose this problem doesn't come from the splitters.

My main problem is that, no matter if I use 25 or 50 as fps value in mkvmerge, in both cases ffms will double the number of frames.
Here's an example : a short video stream of 325 frames, muxed into mkv, that FFVideoSource will input as 650 frames. When I open the avs script with VirtualDub, the video is jerky, as if all groups of 3-4 frames were duplicated.
* The video stream demuxed with DGDecNV under windows, 24.4 MB
* The 50 fps mkv file, 24.4 MB (frame doubling problem is the same with default framerate of 25 fps, BTW)

Here's an easy workaround to get the right number of frames and the right framerate :

FFVideoSource("video_dgdecnv.50fps.mkv").SelectEven().AssumeFPS(25)

AssumeFPS(25) is needed because otherwise avs2avi, x264, etc., will detect a framerate of 25.155 (50.310 without SelectEven() ).

So, am I doing something wrong with these files ? Is there a way to handle them correctly with ffms2 ? Or is there a bug in the software ?

[TheFluff]

In cases like yours FFMS2 will decode each field into its own frame (because that's how libavcodec's h264 decoder does it). You can try playing around with its rffmode parameter, but really, the solution you already found is about as correct as it gets.

[ganymede]

With rffmode=1, the number of frames is still doubled (650 instead of 325), and FPS value stricly set to 25, so I still have to add SelectEven().AssumeFPS(25). Rffmode=2 is irrelevant.
Anyway, thank you TheFluff for your explanations.
I hope some day libavcodec will be frame-accurate with h.264 streams in mpeg-ts container... Using aformentioned workarounds by then.