Encoding damaged video with ffmpeg : audio desynchronisation

[Kivenkantaja]

Hello,

a few years ago, I've captured (with Huffyuv codec) movies from an old camera (analog source). Then, I converted them to DV format (audio in FLAC), which I call "masters".

However, some of the sources where damaged...

Now, I want to convert my masters (DV) to accessible standards (x264) to share with family.

I use AviSynth for several treatment (Trim, TomsMoComp, Crop, LanczosResize, ConvertToYV12, Convolution3D, Blur) and encode them with ffmpeg.

It's working well with non-damaged sources.

But, with the damaged sources, the audio is coming much later than the video.

I guess that the damaged framed are "dropped" or something like that...

I could use some Delay() function on my avs Script but that's not a relevant option...

I tried some -vsync, -async, -r 25 option, without result (anyway, I'm not sure is I'm using them well)

As a test, I converted directly the DV sources to x264 : I don't have any desynchronisation. So there is something with the avs Script and the ffmpeg command I guess.

My ffmpeg command line is the following :
ffmpeg -i !SRC! -map 0 -c copy -c:v libx264 -preset faster -crf 20 -profile:v high -level:v 4.1 -c:a:0 libfdk_aac -b:a 128k !DST!


Actually, I don't where to start... Any clue is welcome !

Thanks in advance !

[foxyshadis]

Post an exact script that you use. It's probably the input plugin causing it.

[fvisagie]

You could also compare results opening your script in VirtualDub and/or a media player. That should provide some indication as to whether the desync is originating in Avisynth or ffmpeg in this case.

[Kivenkantaja]

Thanks for your repplies !

I've launched my avs with Media Player Classic and... There is no desynchronisation... So it's coming from ffmpeg... Or an incompatibility between ffmpeg and a plugin ? is it possible ?

Anyway, here is my script :

# Variables
megui_path = "C:\MeGUI_2418_x86\"
final_width = 640
final_height = 480
file_path = "Master DV+FLAC.mkv"
text = "Text"

src = DirectShowSource(file_path)

# ---- BlankClip ----
blank = BlankClip(length=25 * 5, width=final_width, height=final_height, pixel_type="YV12", fps=25, color=$000000, channels=2)

blank = blank.Subtitle(text, first_frame=0, last_frame=25 * 5, font="Palatino Linotype", size=48, text_color=$FFFFFF, halo_color=$000000, align=5, spc=0, lsp=0)
blank = blank.FadeIO2(25, color=$000000)
# ---- !BlankClip ----


# ---- Trim ----
trimmed = src.Trim(8, 51764)
# ---- !Trim ----


# ---- Deinterlace ----
LoadPlugin(megui_path + "tools\avisynth_plugin\TomsMoComp.dll")
tomsmo = trimmed.TomsMoComp(1,5,1)
# !Deinterlace ----


# ---- Crop + Resize ----
cropped = tomsmo.Crop(12, 2, -12, -16)
resized = cropped.LanczosResize(final_width, final_height)
# ---- !Crop + Resize ----


# ---- Denoise ----
yv12 = resized.ConvertToYV12()
LoadPlugin(megui_path + "tools\avisynth_plugin\Convolution3DYV12.dll")
convo = yv12.Convolution3D("vhsBQ") # Heavy Noise
blurred = convo.Blur(0.40) # Fait grandement gagner en compressibilité
#sharped = blurred.MSharpen(240)
video = blurred
# ---- !Denoise ----

video = video.FadeIO2(25, $000000)

return blank + video

Thanks in advance !

[fvisagie]

I've used ffmpeg to encode straight from AVS the way you describe many times without problems. Therefore, given the result of your comparison with MPC the problem probably lies with ffmpeg or its command-line. Assuming you're using a fairly recent ffmpeg build (?), the problem is therefore possibly the command-line.

Originally Posted by Kivenkantaja

My ffmpeg command line is the following :
ffmpeg -i !SRC! -map 0 -c copy -c:v libx264 -preset faster -crf 20 -profile:v high -level:v 4.1 -c:a:0 libfdk_aac -b:a 128k !DST!

Unless you have very specific reasons not mentioned, there seems to be some redundancy and/or confusion here. It might be useful to start off with the most basic possible command-line, hopefully get that working and then build up from there.

If I remember correctly, ffmpeg normally includes only the first video and audio streams (and also excludes subtitles?). '-map 0' includes everything, but you specify audio encoding for only 1: ':0'. The initial '-c copy' is superseded later, unless there are more audio streams. In that case potential trouble could arise if those default to a codec other than libfdk_aac. Does your input really have multiple video and/or audio streams? To some extent defaulting would be influenced by your choice of output file extension. What container format are you using?

How about starting off with:

ffmpeg -i !SRC! test.mp4

Unless I'm mistaken that should default to H.264 video and AAC audio. If it doesn't, add the appropriate one from '-c:v libx264' or '-c:a libfdk_aac'.

Before adding back your formatting options, if need be next play around with sync-related options. In addition to the ones you mention, you can use '-map 0,0:n' to select the n+1th stream(*) of the input file as synchronisation source. Try both video and audio if necessary.
*: like the other map option parameters, n counts from 0

If the problem turns out to be encoding-related and you don't get it fixed here, there is also the dedicated "MPEG-4 AVC / H.264" encoding subforum here.

[fvisagie]

PS. If all else fails, you could try making intermediate lossless encodes which you then encode with ffmpeg instead. For the lossless encodes you could use e.g. VirtualDub with the Lagarith codec. If you have very many damaged files you can use VirtualDub's batch processing to create the intermediate files, and Windows' "for %i in (*.avi) do ffmpeg -i %i ..." syntax to batch encode those.

[Kivenkantaja]

Hi,

I'm using FFMPEG 2.2.4 32bits non-free (downloaded on orbroot.fr). I could try with the Zeranoe version eventually...

I've "cleaned" the command line (the "map" options were remains from another generic command line I use...). I have one audio track and one video track, that's all.

But the following command lines gave me the same desynchronisation :

ffmpeg -i !SRC! test.mp4
ffmpeg -i !SRC! !DST!_normal.avi
ffmpeg -i !SRC! -c:v ffv1 -qscale:v 0 !DST!_ffv1.avi
ffmpeg -i !SRC! -c:v huffyuv !DST!_huffyuv.avi

Finally, I encoded the avs with VirtualDub 1.10.4 with Lagarith, and I have the same problem ! And I'm not dreaming : I have no desynchronisation with MPC...



I will try something with -vsync and -async option (I've already tried it but with my map options, maybe it was not the good way to use these).

Thank you again for your help, I really appreciate.

[fvisagie]

Originally Posted by Kivenkantaja

I will try something with -vsync and -async option (I've already tried it but with my map options, maybe it was not the good way to use these).

If those don't work, there are many more sync-related options to find if you read carefully through ffmpeg's documentation. The vanilla Zeranoe build you mention is also worth comparing results with.

However, it's perhaps time for you to post your problem along with a short test clip to reproduce it with on the encoding forum. ffmpeg.org also links to an email list of their own failing any solution here.

[Kivenkantaja]

Finally I get it !!!

In the avs :
I use :

src = DirectShowSource(file_path, convertfps=true, pixel_type="YV12")

Instead of :

src = DirectShowSource(file_path)

(and btw, "pixel_type="YV12"" replace the .ConvertToYV12() function I was using later in the script)

convertfps = false (in v2.56): When setting it to true, it turns variable framerate video (vfr) into constant framerate video (cfr) by duplicating or skipping frames. This is useful when you want to open vfr video (for example mkv, rmvb, mp4, asf or wmv with hybrid video) in AviSynth. It is most useful when the fps parameter is set to the least common multiple of the component vfr rates, e.g. 120 or 119.880.

Thanks again for your help, it really helped me !

[fvisagie]

It always adds an extra little glow when you found and fixed the problem yourself, well done!

Thanks for coming back to post the solution, by the way. Now others can benefit from your experiences, too.

EDIT: I must say I'm a little puzzled by the fact that "convertfps=true" seems to have been the fix for DV video which is supposedly constant framerate. Or, do you mean "digital video", instead of "encoded with the DV codec"? Either way, glad you found a fix!

[Kivenkantaja]

Thanks and you're welcome !

By DV I mean : http://en.wikipedia.org/wiki/DV
The FourCC is "dvsd"

Maybe I missed something when I converted from HuffYUV to DV...


Note that with that "convertfps=true" parameter, the encoding time takes almost the double time...

[poisondeathray]

Originally Posted by Kivenkantaja

Note that with that "convertfps=true" parameter, the encoding time takes almost the double time...

Any reason why AVISource() isn't used ? A free VFW DV decoder is cedocida

DirectShowSource is usually the last resort , given the multiple potential problems and inconsistencies

[Kivenkantaja]

My "masters" are DV+FLAC into MKV (sorry fvisagie, I forgot to answer you about that).

I wanted to try something original

Finally, it's not taking always the double of time... It takes more time when some frames need to be added I guess. Anyway, I will not do lots of benchmarks since it works

Edit : "it's *not* taking always..."

[fvisagie]

Originally Posted by Kivenkantaja

My "masters" are DV+FLAC into MKV (sorry fvisagie, I forgot to answer you about that).

I wanted to try something original

Finally, it's taking always the double of time... It takes more time when some frames need to be added I guess. Anyway, I will not do lots of benchmarks since it works

It might be worthy doing _one_ more test with the AVI container instead of MKV and using AVISource() as suggested by pdr. I'm not sure how happily AVI works with FLAC, so this might extend to two tests, the additional one with audio converted to PCM, MP3, AC3, etc.

[creaothceann]

Btw. there's also DSS2.

[Kivenkantaja]

Hello,

It seems that AVI does not work with FLAC...

And opening an AVI (Huffyuv + WAV) with AVISource() does not resolve my problem... I guess that my first conversion (from raw (Huffyuv) to DV) was not very relevant ...

So I did the work with my previous solution. By the way, I've written a mistake :
Finally, it's **NOT** taking always the double of time... It takes more time when some frames need to be added I guess.

Thanks again for the help.

[LemMotlow]

Originally Posted by Kivenkantaja

It seems that AVI does not work with FLAC...

And opening an AVI (Huffyuv + WAV) with AVISource() does not resolve my problem... I guess that my first conversion (from raw (Huffyuv) to DV) was not very relevant ...

Do people still do this? Why?

I'll echo previous response: see post #12. Now that you've already screwed up going analog->DV for apparently no reason, work that nearly-ruined analog capture as lossless media, man. Jeez, I can't help it -- It's like people behave as if they never heard this before.

Otherwise: never mind.

[Kivenkantaja]

Dear Lem Motlow,

I'll try to explain my choice (which seems to be dumb, as if I was behaving like I never heard this before, which the case).

When I captured the videos, a few years ago, some tutorials were mentioning lossless codecs like Huffyuv.

Unfortunately, Huffyuv files are quite big. And I wanted to keep some kind of "masters" videos, but I wanted also save disk space...

After a few research, it appeared that DV was a good compromise : a good compression ratio, saving space and keeping the maximum of visible information for the human eye.

So, people like me still do this. But if you give me a good advice with courtesy, I will learn from my mistake and take a better choice for my next capturing sessions

Sincerely

[LemMotlow]

OK, point taken. Indeed, huffyuv files are bigger than DV. But the assumption being made is that DV and VHS use the same color matrix and are in most respects, more or less, "alike". But they aren't. So if file size was your priority you achieved what you wanted. In the past I gave DV a try. Never again, not with VHS. DV was never designed for corrections / conversion / re-encoding to other formats. That doesn't mean it can't be done, but it's a pain to clean up. What gets lost in color resolution and lossy encoding doesn't come back later.

I realize that there are many who worship VHS->DV. So be it. I don't see it recommended by those who understand the two media more deeply. But if that's what you have, that's you must work with. Meanwhile, I think earlier advice would be the best method: decode that to lossless AVI (the cedocida codec is a good choice and works with AviSource) and use the other corrections that worked. Losslessly compress the working files with huffyuv or Lagarith, and do what you have to do. You don't have to keep the intermediate working files. Likely you'll want Lagarith for working with YV12, as many versions of huff work only with RGB or YUY2. I don't save all my captures, but I do save captures made for those who give tapes to me for work. The captures I save are lossslessly compressed Lagarith AVI on external hard drives or given to the tape's owner.

As someone has stated here (somewhere, I don't recall where), variable frame rate is a solution in search of a problem. Sorry, but when I see posts like this I do get a little bent out of shape, especially when I see these posts all over the place. I can understand how the usual website targeting the "average consumer" gives tangled advice about this stuff from people who don't know what they're talking about. I ran into a lot of that when I first started this crazy activity, and nothing I'd like better than to have serious but brief discussions with those who write that stuff. But having had many such videos presented to me for work, all I can think of is how much time and trouble are involved and how much better the final results could often be.

Anyway, good luck. No sense wasting time trying to undo what's already done.