methods for opening MP4 and MKV in avisynth

[halsboss]

Hi, I searched but didn't manage to find anything concise... I have coreAVC and the haali that comes with it.

Can someone please clarify whether these are the latest methods of opening MP4 or MKV in AVIsynth ?

i) demux using MKVToolnix's MKVextract to extract to elementary streams and then run DGAVCDec over the elementary video and then open it with DGAVCDecode.dll in avisynth

ii) use DirectShowSource("InputMP4orMKV",fps=theFPS,convertfps=true,audio=false) directly on the MP4 or MKV and haali and coreAVC will take care of it (MKVinfo should yield the framerate,.. you should know it anyway).

Do both methods work OK with temporal processing ?

There must be a reason neuron2 has created the DGAVCDec tool http://forum.doom9.org/showthread.php?t=122598

Anyone care to indicate their preference for getting the video into avisynth and why ?

[buzzqw]

other way could be FFmpegSource

BHH

[actionman133]

DGAVC seems to have better memory management and access than DirectShowSource, as (I imagine) the majority of its source stems from his excellent DGDecode for MPEG-2.

DirectShowSource works okay, but I have had problems in the past where it can't reliably access the right frame and using seek=false is just cumbersome, memory hungry and time consuming...

That's my experience...

[Guest]

My NV tools can open MKV directly, e.g., DGAVCDecNV.

[mavinashbabu]

Hi,

Not sure if it is related to my same question http://forum.doom9.org/showthread.ph...19#post1285419 here

is this the same that is being discussed in this thread.

EDIT: My Question has been answered not sure if it applies here http://forum.doom9.org/showthread.ph...24#post1285424

Thanks
Avinash

[halsboss]

Originally Posted by neuron2

My NV tools can open MKV directly, e.g., DGAVCDecNV.

Thanks, am browsing the thread DGAVCDec to look for it and came across this

Originally Posted by neuron2

Use DirectShowSource() or DSS2().

It seems there are minor "paff" (?) issues with DGAVCDec. DGAVCDecNV is here http://forum.doom9.org/showthread.php?t=141104

Just wondering, what are the relative pros/cons of DirectShowSource and DGAVCDec(NV) to help people choose amongst the options ?

Given I have coreavc and haali installed, does directshowsource have drawbacks ? Like this ... not sure if was related to the codec (eg coreavc or something else)

Originally Posted by actionman133

DGAVC seems to have better memory management and access than DirectShowSource... DirectShowSource works okay, but I have had problems in the past where it can't reliably access the right frame and using seek=false is just cumbersome, memory hungry and time consuming...

edit: ah, came across this. I assume that means DSS isn't frame-accurate ? Pity, I like coreavc.

Originally Posted by neuron2

the main purpose of using CUDA is to get correct decoding for the streams that are not handled by libavcodec, and to access some of the post-processing capabilities of CUDA. Also, to add support for VC-1 in the future.

Originally Posted by saint-francis

Does CoreAVC correctly handle the sources that libavcodec doesn't? If so then the advantage DGAVCdecNV has over it with directshowsource is frame accuracy?

Originally Posted by neuron2

Yes, that is the point of DGAVCDecNV, to be a frame accurate Avisynth source filter. Many people also find DGAVCIndexNV to be a useful little app in its own right for quickly inspecting AVC streams, getting PIDs, seeing what audio is present, etc. The NV support also brings you the PureVideo deinterlacing. And it will bring you VC-1 support in the future. For some it also brings a performance inprovement when transcoding. For fast systems, that will be modest.

[Blue_MiSfit]

DSS2 works very well - it's an improved DirectShowSource.

I use it for my BluRay encodes all the time

neuron2's tools are excellent as well, provided you have a half decent nVidia card. They're particularly handy for me when it comes to handling true 1080i60 content. PureVideo's bob deinterlacer is fairly solid, and runs real-time on the GPU. Compare that to glacial speeds (and better quality, but not by night and day) from TempGaussMC or MCBob.

~MiSfit

[halsboss]

Originally Posted by Blue_MiSfit

DSS2 works very well - it's an improved DirectShowSource. I use it for my BluRay encodes all the time

Excellent ! I used to have a link to it and tried it out for another matter which turned out to be a packed-bitstream related issue, so I'll have to find it again I wonder if there's a reason it isn't included "with" an avisynth install like DirectShowSource is ?

edit: found this

Originally Posted by Blue_MiSfit

... then using DSS2(...) to load the MKV. DSS2 is part of avss.dll, which can be found in your installation directory for haali media splitter.

and this thread discussing DSS2 http://forum.doom9.org/showthread.php?p=1093911 and http://forums.animesuki.com/showthread.php?t=48608

DSS2 is Haali's implementation of Directshowsource. It has the advantage of being frame-accurate, but only supports VFR by converting it to CFR; at least that was the case when I tried it last time, it might have changed.
and
Well if this is going to be turned into a general DSS2 thread... the syntax is:

dss2(string infile, float fps)

i.e.

loadplugin("avss.dll")
dss2("X:/test.mkv",fps=23.976)

Unlike the original DSS, it does not support loading audio. It also always converts VFR to CFR; i.e. it acts like the original DSS with convertfps=true does.
also
Avisource uses VfW codecs and the VfW framework only. Think of it like VirtualDub but in Avisynth. As long as the AVI in question doesn't have any particularly funky hacks, it will always have frame-accurate seeking.
Directshowsource uses Directshow codecs and is hence the Avisynth equivalent of Windows Media Player. It's usually NOT frame-accurate, but it does support VFR, kind of; either through converting to CFR (convertfps=true) or by just reading all frames and ignoring timecodes.
DSS2 is Haali's implementation of Directshowsource. It has the advantage of being frame-accurate, but only supports VFR by converting it to CFR; at least that was the case when I tried it last time, it might have changed.

BUT

Originally Posted by Zwitterion

There are some problems with it, however (M2TS files from Blu-ray, using Haali's splitter and ffdshow in the graph):
- artifacts when seeking in AVC video. They disappear after a few frames.
- When seeking in VC-1 video, sometimes it shows two or three duplicate frames before continuing in the stream.
- It can't load graph files (This isn't a bug, but it'd be a nice feature).
So you can't rely on perfect picture quality in the first few frames after seeking. But at least you know that it is the right frame unlike with DSS, where sometimes the whole stream gets shifted by a few frames.

Originally Posted by neuron2

It just means you have seeked to an open GOP. That's the price you pay for frame accuracy. If you suppress the orphaned frames, then you haven't really seeked to the frame number requested. This is why the DG tools back off by one GOP and decode all the frames necessary to correctly decode the seeked-to frame.

Originally Posted by neuron2

(DSS2)... relies on the stream having completely accurate and continuous timestamps throughout the file, and of course raw streams don't have timestamps. Just clarifying the different niches for DSS2() versus DG. I think the SPS/PPS injection heuristics are a bit better in the DG tools also, which can be relevant for some streams.

[halsboss]

And buzzqw and LoRd_MuldeR kindly pointed out ffmpegsource

Originally Posted by LoRd_MuldeR

http://forum.doom9.org/showthread.php?t=127037

in the dev thread.

Not sure if this is relevant.. they both start with FF

Originally Posted by Flaarn

http://forum.doom9.org/showthread.php?t=120317 As poisondeathray suggested try to move away from ffdshow for the time being, until it handles this content correctly.

I'll try to collate/summarise in a post at the end, with links, after people in the know have opportunity to put their 2c. No doub't that'll need some correction too as I try to read between the lines of the threads and the posts here.

edit: Ps does this forum have a facility for creating a table ?

[halsboss]

So far, given the above, for frame-accuracy in avisynth temporal processing on an MKV or MP4 source then seems like the options are

- DGAVCDecNV can do elementary streams and MKV, but not MP4. It's for NVIDIA card owners only.
- DGAVCDec has more limitations than the NV version (dependency on libav library?)
- possibly ffmpegsource however I'm not sure (neuron2 has some things to say in his DG threads on the libav library and is using an older version, can someone to interpret that and it's effect, if any, on ffmpegsource ?)
- not directshowsource as it's not frame accurate
- not DSS2 as there are artifacts when seeking in AVC video (or there were, when the post was made)

[TheRyuu]

Originally Posted by halsboss

- DGAVCDecNV can do elementary streams and MKV, but not MP4. DGAVCDec has more limitations than the NV version. It's for NVIDIA card owners only.

Because it uses CUDA for decoding (and thus only runs on Nvidia cards, but is faster for decoding). It also costs money.

Originally Posted by halsboss

- possibly ffmpegsource however I'm not sure (neuron2 has some things to say in his DG threads on the libav library and is using an older version, can someone to interpret that and it's effect, if any, on ffmpegsource ?)

I don't really follow (understand the question?). ffmpegsource is built using the current ffmpeg version at the time of compilation.

Originally Posted by halsboss

- not DSS2 as there are artifacts when seeking in AVC video (or there were, when the post was made)

Never had a problem with this, even with m2ts BD streams... You shouldn't rule out dss2 as a viable option, although with mkv/mp4 I'd use fffmpegsource over dss2 probably.

The options for opening a mkv or mp4 and be frame accurate are:
-ffmpegsource
-dss2

I don't see an issue with either of those and I'm not really sure what exactly the problem is here...

[halsboss]

Originally Posted by TheRyuu

I don't really follow (understand the question?). ffmpegsource is built using the current ffmpeg version at the time of compilation.

Well, pending clarification from someone who knows, as I understand it ffmpeg is built using the libav library. Neuron2 says he can't use new versions for a good reason :-

Originally Posted by neuron2

I cannot use the newer versions because they are broken for accurate frame random access.

which in turn implies that ffmpegsource might also be broken too for frame accurate random access (eg possibly with temporal processing). I don't know positively one way or another about ffmpegsource because of that. Probably it's fine, but how to know for sure and not heresay? What are its Pro's and Con's compared to DGAVCDec(NV) ... eg can it handle all the same things just as well and avoid the reported DSS2 issues too?

Originally Posted by TheRyuu

Never had a problem with this, even with m2ts BD streams... You shouldn't rule out dss2 as a viable option

But how should I rule it in or out ? Some people say there's problems with DSS2 and frame seeking in terms of some individual frames being returned as gibberish if Seek'd in the middle of a GOP, another says maybe there isn't a problem, everyone positive sounding in their statements.

Originally Posted by TheRyuu

The options for opening a mkv or mp4 and be frame accurate are:
-ffmpegsource
-dss2

I see you omitted DGAVCDec(NV) probably because it wasn't free, and included one with reported errors and another with a possible questionmark over it. I'm very nearly decided to fork out a couple of peanuts to donate to acquire the NV since it just about seems like the best of the options without conflicting reports. I bought coreavc codec for a couple of peanuts for the same reason. There seems no "agreed, reported by all as reliable" alternative.

Hopefully you see the problem now, with conflicting reported errors and "no there isn't"s for some of the s/w on the list... what IS the situation when information apparently conflicts ? If I had the skills and samples I'd test them all, but I don't.

[Blue_MiSfit]

If you have an nVidia card, don't mind spending just a few measly bucks, and don't mind spending the time to index any source you want to process, neuron2's DG***IndexNV tools are by far the best option. They are extremely reliable, give you excellent free deinterlacing, and are perfectly frame accurate, even when seeking inside a GOP.

If on the other hand you just need to process BluRay or other H.264 MKV / MP4 sources straight through without any trims or seek operations, and don't want to spend a dime, then there is absolutely nothing wrong with DSS2 at all. It works extremely well, and I've done literally hundreds upon hundreds of BluRay backups using this method

~MiSfit

[TheFluff]

just fyi ffmpegsource has never used libavformat to open mkv's, hth

e: afaik there has never been any particular frame accuracy issues with mp4's and libavformat

[halsboss]

from the dev forum (slightly abbreviated) ...

Originally Posted by Myrsloik

Libavformat is one big random access issue just waiting to happen so that sounds about right. The difference is most likely how we treat it. FFMS2 seeks, looks at the returned timestamp, goes back further if it wasn't the intended frame to seek to and so on until it finally succeeds or throws an error (default behavior). Sometimes lavf is so inaccurate it can't match up. That's life. However FFMS2 always goes back to the previous GOP (if it succeeds at all) so that is not an issue. What DSS2 does I'm not really sure of.
I think they've proposed a new seeking API as a SoC project this year though...

[halsboss]

EDIT Dec 2009, this summary is not accurate and is out of date... refer the the following 15 or so posts for clarification.

In summary, as at May 2009,

1. DirectShowSource has problem returning "good" frames when seeking. May be OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... fine if you don't mind some glitches. Opens MKV, MP4 provided you have the right splitter and codec installed. Free.
2. DSS2 (DirectShowSource2) has a problem returning "good" frames when seeking. Appears OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... fine if you don't mind some glitches. Opens MKV, MP4 provided you have the right splitter and codec installed. Free.
3. ffmpegsource tries its heart out but is not guaranteed to seek/return the frame you think you need. Appears OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... seems like a very handy tool and fine if you don't mind the possibility of an occasional glitch. Opens MKV, MP4. Free.
4. DGAVCDec works. Is video-card independent. Is based on an older version of the libav library where seeking issues are not apparent, but as a consequence it handles less "types" of AVC streams than the "NV" version, or ffmpegsource, or a new libav library version could if it seeked properly. Can't have it both ways I guess. Have to demux MKV and MP4 to elementary streams before using it, so team it with a free demuxer. Free.
5. DGAVCDecNV works. Offloads processing to the GPU. Works only with Nvidia 8xxx series and above video cards with VP2/3. ATI seems to have made things challenging and hence left their customers "in the cold" a bit. Is currently the most reliable solution if you want to process video with trims or seek operations and multi-threaded processing or a range of AVC "stream types". Opens MKV. Have to demux MP4 to elementary streams before using it, so team it with a free demuxer. Costs a trivial few peanuts of donation to the author; I forked out... and now I don't have to wonder "what glitches are in my output video because I was a cheapskate".

Happy to see updates... opinions welcome; it seems there are a few "either way" on some things

[buzzqw]

@halsboss

please update the first post too... this thread could be a faq

BHH

[halsboss]

Sure, not sure what you'd like though. PM me and I'll do it.

[TheFluff]

Originally Posted by halsboss

In summary, as at May 2009,

1. DirectShowSource has problem returning "good" frames when seeking. May be OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... fine if you don't mind some glitches. Opens MKV, MP4 provided you have the right splitter and codec installed. Free.
2. DSS2 (DirectShowSource2) has a problem returning "good" frames when seeking. Appears OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... fine if you don't mind some glitches. Opens MKV, MP4 provided you have the right splitter and codec installed. Free.
3. ffmpegsource tries its heart out but is not guaranteed to seek/return the frame you think you need. Appears OK for "straight through without any trims or seek operations" and I guess that means without multi-threaded processing either. Want to process the video in some way... seems like a very handy tool and fine if you don't mind the possibility of an occasional glitch. Opens MKV, MP4. Free.
4. DGAVCDec works. Is video-card independent. Is based on an older version of the libav library where seeking issues are not apparent, but as a consequence it handles less "types" of AVC streams than the "NV" version, or ffmpegsource, or a new libav library version could if it seeked properly. Can't have it both ways I guess. Have to demux MKV and MP4 to elementary streams before using it, so team it with a free demuxer. Free.
5. DGAVCDecNV works. Offloads processing to the GPU. Works only with Nvidia 8xxx series and above video cards with VP2/3. ATI seems to have made things challenging and hence left their customers "in the cold" a bit. Is currently the most reliable solution if you want to process video with trims or seek operations and multi-threaded processing or a range of AVC "stream types". Opens MKV. Have to demux MP4 to elementary streams before using it, so team it with a free demuxer. Costs a trivial few peanuts of donation to the author; I forked out... and now I don't have to wonder "what glitches are in my output video because I was a cheapskate".

Happy to see updates... opinions welcome; it seems there are a few "either way" on some things

you got about half of them wrong and turned this into a neuron2 advertising thread, congratulations

• DSS2 should always be frameaccurate (depending of course on your system's directshow splitter/decoder chain for the format in question) but always converts to CFR. If this is a problem don't use it.
• ffms2 is usually frameaccurate; with mkv it's probably always frameaccurate (after a bug that made it drop the first or second frame was fixed last week); with mp4 lavf is probably good enough to be frameaccurate for all sane cases.

it's funny how you mention the open gop issue that can happen with m2ts as if it was relevant for mkv/mp4; it's also funny how you think you can determine the truth only by reading doom9 posts

[Guest]

Originally Posted by TheFluff

[*]DSS2 should always be frameaccurate

False. If the timestamps are not fully correct and continuous it fails. It also fails for elementary streams.