LAV Filters - DirectShow Media Splitter and Decoders

[madshi]

It's always the first 1080p24 h264 track... ;)

[sneaker_ger]

Ah, OK. Sounds reasonable.

/edit:
And what about multiple views? Or are those not supported with 3D?

[madshi]

AFAIK, Blu-Ray only supports 2 views (3D), nothing more. If there were multiple views, there would still be one normal h264 stream (the first one in the file), plus one or several extra MVC streams, which would all be marked by the splitter as either "right eye" or "MVC".

[sneaker_ger]

Sorry, I didn't mean multiple views as in "more than stereo 3D", but like for alternative scenes. For example the Star Wars DVDs have the "A long time ago, in a galaxy far far away"-text in multiple languages as different video streams. Can this also be done in 3D Blu-Ray? Or is this realized by separate m2ts files?

[madshi]

Blu-Ray does this via seamless branching = different m2ts/ssif files.

[sneaker_ger]

I see. Guess you've answered all my questions, then.

[Boltron]

Basic question regarding audio playback...

There is a lot of discussion regarding the playback of audio, Arcsoft Audio HD, decoders, 96/48KHz, downmixing etc...

I use an HTPC exclusively for playback of MKVs. I use MPC-HC, ffdshow video decoder, LAV Splitter/Audio. I have an ATI 5670 video card HDMI connected to my Denon AVR-791. Audio is over HDMI with bitstreaming. My MKV movies are mostly ripped from bluray in full 1080p without compression and I strip out all audio tracks except for the best track available (DTS-MA, TrueHD, DTS, AC3).

Now my question.

As I bitstream with LAV Audio, the way I understand it, the audio info is just passed over the HDMI to the AVR where the DAC/Audio processing happens. There is no decoding done by LAV Audio (e.g. 96/48KHz, downmixing), it is a simple passthrough. As such I do not need to concern myself with many of the topics seen here lately regarding Arcsoft Audio HD, decoders, 96/48KHz, downmixing. Is this assumption correct?

My ultimate goal is the best possible no compromise playback.

Thanks

[SamuriHL]

If you're bitstreaming, you're good. The audio is passed unaltered to the receiver. Done. :)

[nevcairiel]

Since the topic was discussed in some length today, i did some testing on some files in my sample library to figure out how much one can trust container timestamps for audio streams.

I measured the jitter of the incoming timestamps to my calculated timestamps that assume the stream is continous.

The values in the following tables are all in DirectShow reference time, which has the unit 100ns, that means a value of 10.000 is one millisecond, a value of 10.000.000 is one second.
All tests are with LAV Splitter, of course. I do not vouch for any other splitters timing accuracy.

MKV
Format Min Max Avg Status Muxer

AAC 0 10000 4992 OK

AC3 0 0 0 Perfect

MP3 0 0 0 Perfect

DTS 0 140000 23828 Acceptable mkvmerge v2.4.0 (most DTS files tested are exactly like this)
0 760000 90500 Acceptable mkvmerge v4.5.0

DTS-HD MA 0 580000 126125 Bad mkvmerge v4.1.1
0 1970000 420455 Bad mkvmerge v4.1.1
0 10000 9066 OK MakeMKV 1.6.0

TrueHD 0 15000 8600 OK MakeMKV 1.6.0
0 15000 8816 OK MakeMKV 1.6.7
0 5000 2500 OK mkvmerge v3.1.1

From my limited test set, it would appear that mkvmerge fails at muxing DTS, as it doesn't seem to set a proper duration for the frames, or something to that effect. MakeMKV on the other hand is nearly perfect, a maximum jitter of 1ms.
After adding the default duration field to the DTS MKVs created by mkvmerge, the values did change slightly, however the problem was not resolved.

Note that in all MKV tests, the value frequently still reaches 0, which means there is no static sync offset, and its not drifting, its just jittering that badly.

AAC is OK, AC3 and MP3 are perfect, TrueHD is also OK.

Now in comparison the mpeg container, since the results were basically the same, i summed up TS (both TS and m2ts) and PS (mpg, vob, evo)
MPEG-TS/PS
Format Min Max Avg Status

AAC 0 0 0 Perfect
LATM AAC 0 0 0 Perfect

AC3 0 0 0 Perfect

E-AC3 0 320000 160000 Acceptable, jitter is caused by the dependent substream thats not supported by the decoder, 5.1 streams will be fine
160000 160000 160000 16ms sync offset, otherwise perfect. (EVO)

MP2 0 0 0 Perfect

DTS-HD MA 0 0 0 Perfect

TrueHD 0 0 0 Perfect

LPCM 0 0 0 Perfect

Well this looks great, doesn't it? :)
For some reason the E-AC3 5.1 stream from an EVO has a 16ms sync offset, but i consider 16ms to be acceptable. Maybe my parser is caching one frame too much, and thus causing a delay.

So, i come to a short conclusion:
Adding a "safety net" that catches audio gaps is possible, at least for MPEG containers it should be trivial, just resync if we go above maybe 20ms, maybe a bit more to be safe.

For MKV, its a whole different story.
The default MKV timebase is so coarse (just millisecond precision), that the frame time of DTS cannot be accurately represented, so it seems to be cut off (10.6666 becomes 10), and frequently we get one frame with a big change in the time to resync the timestamp with the Video. This can be as big as 76ms, or even nearly 200ms in some DTS-HD files. MakeMKV is smarter there, and builds up durations like 11, 11, 10, 11, 11, 10, or something similar, which do not create a big offset.

So, how do you detect a gap in the audio in this situation? I can't exactly rely on the incoming timestamps, except if it reaches insanely high jitter values (>250ms maybe). I could keep checking the jitter offset, and if it fails to reach 0 for a while (maybe a timespan of 1s? 5s?), i use the lowest value encountered as the new offset.

Ideas, suggestions, anyone?

[madshi]

Ideas, suggestions, anyone?
My ideas:

(a) Report this problem to Mosu, so he can fix the problem in mkvtoolnix. He's quite willing to fix bugs. Of course that won't help with the files out there which are already broken. But it will help in future.

(b) For TS/m2ts I still think simply using the container timestamps would be the most straightforward way. You wouldn't even need a safety net for TS/m2ts then.

(c) For MKV you could first of all use your own timestamps. You'll need a safety net to account for gaps/overlaps. For that you could measure the arithmetic mean of the last [x] "container timestamp - calculated timestamp". As long as there's no audio gap/overlap, the mean should be very near to zero. If the mean gets too far away from zero, there likely was a gap or overlap which you need to correct for.

[nevcairiel]

(a) Report this problem to Mosu, so he can fix the problem in mkvtoolnix. He's quite willing to fix bugs. Of course that won't help with the files out there which are already broken. But it will help in future.

I'll certainly try that, i'll have to find a neat way to show the problem though, and reproduce with the latest mkvmerge to begin with.


(b) For TS/m2ts I still think simply using the container timestamps would be the most straightforward way. You wouldn't even need a safety net for TS/m2ts then.

Due to the parser, which buffers data internally for one frame, and me buffering PCM data before output, its not as simple as it sounds.
However, i already did all the legwork with the code which determines the jitter offset, so i can just use my reference value for the timestamp. ;)


(c) For MKV you could first of all use your own timestamps. You'll need a safety net to account for gaps/overlaps. For that you could measure the arithmetic mean of the last [x] "container timestamp - calculated timestamp". As long as there's no audio gap/overlap, the mean should be very near to zero. If the mean gets too far away from zero, there likely was a gap or overlap which you need to correct for.

The mean will never reach zero, thats basically what the Avg column is above, except its over my whole sampling period, and not a shorter timespan.

"container timestamp - calculated timestamp" will look something like this (in milliseconds)
Frame1: 0
Frame2: 1
Frame3: 2
...
Frame77: 76
Frame78: 0
...

Thats why i had the idea to keep watching for the lowest value in a given timespan, and sync to that.

[robpdotcom]

Not sure if this is relevant, but mkvmerge began writing the Default Duration of DTS files a few versions back. All files made before (at least, the 50 or so that I tested), can be fixed via mkvmerge's Header Editor (by adding 10666666 for the default duration).

It solved the DTS-HD + mkvmerge + Haali problem.

[nevcairiel]

That does not actually fix the timestamps of the frames, just the duration (which is not relevant for me)

[robpdotcom]

OK, just thought I'd throw it out there. I'm usually not any help, but sometimes I still like to try.:p

[madshi]

The median will never reach zero, thats basically what the Avg column is above, except its over my whole sampling period, and not a shorter timespan.

"container timestamp - calculated timestamp" will look something like this (in milliseconds)
Frame1: 0
Frame2: 1
Frame3: 2
...
Frame77: 76
Frame78: 0
...

Thats why i had the idea to keep watching for the lowest value in a given timespan, and sync to that.
No no, I meant it differently. Some posts ago you said:

> i have seen deviations of 300ms, then a few
> samples later its -300ms, and it just swaps
> around like that all the time - 100% constant
> and predictable

So basically you have something like this:

Frame1: -300ms
Frame2: +300ms
Frame3: 0ms
Frame4: -300ms
Frame5: +300ms
Frame6: 0ms

The simple average of these deviations should be pretty much zero. If that's the case, audio and video should be in sync. If the simple mean (not median!) goes up to e.g. 50ms then there appears to be a gap of 50ms which you need to correct for.

[nevcairiel]

Ah i typo'ed mean into median, oh well.

Anyway, those pattern are only observed on some TrueHD files, but i compensate for that automatically now.
As you can see in the data above, the big problem is DTS, and its problem is that the timestamps slowly drift apart (due to cutting off 10.66666ms to 10ms), and then one frame comes along to sync them again. There is no negative deviation to form a proper "0". If i were to average the deviations, i would add audio delay thats not supposed to be there. As long as the sync-correction-frame still corrects the deviation to 0 again, i'm still in perfect sync.

[ontherocks]

Yes with the splitter DD-THD bitstreaming worked.
However DD+ showed as DD in my Onkyo HT-R380. I guess its not bitsreaming DD+ properly.

[madshi]

Ouch, that is really bad. Maybe you can detect that sync is drifting in 10.666 / 10.0 speed and ignore that - unless that one correction frame doesn't come at all? To be honest, I'm not really sure how to handle this. Maybe the best solution would be to naively use the container timestamps and blame the broken video file / muxer for any problems caused by that? Have you checked what the other MKV splitters do? I bet they simply output the container timestamps. Seems to work well enough?

[SamuriHL]

That receiver doesn't claim to support DD+ decoding.

[nevcairiel]

Ouch, that is really bad. Maybe you can detect that sync is drifting in 10.666 / 10.0 speed and ignore that - unless that one correction frame doesn't come at all? To be honest, I'm not really sure how to handle this. Maybe the best solution would be to naively use the container timestamps and blame the broken video file / muxer for any problems caused by that? Have you checked what the other MKV splitters do? I bet they simply output the container timestamps. Seems to work well enough?

LAV Splitter also only outputs container timestamps. But this is not a splitter problem, its the audio decoder. :)

Other audio decoders are not handling this very well. The MPC-HC decoder just resyncs timestamps everytime it drifted more then 100ms, which in the DTS-HD case will send that jitter to the audio renderer, which either trys to compensate, or you hear minor sound glitches. ffdshows jitter correction is just flawed to begin with.

What i'm thinking right now is keep watching out for the smallest absolute value in a given timespan, and if thats still close to 0, its still in sync. If there was a gap, or overlap, the absolute value will not be 0 anymore (some huge positive number for overlap, negative for gaps)

[ontherocks]

That receiver doesn't claim to support DD+ decoding.

http://img155.imageshack.us/img155/9504/clipboard01tm.png

And also a photograph showing DD+ decoding would suffice?

[nevcairiel]

A photograph with DD+ decoding would suffice?

The tech spec page lists DD-Plus, SamuriHL is just not looking close enough.

Anyhow, i have a Onkyo TX-SR607, and DD+ works just perfectly for me - i would assume its somewhat similar decoding units in those two. ;)

[nevcairiel]

To illustrate the DTS timing problem, here a quick dump of the log output with the timing logging active..

http://pastebin.com/raw.php?i=S3fXZKUi

Note that i buffer 3 decoded DTS samples before delivery (to form 32ms samples), so you'll not see the 10ms increases
I just noticed something else in there - the timestamps are not even continous, they go backwards sometimes, which causes those really big jumps in the offsets.

[ontherocks]

Anyhow, i have a Onkyo TX-SR607, and DD+ works just perfectly for me - i would assume its somewhat similar decoding units in those two. ;)

If possible could you describe your setup.
Its really annoying & frustrating that I am not able to get DD+ working since a long time.
Struggled a lot with ffdshow, then concluded that it hasn't been implemented yet.
I have discussed about it here http://forum.doom9.org/showthread.php?p=1494674#post1494674
Got excited when I got to know about LAV, still I am back to square one.

[SamuriHL]

The tech spec page lists DD-Plus, SamuriHL is just not looking close enough.

Anyhow, i have a Onkyo TX-SR607, and DD+ works just perfectly for me - i would assume its somewhat similar decoding units in those two. ;)

Yea I missed that page. The page I saw only listed DD, TrueHD but it didn't have DD+. Awesome. :) No idea why it's not working, either. Works fine with my Pio's.

[Skinleech]

Yea I missed that page. The page I saw only listed DD, TrueHD but it didn't have DD+. Awesome. :) No idea why it's not working, either. Works fine with my Pio's.

Works fine on my 5450 to an Onkyo 876 also.

[nevcairiel]

If possible could you describe your setup.

Nothing special.
NVIDIA GTS450 for HDMI out, connected to the receiver.
Using ReClock, but tested DD+ also without - but if you're not using it yet, try with it too (or without :d).

[noee]

DD+ good here on LAV->Reclock->HD5670->HDMI->TX-NR905

[nightfly]

Works fine on my 5450 to an Onkyo 876 also.

And my Denon 988. Just got a new AVR though a Yamaha V867 and will see how that pans out.

[ontherocks]

One basic question.
How do I know that LAV splitter is working? Is there an indication in the taskbar or something?

[nevcairiel]

Well, in MPC-HC you would see it in the filter list. (Right Click the player -> Filters). It may also be hiding as the last entry with the name of the file, you can find out by opening the property page from there.

[nevcairiel]

New buildzies

http://files.1f0.de/lavf/LAVFilters-0.25-38-gfd5a9b3.zip

Highlights (only LAV Audio changes):
- A first draft of automatic A/V syncing when it detects an actual offset between our calculated timestamp and the timestamps from the incoming data. I *hope* this doesn't break anything, but i only tested MPEG-TS and MKV, other containers may go wildly wrong. The code is also very conservative, it may require up to 5-6 seconds for it to catch on to the sync problem, but IMHO thats alot better then having it re-sync a working video and causing issues.
- The ability to use the ArcSoft dtsdecoder directly through LAV Audio, just drop "dtsdecoderdll.dll" in the LAV Audio directory, and lean back for full DTS-HD enjoyment. On the LAV Audio status page, the ffmpeg decoder is called "dca", and the ArcSoft decoder is called "dts" (or postfixed with the profile), so you can see if its active.

Please note: Do not ask where to get the ArcSoft decoder. If you own TMT, its in the TMT folder under Codecs. If you don't, you should buy it if you want to use their decoder. This is *NOT* the place to try to score it for free. I respect their work, and so should you.

[SamuriHL]

New buildzies

http://files.1f0.de/lavf/LAVFilters-0.25-38-gfd5a9b3.zip

Highlights (only LAV Audio changes):
- A first draft of automatic A/V syncing when it detects an actual offset between our calculated timestamp and the timestamps from the incoming data. I *hope* this doesn't break anything, but i only tested MPEG-TS and MKV, other containers may go wildly wrong.
- The ability to use the ArcSoft dtsdecoder directly through LAV Audio, just drop "dtsdecoderdll.dll" in the LAV Audio directory, and lean back for full DTS-HD enjoyment. On the LAV Audio status page, the ffmpeg decoder is called "dca", and the ArcSoft decoder is called "dts" (or postfixed with the profile), so you can see if its active.

Please note: Do not ask where to get the ArcSoft decoder. If you own TMT, its in the TMT folder under Codecs. If you don't, you should buy it if you want to use their decoder. This is *NOT* the place to try to score it for free. I respect their work, and so should you.

Absolutely wonderful! This means, for those that aren't fully understanding, that you no longer need to register the ASAudioHD.ax file to get this to work. You just need to drop the dtsdecoderdll.dll file into the LAV Audio dir and you're good to go. Full bitdepth decoding of DTS-HD MA. WOO HOO!

I agree with Nev's comments about buying TMT if you use this. That will support their future development and allows us to use a really well developed decoder. Seems like a fair exchange. :) This is truly awesome. Thanks, Nev!

[idaho_mark]

I feel a ripple in the force!! Way to go - thanks much!

[BatKnight]

Is it specific to any TMT version? Because I've dropped the 1.1.0.0 and the 1.1.0.7 and neither work.
I've unregistered LAV and registered again and still only show dca on the format page.. Not dts there.

Am I missing anything?

Bat

[SamuriHL]

You misread. It will show that on the STATUS page when PLAYING a file. :)

[BatKnight]

Oh, sorry.
Yeah, that's it. It shows dca with using LAV and dts-hd ma when using Arcsoft.

Bat

[SamuriHL]

Then it's working. :) Man, such a horrible setup process. LOL!

[ontherocks]

To guys with ATI Radeon HD 5XXX series and Onkyo receivers........
Do you really see the http://img811.imageshack.us/img811/8278/clipboard02eo.png light on the receiver while playing DD+ streams?

[Mercury_22]

AVC playback it's broken in some m2ts files with MPC-HC's internal decoder = black screen and audio
Changing the decoder to another one or changing the splitter to LAV's "official Version 0.25" or to MPC-HC's internal it's solving the problem

Sample (http://www.megaupload.com/?d=DUE6Q94R)

[nevcairiel]

Don't you know better then "some m2ts files"? :)

[Mercury_22]

Don't you know better then "some m2ts files"? :)

:)Sorry I've forgot the link
Fixed :p

[Village Guy]

I am presently decoding DTS-HD Master using LAV Splitter + Audio decoder and bitstreaming to my Denon AVP successfully.

Am I to assume that this combination is not actually passing the full audio resolution and adding the Arcsoft decoder does?

Would someone mind clarifying this for me.

Thanks.

[nevcairiel]

:)Sorry I've forgot the link
Fixed :p

I switched to official 0.25, and that file never plays properly with MPC-HC decoder. Although i don't get a black screen, i get some corrupted image.
MPC-HC internal splitter gives me black screen.

I think the decoder just doesnt like the file..

I am presently decoding DTS-HD Master using LAV Splitter + Audio decoder and bitstreaming to my Denon AVP successfully.

Am I to assume that this combination is not actually passing the full audio resolution and adding the Arcsoft decoder does?


Well are you bitstreaming, or decoding? :)
If you bitstream, that is your receiver does the decoding (you ticked the HDMI bitstreaming checkboxes in LAV Audio config), then you do not need to do anything, you'll get the full quality.

[Mercury_22]

I switched to official 0.25, and that file never plays properly with MPC-HC decoder. Although i don't get a black screen, i get some corrupted image.
MPC-HC internal splitter gives me black screen.

Official 0.25 & MPC-HC's (1.5.2.3104) AVC (DXVA) internal it's playing fine here :confused:
Nevcairiel-LAVFSplitter-0.25-39-gd1a6c61 & MPC-HC's (1.5.2.3104) AVC (DXVA) it's broken = audio + black screen

I'm using VS2010SP1 and gcc461 for both (Lav & MPC)

P.S. The file it's straight from blu-ray (Hi-Definition Reference Disc)

EDIT NON-DXVA it's working with Nevcairiel-LAVFSplitter-0.25-39-gd1a6c61 too !:confused:

[Village Guy]

Well are you bitstreaming, or decoding? :)
If you bitstream, that is your receiver does the decoding (you ticked the HDMI bitstreaming checkboxes in LAV Audio config), then you do not need to do anything, you'll get the full quality.

I appreciate the clarification :thanks:

[nevcairiel]

Official 0.25 & MPC-HC's (1.5.2.3104) AVC (DXVA) internal it's playing fine here :confused:
Nevcairiel-LAVFSplitter-0.25-39-gd1a6c61 & MPC-HC's (1.5.2.3104) AVC (DXVA) it's broken = audio + black screen

I'm using VS2010SP1 and gcc461 for both (Lav & MPC)

P.S. The file it's straight from blu-ray (High definition reference disc)

Ah, i tested with software mode. And it is broken there, which does indicate that the file is wonky.

I'll test a bit.
Non-DXVA is broken for me with both the MPC-HC splitter and LAV Splitter.
DXVA is only broken with LAV Splitter.

I did try official 0.25, and no change. Maybe some change in ffmpeg.

Edit:
Nope, no changes, it remains broken. I blame the MPC decoder, its really far from usable these days.

[nevcairiel]

So i figured something out. Setting the MPC-HC decoder to 1 thread fixes software playback with LAV Splitter. I didn't think the MPC-HC decoder used any threads. o.O

MPC-HC Splitter: Software broken, DXVA working
LAV Splitter: Software working, DXVA broken

I still couldn't get 0.25 working..

this is weird. :P

[Mercury_22]

Changing decoder thread numbers make no difference here
Still it's very weird that 0.25 it's not working for you

[nevcairiel]

I fixed it for now, will need some proper fix later.