LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

So it is somehow splitter related.... Interestingly the mpc splitter also fails with all audio filters (including the internal mpc audio filter) except lav. Is there any explanation? At the momemt haalis way of fixing it in the splitter seems better as it is compatible with all audio filters---

Haali basically decodes LPCM to "normal" PCM, which avoids the problem - so the LPCM decoders in the audio filters you listed do not get used, and the bugs do not show.

This is not a "fix" in my book, and i will out of principal not do such hacked workarounds just to avoid broken decoders, especially when there is a working decoder which also works with the MPC-HC splitters.

[nevcairiel]

TBH, I'm really only interested in making it work with the Sage demux; it uses the right subtype ({EB27CEC4-163E-4CA3-8B74-8E25F91B517E}) :)

Is this from Blu-ray content?
If it is, does decoding TrueHD work with LAV Audio?

On Blu-ray, TrueHD is interleaved with AC3 frames for backwards compat. Most MPEG-TS splitters do not remove those AC3 frames, and if they do not, they completely confuse LAV Audio - which will also show when decoding.

Anyhow, if decoding does not work, its basically the same issue the MPC-HC MPEG splitter has, except that the MPC-HC one additionally sends the wrong media type. :p
I will have to write my own parser to separate the TrueHD frames from the AC3 frames in those streams .. i'll get to it eventually, its somewhere on the list..

[andyvt]

If it is, does decoding TrueHD work with LAV Audio?


It does not with BD (M2TS), it does with MKVs.


Anyhow, if decoding does not work, its basically the same issue the MPC-HC MPEG splitter has, except that the MPC-HC one additionally sends the wrong media type. :p
I will have to write my own parser to separate the TrueHD frames from the AC3 frames in those streams .. i'll get to it eventually, its somewhere on the list..

So this would be a good place to start if I can't get Sage to address it in their splitter?

Do you know off-hand if the MPC-HC or ffdshow decoders parse out the AC3 frames?

[nevcairiel]

Both do. They both have been designed in This circle of bugs that are fixed at the wrong end.

[Portioli]

it does something more than simple Decoding TrueHD (both .m2ts .mkv)


http://i.imgur.com/XMETF.jpg


and the media info`s

http://i.imgur.com/KnMd7.jpg

[Andy o]

That's ReClock doing it. Check "slave reference clock to audio" and set the media adaptation speed to "original" for no resampling.

[Portioli]

fixed, and it works.
does this mean that lav audio + reclock via wasapi can decode bitferfect hd audio?

[Andy o]

Yes, but I don't know why you'd wanna do that. Reclock basically guarantees frame error-free playback when decoding.

[Portioli]

because i think Windows/MPC default audio renderer downsamples the HD Audio

[Andy o]

It doesn't when you're using WASAPI exclusive mode, like you already are with ReClock. Even if you're not using exclusive mode, the Windows mixer won't resample if you set it to the same sample rate. If you set it to a higher rate, it will upsample. BTW, that "bit-exact" message means that whatever ReClock is outputting, after processing, is being delivered "bit-exact" to the driver.

[Portioli]

well , if i want bitfercet i have to set the driver`s output , default format to the files bitrate/samplerate?
do all audio filters decode without downsampling?

[Andy o]

well , if i want bitfercet i have to set the driver`s output , default format to the files bitrate/samplerate?
Yes, but then you don't get ReClock's main advantage which is video without dropped/repeated frames. Depending on how close your video card is to the file's fps, you can get many, or just one or two per movie. From what I can see you're running your display at 59.999 (60 for all practical purposes), while most content is either 29.97 or 59.940, this leads to many frame errors in the duration of a regular video/movie.

I don't know if your display can do 24p, but in that case, the frame repeats/drops are more noticeable.

You should ask yourself again why you need bit-perfect, cause the reasons you gave above don't really apply here.

do all audio filters decode without downsampling?
No, the commercial ones do, but there's no reason to use them for other than DTS-HD decoding, and for that, the Arcsoft decoder is the only one you can use. Some versions of it do downsample, so you need to make sure you're using the one from build .185 of TMT3.

[Casshern]

Haali basically decodes LPCM to "normal" PCM, which avoids the problem - so the LPCM decoders in the audio filters you listed do not get used, and the bugs do not show.

This is not a "fix" in my book, and i will out of principal not do such hacked workarounds just to avoid broken decoders, especially when there is a working decoder which also works with the MPC-HC splitters.

Ok, that explains it. What you are saying is that ffdshow lpcm and mpc lpcm decoders are broken. Good to know - maybe I will get rid of ffdshow audio decoding all together, as i never liked its jitter correction feature which seems somewhat broken. Some streams will even only play if it is enabled....... did you use part of that jitter correction routine in your lav audio decoder?

[SamuriHL]

I've appeared to have lost the ability to display subtitles using ffdshow. :( Anyone else having issues? This seems bad.

EDIT: Appears to be ffdshow and cyberlink decoder not playing well together. Switched to ffdshow on that machine and it's working fine. sigh. No idea what's up with that.

[ppp0941]

will there be thumbnail feature?:helpful:

[nevcairiel]

did you use part of that jitter correction routine in your lav audio decoder?

Nope, there is no real need for jitter correction, unless the audio stream is not continuous (broken files), but i've not yet encountered one file that would be like this... If i find one i can probably fix those cases as well.

So anyway, when decoding, i do not trust the timestamps of the source filter at all, i rather simply decode, and afterwards i know *exactly* the duration of the decoded data, and with that i can perfectly track the time.

[madshi]

If you need broken files, just record some TV programs and check them with "eac3to source.ts -check". Once in a while you'll get a file that's broken.

It might make sense to compare the container timestamps to your self-calculated timestamps to make sure they don't deviate too much. If you calculate a difference of more than 20ms there's likely something wrong, I'd say. If you don't do this, you might get audio/video desync problems with broken files.

[nevcairiel]

20ms is not enough, especially when you're dealing with a stream from a MKV file, the timestamps are so coarse that they deviate quite alot - especially on TrueHD, which has a sample size of 0.83ms, i have seen deviations of 300ms, then a few samples later its -300ms, and it just swaps around like that all the time - 100% constant and predictable. At first i thought there would be a problem, but with the very constant pattern, i just attributed it to too coarse timestamping.

[madshi]

P.S: Another option would be to take a "good" TS or m2ts and randomly overwrite a chunk of maybe 50KB in the middle of the file with zeroes, or with random data. Or if you want to simulate an audio problem, only, you can search for the AC3 header signature and just zero out the AC3 header of maybe 5 consecutive AC3 frames in the TS/m2ts stream. That way the video should play fine but audio should have a gap.

[madshi]

Oh well, I didn't know that MKV timestamps are so bad. eac3to calculates its own timestamps (just like you do), but keeps track of the original timestamps, too. Any audio gaps/overlaps are then removed in a second pass. Of course I have it much easier than you in this case, cause you can't do a second pass, of course.

Maybe you should treat MKV different to TS/m2ts? Because MKV is pretty much guaranteed to have no audio gaps/overlaps (unless the muxer totally crapped out). While with TS corruption/gaps/overlaps are quite usual, because satellite reception isn't always 100% reliable. Also, from my experience, TS/m2ts audio timestamps are pretty much perfect. So for TS/m2ts I think you could probably rely on the container timestamps?

[nevcairiel]

Well, there isn't an actual problem right now to fix, its all theoretical at this point.
I'll probably create some broken ts file and see what happens. I'm also not sure how the timestamps of TrueHD in m2ts look, as the TS timebase of 1/90000 is still too coarse to perfectly timestamp audio of 0.833ms each sample. (still alot better then 1/1000 MKV timestamps)

Playback right now is fine, its just the handling of corrupted streams that needs some work - and if its just broken samples, i can detect those and re-sync against the container timestamps after such a decoding failure. Now if there is frames completely missing, thats another deal entirely.

There is another aspect to consider. Internally i buffer PCM samples until i reach a certain buffer length, because some post-processors are not particularly happy when they get samples of 0.8333ms each - so what i do is collect samples until i have at least 16ms (and in addition try to collect enough samples so the duration can be expressed as an integer), and during buffering my internal time tracking does not advance.

But like i said, playback right now is perfect, and just from my personal opinion: screw DVB recordings, TV is just crap anyway. :)

[Casshern]

If you basically throw the original time stamps away, I tend to agree with madshi. There needs to be a mechanism to ensure that audio and video stays in sync. Its a shame that in some cases the timestamps are unreliable. Its the same with video where everybody invents their own way of deriving the real frame rate from the stream. For audio this would mean averaging over a large enough amount of samples and occasional resyncing with video. One would hope this would increase error resilience.

20ms is not enough, especially when you're dealing with a stream from a MKV file, the timestamps are so coarse that they deviate quite alot - especially on TrueHD, which has a sample size of 0.83ms, i have seen deviations of 300ms, then a few samples later its -300ms, and it just swaps around like that all the time - 100% constant and predictable. At first i thought there would be a problem, but with the very constant pattern, i just attributed it to too coarse timestamping.

[madshi]

Well, there isn't an actual problem right now to fix, its all theoretical at this point.
I'll probably create some broken ts file and see what happens. I'm also not sure how the timestamps of TrueHD in m2ts look, as the TS timebase of 1/90000 is still too coarse to perfectly timestamp audio of 0.833ms each sample. (still alot better then 1/1000 MKV timestamps)

Playback right now is fine, its just the handling of corrupted streams that needs some work - and if its just broken samples, i can detect those and re-sync against the container timestamps after such a decoding failure. Now if there is frames completely missing, thats another deal entirely.
I don't think it's so theoretical. From my own broadcast recordings, about 1 out of 5 is in some way corrupted. Corruption can be just a couple of wrong bits, or larger blocks of random data. Depending on the type of corruption, anything can happen. E.g. when the AC3 decoder receives a frame to decode and the header signature isn't correct, the decoder (or parser) will likely skip the damaged data and simply wait for the next proper frame. The net effect is an audio gap. For video the probability of a missing frame is smaller cause the data/header ratio is different. Also, even if video and audio frames are both damaged, there's still a high chance that they will run out of sync because 1 video frame has a different runtime compared to 1 audio frame. Sure, if you resync after every problem, that might fix the majority of problems. But since the corruption is random, I would be afraid that in some cases you might not detect the corruption, but still there could be an audio gap/overlap.

For m2ts the Blu-Ray spec defines how big TrueHD data blocks should be, IIRC. I think the blocks are defined in such a way that they match the timestamps somehow, but I'm not 100% sure right now. Should be easy to double check. IMHO, the best solution would be for you to check how reliable TS/m2ts timestamps are. If they are as reliable as I think they are, the easiest and best solution would be to use the TS/m2ts timestamps as they are instead of using your own timestamps. Of course for MKV I would keep the current logic, but maybe add some kind of safety net.

Just my 2 cents, of course. Feel free to ignore... :)

[nevcairiel]

Only TS should suffer from this high corruption probability, right?
Even m2ts on Blu-rays should be mostly fine again, as i would hope they are muxed somewhat sanely, and all non-mpeg formats are typically written by "offline" muxers, and not dumping broadcasts onto disc.

I can of course detect if the source filter claims to have opened a .ts file, and if it did, sync against the source timestamps every time i deliver a PCM sample, or something to that effect.

Edit: A quick test with some m2ts look good, the timestamps appear to be perfect. But i do not have a TrueHD sample right now, only checked AC3 and DTS. More testing this afternoon or over the weekend. My sample collection on the work laptop is quite limited. ;)

[madshi]

Yes, only TS should suffer from high corruption probability, I'd say. However, some of the freeware muxers allow converting TS files to m2ts, so a TS->m2ts file could be corrupted, too. Also, if the timestamps are perfect, there's little reason to not use them. :) So you could simply use the container timestamps for TS and m2ts, while using your own calculated timestamps for other containers. Alternatively you could also choose the timestamps based on the container + audio format. E.g. you could use the container timestamps only for TS + AC3/E-AC3/DTS/MP2, but not for TrueHD or DTS-HD. Broadcasts never use TrueHD or DTS-HD, so TrueHD and DTS-HD have a rather low probability of being damaged.

[ontherocks]

Is there a Readme or tutorial on this?

My goal is just bitstreaming no more no less.
So I installed just the Audio Decoder (install_audio.bat).
I went to MPC-HC View-->Options-->External Filters-->Add Filter and selected "LAV Audio Decoder". Then set it to "Prefer".

http://img806.imageshack.us/img806/7591/capture13052011141300.png

Then double clicked "LAV Audio Decoder" to bring up the options. Selected all bistreaming options.

http://img808.imageshack.us/img808/7044/capture13052011141033.png

Then deselected all the internal filters in MPC-HC.

http://img508.imageshack.us/img508/5892/capture13052011141126.png

Now if I play any DTS stream (DTS, DTS-HD) it bistreams fine to my Onkyo HT-R380.
But if I play any DD HD stream (DD-THD, DD+) it doesn't bitstream them. The receiver seems to go in a loop trying to find the input via HDMI.
The files I am trying to play are from http://www.demo-world.eu/trailers/high-definition-trailers.php

Am I missing something? Is there anything else I need to install/do?

[nevcairiel]

Are you using LAV Splitter as well, or another splitter, like Haali?

TrueHD bitstreaming currently only works reliably with LAV Splitter, especially out of mpeg-ts files (.ts, .m2ts)

I should probably write some more extensive guide or something. Anyone else want to do this maybe? :p

[sneaker_ger]

Maybe you should treat MKV different to TS/m2ts? Because MKV is pretty much guaranteed to have no audio gaps/overlaps (unless the muxer totally crapped out)

There are lots of mkvs with gaps in audio. It can easily happen, for example on appending files.
I don't think just assuming that files don't have gaps is a good idea (just like assuming files run with a constant framerate).

[madshi]

There are lots of mkvs with gaps in audio. It can easily happen, for example on appending files.
:(

So some kind of audio/video sync safety net is necessary, it seems.

[nevcairiel]

IMHO, if some tool merges two mkv files, it should make sure the stream is continuous at the merge point.

Anyway.
I changed my timestamp difference detection to detect the actual jitter of the decoded audio, and so far results look pretty good, even on MKV. I'll run some extensive tests on all kinds of formats later, and post my findings, in case you're interested.

[Thunderbolt8]

it does something more than simple Decoding TrueHD (both .m2ts .mkv)

[PICS]

Im wondering, how can it says bit perfect output, while reclock is actually slowing down or speeding up in that moment you played those files? (its not set to speed: original & locked)

[sneaker_ger]

IMHO, if some tool merges two mkv files, it should make sure the stream is continuous at the merge point.

Although I agree that it may be desirable, e.g. mkvmerge can't do it (without destroying sync) and that's pretty much the mkv muxer.

[Jasch]

Hi i have a little Problem mit the MKV Splitter, it is reporting very often the wrong Framerate to my Player(DVBViewer).
Most of the Time it is reporting 23.97FPS ,but it should be 25FPS. If i use Reclock , in Reclock it is detected as 25, MPC Splitter is also working correct and is reporting 25.
The strange thing is, with some Files it is working some not.
Since i am using a little Plugin in DVBViewer wich is switching Refreshrate according to inputmaterial, this goes wrong.

If you need more things, sampels...please let me now.
(Sorry for my spelling)

[madshi]

I'll run some extensive tests on all kinds of formats later, and post my findings, in case you're interested.
Sure we are... :)

[ontherocks]

Are you using LAV Splitter as well, or another splitter, like Haali?

TrueHD bitstreaming currently only works reliably with LAV Splitter, especially out of mpeg-ts files (.ts, .m2ts)

I should probably write some more extensive guide or something. Anyone else want to do this maybe? :p

No, I don't have anything else on my computer. No other splitter, no other player (other than default WMP). Just MPC-HC and LAV Audio Decoder.
Do I need to install the splitter (install_splitter.bat) as well along with the decoder?
or
just the filter is enough ? But then I guess with just the splitter I won't get the options to bitstream.

If I am able to figure out how to go about it correctly I will write the guide myself.

[nevcairiel]

You'll need both.
Add LAV Splitter to the external list as well, and it should hopefully work.

In the future, LAV Audio should eventually also work with the default MPC-HC splitter, but then again, there is no really good reason to still use it anyway. :)

[madshi]

In the future, LAV Audio should eventually also work with the default MPC-HC splitter, but then again, there is no really good reason to still use it anyway. :)
There is one: There's a patch available for the MPC-HC splitter, enabling it to output 3D content. There's no such patch available for the LAV splitter yet. There are no 3D decoders and renderers available yet, but that can change quickly these days.

[nevcairiel]

There is one: There's a patch available for the MPC-HC splitter, enabling it to output 3D content. There's no such patch available for the LAV splitter yet. There are no 3D decoders and renderers available yet, but that can change quickly these days.

If it does change, i'll look into it.

Do you know how thats used in a player anyway?
Does the player open a second instance of the splitter for the second stream on the Blu-ray?

Thats how i figured it would work, after reading the patch, but i never really investigated.

I also wonder how 3D in Matroska will work..

[ontherocks]

You'll need both.
Add LAV Splitter to the external list as well, and it should hopefully work.

In the future, LAV Audio should eventually also work with the default MPC-HC splitter, but then again, there is no really good reason to still use it anyway. :)

Ok. Now I can see the following in External Filters.
Should I Add both of them?

LAV Splitter
LAV Splitter Source

[nevcairiel]

Doesn't hurt to add both, but you'll most likely only need "LAV Splitter".

[madshi]

If it does change, i'll look into it.

Do you know how thats used in a player anyway?
Does the player open a second instance of the splitter for the second stream on the Blu-ray?

Thats how i figured it would work, after reading the patch, but i never really investigated.
For Blu-Ray the left and right eye video streams are encoded as totally separate video streams, but in the same m2ts file, similar to how e.g. main video and PIP video tracks are separate video tracks in one m2ts file. So the splitter could simply export 2 video output pins, one for the normal left eye stream (compatible with any h264 decoder) and one for the right eye stream (needs a special h264 MVC decoder). Should be very easy to do. From what I've heard, 3D camcorders output a single video stream, containing both the left eye and right eye slices.

I don't see the need to create 2 splitter instances.

[sneaker_ger]

For Blu-Ray the left and right eye video streams are encoded as totally separate video streams

No, the left eye is normal h.264 (for backwards compatibility) , but the right eye cannot be decoded without the left eye AFAIK.

[ontherocks]

Doesn't hurt to add both, but you'll most likely only need "LAV Splitter".

I added only "LAV Splitter" (prefer minimalistic setups :) )

http://img847.imageshack.us/img847/3031/capture13052011180824.png

I think the default options should be fine as far as bitstreaming is concerned.

http://img714.imageshack.us/img714/5294/capture13052011180419.png

[nevcairiel]

For Blu-Ray the left and right eye video streams are encoded as totally separate video streams, but in the same m2ts file, similar to how e.g. main video and PIP video tracks are separate video tracks in one m2ts file. So the splitter could simply export 2 video output pins, one for the normal left eye stream (compatible with any h264 decoder) and one for the right eye stream (needs a special h264 MVC decoder).

I see.
For some reason i thought the one eye was in the normal m2ts file (for 2D backwards compat), and the second stream would be in the ssif file, and only be opened when required.
So the ssif contains both eyes again? Interesting..

I'll probably bring this up again in a separate topic when i'll start to work on this.

No, the left eye is normal h.264 (for backwards compatibility) , but the right eye cannot be decoded without the left eye AFAIK.

Thats what he said. :p

[nevcairiel]

I think the default options should be fine as far as bitstreaming is concerned.

You might want to fill in the audio language field, so you end up with the proper languages, in case your media has multiple streams. :)

[sneaker_ger]

Thats what he said. :p

If you regard one stream relying on the other as "totally separate streams" that is. ;)

[madshi]

No, the left eye is normal h.264 (for backwards compatibility) , but the right eye cannot be decoded without the left eye AFAIK.
From a decoder point of view you're right. I was talking from a splitter point of view. LAV Splitter can treat the left and right eye stream as totally separate streams. As long as the timestamps are left intact, the MVC decoder will have no problems.

For some reason i thought the one eye was in the normal m2ts file (for 2D backwards compat), and the second stream would be in the ssif file, and only be opened when required.
So the ssif contains both eyes again? Interesting..
One example:

00000.m2ts: left eye stream + audio tracks
00001.m2ts: right eye stream, nothing else
00000.ssif, left eye stream, right eye stream + audio tracks

So basically, whenever there's a *.ssif file, you can safely split the ssif file instead of the m2ts file(s). It's 100% the same format, the only difference is that the ssif file contains all tracks.

Edit: The main reason for not having all tracks in the m2ts files is the Blu-Ray bitrate limit. The bitrate for ssif files is often over the max 2D limit. So in order to keep compatability with 2D Blu-Ray players, only ssif files are allowed to surpass the 2D bitrate limit.

[sneaker_ger]

From a decoder point of view you're right. I was talking from a splitter point of view. LAV Splitter can treat the left and right eye stream as totally separate streams.

But it still has to recognize them as a left+right eye belonging together, right?

[madshi]

But it still has to recognize them as a left+right eye belonging together, right?
The splitter doesn't actually have to know that left and right eye streams belong together. It should use an appropriate media type for the right eye stream, though, so that nobody mistakes it as a left eye stream.

[sneaker_ger]

So, how does the decoder know which h.264 stream is the left eye?