LAV Filters - DirectShow Media Splitter and Decoders

[Andy o]

Any other movie that you're having trouble with? I already saw D9 so I wanna rent something I haven't watched... 2 birds one stone you know.

[SamuriHL]

I'm having all kinds of problems technically. But I think a lot of it is my own issue to deal with. For MC16 I have to use ffdshow for subtitle rendering. And I think that's got some issues with the new forced sub option. I mean, with the chain I've got going on, it's not surprising that it's having problems. However, even with MPC-HC's subtitle renderer, forced subs aren't working flawlessly. Salt didn't seem to work right for me, either. So I don't know if it's a bug or if it's something in my config.

[Andy o]

According to that spreadsheet, Avatar has flagged forced subs too, just like D9, Salt and Paprika, so these should be the same. I'm gonna choose one of the movies of that sheet.

[SamuriHL]

Ok, forget everything I've said. I have no idea what the HELL happened. It *DOES* work with MPC-HC's subtitle renderer. It does NOT work with ffdshow subtitle renderer. This situation sucks, but, it is what it is. It means I can't use this in MC16. But District 9 works with the MPC-HC sub renderer. Now if only I could use that in MC16. :( I so wish Nev would write us a proper subtitle renderer. :D :p

[SamuriHL]

According to that spreadsheet, Avatar has flagged forced subs too, just like D9, Salt and Paprika, so these should be the same. I'm gonna choose one of the movies of that sheet.

Yea, see my latest post. It works fine with MPC-HC's sub renderer. So, blank my life, but, that's just how it goes.

[Andy o]

The JRiver folks are also very responsive to customer requests, aren't they? I tried their player a long time ago and liked it, but I really didn't need it.

[SamuriHL]

Yea they are. ffdshow just ain't working. No matter what I try. Not for this. We so need a better sub renderer.

[robpdotcom]

Correct, MakeMKV does NOT set flags for forced subs - you must load the file into mmg's header editor to set the forced flag (which only takes a few seconds).
Posted before I saw your post. It does seem to be preserving the flags. I tried with 1.6.8, but I haven't seen this specific issue on the release log.

I think we're talking about 2 different things. It does preserve the flags within the pgs stream, but does not set the forced flag for the track inside the mkv. LAVSplitter needs the latter. I suspect that if LAVSplitter is automatically loading the track, and you have not set the flag with mmg's header editor, then it is probably just loading the first track.

BTW: Another, more recent movie with forced subs: The Tourist.

OT: In MakeMKV, you can always select the option to extract forced subs - if there are none, it will not create the extra track.

[Andy o]

You still have to turn on the subs even if you wanna see only the forced captions within the normal sub track. I don't see much value over having an actual forced full track, but you can set the flag quickly with mkvmerge, without remuxing. I don't know what LAVF Splitter does with such a track, or MPC-HC's sub renderer, for that matter.

[robpdotcom]

I'm confused.

Are you saying that you have a pgs stream, containing all normal subs (plus individual forced subs), and somehow you are able to display only the forced subs?

[SamuriHL]

That's the point of what Nev just added. :)

[robpdotcom]

Ooooh. Looks like I missed a few pages here. :rolleyes:

I hope this clears up my confusion a bit:

Will LAVSplitter eventually be able to display the forced subs only from PGS subtitle streams? Is that even something a splitter could do, or would that be something for the subtitle renderer?Thats something a sub renderer should be doing. I don't interpret any streams, i just send them to the decoders to do their magic with.

Personally, i have only seen BluRays where the forced subs are in their separate track, though. (Sadly, you cannot detect if they are forced subs just from the track. <.<)

Very cool stuff.

[SamuriHL]

Yea, um, Nev's had a....change of heart. :D

[robpdotcom]

This, combined with madVR being able to use MPC-HC's internal sub renderer makes things soooo much sweeter.

BIG, BIG Thanks Nev!

[SamuriHL]

Amen to that. I just wish it would work with ffdshow's sub renderer so I can continue to use MC16. Case in point I'm watching The Crow (japanese import) in MC16 with LAVF Splitter/ffdshow audio/cyberlink video decoder/madVR as we speak. Right off the disc. Fortunately no subs for this one, but, I'd LOVE to get the forced sub thing to work.

[robpdotcom]

Just tested this with two mkv's: The Tourist, with which I did not extract a separate, forced sub stream (so, the forced subs are embedded in the normal subs stream); and Avatar which I had previously ripped by extracting the extra forced stream (flagged as forced by mmg).....

Both work perfectly by setting Subtitle Selection Mode to "Only Forced Subs" and Blu-ray Subtitles to "Deliver only Forced Subtitles".

I can see though, how his will be tricky to select full subtitles in a pgs stream....

Would it be possible for LAVSplitter to create a virtual "forced" track, which will load automatically, then you can then select the full subs, for example, via Right Click > Navigate, or even by the player's "next subtitle" keyboard shortcut ("S" in MPC-HC)?

[SamuriHL]

Nev is planning on some enhancements to make it easier to use.

[jmone]

SamuriHL - I DO GET Subs working fine in MC16 to start with then for some reason they stop displaying after about 30 sec with my latest filter config. It has to be either FFDSHOW or the way LAVSplitter works with FFDSHOW for these. I posted the filter info a few post earlier at http://forum.doom9.org/showpost.php?p=1496491&postcount=1978 + I've tried all sorts of various options in ffdshow, etc but I've no idea why it would work for a bit then simply stop displaying them. I'm testing with D9 and Avatar both off the disc and muxed to various containers. I'm also sure I had an earlier config working fine....but I can not seem to work out what it was now (so may bits have changed).

[Andy o]

If you're using mkvs though, you always have the option to choose which forced subs you wanna rip to a separate track, like I did with Paprika.

[SamuriHL]

Sigh. This isn't a good situation for us. I wish we could have the sub rendering code from mpc in mc16.

[SamuriHL]

Yes, Andy, but on rented discs that's not a viable solution.

[Andy o]

Oh sorry I didn't mean you guys, I was answering to robpdotcom who was talking about mkvs:
Both work perfectly by setting Subtitle Selection Mode to "Only Forced Subs" and Blu-ray Subtitles to "Deliver only Forced Subtitles".

I can see though, how his will be tricky to select full subtitles in a pgs stream....
Should have used the quote button.

[jmone]

Subtitle Dropping Issues: Using D9 as the test disc. Start playback at Chapter 4 (time code 20:33) and the Subs work fine from here till 21:15, they should start again at 21.25 but don't appear. It always happens exactly at that point. I've tried multiple FFDSHOW builds (clid's Generic, clid's ICL10, XhmikosR's icl 12). Issue happens using both madVR and EVR so it is not the Video Renderer.

EDIT: Don't know why I did not test this early! If I leave LAVSplitter with "Full Subtitles", and "Only with matching language" / "Deliver only Forced Subtitles" unchecked then there is no issue with the subtitles dropping (of course I now get all subs not just forced subs). It looks like a bug in LAV reliably delivering just forced subs with FFDSHOW.

[jmone]

Thats nothing a decoder would do. Thats all the renderers doing, and ReClock is the only reliable WASAPI (=exclusive) renderer i know.
But even when using WASAPI, it doesn't magically switch output configurations - ReClock has an option to do this for you ("Set matching speaker configuration")

FYI - MC16 support WASAPI and WASAPI Event Style with options for:
- Exclusive Mode
- Flush devise buffers (start up and on pause)
- Present 24-Bit data in 32 Bit package

Also enables the use of their DSP including stuff like:
- Output Format (resampling/bitdepth, channel mixing etc)
- EQ
- Video Sync (keep audio in sync with the video by resampling) - thanks to madshi for helping out on this one!
- Room Correction
- other stuff!

The only downside is it only works in MC16 (it is not a seperate DirectShow filter).

[nevcairiel]

Sigh. This isn't a good situation for us. I wish we could have the sub rendering code from mpc in mc16.

How about DirectVobSub? It should be the same renderer as MPC-HCs, except that it applys it onto the decoded image, not the rendered image. At least with Blu-rays, that are 1080p from the start, you don't get scaling issues - just chroma upsampling issues. :p Nothing that ffdshow wouldn't also suffer from.

And what the hell, while i'm sleeping you guys produce 2+ pages of comments.

Anyhow, i have alot of discs myself, including Salt, D9 and several others.
From the list in that AVS thread, forced subs inside the normal track are far more common, it seems. I do however have some discs that also have separate tracks for the subs (I don't remember exactly, but i watched one just 2 weeks ago or so - The Expendables it might have been)

The filtering code is pretty dumb right now, i only tested on that one disc - which worked fine. But i can imagine situations where it would cause some issues - not sure if those actually exist in real-life though.

What the filtering does is produce "holes" in the sub stream, usually they seem to be numbered sequentally, now some are missing - maybe ffdshow can't deal.

@MakeMKV:
Flagging the tracks as forced is not important in this scenario. I would however be surprised if it modifys the flags in the PGS frames, just because simply copying is so much easier then rewriting them - so it should preserve those flags.

[jmone]

How about DirectVobSub? Unless I've missed something, it does not have PGS supports which makes it kinda pointless these days. The benefit of FFDSHOW is they added PGS support at the end of last year.

And what the hell, while i'm sleeping you guys produce 2+ pages of comments.

Imagine what would happen if you slept in!

What the filtering does is produce "holes" in the sub stream, usually they seem to be numbered sequentally, now some are missing - maybe ffdshow can't deal.

EDIT: That is exactly what it feels like from my end. The FORCED subs stop displaying once a "normal" sub has been supressed.

[jmone]

As an idea - instead of producing holes, could you present blanks instead?

[nevcairiel]

Unless I've missed something, it does not have PGS supports which makes it kinda pointless these days.

It uses the same logic to render subs as the MPC-HC internal renderer - it just looks like there is a missing media type for the DirectVobSub subtitle pin - i'll test later and possibly add it.
The only "problem" is that it doesn't support NV12, only YV12 - so you wouldn't be able to use it with LAV CUVID, unless i manage to add support for that too.

(Edit: Adding NV12 support to DirectVobSub seems hard, would probably be easier to add YV12 output to LAV CUVID.....)

I'm also pondering if it would be possible to write a sub renderer using the MPC-HC rendering code which would hook into the sub rendering interface of madVR, basically how MPC-HC works, just as a standalone filter.
It would however be limited to madVR (and the Haali renderer).

As an idea - instead of producing holes, could you present blanks instead?

Maybe, maybe not. I don't know that much about the PGS format itself..
I'll test with ffdshow when i get home from work.
Maybe its an easy fix in ffdshow sub rendering code...

[Andy o]

Sigh, good news and bad news. Good news, the Tourist works fine, just like Paprika did. Bad news, I spent my free rental on The Tourist.

[nevcairiel]

From what i gathered from the last two pages - the subtitle filtering works fine in principal, as long as you use the MPC-HC renderer, right? Any discs (or mkvs) it failed on? (ignoring ffdshow issues for now)

[Andy o]

I'm using the MPC renderer, yes. The only available titles I have right now with forced subs are those two, Paprika and The Tourist. I think we can add Avatar and District 9 to the list of working titles too.

[Nexin]

@nevcairiel Some Praise and idea for inclusion into ffdshow asap and keep in sync with versions <> (http://forum.doom9.org/showthread.php?p=1496618#post1496618)

Thanks to you both for the development and updating of LAVSplitter. Moreover making LAVSplitter available for us all to use :)

[nevcairiel]

I don't think thats a good idea. ffdshow is a decoder, adding a splitter in there is weird .. (also, i have my own Audio Decoder, which would conflict with ffdshow - and i rather distribute both)
Additionally, i do not care for the extra work keeping it in sync.

Then there is the whole ffmpeg version issue...

Just install both packages, i don't think thats too much to ask.

[jmone]

It uses the same logic to render subs as the MPC-HC internal renderer - it just looks like there is a missing media type for the DirectVobSub subtitle pin - i'll test later and possibly add it.
The only "problem" is that it doesn't support NV12, only YV12 - so you wouldn't be able to use it with LAV CUVID, unless i manage to add support for that too.

(Edit: Adding NV12 support to DirectVobSub seems hard, would probably be easier to add YV12 output to LAV CUVID.....)

I'm also pondering if it would be possible to write a sub renderer using the MPC-HC rendering code which would hook into the sub rendering interface of madVR, basically how MPC-HC works, just as a standalone filter.
It would however be limited to madVR (and the Haali renderer).



Maybe, maybe not. I don't know that much about the PGS format itself..
I'll test with ffdshow when i get home from work.
Maybe its an easy fix in ffdshow sub rendering code...

All good ideas, and I would plug for the following, but given your time is precious, I'd also put them in this order:
1) Mod LAVSplitter to output a (forced) SUB stream that is compatible with a wide range of decoders as possible (incl FFDSHOW) OR if it already does then it is a bug in FFDSHOW that needs to be fixed
2) Mod DirectVobSub to support pgs
3) Write a new Sub Renderer (happy that it would only support madVR but that means you could not use it with DVD's at present).

OR - since it is a rendering role, and Madshi has been kicking back recently get him to build it into madVR :p

[nevcairiel]

1) Mod LAVSplitter to output a (forced) SUB stream that is compatible with a wide range of decoders as possible (incl FFDSHOW) OR if it already does then it is a bug in FFDSHOW that needs to be fixed

I'll try to fix it in LAV Splitter, maybe there is something simple to be done.
Maybe i can output dummy, blank packages .. or adjust the id in the packages .. i'll look into it.

2) Mod DirectVobSub to support pgs

That'll be easy. In addition i'll add YV12 output to LAV CUVID so those two can be used together.

3) Write a new Sub Renderer (happy that it would only support madVR but that means you could not use it with DVD's at present).
I don't think i'll be doing this, its a whole new world of pain.. and, no practical use for me, i use MPC-HC. :p

[jmone]

Sounds good to me for 1 and if that fails 2 + I agree on your comments for 3 (unless an earlier hint of LAVVideo was true in which case.....) Anyway, Filters are like bad habits, I'm always trying to cut down the number I have!

[nevcairiel]

There will be a LAV Video, but it won't do subtitles.

[madshi]

OR - since it is a rendering role, and Madshi has been kicking back recently get him to build it into madVR :p
Who's Madshi? What's madVR? Huh? What are you talking about? :confused: :p

[fastplayer]

There will be a LAV Video, but it won't do subtitles.
...and so the circle is complete. :D

[joeydrunk]

So mochc dies.allow.the flags through than. What exactly are you saying about this particular situation? When using makemkv we just have to make sure and always check the forced sub box if we want to be able to watch forced only and regular subs separately? Just tested this with two mkv's: The Tourist, with which I did not extract a separate, forced sub stream (so, the forced subs are embedded in the normal subs stream); and Avatar which I had previously ripped by extracting the extra forced stream (flagged as forced by mmg).....

Both work perfectly by setting Subtitle Selection Mode to "Only Forced Subs" and Blu-ray Subtitles to "Deliver only Forced Subtitles".

I can see though, how his will be tricky to select full subtitles in a pgs stream....

Would it be possible for LAVSplitter to create a virtual "forced" track, which will load automatically, then you can then select the full subs, for example, via Right Click > Navigate, or even by the player's "next subtitle" keyboard shortcut ("S" in MPC-HC)?

[jmone]

Who's Madshi? What's madVR? Huh? What are you talking about? :confused: :p

Some slacker!

Seriously, would not sub rendering be a logical inclusion for your renderer?

[tetsuo55]

Although i agree that the video-renderer should blend-in the rendered subtitles I don't think it should also do the subtitle-rendering itself.

We still need a standalone subtitle renderer that writes to a buffer of ready-to-use subtitles. And renderers that understand how to use the images in the buffers.

For it to work like the internal mpc-hc subtitle renderer the video-render also has to fill a few buffers with data, so the subtitle-renderer can use the right timing.

[SamuriHL]

A stand alone filter is badly needed. However, for now i'd just like to get forced subs working in mc16. So whatever is the easiest path for that right now makes the most sense. Sub rendering seems like the biggest pita for devs to work on. But the effort is appreciated.

[madshi]

Seriously, would not sub rendering be a logical inclusion for your renderer?
I generally don't comment on future versions/plans... :)

We still need a standalone subtitle renderer that writes to a buffer of ready-to-use subtitles. And renderers that understand how to use the images in the buffers.

For it to work like the internal mpc-hc subtitle renderer the video-render also has to fill a few buffers with data, so the subtitle-renderer can use the right timing.
There's the ISubRenderCallback logic which the MPC-HC internal subtitle renderer uses to communicate with the Haali Video Renderer and with madVR. This ISubRenderCallback logic could also be used by a standalone subtitle renderer. Of course support for this would then have to be added to the MPC-HC EVR/VMR renderers.

[clsid]

@nevcairiel
It would be cool if you could have a quick look at the ffdshow subtitle code. It is probably some kind of stupid timing or buffering issue.

[Nexin]

I don't think thats a good idea. ffdshow is a decoder, adding a splitter in there is weird .. (also, i have my own Audio Decoder, which would conflict with ffdshow - and i rather distribute both)
Additionally, i do not care for the extra work keeping it in sync.

Then there is the whole ffmpeg version issue...

Just install both packages, i don't think thats too much to ask.

Ok didn't know would be a problem if they were to be combined.

[sneaker_ger]

I haven't read up on all the subtitle stuff, but if you plan to include ASS support please use libass (http://code.google.com/p/libass/) instead of vsfilter/MPC-HC, it's like 10 times faster and is actively developed and no one has build a DS filter with it yet.

[nevcairiel]

I noticed that the previous subtitle filtering was not properly implemented, when looking at the stream of data packages going out. I think i have corrected this, it does seem to work fine in ffdshow now, but i again only tested on one disc.

http://files.1f0.de/lavf/LAVFilters-0.24-11-g746ae26.zip

PS: DirectVobSub worked just fine for PGS for me.
PPS: That build leaks some memory when watching PGS subs .. i already corrected it, but i cba to upload a new build right now. Its nothing extreme, over the whole movie maybe a few megs..

[SamuriHL]

Awesome! I'll test this out sometime today. I have a bunch of work meetings right now so I can't test it at the moment but will definitely get to it. Thanks so much!

P.S. What version of DirectVobSub are you using? Do you have a link?

[nevcairiel]

I compiled my own from the latest MPC-HC source tree. Its included in the latest MPC-HC standalone filters, found here: http://xhmikosr.1f0.de/index.php?folder=bXBjLWhj - its VSFilter.dll

For compiling it, just open filters_2010.sln in the mpc-hc source tree.