xy-VSFilter Project (High Performance VSFilter Compatible Subtitle Filters)

[madshi]

There does not seem to be an elegant solution for the auto-loading problems, and most suggested workarounds involve assistance from the player. So I suggest to keep things as simple as possible: give the player complete control of loading a subtitle provider (when needed) and limit the interface to the communication between providers and consumers.

In other words:
* A consumer does not do anything related to loading. It only communicates with providers.
* A provider should not attempt to auto-load itself unless it is able to load itself for external subs (which implies it connects to the video pin). This means that a filter like XySubFilter should have a merit equal to MERIT_DO_NOT_USE.
* The player is responsible for loading (non-video handling) subtitle providers such as XySubFilter. All players that are capable of using a non-standard video renderer are already customizing the graph building, so it should be rather trivial to add for player developers.
All of that could be achieved by my suggested modification of the new subtitle header. Media players who prefer your suggested behaviour would simply have to call "EnableSubtitleAutoLoading(false)" before building the graph.

The problem with following your suggestion by default is that auto-loading would stop working for every single media player out there, until they manually add support for it. And that is a situation I don't like at all. If a media player is willing to add support for the new subtitle interface, then it should be easy for the media player to call "EnableSubtitleAutoLoading(false)" before building the graph. And that would fulfil most (all?) of your wishes.

Adding a video input pin to XySubFilter is a bad idea, IMHO, because it wouldn't be used for any purpose (other than to get loaded), and it would add complication to XySubFilter development, and maybe harm DXVA decoding compatability.

It is theoretically possible that multiple providers are in the graph. In case of external subs, this means a sub file could be loaded twice. This could be prevented by letting the (first) provider setting a mutex, signaling to other providers that a file is already loaded (and thus should be skipped). The mutex could be for example "MadSubInterface" + HashOf(subtitle_filename). The reason for mutexing individual files, is because providers might only support a subset of all formats, or use different auto-loading rules and source locations.
Yes, that sounds like a useful addition. Instead of explaining the mutex logic in the header, I'd prefer to add a little helper function, though, to make things easier for the media player devs.

I suggest adding a function to XySubFilter (or maybe all non-video handling providers) that allows the player to instruct it to add an external subtitle file. This could be used by for example MPC-HC to load a downloaded subtitle file, or a file that is added through drag&drop. Such a direct function would not require any dynamic DirectShow graph handling (and the complexities that implies). Then XySubFilter could be used as a full ISR replacement in the case where madVR is used as video renderer.
Yes, that would be useful. It would be nice to have this as part of the interface, though, so that it isn't XySubFilter specific but could also be used by/with other providers.

[clsid]

All of that could be achieved by my suggested modification of the new subtitle header. Media players who prefer your suggested behaviour would simply have to call "EnableSubtitleAutoLoading(false)" before building the graph.

The problem with following your suggestion by default is that auto-loading would stop working for every single media player out there, until they manually add support for it. And that is a situation I don't like at all. If a media player is willing to add support for the new subtitle interface, then it should be easy for the media player to call "EnableSubtitleAutoLoading(false)" before building the graph. And that would fulfil most (all?) of your wishes.

The players that are currently capable of using madVR (the only existing consumer atm) will all have to deal with the new situation one way or another. Most have an ISR and/or smart graph building. They all should have situations in which they need to block the interface to prevent handling subs twice.

Opt-out:
[-] Players might show subs twice until player is capable of blocking the use of the interface.

Opt-in:
[+] Simpler interface.
[-] Player can't use the interface until it adds support for loading a provider.

I don't expect much problems getting the players to implement the loading functionality. XySubFilter is still in early beta stage, so there is plenty of time. MPC-HC and MPC-BE will be able to add it very quickly. PotPlayer and KMPlayer will no doubt 'borrow' that code as they usually do. The ZoomPlayer dev (Blight) is Doom9 member, so I expect no problems there either.

Adding a video input pin to XySubFilter is a bad idea, IMHO, because it wouldn't be used for any purpose (other than to get loaded), and it would add complication to XySubFilter development, and maybe harm DXVA decoding compatability.I wasn't suggesting to add a video pin. I think you misunderstood/misread.
The MERIT_DO_NOT_USE idea is for the subtitle pin. To prevent it from loading itself for embedded subs.

[madshi]

Opt-out:
[-] Players might show subs twice until player is capable of blocking the use of the interface.

Opt-in:
[+] Simpler interface.
[-] Player can't use the interface until it adds support for loading a provider.
You didn't list the key advantage of Opt-in, though, which is:

[+] Makes external subtitle tracks work automatically with every media player.

Please don't under-estimate that. Maybe some day madVR will work in media players which don't explicitly support it. Maybe LAV Video Decoder will become a subtitle consumer and thus make the use of XySubFilter possible in all media players. Furthermore I think it's easy enough in most (all?) media players to simply disable the internal subtitle rendering engine, so the disadvantage of "Opt-out" you mentioned is a simple configuration issue the end-user can take care of himself.

I very much understand your point of view, though, and if the majority of developers prefers removing auto-loading in its current form, I will bow to the majority. I just think it really needs to be carefully considered. And my suggestion to support disabling of the auto-loading by the media player still stands. Actually, I've just released madVR v0.86.10, which already supports my suggested method to disable auto-loading...

[xabregas]

SO i use mpc int subtitle renderer and xysub filter doesnt yet support srt subtitles to be rendered outside the movie window. Can i use xy-vsfilter with extend to 16:9 option and still have better performance than with MPC internal subtitle renderer?

TIA

[clsid]

Well, I suppose since either solution requires at least some effort from player developers, your opt-out solution would be fine too. But perhaps a mutex would be much simpler for players to implement?

I think it would be best to change the auto-loading of the providers by the consumer from a requirement to a recommendation. Then at least a consumer has the freedom to decide if auto-loading is desired in any given situation. The function/mutex to disable auto-loading is of course mandatory.

Perhaps consumers should also create a mutex to signal their presence. Suppose that LAV Video becomes a consumer in the future. Then for example (xy-)VSFilter could check for such a mutex and decide to refuse to join the graph.

Speaking of mutexes, they should contain a unique ID of the current graph. Otherwise undesirable behavior might occur with multiple simultaneous graphs.

[clsid]

SO i use mpc int subtitle renderer and xysub filter doesnt yet support srt subtitles to be rendered outside the movie window. Can i use xy-vsfilter with extend to 16:9 option and still have better performance than with MPC internal subtitle renderer?Yes. However, the subtitles will be rendered at the resolution of the video (expanded to 16:9) while the ISR can render at the resolution of your screen. So subs may be a bit less sharp for low resolution videos.

@cyberbeing
Can an "expand to 16:10" option be added to xy-VSFilter? Many people have monitors with such an AR.

[DarkSpace]

Nice work on the Beta so far!
However, I get a crash on certain files. I have not yet been able to create a sample which reproduces the crash, so I hope that uploading the Debug Log and Crash Report from madVR helps solve the issue. If you need more information or a sample file, I'll be happy to provide more information and I can upload a file after the weekend (unless I manage to create a small sample that reproduces the issue in the meantime), but right now, my Internet is slow.
Could the crash maybe related to Issue 153 (http://code.google.com/p/xy-vsfilter/issues/detail?id=153)? I made sure to have all the Debug Symbols in the respective filters' directories when creating the logs (https://mega.co.nz/#!h8hjAbbA).
The crash happens with both madVR 0.86.9 and 0.86.10.

[thewebchat]

Which queue in madVR drops first?

Do you have any Flush & Wait settings enabled? If so, try setting all to "Don't Flush".

What CPU, GPU, OS?


I'm not seeing anything particularly abnormal as far as CPU usage on the second sample on my System.
704x480->853x480 = 1% CPU Usage by XySubFilter
704x480->2560x1440 = 3% CPU Usage by XySubFilter

The first sample likely just caches well by chance.
Just to make sure, I'll have our dev double check that nothing strange is going on with second sample.

Change flush does absolutely nothing. I see the subtitle queue instantly drop to 1 as soon as the first subtitle event activates. I observe average 8% CPU usage windowed and 30% CPU (one core completely saturated) in fullscreen.

CPU: Intel Ivy Bridge M @ 2.3 GHz
GPU: Nvidia GT650M
OS: Windows 8

Regardless of my hardware, the subtitle renderer should never cause the video renderer to drop frames. If the subtitle renderer can't meet its deadlines, it should drop *subtitles* instead of causing unwatchable video.

Edit: I don't observe any performance problems when pre-scaling the subtitles to 2560x1440 using Aegisub. The example file http://www.sendspace.com/file/iybf50 plays without any issues while the original file chokes Xy.

[Moragg]

Moragg, curious, which CPU do you have? That's the same sample my CPU can pass with a queue of 64.

edit: xy-VSFilter vs xySubFilter: 5/64 and 11/64 for subtitle queue respectively, so not that much of a difference.

Also, about multithreading, wouldn't it be possible to say, split the screen in 4 subpictures and let each thread render one of them? I know multithreading isn't that easy from my tries to implement it for a much smaller project, but it would really be a huge benefit to have it. Especially for AMD users, since multithreading is where their CPUs excel at.

My CPU is a PhenomII 1055T - 2.8Ghz 6 core, so not very good at single threaded stuff.

While I would like multithreading "just because" it'll only make any difference on stupidly heavy typesetting, so I'm not sure it's really worth the effort.


That sounds very good, actually I'm surprised that it performs this well.


Appears I was badly mistaken, previous attempts to play this with xy-vsfilter got the result I mentioned, but a retry gets only 22 dropped frames with xy-vsfilter. xy-subfilter is still better at ~10 dropped frames though.

One thing I am confused about: why should the size at which I playback affect the subtitle queue? With no upscaling/downscaling I get perfect playback, but playing back at fullscreen gives me dropped frames.

[sneaker_ger]

One thing I am confused about: why should the size at which I playback affect the subtitle queue? With no upscaling/downscaling I get perfect playback, but playing back at fullscreen gives me dropped frames.

Since XySubFilter renders at the target resolution (by default) it will have more work to do in fullscreen compared to rendering for only a small target window, I guess?

[KoD]

A quick question:

I've noticed that people said xysubfilter can be used with DXVA native. However, I don't seem to be able to use DXVA native, in ZoomPlayer, when using LAV video for decoding H264, xysubfilter, and madVR v0.86.10. As soon as I unregister xysubfilter, DXVA native works (there are no other subtitle filters registered on the system).

I have disabled "Smart play" in ZoomPlayer to be able to see subtitles. xy-vsfilter is unregistered, so only xysubfilter is registered. XYSubFilter is configured with the default settings.

Are there any specific requirements to be able to use DXVA native (in ZoomPlayer) ?

[Pulstar]

Yes. However, the subtitles will be rendered at the resolution of the video (expanded to 16:9) while the ISR can render at the resolution of your screen. So subs may be a bit less sharp for low resolution videos.

@cyberbeing
Can an "expand to 16:10" option be added to xy-VSFilter? Many people have monitors with such an AR.

I'm reverting to xy-VSfilter because of this.

[turbojet]

ASS subs were the reason for style not working, would be nice to force a style on them in the future. Styles are working fine on other text subs.

Thinking way ahead would there be any consideration in resizing image based subs (sup, vobsub, etc.)? Bdsup2sub can do this but it seems like a lot more work than is necessary.

[Moragg]

If possible, a feature I'd like is to turn the horrible yellow vobsub into something better - like a simple white font.

[Soukyuu]

I'm not quite sure, but isn't vobsub basically bitmaps? I don't think you can restyle them.

[Moragg]

Okay, I read up a bit and it looks like each subtitle is rendered as its own bitmap - didn't think anyone would do that as it seems horribly inefficient.

Maybe incorporate some on-the-fly character recognition and replacement? Or - slightly (a lot?) easier to do - force vobsubs to be rendered in black and white. What I hate most of all is the yellow colour, get it white and I'd be much happier.

[turbojet]

They are images so no chance to change font but the colors are controlled by a palette thats configurable in *vsfilter. I'm not sure it works, nothing handy to test atm but vaguely remember mpc isr working fine to change colors.

Scaling options move image subtitles, the only thing that's missing that should be possible is resizing/zooming.

[onomatopellan]

Are there any known plans to make more xySubFilter subtitle consumers? As I can't use madVR because of my intel integrated graphics I'm using latest MPC-HC with LAVFilters and EVR-CP as video renderer and I'd like to use xySubFilter.

[filler56789]

If possible, a feature I'd like is to turn the horrible yellow vobsub into something better - like a simple white font.

Just for the record --- there ARE people who think quite the opposite, i.e., "simple white fonts" do not have sufficient visual contrast, and we prefer yellow subtitles to white subtitles. No, we don't like to manually edit the color palette in the .IDX files, but this is what we have to do now and then.

[cyberbeing]

Change flush does absolutely nothing. I see the subtitle queue instantly drop to 1 as soon as the first subtitle event activates. I observe average 8% CPU usage windowed and 30% CPU (one core completely saturated) in fullscreen.

CPU: Intel Ivy Bridge M @ 2.3 GHz
GPU: Nvidia GT650M
OS: Windows 8

Hopefully we can find a way to optimize the effect in this script a bit more.

Regardless of my hardware, the subtitle renderer should never cause the video renderer to drop frames. If the subtitle renderer can't meet its deadlines, it should drop *subtitles* instead of causing unwatchable video.

Open a bug report on madshi's bugtracker (http://bugs.madshi.net/my_view_page.php). madVR is in charge of requesting and displaying subtitle bitmaps, so you have that reversed. If madVR cannot meet it's deadlines it should either A) not be requesting new subtitle bitmaps or B) dropping existing subtitle requests in favor of not dropping video frames. Currently madVR is likely designed to drop video frames when subtitle request are late, so you'd need to request this be changed.

Edit: I don't observe any performance problems when pre-scaling the subtitles to 2560x1440 using Aegisub. The example file http://www.sendspace.com/file/iybf50 plays without any issues while the original file chokes Xy.
You've not scaled \be in that sample though. If you replace every \be1 with \be3 or \be4 does it still play fine?

I've noticed that people said xysubfilter can be used with DXVA native. However, I don't seem to be able to use DXVA native, in ZoomPlayer, when using LAV video for decoding H264, xysubfilter, and madVR v0.86.10.

This problem cannot be caused by XySubFilter, since we never connect to a video pin or do anything else which would interfere with DXVA. At least that we are aware of. If DXVA is not working, it must be a madVR bug or ZoomPlayer bug.

ASS subs were the reason for style not working, would be nice to force a style on them in the future. Styles are working fine on other text subs.
You were using xy-VSFilter (VSFilter.dll) then? The reason this was disabled in xy-VSFilter is because the existing VSFilter override behavior for how this was triggered and applied to ASS subtitles was rather annoying.

Have you tested our new style override dialog for ASS subtitles in XySubFilter?

Thinking way ahead would there be any consideration in resizing image based subs (sup, vobsub, etc.)? Bdsup2sub can do this but it seems like a lot more work than is necessary.

In xy-VSFilter (VSFilter.dll) no, there are no plans to add resize functionality for image-based subtitles.

With XySubFilter, madVR is in charge of resizing all image based subtitles.


Okay, I read up a bit and it looks like each subtitle is rendered as its own bitmap - didn't think anyone would do that as it seems horribly inefficient.
Exactly the opposite, multiple smaller bitmaps is often more efficient than combining everything into one massive bitmap.

Maybe incorporate some on-the-fly character recognition and replacement?
What do you mean by this?

Or - slightly (a lot?) easier to do - force vobsubs to be rendered in black and white. What I hate most of all is the yellow colour, get it white and I'd be much happier.
If possible, a feature I'd like is to turn the horrible yellow vobsub into something better - like a simple white font.

Possibility for the future.

In the meantime, you could always force a custom style for VOBSUBs by enabling "custom colors" in the IDX file and editing the style to your preferences. This is something VSFilter has always supported, but rarely used for some reason.

Are there any known plans to make more xySubFilter subtitle consumers?

No known plans at the moment, but MPC-HC at least seems to be considering the possibility.

[cyberbeing]

MPC-HC has merged a change into the latest nightly to allow both xy-VSFilter (VSFilter.dll) & XySubFIlter to co-exist a bit better. This change will block VSFilter.dll from being used with madVR when XySubFilter is installed, while still allowing VSFilter to remain installed and be used with other video renderers. MPC-HC nightly builds can be found here (http://nightly.mpc-hc.org/)

[sneaker_ger]

I kinda liked the older revisions better: just block xy-vsfilter under external filters if you want to use XySubFilter, or set to "prefer" if you do in fact want to use it. Now, if you want to quickly switch to xy-vsfilter for a test you have to actually un-register XySubFilter. Well, it's a good enough quick fix for beginners...

Okay, I read up a bit and it looks like each subtitle is rendered as its own bitmap - didn't think anyone would do that as it seems horribly inefficient.
Exactly the opposite, multiple smaller bitmaps is often more efficient than combining everything into one massive bitmap.

A typical bitmap based subtitle for a cinema movie is maybe a few Megabytes in size. This is still an order of magnitude bigger than text based ones (unless you stuff your mkv with fonts!), but compared to the video and audio data it's almost negligible. Or maybe he means "one bitmap for every video picture" instead of "for every line"? The former would indeed be more inefficient, but DVD and Blu-Ray use the latter, i.e. one subtitle line that spans over 10 seconds uses only one bitmap. At least Blu-Rays even support simple effects like fading for PGS subs without creating a new picture for every frame of the effect. Also, some RLE compression is used (and zlib in mkv, though optional).

What do you mean by this?

He means real-time OCR (https://en.wikipedia.org/wiki/Optical_character_recognition), i.e. internally converting to text based subs and then rendering with whatever style (font, color,...) you want. Everyone who has some experience with converting bitmap based subs to text based ones will know that this would be a very ambitious feature - even the programs that are specifically designed for the job and have had years of development still require lots of user-interaction regularly, so it's not really suited for full automation/real-time (at least with existing free libraries).

[cyberbeing]

I kinda liked the older revisions better: just block xy-vsfilter under external filters if you want to use XySubFilter, or set to "prefer" if you do in fact want to use it. Now, if you want to quickly switch to xy-vsfilter for a test you have to actually un-register XySubFilter. Well, it's a good enough quick fix for beginners...
Well this is one of those things where if someone installs XySubFilter, there is an expectation they actually want it to be used with madVR. If you just want to see what something would look like with VSFilter.dll style rendering, you could always enable the "Render to Original Video Size" option in XySubFilter.

Maybe you could request that MPC-HC add a registry key to toggle this behavior, like the hidden key they use to toggle the ISR blocking VSFilter behavior.


one subtitle line that spans over 10 seconds still uses only one picture.

The subtitle interface actually works in this way already. Each bitmap is given a unique id as well as a start/stop time.

We also have an advanced multi-level caching mechanism in xy-VSFilter/XySubFilter which makes rendering identical portions of text subtitle lines use nearly zero CPU time. This caching is what makes Libass so much slower than xy-VSFilter at times. Libass is forced to re-render identical computational heavy operations every frame, while we render once and then pull it from our cache for future frames.


He means real-time OCR (https://en.wikipedia.org/wiki/Optical_character_recognition), i.e. internally converting to text based subs and then rendering with whatever style (font, color,...) you want. Everyone who has some experience with converting bitmap based subs to text based ones will know that this would be a very ambitious feature - even the programs that are specifically designed for the job and have had years of development still require lots of user-interaction regularly, so it's not really suited for full automation/real-time (at least with existing free libraries).

...yeah, probably not going to happen. VSFilter traditionally has always converted VOBSUB bitmaps to vector though.

From what I can tell, all image based subtitles formats (VOBSUB/PGS/DVB) have a color palette, but the hard part would likely be knowing which color of the palette to modify. Overall since our focus has mainly been on text subtitles, any features such as this would best be requested of MPC-HC or MPC-BE who actually work on improving the code for image based formats.

[sneaker_ger]

Well this is one of those things where if someone installs XySubFilter, there is an expectation they actually want it to be used with madVR. If you just want to see what something would look like with VSFilter.dll style rendering, you could always enable the "Render to Original Video Size" option in XySubFilter.

Maybe you could request that MPC-HC add a registry key to toggle this behavior, like the hidden key they use to toggle the ISR blocking VSFilter behavior.

Yes, that would be nice maybe. But I know this is still being discussed and I'm currently sticking to an older MPC-HC revision anyways because the current ones override LAV Filter's stream selection logic.

[cyberbeing]

How to take madVR screenshots with MPC-HC which include XySubFilter subtitles (http://forum.doom9.org/showpost.php?p=1637684&postcount=19673)

[cyberbeing]

Yeah vivan, as you discovered yourself (https://code.google.com/p/xy-vsfilter/issues/detail?id=105) in your edit, both .\sub unable to be created and the default path entries unable to be deleted are behaviors inherited from VSFilter which have never been fixed or modified.

Also I would like to remove "c:\subtitles" path, but Remove button is disabled.

http://imageshack.us/a/img594/6916/bq21.png

No idea why this is, but this was what Gabest set as the default behavior when he originally created VSFilter and nobody ever thought to change it. Eventually, this legacy search path behavior will likely be scraped and replaced with something more feature rich. Until then, it is how it is.

[Moragg]

Just for the record --- there ARE people who think quite the opposite, i.e., "simple white fonts" do not have sufficient visual contrast, and we prefer yellow subtitles to white subtitles. No, we don't like to manually edit the color palette in the .IDX files, but this is what we have to do now and then.

I normally watch anime, and yellow just doesn't work - since the backgrounds are often bright and have a similar texture it clashes badly (in my eyes) and makes it harder to read.

The argument is a lot stronger for live-action movies, but even there I find them too intrusive, and white wouldn't upset my viewing as much. But here I would also say that yellow could be a better option in many circumstances.


Exactly the opposite, multiple smaller bitmaps is often more efficient than combining everything into one massive bitmap.


But much less efficient than a font and text and video.




In the meantime, you could always force a custom style for VOBSUBs by enabling "custom colors" in the IDX file and editing the style to your preferences. This is something VSFilter has always supported, but rarely used for some reason.

Didn't know about this, will definitely check it out, thanks.




...yeah, probably not going to happen.


Shame, didn't realise it was that difficult to do.

[cyberbeing]

But much less efficient than a font and text and video.
Depends on the font, some complex font outlines are extremely computationally heavy to render. ;)

[Telion]

Can you please implement a feature to save & reopen last selected properties tab? It will be especially handy considering the current workaround for taking screenshots with subtitles. Timing corrections will also benefit from this.

[cyberbeing]

We'll look into it, I created Issue #158 (http://code.google.com/p/xy-vsfilter/issues/detail?id=158). In the future, feel free to make such feature requests directly on our bug tracker.

[Telion]

OK, thank you.

[TheSkiller]

At least Blu-Rays even support simple effects like fading for PGS subs without creating a new picture for every frame of the effect. Also, some RLE compression is used (and zlib in mkv, though optional).Just for the record, DVD has a fade feature as well (which uses just one subpic for the period of a subtitle) and the bitmaps are RLE compressed as well. :)
There are also wipe effects (used for karaoke).


One thing to consider when choosing to make subtitles "white" is if you intend to display them on a TV you should not make them 100% opaque and 100% white because it will look blindingly bright if the movie background is dark and you're watching in a dark room.
Better use a light grey color and make the subtitles slightly translucent so that they can adapt to the brightness of the movie background to some degree. My recommendation based on my exhaustive subtitle creations is a color of around 230,230,230 and an opacitiy of 66-75%.

When it comes to visibility I'm 50/50 about white vs yellow. Both can be read equally well from all kinds of backgrounds if only the (black!) outline is thick enough which from what I've seen is rarely the case unfortunately. My recommendation for outline width: about as thick as a stroke of the characters used.

[filler56789]

I normally watch anime, and yellow just doesn't work - since the backgrounds are often bright and have a similar texture it clashes badly (in my eyes) and makes it harder to read.

The argument is a lot stronger for live-action movies, but even there I find them too intrusive, and white wouldn't upset my viewing as much. But here I would also say that yellow could be a better option in many circumstances.

Ideally, the "releasers" :rolleyes: should offer BOTH white and yellow(/green/whatever) subtitles. And just for the notes again, it's not a matter of "anime vs. live action", it's just the fact that different eyeballs and/or different visual cortices will perceive brightness, contrast and color saturation differently ;)

But I agree, it would be very-nice if (xy)VSFilter had an option for easily overriding the color palette from the IDX files. While this doesn't happen, you could give a try to Mplayer --- because Mplayer completely ignores the color palette from the bitmap subs and gives you "simple white fonts" all the time :(

[michkrol]

Thanks for your effort to bring us this software, works great.

While testing various files, I've encountered a rather curious behavior. I have a file that has crazy subs effects (around 10k lines in ~7 seconds), after adjusting madVR CPU queue to 24 this effects plays fine on my PC, with no dropped frames, but ~25 seconds later (with only simple text on screen) the playback freezes completely for a few seconds, also freezing the player controls, the sound plays fine. The playback then recovers with ~150 dropped frames.

What's curious is it only freezes if you play the file without pausing and seeking. If you seek somewhere between the heavy effects and the moment it freezes the playback goes over this part flawlessly as expected. It might be caused by the caches being emptied all at once.

I'm not opening a bug report, since I would like to confirm first, whether it's XySubFilter or madVR causing this.

Below is my system config:
Intel Core i5-3570K
Intel HD4000
16GB RAM
Windows 7 x64
MPC-HC 1.7.0.7646 (f6f1b82) (official nightly build)
LAVFilters 0.58.1 (official release)
XySubFilter 3.0.546 (official beta release)
madVR v0.86.10 (official release)


How to reproduce the problem:

Set madVR CPU queue to 24, to get smooth subtitles effects playback. (Of course it will differ between PCs, just don't set it to max.)
Open the file in windowed MPC-HC at movie resolution (1280x720).
Open the file.
The title with effects plays smoothly.
Continue playback without pausing and seeking.
Around 0:33 playback freezes, recovers for a split second, and freezes again. This takes around 7 seconds in total for me.
Playback recovers and the rest plays smoothly.


Link to the file in question:
http://www.mediafire.com/download/cl9plddqpfa0c80/Shingeki_subs_test.zip
I've replaced the video with something pointless (with same parameters as original) and file is zipped to save bandwidth (the subs are over 35MB and compress quite well).
I know the subs in this file are crazy, but inspecting this might help improve performance in other extreme cases.

Thanks for your time and please tell me, if I've overlooked some important info.

[sneaker_ger]

I can reproduce the performance drop at 0:33 - really strange.

[Soukyuu]

I must admit I snickered when I saw the filename. So I guess Commie's typesetting make intel CPUs stutter as well :P
Before that, Koharubi's bakemonogatari ep02 @1080p was my benchmark for subtitle renderer performance, but xySubFilter masters that one thanks to the cache (it had a half second alternation of two typesets covering the whole screen)

[turbojet]

You were using xy-VSFilter (VSFilter.dll) then? The reason this was disabled in xy-VSFilter is because the existing VSFilter override behavior for how this was triggered and applied to ASS subtitles was rather annoying.

Have you tested our new style override dialog for ASS subtitles in XySubFilter?

It was subfilter, forcing default style had no effect, is there something I'm missing?

In xy-VSFilter (VSFilter.dll) no, there are no plans to add resize functionality for image-based subtitles.

With XySubFilter, madVR is in charge of resizing all image based subtitles.

Right but madvr needs to be instructed to zoom in/out and subfilter needs an option for zoom level, for example: Double the size of image subs zoom 2.0x, reduce them by half, zoom 0.5x

Wouldn't it make more sense to keep all subtitle options out of madvr to reduce user confusion? There currently is one but wouldn't it be better for subfilter to instruct madvr how to handle that code instead?

[ikarad]

What are the differences between xy-vsfilter and Xysubfilter?

[Moragg]

Link to the file in question:
http://www.mediafire.com/download/cl9plddqpfa0c80/Shingeki_subs_test.zip
I've replaced the video with something pointless (with same parameters as original) and file is zipped to save bandwidth (the subs are over 35MB and compress quite well).
I know the subs in this file are crazy, but inspecting this might help improve performance in other extreme cases.
.

I have the original version of that - 90K lines in 7 secs :devil:

But if you want help, press ctrl+j to bring up madvr info, and look down to the queue information. During playback watch the queues, problems usually occur when they fall to very low values (0) - you'll likely find frames dropped/delayed when that happens. If so, try increasing those queue sizes.

If not it may be a memory flushing thing which I have no clue whatsoever about.

Edit: I can reproduce the problem, both in the file you gave me and in Commie's Shingeki ep1v2 - the one with 90k lines - a completely different file. Same thing happens, video freezes and subtitle queue just empties. Other queues are fine.

What are the differences between xy-vsfilter and Xysubfilter?

Xy-vsfilter is the old one and has a stable release, xy-subfilter is the new, improved version but is currently in beta testing, and can only be used with madvr.

[ikarad]

Xy-vsfilter is the old one and has a stable release, xy-subfilter is the new, improved version but is currently in beta testing, and can only be used with madvr.
1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?

[Blight]

This is just a heads-up to Zoom Player users,
I plan to add support for this interface in the next release, you will probably see a beta within the next month as I'm currently focusing most of my time on Zoom Commander and other non-coding chores.

[Moragg]

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?

1)not as far as I know. Hasn't for nearly a year, with xysubfilter out I doubt anyone will bother.
2)No.
3)Not entirely sure, but perhaps: better colour accuracy, faster. I know it renders subs at a different res. And in madvr you can make it optimize performance if you have a slow CPU.

[Asteague]

Any reason why xySubFilter would work flawlessly for files with subtitles in the container, but refuses to show up for files with only external subs? In some rare cases it does show up, but I have no idea why it won't always show up.
Any ideas?

My setup:
Win 8 x64
MPC-HC 1.7.0.7646
madVR 0.86.10
LavFilters 0.58.1
xy-vsFilter not installed (didn't work even if installed)

[cyberbeing]

Thanks for your effort to bring us this software, works great.

While testing various files, I've encountered a rather curious behavior. I have a file that has crazy subs effects (around 10k lines in ~7 seconds), after adjusting madVR CPU queue to 24 this effects plays fine on my PC, with no dropped frames, but ~25 seconds later (with only simple text on screen) the playback freezes completely for a few seconds, also freezing the player controls, the sound plays fine. The playback then recovers with ~150 dropped frames.

Issue #159 (http://code.google.com/p/xy-vsfilter/issues/detail?id=159)

I noticed this exact same issue with the original script in your sample the other day. Not sure what causes it at the moment, we are looking into it.

I'll attach the sample you provided to the issue, but at this point I'm more interested in obtaining any other scripts which are able to trigger this bug.

[sneaker_ger]

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?
1)not as far as I know. Hasn't for nearly a year, with xysubfilter out I doubt anyone will bother.
2)No.
3)Not entirely sure, but perhaps: better colour accuracy, faster. I know it renders subs at a different res. And in madvr you can make it optimize performance if you have a slow CPU.

1.) Development of xy-vsfilter has been continuing in the background, even though the last official version is 8 months old. It is not dead and new official versions will come.
2.) Correct. (no)
3.)
The main difference between the two is that xy-vsfilter sits in the DirectShow chain as a link between video decoder and the video renderer. It overlays subtitles onto the video frames it receives from the video decoder and then passes the result on to the video renderer.
This is has following disadvantages for the typical end-user:
- the subtitles will be rendered at the video resolution (and with chroma sub-sampling if applicable), so for videos whose resolutions are much lower than the display resolution they may look blurry or aliased.
- DXVA2 (native) does not work because it requires a direct connection between the video decoder and the video renderer.
- rendering in the black bars can be problematic, because xy-vsfilter does not really know the target rectangle and changing resolutions during runtime may introduce some problems.

XySubFilter on the other hand renders at the display resolution (without chroma sub-sampling) by default, which makes the subtitles always look sharp. It is not connected to the video decoder at all, but only sits between the splitter and the renderer. Since the video decoder can now directly connect to the video renderer again DXVA2 (native) will work fine.

Concerning colors: xy-vsfilter will also have "correct" color handling (similar to XySubFilter) - or rather has the most important stuff already. This is more of an advantage of XySubFilter over MPC-HC's internal renderer (this is related to historic limitations of the old vsfilter, which didn't know anything but limited range BT.601 colors, resulting in obscure workarounds which break colors with MPC-HC's internal renderer/VLC and a bunch of others. A new extension of the ASS "spec" is supposed to harmonize the different renderers again to some extent.)

Any performance differences should ideally only be related to the different resolutions/color spaces at which they render by default - so new optimizations will likely be used in both versions. The madVR performance trade off option should not make XySubFilter faster than xy-vsfilter - it just relegates some color conversion stuff from madVR to XySubFilter, which uses some simpler conversion on the CPU instead of the GPU like madVR (nothing to worry about - that's why it's on by default. cyberbeing's more technical explanation (http://forum.doom9.org/showpost.php?p=1636975&postcount=19564)). xy-vsfilter always has to do these things itself since it does not pass on the subtitles through a separate pin to the renderer but overlays them onto the video itself.

About rendering in the black bars:
While the new subtitle interface that connects XySubFilter and madVR has been designed with enabling subtitle rendering in the black bars in mind (if the user wishes so), this has not yet been implemented.

[cyberbeing]

It was subfilter, forcing default style had no effect, is there something I'm missing?
XySubFilter only overrides the styles defined in the header of a script, it does not strip style override tags. Is that it?

If not, please provide a sample which easily reproduces the issue.


In xy-VSFilter (VSFilter.dll) no, there are no plans to add resize functionality for image-based subtitles.

With XySubFilter, madVR is in charge of resizing all image based subtitles. Right but madvr needs to be instructed to zoom in/out and subfilter needs an option for zoom level, for example: Double the size of image subs zoom 2.0x, reduce them by half, zoom 0.5x

madVR doesn't need to be instructed. XySubFilter always passes bitmap subtitles at original resolution and madVR scales them as needed automatically.

Wouldn't it make more sense to keep all subtitle options out of madvr to reduce user confusion? There currently is one but wouldn't it be better for subfilter to instruct madvr how to handle that code instead?

I'm a bit confused. If you are requesting that madVR be instructed to apply arbitrary non-default scaling to bitmaps, our subtitle interface does not support that currently. I'd suggest explaining your feature request to madshi in the madVR thread.


What are the differences between xy-vsfilter and Xysubfilter?

XySubFilter
Requires madVR (or other Subtitle Consumer supporting SubRenderIntf.h (http://madshi.net/SubRenderIntf.h))
Higher Quality
RGB Subtitles
Renders subtitles after scaling is performed by the video render (allows high resolution subtitles)
Subtitles alpha-blended on the GPU
Image based subtitles are scaled to any video resolution
Works with DXVA Decoding
Allows Deinterlacing via bitstream flags
Compatibility option to support anamorphic video
Allows all colorspaces supported by the video renderer to be used

xy-VSFilter (VSFilter.dll)
Works with any video renderer.
Lower Quality
Usually YCbCr Subtitles
Renders subtitles prior to scaling at original video resolution
Subtitles alpha-blended on the CPU
Image based subtitles are not scaled and must match video resolution
Breaks DXVA Decoding
Blocks Deinterlacing bitstream flags
Requires all subtitles scripts be compensated for VSFilter behavior with anamorphic video
Only a more restrictive set of colorspaces supported by xy-VSFilter can be used by the video renderer



1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?

1) An official update to xy-VSFilter (VSFilter.dll) is planned after we release a stable build of XySubFilter. Though after that point, the future of legacy VSFilter.dll releases from us is unknown.

2) Update to the latest MPC-HC x86 nightly build (http://nightly.mpc-hc.org/) and both filters can co-exist, but it is not required to have both installed. XySubFilter will be used with madVR, while xy-VSFilter will be used by all other renderers.

Any reason why xySubFilter would work flawlessly for files with subtitles in the container, but refuses to show up for files with only external subs? In some rare cases it does show up, but I have no idea why it won't always show up.
Any ideas?
...
xy-vsFilter not installed (didn't work even if installed)

Did you remember to disable the MPC-HC ISR by unchecking "Auto-load subtitles" under "Playback" in MPC-HC settings?

Are you positive the external subtitle files have an identical file name to your video? Please double-check by renaming one to the other with copy/paste (not including file extension).

[turbojet]

XySubFilter only overrides the styles defined in the header of a script, it does not strip style override tags. Is that it?

If not, please provide a sample which easily reproduces the issue.
http://www.sendspace.com/file/bt4zd8 I'm not 100% sure what format these subs are in, it's been awhile. They have color but extract fine as srt, could be colorized srt. Known srt subs tested style fine in subfilter.

madVR doesn't need to be instructed. XySubFilter always passes bitmap subtitles at original resolution and madVR scales them as needed automatically.

This is concerning the question about considering zooming bitmap subs in the future, many are very small. No subtitle renderer does it yet but I'm pretty sure it's possible to do it like bdsup2sub does. Madvr might be able to help out but the options wouldn't belong in madvr.

I'm a bit confused. If you are requesting that madVR be instructed to apply arbitrary non-default scaling to bitmaps, our subtitle interface does not support that currently. I'd suggest explaining your feature request to madshi in the madVR thread.

As vanilla madvr has nothing to do with subtitles doesn't subtitle options such as the current optimize seem out of place? I'm suggesting have the option in xysubfilter that triggers code in madvr.

[Aleksoid1978]

Question to developers:
1 - can you merge this 2 menu items section :
http://i.imgur.com/1CVrmda.png
Then it would be convenient to switch the built-in and external subtitles in the menu
2 - Where can be find source(full or just for interfaces)? It's need for better integration with MPC-BE.

[Toku]

I have a file that causes both MPC and madVR to crash when using XySubFilter but works fine when using xy-VSFilter, no crash occurs when subtitles are disabled
https://imageshack.us/a/img854/7515/7chq.png
10 Second sample- here (http://darkwalker31.poseidon.feralhosting.com/%5bKaylith-Music%5d%20supercell%20-%20Hakushukassai%20Utaawase%20%5b1080p%5d%5b4E39737B%5d%20cut%20(1)-001.mkv)

[cyberbeing]

http://www.sendspace.com/file/bt4zd8 I'm not 100% sure what format these subs are in, it's been awhile. They have color but extract fine as srt, could be colorized srt. Known srt subs tested style fine in subfilter.
Yes, those are SRT subtitles. I don't see the problem though, our style override dialog functions just fine on that sample.

Override Style
1) Open your sample (http://www.sendspace.com/file/bt4zd8) in MPC-HC

2) Pause video

3) Right-click MPC-HC window -> Filters -> XySubFilter -> Properties

4) Click "Styles"

5) Click the "Default" tab

6) Edit to bogus value like "Border Width = 10"

7) Click "Ok" to style Dialog

8) Click "Apply" to main setting dialog [note: the need to press Apply here first seems to be a bug]

9) Click "Okay to main setting dialog

10) Style with "Border Width = 10" is applied to subtitles.

OR

Force Default Style
1) Open your sample (http://www.sendspace.com/file/bt4zd8) in MPC-HC

2) Pause video

3) Right-click MPC-HC window -> Filters -> XySubFilter -> Properties

4) Click "Styles"

5) Click the "Global Default" tab

6) Edit to bogus value like "Border Width = 10"

7) Click "Ok" to style Dialog

8) Check "Force Default" in main setting dialog

9) Click "Okay to main setting dialog

10) Style with "Border Width = 10" is applied to subtitles.

11) Exit MPC-HC

12) Re-open your video

13) "Border Width = 10" is still applied to subtitles


This is concerning the question about considering zooming bitmap subs in the future, many are very small. No subtitle renderer does it yet but I'm pretty sure it's possible to do it like bdsup2sub does. Madvr might be able to help out but the options wouldn't belong in madvr.
We currently have no intentions of adding a bitmap resizing function within XySubFilter. The actual resizing code should exist in madVR, even if we ourselves trigger use of such code.

As vanilla madvr has nothing to do with subtitles doesn't subtitle options such as the current optimize seem out of place? I'm suggesting have the option in xysubfilter that triggers code in madvr.

madVR has much to do with the subtitle interface used by XySubFilter. Unless madVR adds such code to madVR, and offers a way for us to trigger such functionality, there is nothing we can do. You need to make a case to madshi for why he should support such a feature in madVR and the subtitle interface.