xy-VSFilter Project (High Performance VSFilter Compatible Subtitle Filters)

[cyberbeing]

Some time ago, I complained about performance problems with xySubFilter in full resolution mode. I'm still seeing occasional heavy frame drops with xySubFilter in the beta 2 release.

Beta 2 doesn't contain any new performance optimizations, so I guess that's to be expected. I looked back in the thread only see a single post from you though, and it was a response to someone else. If you find scripts which are causing performance issues, please upload samples so we know what still needs to be optimized. Increasing madVR's CPU queue can help in that regard to a certain extent, but certain computationally heavy operations will always be slow. For Beta 3, I'm hoping to put more focus on performance improvements.

Looking at the execution of mpc-hc.exe under Process Explorer, I notice the GPU memory usage fluctuating heavily as though a lot of dynamic allocation was being performed. Is this expected?
That's probably a better question for the madVR thread, or MPC-HC thread (if using EVR-CP's consumer). All XySubFilter does is send bitmaps when requested, beyond that GPU memory allocations are in the hands of the subtitle consumer. You could try adding a DWORD value "MAX_BITMAP_COUNT" under HKEY_CURRENT_USER\Software\Gabest\XySubFilter\Performance\ and play with the value to see if it changes this behavior at all. The default value is 8 (decimal), but not a lot of testing has been done to determine if this is the optimal value in terms of CPU & GPU load.

[Kado]

@cyberbeing

Regarding cache those values were me messing around as they were at auto(511) before, also even without ffdshow it crashes.

Did some tests and the file in folder "[2]/InsaneBorder" does not crash MPC-HC.

Cheers!

Kado

[cyberbeing]

Thanks, I'll pass it along to our developer. That build reverted the two insane border support commits [1] (https://github.com/Cyberbeing/xy-VSFilter/commit/fc3b925f403f1b8d854a6067d20e4c4ed7c5034e) [2] (https://github.com/Cyberbeing/xy-VSFilter/commit/0d1b62494d61b53869fbc55aa5ee32beee38b60e), which was a separate enhancement from the bugfix for Issue #166 (http://code.google.com/p/xy-vsfilter/issues/detail?id=166) which should still be fixed.

I've created an entry on our bug tracker as Issue #181 (http://code.google.com/p/xy-vsfilter/issues/detail?id=181), and attached the XySubFilter Test Build you've stated resolved your crash.

[Kado]

Very well. Thanks for your support!

Cheers!

Kado

[wOxxOm]

This is expected behavior. Only subtitles found within the search paths, with identical file names will be loaded.

How do I make it work with DVD/bluray playback in MPCHC?

[cyberbeing]

For standard DVD disc playback, I'm unsure it's possible with VSFilter-based filters in general unless you find a splitter which exposes a subtitle pin. DVD disc subtitles are normally rendered by the Video Decoder. The VSFilter VOBSUB format requires ripping subtitles from DVDs, at least as far as I'm aware. DVD playback is not my area of expertise. You may get a better answer in the MPC-HC thread, as the same would apply to their version of VSFilter.dll.

For Blu-ray playback, XySubFilter should be used normally to render the PGS subtitles directly embedded in the TS files on the disc, assuming LAV Splitter is used and no other subtitle renderer is enabled.

[wOxxOm]

For standard DVD disc playback, I'm unsure it's possible with VSFilter-based filters in general
Well, I meant it was previously possible (with xySubFilter) to just drag'n'drop any srt/ass subtitle in the MPCHC window while watching a DVD. Now it's no longer possible with xySubFilter, I assume?

Also, as of now it's not possible to start watching a video without any subtitles (either internal or external) and later drag'n'drop/load arbitrary subtitles with xySubFilter, right?

[cyberbeing]

Well, I meant it was previously possible (with xySubFilter) to just drag'n'drop any srt/ass subtitle in the MPCHC window while watching a DVD. Now it's no longer possible, I assume?

Nothing has changed as far as subtitle loading is concerned in BETA2. As far as drag'n'drop support for XySubFilter, this is only functional in MPC-BE I believe. If you drag a subtitle into MPC-HC it enables the ISR on-the-fly and XySubFilter will not be used, even in previous versions.

Also, as of now it's not possible to start watching a video without any subtitles (either internal or external) and later drag'n'drop/load arbitrary subtitles with xySubFilter, right?

Same as above, you need to use MPC-BE. XySubFilter should always be present in the graph, assuming the video has an audio track and subtitles exist. Otherwise, you can set it to "Always Load".

With MPC-HC, you'd actually need to open up XySubFilter settings during playback and use Main -> Open... to select your subtitle.

[wOxxOm]

I think MPC-BE now behaves the same as MPC-HC and just uses ISR for drag'n'dropped subtitles. It's rather disappointing not having a way to force xySubFilter to load always...

[cyberbeing]

I think MPC-BE now behaves the same as MPC-HC and just uses ISR for drag'n'dropped subtitles. It's rather disappointing not having a way to force xySubFilter to load always...

No, I'm 100% positive drag'n'drop support is functional with XySubFilter in MPC-BE. He specially designed it to always use VSFilter/xy-VSFilter/XySubFilter if they are active in the graph, instead of the ISR. You can confirm this by opening up VSFilter/xy-VSFilter/XySubFilter settings after dragging a subtitle file into MPC-BE, as it will show the subtitle file you just dragged as being open.

Only MPC-HC loads the ISR on-the-fly and doesn't use XySubFilter with drag'n'drop.

[wOxxOm]

Huh, just tried MPCBE 4580 on several subtitle-less files, DVD as well, and no, xySubFilter doesn't get loaded in filter graph after drag'n'drop, thus I guess ISR is used.

[cyberbeing]

Huh, just tried MPCBE 4580 on several subtitle-less files, DVD as well, and no, xySubFilter doesn't get loaded in filter graph after drag'n'drop, thus I guess ISR is used.

XySubFilter needs to already be present in the graph (idle) for that to function. There is no way for it to be added to the graph on-the-fly. You may need to set XySubFilter to "Always Load" in settings to ensure it is present in the graph.

[wOxxOm]

You may need to set XySubFilter to "Always Load" in settings to ensure it is present in the graph.
Of course it's set to 'always', I'm sorry for not mentioning it right away. And dx merit is 'preferred+2'. And it's not blocked in MPCBE's external filters page.
As far as I know MPCBE guys reused the new MPCHC ISR code (or parts of it) and probably decided to mimick its behaviour regarding autoloading of xySubFilter (or lack thereof) as well.

[cyberbeing]

Not sure why this isn't working for you. The MPC-BE guys explicitly added support for XySubFilter/xy-VSFilter a long time ago, and change the ISR behavior so it wouldn't conflict. I've tested it in the latest stable releases and nightly of MPC-BE and it continues to function as expected. XySubFilter has never been loaded by the media player. With XySubFilter Beta1, it was loaded by madVR. With XySubFilter Beta2, DirectShow loads XySubFilterAutoLoader which then loads XySubFilter if needed based on settings.

Do you have the ISR disabled in MPC-BE? Auto-load subtitles unchecked? (http://i1.someimage.com/QvsKanJ.png)

XySubFilter set to Always Load (http://i1.someimage.com/1yPcOvW.png)

This shouldn't be needed, but have you tried setting XySubFilterAutoLoader to Prefer in MPC-BE External Filters? (http://i1.someimage.com/NT8MNei.png)

Remember that XySubFilter can only be loaded into the graph when the video is initially opened. If you are saying it's not entering the graph at all, even when doing all of the above, you may need to provide a sample or more details about the directshow filters you have installed. If you download that logging build (https://www.mediafire.com/?bxixb7x9gbo7v4n) I posted eariler and upload a log, it may give a hint of why it's failing to add XySubFilter to the graph.

[wOxxOm]

cyberbeing, thanks, running install.bat helped, I didn't expect it was needed after updating the dll, so in case I'm not the only dumb one: how about adding a notice in bold red to remind that beta2 *must* be re-registered by running the install bat file?

[cyberbeing]

I did have a note in my Beta2 release post about it, but I've taken your suggestion and made it bold red (http://forum.doom9.org/showpost.php?p=1672047&postcount=324). I'll also add a notice to the first post of the thread.

[Soukyuu]

I have sent you a PM cyberbeing, with a sample file that stutters badly when attempting to display the first subtitle line, which only happens with xy-subfilter beta2. I did not test it with beta1, but I did test it with mpv and it doesn't happen there, so it shouldn't be a limitation of my AMD CPU (single-threaded bottleneck).

I re-registered the beta2 dll just to be sure but that didn't change anything.

[cyberbeing]

Cannot seem to reproduce this. Does it occur with xy-VSFilter 3.0.0.211 or the MPC-HC ISR? Does it occur when using XySubFilter with EVR-CP? What is your CPU usage like on each thread without any subtitle renderer active/installed? If you lower madVR's CPU queue to something like 24, does anything improve? I'll try to debug this with you via PM.

[Soukyuu]

I think it might be a VRAM issue (the one you warned me about previously), setting queue from 32 down to 24 didn't change anything, but setting it to 14 solves it. I am not sure why it doesn't trigger it with xy-VSFilter or ISR. Basically, a combination of madvr debanding triggering + high queues + xy-subfilter is what kills playback for me on that scene.

[cyberbeing]

You mentioned via PM, that you thought this issue was being triggered by madVR's debanding fade detection. Does using constant debanding, with fade detection disabled in madVR 'trade quality for performance' settings actually resolve the issue? From what you've stated so far, this doesn't sound like a XySubFilter bug especially if you say you're running out of GPU VRAM? I'll pass on your madVR debug log to madshi and see what he thinks.

[Soukyuu]

I actually have "do not re-render frames when fadeout detected" enabled, so I guess it's not even the fade detection, but debanding itself.
Disabling it completely solves the issue (as does lowering the GPU queue), it is using the "low" preset otherwise.

What I see on the Ctrl+J OSD is that it freezes completely for a few seconds, then the next moment it updates and I think I see the render queue filling up from 5 to 24, while the backbuffer queue is 0-0/8. I also see that with GPU queue set to 24, the rendering time is twice the time (~40ms) compared to when I lower it to around 10 (~20ms). I am using windowed mode without overlay, but it happens in both the overlay and the fullscreen exclusive mode. The freeze is less severe in exclusive mode, more of a stutter, but is the same in overlay mode.

I do think it's merely a problem of my GPU running out of VRAM, I only have 896MB of it and win8.1 already eats about 150MB of it for it's crippled desktop composition. That would explain why exclusive mode is not as bad, I guess. I don't know why xy-VSFilter doesn't have the problem, maybe because it's not passing the subtitles directly through madVR. Lowering the GPU queue but keeping CPU queue high solves the problem for me, I have both low-ms rendering times and some insurance in case subtitles become heavy enough for my crappy CPU to bottleneck me.

Anyway, here is the log xy-SubFilter created: click (https://dl.dropboxusercontent.com/u/19330332/XySubFilter_knk.log).

Edit: and here (https://dl.dropboxusercontent.com/u/19330332/madVR%20-%20log_knk.7z) are some madVR logs. I tried my best to get as close as possible to the point it happens, but it looks like it only stutters if I jump about 2-3 seconds before the event, otherwise it plays it smoothly. I have made 3 attempts, I guess you could start looking at the small one then work up if you don't see anything.

[cyberbeing]

With XySubFilter, madVR does need to upload subtitle bitmaps to the GPU VRAM which has an impact on GPU performance. There is also an additional GPU overhead in regards to shader passes required + subtitle alphablending. With xy-VSFilter, there is no additional GPU overhead since subtitles are blended into video frames before being passed to madVR.

There shouldn't be any problem with having a lower GPU queue set in madVR, as long as you aren't overburdening your GPU core. For both XySubFilter & xy-VSFilter, CPU queue is more important.

[madshi]

Which kind of subtitles are we talking about here? SRT? ASS? PGS? The GPU RAM consumption of SRT subtitles should be extremely small. Animated ASS subtitles require more GPU RAM because the subtitle change all the time. However, XySubFilter is supposed to only send madVR the subtitle rectangles which actually have content in them, which should save a lot of RAM. PGS is weird in that it always sends the full screen size, but it shouldn't change all the time, so PGS probably shouldn't be a RAM size problem, either.

In any case, there's not much I can do about this. GPU RAM consumption is what it is, and I don't think there's anything I can optimize there. If you're running out of GPU RAM, the easiest workaround is to reduce the GPU queue size. Alternatively you could buy a GPU with more RAM. I don't know if XySubFilter can be optimized to send the subtitles bitmaps with less RAM consumption. Maybe it is already as optimized as possible, I don't know...

[nevcairiel]

PGS is weird in that it always sends the full screen size, but it shouldn't change all the time, so PGS probably shouldn't be a RAM size problem, either.

While some discs do that, not all do, and its not an inherent property of the PGS format (unless its a thing the PGS decoder in xy-vsfilter does?)

[madshi]

I'm not sure. I just noticed that once, it could have been the disc.

[cyberbeing]

Which kind of subtitles are we talking about here? SRT? ASS? PGS?
ASS subtitles. Though from his XySubFilter log, they are just simple unanimated dialog lines being displayed at the beginning. Nothing special.

While some discs do that, not all do, and its not an inherent property of the PGS format (unless its a thing the PGS decoder in xy-vsfilter does?)
xy-VSFilter uses the MPC-HC PGS decoder, so the output limitations are the same. At least in my experience, the MPC-HC PGS decoder always outputs 1920x1080 subtitle bitmaps for PGS from Blu-rays.

[madshi]

ASS subtitles. Though from his XySubFilter log, they are just simple unanimated dialog lines being displayed at the beginning. Nothing special.
But if they're not animated, then even a GPU queue of a hundred frames should just end up uploading 1-3 different bitmaps to the GPU!? How can that cause problems with the GPU RAM? Is it possible that XySubFilter does not reuse the same bitmap ids if the subtitle images don't change? Of course if XySubFilter sends a new image for every video frame, although the subtitles don't change, that would increase GPU RAM consumption quite a lot. Do you have logs from XySubFilter which tell us whether it reuses the bitmaps ids, as long as the subtitles don't change?

[Soukyuu]

It just seems to be an unlucky coincidence of an older GPU being run with settings that nearly max it out, the debanding triggering using some more ram and subtitles pushing it over the edge of max VRAM usage. At least from my uneducated point of view. It does not happen with such simple subtitles usually. And I'm fine with keeping the GPU queue low, I don't know why I increased it in the first place.

If you need more logs though, I can provide them.

[cyberbeing]

Do you have logs from XySubFilter which tell us whether it reuses the bitmaps ids, as long as the subtitles don't change?
I'm positive that XySubFilter re-uses Bitmap IDs for identical segments. In the XySubFilter log Soukyuu posted, there is only a single Bitmap ID for each of those lines, and only 2 Bitmap IDs total. I've double checked with my own logs and XySubFilter is definitely behaving as expected. I played back the first 374 unique lines in that script, and XySubFilter generated 374 Bitmap IDs, with each Bitmap ID provided to madVR for multiple frames as needed for display.

XySubRenderFrame::GetBitmap :0 id: 1 dirty_rect: x 322, y 962, w 1276, h 56 pitch:5120 pixels:0DDF36B8

XySubRenderFrame::GetBitmap :0 id: 2 dirty_rect: x 492, y 962, w 936, h 68 pitch:3776 pixels:25A58B50


@Soukyuu

Are you positive you didn't cut off the XySubFilter log before the issue occurred? I do see 364 lines of motion track typesetting, but those would start around 30 seconds after your log ended. Though if your issue occurs before XySubFilter even receives those lines from the splitter, I can't see how they could be to blame.

[Soukyuu]

I am sure I did not cut the log, the line the issue happens at is the
00:00:00,002 00:00:00,002 There was a time when I bore witness to two worlds.
line (line 1059 in my log). The image gets stuck before the line is displayed, then unstucks before the line disappears from the screen, then the next line (1495) is displayed, and I even waited until the third line was displayed (1815) before cutting it off.

[madshi]

I'm positive that XySubFilter re-uses Bitmap IDs for identical segments. In the XySubFilter log Soukyuu posted, there is only a single Bitmap ID for each of those lines, and only 2 Bitmap IDs total. I've double checked with my own logs and XySubFilter is definitely behaving as expected. I played back the first 374 unique lines in that script, and XySubFilter generated 374 Bitmap IDs, with each Bitmap ID provided to madVR for multiple frames as needed for display.
That's good to hear, thanks.

[cyberbeing]

I am sure I did not cut the log, the line the issue happens at is the
00:00:00,002 00:00:00,002 There was a time when I bore witness to two worlds.
line (line 1059 in my log). The image gets stuck before the line is displayed, then unstucks before the line disappears from the screen, then the next line (1495) is displayed, and I even waited until the third line was displayed (1815) before cutting it off.

Ah, so you really were serious when you said it occurred on the very first line of the script. Did you already test the MPC-HC EVR-CP consumer for XySubFilter, to confirm this problem only occurs with madVR? If you monitor with GPU-Z, do you actually see that first subtitle line push you over your GPU VRAM limit with madVR + debanding? How much does VRAM increase on that first line with madVR vs EVR-CP right as your issue occurs? Are there any notable spikes in GPU load or Memory Controller load at the same time?

@madshi
Does some kind of initialization occur when the first subtitle bitmap are sent which has some kind of one-time-only GPU overhead? This sound similar to the issue mark007 experienced on initial load of PGS subtitles (http://bugs.madshi.net/view.php?id=153). Yet this time, we just have simple text dialog lines with tiny bitmaps. Did something regarding subtitle handling change after you added Debanding, DirectCompute dither, or OpenCL NNEDI3? Otherwise, what are your thoughts at this point? Is there anything to suggest XySubFilter specially is at fault for this, or is this just some kind of coincidence that Soukyuu happened to be riding madVR too close to his GPU VRAM limit for awhile now, but somehow only triggered a problem now?

[Soukyuu]

Yes, I did test it with evr-cp and it doesn't hang there. Looking at the VRAM usage, I get the following:
- idle: 214/896MB used
- mpc-hc w evr-cp: 294/896MB
- mpc-hc w madvr, q10: 714/896MB, with a peak of 761MB
- mpc-hc w madvr, q24: 846/896MB, with a peak of 874MB
- mpc-hc w madvr, q24, no debanding: 873MB peak
- mpc-hc w madvr, q24, no subs: 870MB peak

The interesting thing for me is that madvr's peak VRAM usage is at launch, not at the position where debanding/subtitles kick in, as I'd expect.

Looking at memory controller, there are no spikes, the load is around 14-21%. GPU has a "spike" from 86% to 91%, and memory load seems to drop at that point. Maybe it tries to free some VRAM but doesn't manage to? I don't know if there is "swapping" going on, but judging by the memory controller load, it doesn't seem what slows me down.

Considering that I'm at around 20MB under max memory limit, it could be madVR requests more memory than is free and doesn't get it from the OS, which results in stutter/hang.

Also, I'm pretty much sure it's just a coincidence and not really a bug. I think I remember having stutters on some complex typesets, but that was on 720p content and I attributed those to my AMD cpu limiting me. But seeing it happening with a simple line made me wonder.

[madshi]

No subtitle related changes related to debanding, DirectCompute or OpenCL. I did do some subtitle bugfixes recently which involved some small changes, but I doubt they would cause such problems. Since the problem goes away for Soukyuu when using a smaller GPU queue, and since he's really near to using all of the GPU RAM, that sounds like a good enough explanation/solution for me.

[agustin9]

It's possible to put the subtitles outside the video area with madvr yet?

[cyberbeing]

No, not yet. madshi will need to do a major refactor before madVR will be capable of supporting it. You my have better luck putting in a feature request to MPC-HC, asking them to support it in their EVR-CP consumer. Since MPC-HC already supports this with the ISR, they may be able to add such XySubFilter 'outside the video frame' support more easily in their current framework.

[madshi]

FWIW, this is very high on my to do list, together with some other related changes/improvements.

[agustin9]

Great madshi, i will wait for madvr then, thanks

[fagoatse]

Hey there,
just wanted to report 2 issues:

1) the included batch script fails to register the filter in 64 bit windows 8.1 update 1.
A simple "regsvr32 xysubfilter.dll" in elevated cmd did the trick though.

2) for some reason the color space isn't shown during playback in mpc-hc's filters menu. XySubFilter is present there and connected to evr-cp just fine though.

[cyberbeing]

1) the included batch script fails to register the filter in 64 bit windows 8.1 update 1.
A simple "regsvr32 xysubfilter.dll" in elevated cmd did the trick though.
No idea, it must be a security measure of some kind. I'll get around to releasing an innosetup installer for Beta2 eventually, which should resolve such issues.

Does the install.bat from LAV Filters zip releases (http://files.1f0.de/lavf/LAVFilters-0.61.1.zip) also fail? We use the same script. If it does fail, bring it up to Nev in his thread (http://forum.doom9.org/showthread.php?t=156191) and see if he has any ideas.

2) for some reason the color space isn't shown during playback in mpc-hc's filters menu. XySubFilter is present there and connected to evr-cp just fine though.
This is something we changed in Beta2:

Removed connected YCbCr Matrix from filter name

[michkrol]

No idea, it must be a security measure of some kind. I'll get around to releasing an innosetup installer for Beta2 eventually, which should resolve such issues.


The brackets in the filename are the problem. Windows treats them as special characters or something and looks for file named "Install_XySubFilter_", at least for me on Win 8.1 x64. It does work if you either change filename or call the *.bat directly from an elevated command line. Pretty strange, if you ask me, but at least easy to fix ;)

[Soukyuu]

I'm on 8.1 x64 and it did register the .dll without problems? At least it said "registration succeeded".

[cyberbeing]

Hmm... I can reproduce this in folders like Desktop, Downloads, Documents, but not in our our standard C:\Program Files (x86)\xy-VSFilter install directory or some other folders I create. This has to be a strange Windows 8.1 bug regarding running bat files elevated, since I've never heard of parentheses being treated as special characters in regards to filenames.

I'll rename the bat files and replace the archives on GitHub shortly.

Edit: Done. I replaced the 7z archives with zip archives containing the renamed bat files. No other changes.

[Conquerist]

Are there plans for kerning support in xy-vsfilter?

[cyberbeing]

Which subtitle format in particular did you desire kerning support for? Aside from SSF, none actually specify whether kerning should be used or not at the script level.

Edit: Someone pointed out to me that libass already added a script header for toggling kerning in ASS (https://github.com/libass/libass/commit/0d3ddc12295e38cdcb11c79f8dcb5c8d9f1a8b69) many years ago, so I guess we could look into supporting it in that way.

[Conquerist]

I had ASS in mind. I was comparing the rendering output of libass in VLC and xy-vsfilter in MPC-HC, and noticed that libass was applying kerning, but xy-vsfilter was not. As you're saying, it doesn't look like "Kerning: yes" is documented in the SSA specs, but IMHO a consistent output of libass and xy-vsfilter is a good idea.

[cyberbeing]

Libass has kerning disabled by default though, which matches VSFilter. If we decide to add support for kerning for ASS scripts, it would remain disabled by default unless that "Kerning: yes" script header was present.

[Conquerist]

That sounds good to me.

Maybe you can also add an option to the styles editor to force kerning on or off. I think that would be especially useful in combination with the style override feature that was added in XySubFilter.

[kirakami]

can i have latest xy-vsfilter.dll?
edit: got it from cccp :)

[Plorkyeran]

I'd be okay with exposing the Kerning header in Aegisub's UI and eventually enabling it by default in new scripts if xy-VSFilter implements it in a way compatible with libass.