AssRender: inappropriately named libass-based subtitles renderer

[TheFluff]

One fine day a friend of mine wanted to encode something with Avisynth under Wine, on Linux. It went fairly well until he wanted to render subtitles, because Ye Olde VSFilter doesn't work under Wine (too GDI-ish or something, idk).

This gave me an idea and after some false starts I succeeded in cross-compiling libass and its dependencies, and the result is this filter. It renders .asses, the end.

EDIT: I do not maintain this plugin anymore; lachs0r has taken over development and convered it to a C plugin. See his posts at the bottom of this page and the new homepage at http://srsfckn.biz/assrender.

The old OP follows below.





Download
AssRender 0.11
NOTE: the included fontconfig directory MUST be in the same folder as the .dll, or it'll crash!
Note 2: loading something the first time can be pretty slow, especially if you have a lot of fonts installed, because fontconfig needs to cache all your system fonts.

Source code (under MIT license, binaries are under GPL for obvious reasons): assrender_0.11-src.7z


Syntax:

assrender(clip c, string file, int "hinting"=2, float "scale"=1.0, string "charset"="UTF-8", int "loglevel"=-1, string "logfile"="")

Parameter explanations (all parameters except the clip and the input file are optional):

• file: The .ass file to render; no other subtitle format is currently supported.
• hinting: What kind of font hinting to use. Valid values are 0-3; see below for further explanations of this.
• scale: How much to scale the rendered text. Default is 1.0, i.e. 100% scaling (no change).
• charset: The character set of the .ass file, in the standard GNU iconv format (i.e. ISO-8859-1, UTF-16, CP1252, etc are all recognized).
• loglevel: How much diagnostics libass should output to the logfile. Valid values are from -1 to 7, where -1 means nothing, 0 means fatal errors only and 7 means output several hundred kB of internal data structures. 5 is probably sane if you want warnings about things that have gone wrong.
• logfile: Where to write the log. If you set loglevel >= 0, you must specify this too.

Regarding hinting:

• 0 means disable hinting completely. Might be useful with problematic fonts.
• 1 means light autohinting. This is what libass recommends for compatibility.
• 2 means normal hinting.
• 3 means use FreeType's native hinting, which may or may not be buggy.

Normally you'd use 2 or 1, unless things look odd.


Advantages and disadvantages
This filter should mainly have two advantages over VSFilter, namely:
a) it works under wine, and
b) it's probably a lot faster.

On the other hand it has a few limitations, most notably it only supports RGB32 input so far. If someone is sitting on a fast (preferably assembly optimized) routine that can overlay RGBA on YV12/YUY2, feel free to speak up.

Furthermore, libass isn't really bug-for-bug compatible with VSFilter, so it might render stuff a bit differently compared to what you're used to.


Todo

• Expose more libass parameters (aspect ratio compensation and default font comes to mind)
• Make overlaying faster (assembly optimize it?)
• Use Haali's matroska parser to support things like assrender("file.mks", track=2)
• Get rid of the fontconfig configuration directory
• Implement VFR compensation
• Implement an equivalent of VSFilter's MaskSub()
• Steal some subtitles parser code for various formats from Aegisub, convert things to ASS internally and support more subtitle formats than just .ass

Other stuff
It's compiled against a fairly recent libass, this one unless I misremember. I know that technically I'm probably not GPL-compliant because I'm not distributing the source code of the exact versions of freetype/fontconfig/expat/zlib it's linked against (Debian Squeeze's versions including their patches as of three days ago, apt-get source if you want them), but you know, I really don't give a darn.

If you want to compile the source code you need a working mingw32 environment. If you don't want to compile libass and its deps yourself (afaik it's incredibly hard to get libass to build on msys-mingw32 because of the permanent state of autotools hell it is in; personally I didn't bother and just crosscompiled on linux), you can get my compiles. Oh, and you need stdint.h for Visual Studio too just to make it even more annoying to compile.

[Gavino]

Originally Posted by TheFluff

In the meantime if someone knows how to easily convert a PVideoFrame to RGB32 without implementing the conversion myself, tell me and I'll support all input colorspaces and just convert to rgb32 and back again internally.

In your constructor, just env->Invoke("ConvertToRGB32") on the input clip, and similarly to convert back.

[TheFluff]

Nice try (I already attempted that) but I don't have a clip to convert back, I have a PVideoFrame, and I'm not going to change colorspace randomly without telling the user.

[Myrsloik]

Originally Posted by Gavino

In your constructor, just env->Invoke("ConvertToRGB32") on the input clip, and similarly to convert back.

No, noticed how no other filter does implicit colorspace conversions? Same thing applies here. Also converting back implicitly is even worse because then you just lose even more precision instead of letting the filter after implicitly convert to its own preferred format. Just say NO.

If you for some reason want to implement it anyway do something like this in AvisynthPluginInit2:

return new MyFilter(Env->Invoke("ConvertToRGB32", Args[0]), all other filter args go here));

I meant the create function, do never trust coding advice on internet forums

[Gavino]

Instead of constructor, I should have said the 'create' function that calls the constructor. In outline,

static AVSValue Create(AVSValue args, ...) {
  PClip child = args[0];
  child = env->Invoke("ConvertToRGB32", child);
  PClip result = new MyFilter(child, ...);
  return env->Invoke("ConvertToXXX", result);
}

Of course, you need to add error handling and check the original colorspace to know what to convert back to, but this shows the basic approach.

Whether you should do it or not is a separate issue.

[TheFluff]

Posted 0.11; forgot that I had the charset hardcoded to UTF-8, now it has a charset parameter instead. Also flushes the logfile after each write.

[sneaker_ger]

How's the progress going? I'm especially interested in the mkv parsing. (Or someone willing to add such a feature to vsfilter...)

[TheFluff]

There wasn't much interest in this filter so I was like :effort: and never did anything with it. The MKV parsing should be fairly easy to implement actually, but it's of limited use as long as the filter can only be used on RGB32.

[sneaker_ger]

I guess there wasn't much interest without your to do list implemented. Vicious Cirlce ...

[TheFluff]

Fix the RGB32 limitation and I'll get right on that todo list.

[Keiyakusha]

Is it possible to make libass-based directshow filter? I'm surprised that libass wasn't ported to windows yet (in some more or less usable way), especially if it claims to be 50% faster than vsfilter.

[TheFluff]

Certainly, if you assume it's possible to write a VSFilter-like directshow filter. (For most people, including me, it isn't.)

[SledgeHammer_999]

@Keiyakusha why don't you enable sub rendering in ffdshow-tryouts? (or does it contain the same code as vsfilter?) Is it slower?

[Keiyakusha]

Originally Posted by TheFluff

Certainly, if you assume it's possible to write a VSFilter-like directshow filter. (For most people, including me, it isn't.)

I see, thanks. It wasn't actually request for you so no problems. I just wondering why there is no attempts to port it.

Originally Posted by SledgeHammer_999

@Keiyakusha why don't you enable sub rendering in ffdshow-tryouts? (or does it contain the same code as vsfilter?) Is it slower?

Well, I don't know where ffdshow's code came from and actually ffdshow seems to be faster. I do use it sometimes but unfortunately it still gives me some errors like text shown in wrong direction (already reported/confirmed), total corruption in some cases (reported/confirmed) or some characters misplaced a bit so they overlapping with other ones (not reported by me yet, but probably known)

[Mug Funky]

heya, i'm not getting any subs rendered at all?

i'm using an ssa file.

[TheFluff]

Originally Posted by Keiyakusha

I see, thanks. It wasn't actually request for you so no problems. I just wondering why there is no attempts to port it.

I think the two main reasons are that VSFilter is "good enough", and that writing directshow filters in general and intermediate video filters that connect to anything in particular is fucking rocket science that requires a directshow guru.

Originally Posted by Keiyakusha

Well, I don't know where ffdshow's code came from and actually ffdshow seems to be faster. I do use it sometimes but unfortunately it still gives me some errors like text shown in wrong direction (already reported/confirmed), total corruption in some cases (reported/confirmed) or some characters misplaced a bit so they overlapping with other ones (not reported by me yet, but probably known)

ffdshow has its own render implementation. Last time I looked at it, it only supported a rather small subset of the features available in .ass and was very buggy. That was a long while ago though.

Originally Posted by Mug Funky

heya, i'm not getting any subs rendered at all?

i'm using an ssa file.

If you by SSA mean old SSA v4 (not v4+) I'm not sure if libass supports that. Try converting it to ASS, and if that doesn't work either post the script and I'll take a look. I'm not going to put a lot of effort into debugging it though since this filter is in limbo until I get around to it again and fix the rgb32 limitation.

[lachs0r]

So, a lot of things happened in the past few days.

Within two days, I had learned a bit of C and reimplemented AssRender as an Avisynth C Plugin, so it no longer required building with MSVC.
Then, with my limited knowledge, I started working on Fluff’s TODO-list:

• Support more than RGB32 - Almost done. Subsampling still needs work but should be somewhat usable.
• Expose more libass parameters (aspect ratio compensation and default font comes to mind) - Done. Default font doesn’t seem to work because of Fontconfig’s retardedness.
• Make overlaying faster (assembly optimize it?) - How much faster are we going to make it? It already tends to beat VSFilter tenfold…
• Use Haali's matroska parser to support things like assrender("file.mks", track=2) - Maybe sometime in the future.
• Get rid of the fontconfig configuration directory - Done (patched Fontconfig). Also added a parameter for specifying an additional font directory.
• Implement VFR compensation - Done, timecodes v1 and v2 supported.
• Implement an equivalent of VSFilter's MaskSub() - Partially works. Just use AssRender with a BlankClip.
• Steal some subtitles parser code for various formats from Aegisub, convert things to ASS internally and support more subtitle formats than just .ass - SRT support is working, but I forgot stealing from Aegisub :P

See the ChangeLog for further details.

It’s available here: http://luck3r.phicode.de/assrender/

I’ll continue working on it, and maybe we’re also gonna see a working DirectShow transform filter this year so VSFilter can finally rest in peace.

[Mug Funky]

nice one.

it might be worth adding on your site that you need to call it with "load_stdcall_plugin"... if it's there and i missed it, i apologise.

it's working with ssa now. i'm happy

[rapier]

assrender is very, very fast. I'm impressed. With assrender now I can see softsubbed karaoke and typesetting on 720p videos without lagging on a C2D E2160 using GMA950 onboard video. I have some complex karaoke scripts with two thousand lines of code that now runs almost on realtime.

Questions:

1) Is it possible to make assrender even faster?

2) Is it possible to make Aegisub use assrender? Aegisub 2.18 only recognizes VSFilter and VSFilterMod.

3) I can't use assrender on Windows 7 64bit. Says "unable to load C plugin". Other C plugins, like Yadif, works. Is there a way to fix that?

PS.: Sorry for my bad English.

[lachs0r]

Originally Posted by rapier

1) Is it possible to make assrender even faster?

Yes.

Originally Posted by rapier

2) Is it possible to make Aegisub use assrender? Aegisub 2.18 only recognizes VSFilter and VSFilterMod.

No, but that’s fine, since Aegisub supports libass directly (if you find someone to build it with the libass subtitle provider enabled).

Originally Posted by rapier

3) I can't use assrender on Windows 7 64bit. Says "unable to load C plugin". Other C plugins, like Yadif, works. Is there a way to fix that?

Never bothered with 64-bit AviSynth, and at the moment I neither have a mingw-w64 toolchain nor a 64-bit Windows testing environment ready, and this won’t change until I have a better internet connection (this should only take a few weeks).