AviSynth rendered subtitles to BluRay SUP/PGS and BDN XML (v2.08)

[hamletiii]

Hi, ps auxw, I think the new version works. I've only tested the same file, no more color palette exceeding 255 error and color smearing problem. The importing process into Scenarist BD also has been smoother.

As for the second problem with unnecessary splits, is it hard to fix? Scenarist BD is reading these correctly, two windows with one big box and one tiny horizontal strip. The muxed result works fine on the software players (tmt3, powerdvd 8, scenarist QC). But on PS3, lines without characters "g j p q y" (has little tails drop below the line) will show cropped, which mean only the big box gets shown on the screen. Lines with characters "g j p q y" are unaffected. I haven't test with other fonts or languages.

Lastly, one more feature request, when "-s" switch is used, I would like to have both xml files with time splitting and with no time splitting, since disabling and enabling "-s" switch only differ from the xml file generated. Sometimes with -s enabled, muxing could go underflow in scenarist for some reason, while no splitting works fine.

Can you put this function in the next version? Thanks so much for the updates.

[ps auxw]

Originally Posted by hamletiii

Hi, ps auxw, I think the new version works. I've only tested the same file, no more color palette exceeding 255 error and color smearing problem. The importing process into Scenarist BD also has been smoother.

Thanks for testing, good to hear it works now.

As for the second problem with unnecessary splits, is it hard to fix? Scenarist BD is reading these correctly, two windows with one big box and one tiny horizontal strip. The muxed result works fine on the software players (tmt3, powerdvd 8, scenarist QC). But on PS3, lines without characters "g j p q y" (has little tails drop below the line) will show cropped, which mean only the big box gets shown on the screen. Lines with characters "g j p q y" are unaffected. I haven't test with other fonts or languages.

That's a most peculiar behaviour from the PS3. I'll make a fix for that in the next days.

Can you put this function in the next version? Thanks so much for the updates.

This is actually a bit more tricky. Would a two-pass mode be acceptable, where first a log file and PNGs are produced and faster second passes can be used to quickly generate XML/PNG files with different settings?

[hamletiii]

Originally Posted by ps auxw

This is actually a bit more tricky. Would a two-pass mode be acceptable, where first a log file and PNGs are produced and faster second passes can be used to quickly generate XML/PNG files with different settings?

OK, this is harder than I thought. This feature would be useful only in some situation. If you have time to add it, a two-pass process sounds fine. But if you don't, I can absolutely live without this feature. Thank you.

[hamletiii]

Originally Posted by ps auxw

That's a most peculiar behaviour from the PS3. I'll make a fix for that in the next days.

Now I think PS3 might work within spec, other software players are just generous on this one.

Here is a statement from the Scenarist Designer PS (sonic's solution to create Blu-ray menu) tutorial:

"Make sure no layers are less than 8 pixels wide or high."

Although this applies for menus (IGS), I think it maybe works the same way for subtitles (PGS). The ones don't display seems all have 4 pixels in height; while the ones with tails thus display have 16 pixels in height.

[0xdeadbeef]

Originally Posted by hamletiii

As far as my experiments with Scenarist BD, splitting images into two windows could save bandwidth significantly. For a simple two sentences far apart with one sentence on the top of the screen and one on the bottom without splitting could easily spike as high as 9MB (the limit is only 4MB), while same picture splitting into two small regions occupy less than 1MB (well below the 4MB limit). With Tsmuxer, you don't have to worry about this that much since it doesn't check the bandwidth whatsoever, the worst case is that you get some flickering subtitle, but Scenarist BD will complain and refuse to import.

As noted before, bitmaps are RLE encoded in the PG stream. An empty line can be encoded with 2 bytes. With three bytes, you can encode up to 16384 empty pixels. With a correctly implemented RLE algorithm the size of an single subtitle shouldn't be increased by more than 2k (2*1080 = 2160 = 2.1k) just because one line is at the top of the screen and one at the bottom.

A worse scenario is text at the left and right (e.g. Japanese) since than no EOL markers can be used. However even if the subs use the full height, the gap can always be encoded with three bytes per line. So for a maximum of 1080 lines, the gap can be encoded with 3*1080 bytes which equals about 3.1k.
This is about the maximum amount of bytes you can save per frame if you split an image into two composition objects.

In a nutshell: empty pixels don't need a considerable amount of bandwidth. They just increase the initialization/decoding times a little which needs to be considered when calculating the PTS/DTS timestamps of the packets.

Originally Posted by hamletiii

Here is a statement from the Scenarist Designer PS (sonic's solution to create Blu-ray menu) tutorial:

"Make sure no layers are less than 8 pixels wide or high."

Although this applies for menus (IGS), I think it maybe works the same way for subtitles (PGS). The ones don't display seems all have 4 pixels in height; while the ones with tails thus display have 16 pixels in height.

Indeed 8 pixels is the minimum width and height for a composition object defined in an ODS entry.

[ps auxw]

Originally Posted by 0xdeadbeef

As noted before, bitmaps are RLE encoded in the PG stream. An empty line can be encoded with 2 bytes. With three bytes, you can encode up to 16384 empty pixels. With a correctly implemented RLE algorithm the size of an single subtitle shouldn't be increased by more than 2k (2*1080 = 2160 = 2.1k) just because one line is at the top of the screen and one at the bottom.

A worse scenario is text at the left and right (e.g. Japanese) since than no EOL markers can be used. However even if the subs use the full height, the gap can always be encoded with three bytes per line. So for a maximum of 1080 lines, the gap can be encoded with 3*1080 bytes which equals about 3.1k.
This is about the maximum amount of bytes you can save per frame if you split an image into two composition objects.

Indeed. Empty space encoded quite efficiently, but the problems seem to exist after decoding. At least error reports from Scenarist, like this one, lead me to believe this: http://forum.doom9.org/showpost.php?...4&postcount=69

Originally Posted by hamletiii

Although this applies for menus (IGS), I think it maybe works the same way for subtitles (PGS). The ones don't display seems all have 4 pixels in height; while the ones with tails thus display have 16 pixels in height.

Originally Posted by 0xdeadbeef

Indeed 8 pixels is the minimum width and height for a composition object defined in an ODS entry.

Thanks, I will prevent creation of images with either dimension size below 8 in the next version.

[0xdeadbeef]

Originally Posted by ps auxw

Indeed. Empty space encoded quite efficiently, but the problems seem to exist after decoding. At least error reports from Scenarist, like this one, lead me to believe this: http://forum.doom9.org/showpost.php?...4&postcount=69

There is indeed a "decoded object buffer" defined for PG streams and it's 4MB (4*2^20 = 4194304) bytes in size. Then again, this is enough to store two complete frames at full HD resolution (1920*1080 = 2073600 which is roughly 1.98MB). AFAIK only the decoded bitmap data is stored in this buffer - palettes, and other composition information is stored in a separate buffer (composition buffer). The decoding buffer is valid within an epoch, so it can only overflow if there are multiple composition objects per epoch. This is kinda weird as usually there is only one subtitle per epoch and obviously one subtitle frame can't be larger than two full HD frames.

So yes, splitting composition objects can help to safe space in the decoding buffer, but there should be no reason to do this, because there is more than enough space in this buffer as long as you don't do very, very weird things within an Epoch.

Anyway, IMHO the source of the problem is somewhere else. Either Scenarist is somewhat wacky regarding the definitions of Epochs or something else is wrong.

[mp3dom]

The buffer of 4MB could be not enough if you have some subtitles that are consecutive. If you have, let's say, 10 subtitles and every subtitle start immediately at the end of its previous, the epoch will contain more than 1 subtitle (in this example, the epoch will contain 10 subtitles with a composition object made with the largest subtitle). In this case the 4 MB buffer could be not enough (quite easy if you made strange subtitles with lot of graphic or fancy fonts). Subtitles shorter than 2 frames could also lead to a buffer overflow.

[0xdeadbeef]

Hm, I never cared so much about the exact definition of behavior between two epochs. It is possible that subtitles have to be defined within an epoch if they should be displayed directly one after the other without the slightest gap. This isn't really interesting for "normal" subtitles which usually have a certain gap of a few hundred milliseconds between them and thus can be safely stored in separate epochs. Indeed my conception of epochs always was that usually one Epoch contains one subtitle and the rare case of multiple object definitions per Epoch is mostly used to display a 2nd subtitle line with a delay while still showing the first etc.
While there can be up to 8 palettes and 64 composition objects per Epoch, I would be pretty surprised to actually find a SUP stream that contains an Epoch with more than two or three composition objects. With more than 8 objects, they'd also have to share palettes.

However, probably this is the explanation of this whole issue: BDSup2Sub always uses one epoch per subtitle. If the end time of one subtitle equals the start time of the next subtitle, a standalone might introduce a short pause between these two subtitles as it has to clear the decoding buffer etc. between two Epochs. Scenarist however tries to avoid that gap by putting two or more subtitles in one epoch. If there are multiple gapless subtitles though, this leads to issues with palettes and/or a decoding buffer overflow.

[hamletiii]

Originally Posted by 0xdeadbeef

However, probably this is the explanation of this whole issue: BDSup2Sub always uses one epoch per subtitle. If the end time of one subtitle equals the start time of the next subtitle, a standalone might introduce a short pause between these two subtitles as it has to clear the decoding buffer etc. between two Epochs. Scenarist however tries to avoid that gap by putting two or more subtitles in one epoch. If there are multiple gapless subtitles though, this leads to issues with palettes and/or a decoding buffer overflow.

So this explains why sometimes BDSup2Sub's timecode was different from original timecode from avs2bdnxml. Thanks for your informative replies.

[sneaker_ger]

Originally Posted by hamletiii

So this explains why sometimes BDSup2Sub's timecode was different from original timecode from avs2bdnxml. Thanks for your informative replies.

Different timecodes? Are you sure they don't just come from BDSup2Sub's adjustments to synch to the framerate?

[0xdeadbeef]

Yep, I'd think this is unrelated as I talked about the handling of Epochs.
Besides, BDSup2Sub always shifts start/end times to the nearest frame which leads to slightly different times compared to tools which don't care about frame synchronization. Then again, the time codes in the XML format are always frame based, so BDS2S shouldn't adjust the times in this case anyway.

[Solaris]

Hi guys,

Good tool. I am looking to use this function in a little c# program I am writing. However, when I check the PNGs it creates, the fontsize is very small. So I need to set the fontsize, color etc from within my c# prgram. Is there a config or ini file that hold these parameters so that I can modify this?

Thanks in advance for your time.

[hamletiii]

Originally Posted by 0xdeadbeef

Yep, I'd think this is unrelated as I talked about the handling of Epochs.
Besides, BDSup2Sub always shifts start/end times to the nearest frame which leads to slightly different times compared to tools which don't care about frame synchronization.

But isn't this frame synchronization feature intentionally put each composition object under per epoch? There are quite a few Japanese commercial discs have two window slicing with lots of composition objects under one epoch, so I don't think those are odd, avs2bdnxml can create them and import to Scenarist BD perfectly fine. It's just your implementation in BDSup2Sub is trying to create gaps as you said in post#129 which is not desirable for these types of subs.

Then again, the time codes in the XML format are always frame based, so BDS2S shouldn't adjust the times in this case anyway.

I'm referring to frames after the seconds, not the whole time codes.

[0xdeadbeef]

Originally Posted by hamletiii

But isn't this frame synchronization feature intentionally put each composition object under per epoch? There are quite a few Japanese commercial discs have two window slicing with lots of composition objects under one epoch, so I don't think those are odd, avs2bdnxml can create them and import to Scenarist BD perfectly fine. It's just your implementation in BDSup2Sub is trying to create gaps as you said in post#129 which is not desirable for these types of subs.

Having multiple composition object is allowed of course, yet PTS/DTS calculation is already a nightmare for simple Epochs and since the available muxers assume the time stamps to be absolutely corrects, it's up to the subtitle tool to calculate them.
Besides, multiple composition objects are meant for effects like crossfades etc. which are hard to impossible to generically translate to any other format anyway.

These are the two main reasons why BDSup2Sub simplifies the subtitle stream to have one subtitle per Epoch. It doesn't introduce gaps deliberately though - they are introduced by the player's rendering implementation.

Always keep in mind that BDSUp2Sub is a subtitle converter, not a BD authoring tool.

[hamletiii]

Originally Posted by 0xdeadbeef

While there can be up to 8 palettes and 64 composition objects per Epoch, I would be pretty surprised to actually find a SUP stream that contains an Epoch with more than two or three composition objects. With more than 8 objects, they'd also have to share palettes.

Since you mentioned fading, I looked more into the PES structure in Scenarist BD. This is the tree structure showing a subtitle that is capable of one line fading while the other line stay opaque all the time.



The structure shows something called DisplaySet under Epoch which I don't see you mentioned it. Is this something not defined in Blu-ray subtitle spec? Each epoch can have one or two window, then a bunch of DisplaySets. Each DisplaySet has a palette and two composition objects which appears to have the same number as windows. The fading effect (in CompositionObject#2) is caused by altering palette in each DisplaySet, the subtitle picture is always the same png file.

So this DisplaySet seems to be one VERY important element since it simplifies things quite a bit, also gets around the limitations of 8 palettes and 64 composition objects per Epoch?

Also want to share my experience with fading in avs2bdnxml here. The way avs2bdnxml handles fading is by simply rendering each frame as a different picture, so one fading effect could generate lots of pictures. There's some limitations to this though:

1. It's not possible to generate one line fading while the other line stay opaque all the time without flickering. The altering palette method in the example however fades very nicely. I guess it's because the palette info is not stored in the 4MB decoding buffer although the palette alters each frame while the avs way has so many pictures packed together just simply overflows the buffer?

2. Only one line fading with no other line is possible, and the effect is actually pretty decent. But the fading time can not be very long. Some people have suggest to stay within 600ms.

[0xdeadbeef]

Well, as mentioned before, an Epoch doesn't simply have a start and an end, but can have multiple "acquisition" and "continue" events. All of them specify display updates and thus they define display sets. This is an abstract term though which includes several Epoch types and not a stream element.

Anyway, I dunno how this helps here. I don't think that the BDN XML format in its current form is able to store the information that would be needed to define palettes, objects, windows and epochs in a way that would be needed to allow an external converter to accurately convert it to BD-SUP.

Even if someone would define a new XML format to cover all of this, there would be no tool to create such files. Last but not least, someone would have to write the converter with authoring capabilities (calculating all the DTS/PTS timestamps for a complex PG stream with all plausibility and limitation checks will be a nightmare).

[ps auxw]

Sorry for disappearing again. Life is quite busy. For now, I have released avs2bdnxml v1.13, which should fix the problems with tiny (below 8px dimension) pictures and ugly image splits.

@Solaris: avs2bdnxml does no subtitle rendering on its own. You will have to create a subtitle file with your desired font-size etc.. For this, I'd recommend taking a look at Aegisub, or even just SSA/ASS subtitle specification (as far, as you can call those that).

[hamletiii]

Hi, ps auxw, thanks for the update, will test it when I got more time. If everything works, I think this is probably the most intuitive and feature rich Blu-ray subtitle rendering tool out there.

And hopefully you'll get more time in summer to look into SUP support, so that people who want to do fancy Blu-ray subtitles with tsmuxer could benefit from this software as well.

[hamletiii]

OK, this appears to be the fading tags I'm looking for:

<?xml version="1.0" encoding="UTF-8"?>
<BDN Version="0.93" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="BD-03-006-0092b BDN File Format.xsd">
<Description>
<Name Title="Title" />
<Language Code="eng" />
<Format VideoFormat="1080p" FrameRate="23.976" DropFrame="false" />
<Events NumberofEvents="3" Type="Graphic" />
</Description>
<Events>
<Event InTC="00:00:04:02" OutTC="00:00:07:02">
<Graphic>ak_Track_0001.png</Graphic>
<InlineEffect Type="Fade" Duration="00:00:00:05" Anchor="Start">
<Fade FadeType="FadeIn" />
</InlineEffect>
<InlineEffect Type="Fade" Duration="00:00:00:05" Anchor="End">
<Fade FadeType="FadeOut" />
</InlineEffect>
</Event>
<Event InTC="00:00:11:08" OutTC="00:00:14:08">
<Graphic>ak_Track_0002.png</Graphic>
<InlineEffect Type="Fade" Duration="00:00:00:23" Anchor="End">
<Fade FadeType="FadeOut" />
</InlineEffect>
</Event>
<Event InTC="00:00:13:09" OutTC="00:00:17:11">
<Graphic>ak_Track_0003.png</Graphic>
<InlineEffect Type="Fade" Duration="00:00:00:23" Anchor="Start">
<Fade FadeType="FadeIn" />
</InlineEffect>
</Event>
</Events>
</BDN>

The tags are recognizable by Scenarist BD. The first one, ak_Track_0001.png will fade in and fade out; the second and third are cross-fading effect I described in post#136 above. Note on cross-fading, the start time[in bold] of the third picture actually overlaps with the end time of the second picture. Also the actual graphics on the second and the third cannot have the same position. [s]It appears that Scenarist BD automatically crops/splits windows for you, the graphics I tried are all 1920*1020 pictures with no additional width or height tags given as seen in the above text. So upto this point, we are looking at things too hard, at least for Scenarist BD, all you need are full size images with tag <Graphic>. You let Scenarist BD's built-in functions to handle all these cropping/window splitting![/s] In addition, if avs2bdnxml could read the convert the fade tags in *.ass/ssa to fade tags in bdn format, then a proper fading could be achieved instead of the current frame based fading.

[s]So for now, if you want to use Scenarist BD for image splitting, you can just use -a0, -b0 to force avs2bdnxml to output full size png files.[/s] For fading, do NOT add the \fad in your *.ass/ssa files, you can manually pick the pictures and apply the fading tags above after avs2bdnxml finishes image rendering.

Update:

I did a bit more testing. Scenarist BD's built-in functions can only handle cropping with one window and one composition object, it cannot split images into two composition objects. I think this goes back to the decoding buffer error discussion last year, and the whole purpose of having avs2bdnxml to support dual windown splitting. So -a1, b1 options are needed. The fading tags can still be applied to the cropped pictures. In the event that have two graphics:

<Event Forced="False" InTC="00:01:40:08" OutTC="00:01:49:00">
<Graphic Width="713" Height="74" X="96" Y="945">00003008_0.png</Graphic>
<Graphic Width="236" Height="66" X="96" Y="768">00003008_1.png</Graphic>
<InlineEffect Type="Fade" Duration="00:00:00:12" Anchor="Start">
<Fade FadeType="FadeIn" />
</InlineEffect>
<InlineEffect Type="Fade" Duration="00:00:00:12" Anchor="End">
<Fade FadeType="FadeOut" />
</InlineEffect>
</Event>

The two composition objects will fade in and out at the same rate.

But during two events crossfading, these two events can only have one composition object each, otherwise you'll get Error: The number of Graphic Element exceed 2.

Example:

<Event Forced="False" InTC="00:02:51:09" OutTC="00:02:55:17">
<Graphic Width="443" Height="207" X="92" Y="766">00005139_0.png</Graphic>
<Graphic Width="974" Height="140" X="474" Y="62">00005139_1.png</Graphic>
<InlineEffect Type="Fade" Duration="00:00:00:12" Anchor="Start">
<Fade FadeType="FadeIn" />
</InlineEffect>
<InlineEffect Type="Fade" Duration="00:00:00:12" Anchor="End">
<Fade FadeType="FadeOut" />
</InlineEffect>
</Event>
<Event Forced="False" InTC="00:02:54:29" OutTC="00:02:57:16">
<Graphic Width="443" Height="207" X="92" Y="766">00005267_0.png</Graphic>
<Graphic Width="745" Height="128" X="587" Y="62">00005267_1.png</Graphic>
</Event>

The above is invalid because during crossfading, it has four composition objects. While Blu-ray limits 2 composition objects at a time I guess.

Also currently crossfading is possible between two pictures that have close enough colors, if one picture has colors drastically different than the other picture, then the total combined colors will exceed 256, which will make Scenarist BD complain.