Avisynth Script - Personal Subtitles hardcoding...

[northerns74r]

Hi all,
I wanted to share with you this script I did these days, in order to hardcode subtitles in videos starting from .srt files.
I know there are plugins or functions which already do this task (TextSub for example), but I had precise needs and I couldn't use it for my task. So I decied to study bit of Avisynth and something else (like GScript etc)...

Here's what it does: it takes video, .srt file, background image for titles and fuse titles on the video. It can handle collisions e vertical alignment in case of single line title (in my case there aren't subtitle with more than two lines).
The real deal (I couldn't do with TextSub) was the overlaying the background image only when subtitles are on screen. At the end, something which can look a bit more professional...
Now the whole workflow is quite faster than doing all the subs in Premiere pasting from a .doc file. Translating in VisualVobSynch is easier when you get confident with it.


This is how the video looks like after hardcoding...

#LOCAL PATHS
plugins_dir = "C:\Programmi\AviSynth 2.5\plugins\"
video_dir = "C:\Documents and Settings\aelmi\Desktop\105 - CLUB NATION - CHEMICAL - 10-2010 SYNCH.avi"
sottopancia_dir = "C:\Documents and Settings\aelmi\Desktop\sottopancia1280.tif"
subs_dir = "C:\Documents and Settings\aelmi\Desktop\chemicals.srt"

#STUFF NEEDED
LoadPlugin(plugins_dir+"GScript.dll")
LoadModule("avslib", "base", "constants")
LoadModule("avslib", "base", "conversion")
LoadPackage("avslib", "array")
LoadPackage("avslib", "clip")
LoadModule("avslib", "numeric", "rounding")


#IMPORT OF VIDEO, SUBS AND BACKGROUND
c=DirectShowSource(video_dir, audio=false)
fps = c.framerate
dar = 16/9 #ASSUME 16:9
sar = c.width/c.height
par = dar/sar
bg=ImageReader(sottopancia_dir).BicubicResize(c.Width,c.Height)
bgALPHA=ImageReader(sottopancia_dir,pixel_type="RGB32").ShowAlpha(pixel_type="RGB32").BicubicResize(c.Width,c.Height)
subs = import(subs_dir)
offset_iniziale = FindStr(subs,"1")
sotto_subs = ArrayCreate(midstr(subs, offset_iniziale))
ArrayDelimiterSet(CRLF+CRLF)


#SUB PROPERTIES
y_sottopancia = 0
y_sub = 0.825
font_sub = "LT_50543.TTF"
size_sub = 0.0642
text_color_sub = $FFFFFF
halo_color_sub = $303030
align_sub = 5
lsp_sub = 0.017
font_width_sub = 9


GScript("""
if(sotto_subs.ArrayLen()>0)
{
    start_last_frame = -100
    end_last_frame = -100
    text_last_frame = ""
    y_last_sub = y_sub
    
    for(i=0, Floor(Log(sotto_subs.ArrayLen())/Log(10)), 1){
        startloop = int(Pow(10,i)-1)
        offset = i
        if(sotto_subs.ArrayLen()>int(Pow(10,i+1)-3)){
            endloop = int(Pow(10,i+1)-2)
            }
        else {
            endloop = sotto_subs.ArrayLen()-1
        }
        for (j = startloop , endloop, 1)
        {
            
            #CALCULATING IN-OUT FRAME OF THE SINGLE SUB
            single_sub = sotto_subs.ArrayGet(j)
            start_sub_H = MidStr(single_sub, 4+offset,2)
            start_sub_M = MidStr(single_sub, 7+offset,2)
            start_sub_S = MidStr(single_sub, 10+offset,2)
            start_sub_ms = MidStr(single_sub, 13+offset,3)
            start_sub_frame = value(start_sub_H)*60*60*fps+value(start_sub_M)*60*fps+value(start_sub_S)*fps+value(start_sub_ms)*fps/1000
            end_sub_H = MidStr(single_sub, 21+offset,2)
            end_sub_M = MidStr(single_sub, 24+offset,2)
            end_sub_S = MidStr(single_sub, 27+offset,2)
            end_sub_ms = MidStr(single_sub, 30+offset,3)
            end_sub_frame = value(end_sub_H)*60*60*fps+value(end_sub_M)*60*fps+value(end_sub_S)*fps+value(end_sub_ms)*fps/1000
            single_sub_text = MidStr(single_sub, 35+offset)
            
            #HANDLING COLLISIONS
            if(start_sub_frame<=end_last_frame){
                start_sub_frame = start_sub_frame+(start_sub_frame-end_last_frame)+1
            }
            if((int(start_sub_frame)-int(end_last_frame))<10&&start_sub_frame-end_last_frame>1){
                for(k=1,int(start_sub_frame-end_last_frame-1), 1){
                    c_overlaied=c.Trim(int(end_last_frame+k),int(end_last_frame+k))
                    c_overlaied=Overlay(x=0,y=y_sottopancia,c_overlaied,bg,mask=bgALPHA)
                    c=c.Trim(0,int(end_last_frame+k-1))+c_overlaied+c.Trim(int(end_last_frame+k+1),c.framecount)
                    c=Subtitle(c, text_last_frame, first_frame=int(end_last_frame+k), last_frame=int(end_last_frame+k), y=round(c.Height*y_last_sub+y_sottopancia), font=font_sub, size=round(c.Height*size_sub), 

text_color=text_color_sub, halo_color=halo_color_sub, align=align_sub, lsp=round(c.Height*lsp_sub), font_width=round(par*font_width_sub))
                }    
            }
            
            #HANDLING TWO LINES
            break=Findstr(single_sub_text, CRLF)
            if(break!=0){
                single_sub_text=LeftStr(single_sub_text,break-1)+"\n"+Rightstr(single_sub_text, single_sub_text.StrLen-break-1)
            }
            c_overlaied=c.Trim(int(start_sub_frame),int(end_sub_frame))
            c_overlaied=Overlay(x=0,y=y_sottopancia,c_overlaied,bg,mask=bgALPHA)
            c=c.Trim(0,int(start_sub_frame-1))+c_overlaied+c.Trim(int(end_sub_frame+1),c.framecount)
            
            #HANDLING VERTICAL ALIGN OF SINGLE LINE SUBS
            if(break==0){
                y_this_sub = y_sub+0.5*size_sub
            }
            else{
                y_this_sub = y_sub
            }
            c=Subtitle(c, single_sub_text, first_frame=int(start_sub_frame), last_frame=int(end_sub_frame), y=round(c.Height*y_this_sub+y_sottopancia), font=font_sub, size=round(c.Height*size_sub), text_color=text_color_sub, 

halo_color=halo_color_sub, align=align_sub, lsp=round(c.Height*lsp_sub), font_width=round(par*font_width_sub))
               
            start_last_frame = start_sub_frame
            end_last_frame = end_sub_frame
            text_last_frame = single_sub_text
            y_last_sub = y_this_sub
        }
   }
}
""")

audio=DirectShowSource(video_dir, video=false)

return (AudioDub(c,audio))

The script works fine consider my needs.

Do you have any suggestion?

Anyway at the end of the story I have a problem: when VirtualDub encodes the .avs says "out of memory". Do you know why? Where does the code get heavy to run?

Cheers!

[Gavino]

Impressive use of GScript - I think that's the most complex example I've seen!

Is memory usage more or less proportional to the number of subtitles used? It looks like you have an instance of Overlay and Subtitle for each title (and in some cases for each frame of the title?), which can mount up. Not sure if it can be avoided though - perhaps if you know the titles will always be only on a small area of the screen, you could build up a smaller overlay clip, using less memory.

Also, watch out with this code:

c_overlaied=c.Trim(int(start_sub_frame),int(end_sub_frame))
c_overlaied=Overlay(x=0,y=y_sottopancia,c_overlaied,bg,mask=bgALPHA)
c=c.Trim(0,int(start_sub_frame-1))+c_overlaied+c.Trim(int(end_sub_frame+1),c.framecount)

Firstly, the last c.framecount should be c.framecount-1 (or just use 0). More subtly, if you have a subtitle on frame 0, 1, or the last frame your code will not work - the classic Trim problem.

[northerns74r]

First of all thank you for appreciating that... But YOU did the most impressive job! (without the possibility of cycling and other scripting stuff would lose most of his power).

You're right on each you're consideration!
1)Ok for "framecount-1"...
2)I think I can manage titles starting on 1st or 2nd frame by inserting at the beginning of the FOR CYCLE a check which shifts the start to frame 3. Anyway it's quite unusual for my job...
3)It happens that it calls an instance of Subtitle() and Overlay() on each frame when fixing collisions (or covering too short gap between frame, 10 in that code). When gap is too short the previous is lengthen untill the next starts, in order to avoid the 'blinking effect'...

A solution can be - in the collision managing - calculating the gap and overlaying+subtitling for all the gap instead frame by frame.. There could be a for-cycle less!

By the way in your opinion... what is the memory usage related to? I mean... let's say I have a two or three nested for-cycles which all of them scroll along the whole clip, does it need to keep the all video (uncompressed) in RAM?
The clip which gave me that error message was a 5 minutes DV-AVI interview with and around 50 titles... I can't imagine a 2-hours Full-HD clip...

Is there a way to optimize this usage?

And last question: is there the equivalent of LIST instead of ARRAY? The real tricky task is parsing a text file...

Thank you in advance

[TheFluff]

if the "image" you want to overlay as a background is just a semitransparent rectangle (with or without borders and whatnot) then that is fairly easy to do with textsub (although you'll need to use .ass, not .srt. but it's trival to convert between the two.)

hope this helps

[Gavino]

Originally Posted by northerns74r

let's say I have a two or three nested for-cycles which all of them scroll along the whole clip, does it need to keep the all video (uncompressed) in RAM?
...
Is there a way to optimize this usage?

It doesn't keep the whole video in RAM, but each filter used (on each iteration, if in a loop) becomes a separate object in memory at run-time. Overlay and Subtitle are fairly memory-intensive, and the memory needed is proportional to frame size.
So a trick that can be used, if the titles are confined to a specific part of the screen, is to Crop the clip into 5 different parts as follows:

 _________________________________________________
|                                                 |
|                                                 |
|                       A                         |
|                                                 |
|_________________________________________________|
|               |               |                 |
|               |               |                 |
|      B        |       C       |       D         |
|               |               |                 |
|_______________|_______________|_________________|
|                                                 |
|                                                 |
|                       E                         |
|                                                 |
|_________________________________________________|

where C is the area in which the titles are confined.
You can then apply the overlays/titles to the smaller clip C (using less memory, and faster) and put the other parts back at the end using
StackVertical(A, StackHorizontal(B, C, D), E).

And last question: is there the equivalent of LIST instead of ARRAY? The real tricky task is parsing a text file...

I've never used avslib, so I don't know about that. I don't know how memory-efficient it is either. The Avisynth language was never really designed for text parsing, so if TheFluff's suggestion applies, you might be better moving to .ass.

[northerns74r]

Originally Posted by TheFluff

if the "image" you want to overlay as a background is just a semitransparent rectangle (with or without borders and whatnot) then that is fairly easy to do with textsub (although you'll need to use .ass, not .srt. but it's trival to convert between the two.)

hope this helps

I knew that.. I read in these specs that Picture events are allowed, but I never was able to manage it... Furthermore I read somewhere that this function has never been really implemented... That's the reason I moved on scripting.
.ASS won't be a problem because I use VisualVobSynch and it can export .ass as well.
Maybe I'm wrong on that.

Or just in case... do you know if I can mix the two ways (ASS+SCRIPTING)?
I mean... ASS subtitles use the section EVENTS when each title appear with its begin/end timings...
Would be nice doing something like:
for each EVENT do an Overlay() functions or something else... and write it inside .ass file. Am I drunk?

I will try using Gavino's idea - dividing the frame in portions. It seems it definetly saves memory.

Thank you both

[cretindesalpes]

There is a simpler way IMHO, using vsfilter:

• Create a subtitle mask with MaskSub, flip it vertically (because it is rendered upside down for some reason) and convert it to YV12
• Use mt_lutf in "range" mode to produce an all black/all white picture depending on the presence of subtitle pixels on the previous mask
• Use the mt_lutf result with mt_merge to switch between the raw frame and the frame with your overlayed picture
• TextSub the subtitles.

Of course you can use more complex schemes to handle subtitles on different parts of the screen, or to adjust the overlayed picture size.

[TheFluff]

Originally Posted by northerns74r

I knew that.. I read in these specs that Picture events are allowed, but I never was able to manage it... Furthermore I read somewhere that this function has never been really implemented... That's the reason I moved on scripting.
.ASS won't be a problem because I use VisualVobSynch and it can export .ass as well.
Maybe I'm wrong on that.

Or just in case... do you know if I can mix the two ways (ASS+SCRIPTING)?
I mean... ASS subtitles use the section EVENTS when each title appear with its begin/end timings...
Would be nice doing something like:
for each EVENT do an Overlay() functions or something else... and write it inside .ass file. Am I drunk?

I will try using Gavino's idea - dividing the frame in portions. It seems it definetly saves memory.

Thank you both

Picture events are indeed not supported anywhere, but that's not what I suggested you use. There are several ways to accomplish the effect you want; vector drawings is one, using the "solid bounding box" background is another, and using a font that has a square box as one of its characters is yet another.

There are several scripting environments/frameworks intended for use with ASS; the one I have the most experience with is Aegisub's automation, which uses Lua. It has a lot of helper functions and semiautomatic functionality in place that would make what you want (generate a background box for each subtitle line) a matter of maybe ten minutes of scripting for a decently experienced user.

[northerns74r]

Thank you... I will give it a chance!