drawbox function of ffmpeg in avisynth

[Yanak]

Hello,

Not sure if it's the right place to ask about this but will something similar to the drawbox function of ffmpeg will be hard to implement in avisynth ? ( i know about overlay and Blackness/BlankClip in avisynth but i would like something a bit better that could combine and adjust with text sizes used in subtitle function)

In Ffmpeg this is handy, you can auto adjust the size of the box to a given text displayed using text_h & text_w functions of drawtext,
similar thing is possible in php using imageftbbox or imagettfbbox that can calculate the space taken by some text then use imagefilledrectangle function to draw the box behind some text displayed using imagettftext function.

I'm not expert in all this, had to mess a few times with php draw functions and used those things, only used them a bit with ffmpeg but this is very helpful to kind of auto-draw boxes behind some text, can be full color or semi transparent boxes, so I'm wondering if something like this would be possible in avisynth somehow, will be a very neat thing imo, might be just me thinking this will be nice and you guys will throw stones at me for asking something stupid but i ask

So I'm wondering if something allowing us to define the text width and height could be done maybe, then creating a blankclip and overlay will be easier to auto adjust to some text or maybe have a combination of those 2 for a more automated process.

Thanks a lot and thanks again for this nice tool that is avisynth.

[wonkey_monkey]

Your question would be better off in Avisynth Usage but the "textbook" answer that immediately springs to mind is to use crop(...) to split the clip into 9 boxes (well, 5, at a minimum, one either side and split the middle slice into 3), alter the middle one (darken it, blank it), then stack (stackhorizontal and stackvertical) them all back together again.

[Yanak]

Hi and thanks for the answer,

I can do some things already but it needs to be adjusted for each use so it's not exactly what i need, i want something way more precise than this and able to be semi automated.

To achieve it i need a function similar to what ffmpeg or php can do to get the height and width of the text input based on the font used and the font size set for the subtitle function to then be able to draw a semi transparent box just around the text very precisely.

Like i said i'm not sure how hard it will be to add such thing or even if it is possible in avisynth, I know it is possible on some other things like ffpmpeg or even in php wih functions i spoke about in my previous post.

For example some times ago i made me a small batch using ffmpeg to test a bunch of LUT .cube files applied on a single picture ( or entire videos extracting frames from it using ffprobe and making a grid with frames from the video ), the result will bring me the original picture with many different LUT's applied on it and the name of he LUT used will be drawed on the pictures + the semi transparent textbox that is dynamic and auto-adjust to the text input itself, the text font used and the font size used, like here :


all this is automated, i don't need to adjust the black box behind, just a math formula on the batch based on the width and height the text will have using ffmpeg "text_h" & "text_w" functions.

To be able to reproduce something working similarly in avisynth when i use the subtitle function i need it to calculate the width and height the desired text input will take based on the text lenght, the font and fontsize used.

Like i said i'm not sure if this is possible to be implemented or if it is a headache to code something like this in avisynth or even if it is something relevant but having a function like this will be really nice.

Thanks.

[wonkey_monkey]

To be able to reproduce something working similarly in avisynth when i use the subtitle function i need it to calculate the width and height the desired text input will take based on the text lenght, the font and fontsize used.

You might be able to do something with rt_stats or similar (I'm not familiar enough with it to say if you can or can't) whereby you would feed it a subtitle clip and it could tell you the extent of the white pixels on it...

[TheFluff]

If you want a real subtitle renderer with full-featured typography, layouting and text extents calculation, how about you use one? There are several ways to do what you want with .ass-based subtitles and several ways to render them in Avisynth.

[Yanak]

@davidhorman:
thanks i will look into rt_stats documentation tomorrow and see if i can do something with this.

@TheFluff :
It ain't to use on full subtitles, not with subtitle files i mean, more like add a facility to add some background easily on some text i add on my videos using the subtitle functions of avisynth, mostly doing this using some functions and scripts i added inside staxrip with easy implementation there for those and semi-automation, like adding a caption on different parts of the video, selecting easily a portion of the video and extracting frame numbers then having an input textbox to add the text, select font size and position of the text, colors of text and halo etc etc, i have all pretty much automated and just need to input text, select font size, colors and positions mostly.

I'll just need a way to get the text width and height to add a nice background box almost automatically to all this but i understand this might be a specific request, i found some similar questions about this searching on the net before asking here if something could be done, was just wondering as this seems possible in some other domains and tools if this could be done here without much headache.

Thanks.

[TheFluff]

What you're describing is exactly what a subtitle renderer does. You can of course do things the hard way if you really insist, but I strongly believe that by far the easiest way to do what you're talking about is by generating an .ass file, programmatically if you want to. Take a look at Aegisub.

[Yanak]

Thanks a lot for the replies and tips, i have looked into this but it ain't what i need sorry.

I asked thinking it might be possible to get such option implemented since it's something i saw and used in other domains and maybe something that is possible in the language used to code avisynth ( Im' not a coder so i don't know).
Just thinking avisynth could implement something like this natively and hopefully without much headache to code it, that could be handy and allow nice things to be done with/around subtitles and text implementation once avisynth is be able to return the height and width of the desired text input, adding a semi transparent box is only one of the possibilities, probably lot of other things/effects can be done around added text with such known parameters.
Well at least i tried, i guess I'll have to find another way using like suggested rt_stats maybe or something else or just pass and forget about this.

Thanks again for the replies and help.

[poisondeathray]

Re-inventing the wheel. 1 spoke at a time . I'm with Fluffy - ASS subs have a textbox background style. You can make it semi transparent etc.. Lots of power in aegisub




But maybe one approach would be to use masktools to dynamically create a BG mask based on the parameters text/size/font/whatever

You can use mt_rectangle / mt_box to get rough box coordinates, but I don't know of an easy way to get a perfect box using masktools only , without some type of cropping , like autocrop. This means you lose the internal subtitle(x,y) position coordinates, but instead have to use mt_merge or overlay 's x,y positioning coordinates

This is very rough, but I think something like this might be close to what you want, or at least get you started . To debug it for now, you can enter the parameters up top, like font, color etc... . You can probably wrap it into a some function later once it's debugged more thoroughly. You might need to adjust the masktools parameters if you have extreme fonts or sizes

#Parameters
#############################
txt="This is my text, DUDE"
sz=40
ft="arial"
x_pos=100
y_pos=100
bg_color=color_blue
bg_opacity=0.5
txt_color=color_yellow
hal_color=color_black

#############################

orig=colorbars(pixel_type="YV12")

bl=blankclip(orig, pixel_type="YV12")

txtmsk=bl.subtitle(txt, font=ft, text_color=color_white, halo_color=color_white,size=sz)

txtmsk
mt_luts( last,last, mode = "max", pixels = mt_rectangle(6,6,true), expr = "y")
msk=last

msk
autocrop(mode=0, threshold=20)
mt_lut("255") #pure white mask
crpmsk=last

crpmsk
mt_lut("0") #pure black  mask
bl_crpmsk=last

txt_alpha_msk=bl_crpmsk.subtitle(txt, font=ft, text_color=color_white, halo_color=color_white,size=sz)
txtlayer=bl_crpmsk.subtitle(txt, font=ft, text_color=txt_color, halo_color=hal_color,size=sz)

bg_for_txt=blankclip(bl_crpmsk, color=bg_color)


overlay(orig, bg_for_txt, x=x_pos,y=y_pos, opacity=bg_opacity)
overlay(last, txtlayer,  x=x_pos,y=y_pos, mask=txt_alpha_msk)

[Yanak]

Thanks you very much for this,

i cannot test it tonight but will test all this deeply tomorrow and see how it goes and probably mess a bit with it if needed in the next days to adjust a few things but your approach looks nice presented like this.

Thanks a lot for this precious help.

[Yanak]

Had time to test and mess a bit with the script proposed and it works nicely, nice job on that, for this purpose while not 100% perfect or precise as i'd wanted it will do the job nicely, will have to spend time learning a bit more about masktools options to maybe improve it a bit and work on making all this more automated but it's something really nice already, again thanks a lot for the help.

Still wondering about something, the subtitles options have the "align" parameter, for example using align=9 or 3 will put the subs on the top right or bottom right of the screen, using align=5 will perfectly center the subtitles on the middle of the video for the width.

Seeing this i am now wondering even more about this, seems like somehow there is already a built in function in avisynth to know what width the subtitle will take to be able to use the align functions i guess, unless my reasoning is messed up and wrong there is already some sort of internal calculations done to define the length of the text and place it correctly using align functions to place the text on some parts of the screen no ?

Maybe something could be done using the same already built in function in avisynth that is used for align function but instead it will return the values calculated, again if my reasoning is correct here.

Thanks a lot.

[poisondeathray]

Originally Posted by Yanak

Still wondering about something, the subtitles options have the "align" parameter, for example using align=9 or 3 will put the subs on the top right or bottom right of the screen, using align=5 will perfectly center the subtitles on the middle of the video for the width.

Seeing this i am now wondering even more about this, seems like somehow there is already a built in function in avisynth to know what width the subtitle will take to be able to use the align functions i guess, unless my reasoning is messed up and wrong there is already some sort of internal calculations done to define the length of the text and place it correctly using align functions to place the text on some parts of the screen no ?

Maybe something could be done using the same already built in function in avisynth that is used for align function but instead it will return the values calculated, again if my reasoning is correct here.

I doubt using subtitle() 's align would be accurate enough for what you want. You have a grid of 3x3. The BG box will be pretty ugly with overhang on one or more sides depending on your test

Someone should be able to look at the -vf drawbox/drawtext code and translate it directly . Or why not use ffmpeg directly ?

[raffriff42]

Subtitle calls the Windows API function DrawText(... DT_CALCRECT) to determine the bounding rectangle of the text as rendered.

Someone could make the results available to the user somehow, if they were sufficiently motivated.

(EDIT but poisondeathray's solution works great, so I don't see the need)

[Yanak]

@poisondeathray : in most cases i could test it is quite accurate to define the length of the text and place it correctly on screen and it adjusts to the length of the text input.

I can use ffmpeg for some things indeed, for some others i would like to get this directly in avisynth with a easy way to do this, all that is needed is being able to get text height and size, could then also bypass the "align" function and get more control on all this, once you know the width and height of the desired text input it takes just a few simple math formulas in the script that will bring a lot of liberty and new possibilities with all this.


@raffriff42, i was suspecting it was using something like this to define the size of the text and be able to place it correctly, i don't understand the code in the link , sadly this isn't my thing, but as far as i know there is usually 2 ways to define such used space, one takes in account the inter lines spacing and gives less accurate results, especially for the height calculation, other is based purely on the text input and gives really precise results for width and height, not taking in account the interline spaces.


In avisynth i suspect it uses the first one as when for example you use "Align=5" the text is ( as far as i could test) perfectly centered for the width, same free space remains on right and left sides, same exact space in pixels, but for the height it is not perfectly centered, probably due to the interline space parameter of the font that is included in the calculation here, not sure if what i tried to explain is easy to understand or not so a picture is better than words i guess :



The align parameter depending the parameters, especially the font size can bring very nice results and some less nicer due to this method of calculation including the interline ( it's a guess only, don't take this for set in stone, just empiric reasoning) , with quite large font sizes you will have spaces and margins quite big from the bottom of the screen for example, again an example , with font size =30 :

with font size= 100 :

(Edit: here there is no letter like "j,g" but even including this margin for such letters it won't be centered, maybe if it's not the interline space it could be the space for under lined text, i'm not 100% sure and did not tested with under lined text, will have too try if it is possible to be done in a subtitle)

Align parameter uses pre-defined margins, from the documentation :

x = 8 if align=1,4,7 or none; -1 if align=2,5,8; or width-8 if align=3,6,9
y = size if align=4,5,6 or none; 0 if align=7,8,9; or height-1 if align=1,2,3

those "margins" can be bypassed and set in the script too in combination with align parameter, for example : "x=video_width -3, y=video_Height, align= 3"," and will need a few adjustments depending the font sizes used to get the best looking results but it's already looking quite good with align parameter doing a nice job calculating the text space needed.


Like you said it could be interesting to get an option for those values able to be returned to the user, since they are calculated already in this process a function to get those values will not hurt much i think and only offer more possibilities to the end user, after all avisynth+ is "+" so getting more options and possibilities on the top of the existing ones can only be a good thing for the end user, more choices, more control and possibilities ^^

Thanks a lot.

[TheFluff]

Accurately determining the bounding box for a given string when rendered with a given typographic configuration essentially requires rendering that string with all the typographic bells and whistles enabled (kerning, ligatures, font substitution, hinting, etc etc etc - typography is really complex). That's why drawing background boxes tends to be a part of the subtitle rendering itself.

What you want is essentially programmable text layouting, which is prrretty far out of the scope of Avisynth builtins. As previously pointed out, good tools for this purpose do exist - they just aren't intended for use in Avisynth script. See for example Aegisub's scripting module and the text_extents function.

I feel like I keep getting back to this piece of advice over and over on d9, but I'll repeat it again: even though you can do a lot with just Avisynth script, it may, at times, not be the right tool for the job. It is a domain-specific language designed for manipulating filter chains, and it's actually pretty good at that, but it's pretty awkward for pretty much everything else.

[StainlessS]

The Fluff is quite correct, but if you do intend to mess with avs,then a few links to subs threads.

http://forum.doom9.org/showthread.php?p=1658543

http://forum.doom9.org/showthread.php?t=170953

http://forum.doom9.org/showthread.php?p=1801206

https://forum.doom9.org/showthread.php?t=167877



EDIT: This vertically centers (instead of top of text vertically centered)

#Avisource("D:\avs\test.avi")
Colorbars.killaudio

Function SubV(clip c, string text, float "x", float "y", int "first_frame", int "last_frame", string "font", float "size", \
    int "text_color", int "halo_color", int "align", int "spc", int "lsp", float "font_width", float "font_angle", bool "interlaced", \
    bool "vcenter") { # http://forum.doom9.org/showthread.php?p=1628899#post1628899
    c
    GScript("""
        size=Default(size,18)
        vcenter=Default(vcenter,false)
        if(Defined(align) && Defined(lsp)) {
            if((align>=4 && align<=6) && vcenter) {
                    s=RT_StrReplace(text,"\n",Chr(10),Sig=false)
                    Lines = RT_TxtQueryLines(s)                     # Line count with/without final newline
                    y=(Height/2) - ((Lines-1) * (size + lsp/8.0) / 2.0)
            } else if(align>=1 && align<=3) {
                    s=RT_StrReplace(text,"\n",Chr(10),Sig=false)
                    Lines = RT_TxtQueryLines(s)                     # Line count with/without final newline
                    y=Height - ((Lines-1) * (size + lsp/8.0))
            }
        }
    """)
    Subtitle(text,x,y,first_frame,last_frame,font,size,text_color,halo_color,align,spc,lsp,font_width,font_angle,interlaced)
}

VC=True
#VC=False

S="The\nQuick\nBrown\nfox\njumped\nover\nthe\nlazy\ndog.\nThe\nQuick\nBrown\nfox\njumped\nover\nthe\nlazy\ndog.\n"

SubV(S,lsp=0,align=4,vcenter=VC)
SubV(S,lsp=0,align=5,vcenter=VC)
SubV(S,lsp=0,align=6,vcenter=VC)

VC=True


VC=False

Function SubtitleCharSet() { # http://forum.doom9.org/showthread.php?p=1693672#post1693672
    GSCript("""
        S=""
        for(y=0,15) {
            s=s+"$"+RT_Hex(y*16,2)+ " "
            for(x=0,15) {
                n=y*16+x
                c=(n<32) ? "." : Chr(n)
                s=s+c
            }
            s=s+"\n"
        }
    """)
    S
}

colorbars(width=360,height=512).Trim(0,-1)
s=RT_StrReplace(SubtitleCharSet,Chr(10),"\n")
Subtitle(s,font="Courier New",size=32,lsp=0)
return  Last

Fixed Font Courier New Character Set (fixed size both horizontal and vertical)


EDIT: Or a bit of both

Function SubV(clip c, string text, float "x", float "y", int "first_frame", int "last_frame", string "font", float "size", \
    int "text_color", int "halo_color", int "align", int "spc", int "lsp", float "font_width", float "font_angle", bool "interlaced", \
    bool "vcenter") { # http://forum.doom9.org/showthread.php?p=1628899#post1628899
    c
    GScript("""
        size=Default(size,18)
        vcenter=Default(vcenter,false)
        if(Defined(align) && Defined(lsp)) {
            if((align>=4 && align<=6) && vcenter) {
                    s=RT_StrReplace(text,"\n",Chr(10),Sig=false)
                    Lines = RT_TxtQueryLines(s)                     # Line count with/without final newline
                    y=(Height/2) - ((Lines-1) * (size + lsp/8.0) / 2.0)
            } else if(align>=1 && align<=3) {
                    s=RT_StrReplace(text,"\n",Chr(10),Sig=false)
                    Lines = RT_TxtQueryLines(s)                     # Line count with/without final newline
                    y=Height - ((Lines-1) * (size + lsp/8.0))
            }
        }
    """)
    Subtitle(text,x,y,first_frame,last_frame,font,size,text_color,halo_color,align,spc,lsp,font_width,font_angle,interlaced)
}

Function SubtitleCharSet() { # http://forum.doom9.org/showthread.php?p=1693672#post1693672
    GSCript("""
        S=""
        for(y=0,15) {
            s=s+"$"+RT_Hex(y*16,2)+ " "
            for(x=0,15) {
                n=y*16+x
                c=(n<32) ? "." : Chr(n)
                s=s+c
            }
            s=s+"\n"
        }
    """)
    S
}

colorbars.Killaudio
S=SubtitleCharSet
SubV(s,font="Courier New",size=24,align=5,lsp=0,vcenter=True)
return  Last

[raffriff42]

Originally Posted by Yanak

when for example you use "Align=5" the text is ( as far as i could test) perfectly centered for the width, same free space remains on right and left sides, same exact space in pixels, but for the height it is not perfectly centered, probably due to the interline space parameter of the font that is included in the calculation here

I believe that's due to the font descender ('y', 'p', 'q' etc) dimension -- there are no descenders in your example, but room is allowed for them nonetheless. Smarter type tools probably don't have this issue. Subtitle is for "quick and dirty" work. It was never meant for professional typography.

[StainlessS]

I think that top align, aligns top of enclosing box, (incl spacing)
Mid align aligns to base line (bottom of caps letter),
bottom align with bottom of enclosing box. (incl spacing)

Blankclip(width=128,height=128)
s ="yTEXTy"
s1="yTEXT"
s2="TEXTy"
Subtitle(s,x=0,y=0) # align top

X=Width/2  ZX=25
Y=Height/2
Subtitle(s1,align=5,x=x-ZX) # mid align
Subtitle(s2,x=X+0,Y=Y-14.5) # maybe @ default sz 14 to baseline and 0.5 relative center ie height/2 - 0.5(even number of rows) 
Subtitle(s,align=2) # bot align

BicubicResize(width*4,height*4)

M=Last.blankclip(width=32,height=4,color=$FFFFFF)
Overlay(Last,m,x=0,y=height/2-2)
Overlay(Last,m,x=width-32,y=height/2-2)
return last

Leastwise thats how it seems to me

[Yanak]

Hi,

@raffriff42,
yeah i spoke about those in the edit i added under one of the pictures, while i did not used those characters in the example it still won't be perfectly centered with them in the tests i made, might be in case of underlining text is used or something else like the possibility to add accents on capital letters for example like "É" or might be something else, i don't know.

Googling one of the parameters in the link you posted i found this https://msdn.microsoft.com/en-us/lib...(v=vs.85).aspx , plenty of parameters and some like seems to indicate it is possible to fine tune this and get more accurate results for the box, maybe "DT_BOTTOM" will bring better results, well i'm not coder but seems like there is a few options to mess with this.

@TheFluff,
i perfectly understand there is tools done specifically for this, and while it's not the main purpose of avisynth to do such things it's one of his features, a handy one, some things could be improved and offer new possibilities in this domain, like avs+ not long ago allowed the use of non installed fonts.

Here avisynth already have the code used in it to calculate this for the align functions, it already calculates the text sizes, adding a little extra option able to return those already calculated results to the end user would be nice and offer more possibilities, that's all i am saying.

@StainlessS : i will try all those scripts tomorrow and also try to understand them, you guys amaze me with the creativity you have and how all this seems simple for you, myself i struggle a bit more with all this and work more by trial and error. Will spend time tomorrow processing some tests with all this, thanks a lot for the help and scripts, really appreciate all the effort you guys done.

Again thanks to all for all this, i guess now i can only hope one of those days someone will find the idea of making an option returning the already calculated text size using the already present code for this in avisynth might be something that could offer some nice options and worth the effort.

Thank you all.

[raffriff42]

Here's my take, building on poisondeathray's idea.

LoadPlugin(pathBase + "AutoCrop\AutoCrop.dll")

fontface="Arial Bold Italic"
fontsize=32
textcolor=color_yellow
halocolor=color_blue
padding=4
x_adj=6 ## fudge factor
y_adj=6 

msg="""
In the beginning the Universe was created.
This has made a lot of people very angry
and been widely regarded as a bad move.
"""
## next 2 lines needed only if using multiline style above
msg=MidStr(msg, 3)
msg=ReplaceStr(msg, Chr(10), "\n")

orig=Colorbars(pixel_type="YV12")
## /end user params

Blankclip(orig)
Subtitle(msg, 
\   font=fontface, size=fontsize, 
\   text_color=$ffffff, halo_color=$ffffff,
\   lsp=fontsize)

## (faster than mt_rectangle)
mt_expand.mt_expand.mt_expand
mt_expand.mt_expand.mt_expand
GrayScale
AutoCrop(mode=0, threshold=20)
## optional padding
AddBorders(padding, padding, padding, padding)
box=Last
#return box
bl2=Blankclip(box)

txt=bl2.Subtitle(msg, 
\           font=fontface, size=fontsize, 
\           text_color=textcolor, halo_color=halocolor,
\           x=x_adj, y=y_adj, lsp=fontsize)
#return txt

msk=bl2.Subtitle(msg, 
\           font=fontface, size=fontsize, 
\           text_color=$ffffff, halo_color=$ffffff,
\           x=x_adj, y=y_adj, lsp=fontsize)
## expand mask to 0-255 range (there are prob. better ways to do this)
msk=msk.Levels(18, 1, 232, 0, 255, coring=false)
#return msk

## output example 1: center on screen
#orig.Overlay(txt, mask=msk, 
#\           x=orig.Width/2-txt.Width/2, 
#\           y=orig.Height/2-txt.Height/2)

## output example 2: bottom-right
orig.Overlay(txt, mask=msk, 
\           x=orig.Width-txt.Width, 
\           y=orig.Height-txt.Height)
return Last