"Polygon" - polygon drawing, feedback requested

[wonkey_monkey]

Download: polygon0.1.zip
(http://horman.net/avisynth/download/polygon0.1.zip)
This is the very raw beginning of a plugin I'm working on for drawing high quality polygons (mainly with a view to being used as masks) in Avisynth, presented at this early stage mainly as a request for feedback.

It makes use of Windows's GDI+ API, so who knows what problems that might cause

Usage:

1. Define a polygon with pg_define:

poly=pg_define(x1,y1, x2,y2, x3,y3 ...)

# output is a specially formatted RGB32 clip
# floating point values ARE supported

2. Draw the polygon, passing a blankclip (or other RGB32 clip) to set output clip parameters:

pg_draw(blankclip(100,720,480),poly) # 100 frames, NTSC resolution

Improvements to come:


Constructors for specific polygons, e.g. regular, circle/ellipse approximations, etc
Support for non-polygon objects, e.g. lines (representing lines with thin polygons doesn't work well in GDI+)
Animation/tweening between polygons (not sure how well built-in animate will work with GDI+)
Oversampling to overcome deficiencies of GDI+ antialiasing
Other drawing modes, if any are especially useful, like overlaying an existing clip (would require colourspace conversions)
Output to other colourspaces
Translate/rotate/scale existing polygons
Smoothing of polygons
Built-in blurring


Feedback please!

[raffriff42]

Before you get too deep into it, know that Advanced SubStation Alpha (https://en.wikipedia.org/wiki/SubStation_Alpha#Advanced_SubStation_Alpha)already does drawing (http://docs.aegisub.org/3.2/ASS_Tags/#drawing-tags) (circles, rectangles, lines and beziers, but not polygons) plus much more.

[wonkey_monkey]

(circles, rectangles, lines and beziers, but not polygons)

I'm one up on them already, then ;)

[TurboPascal7]

Before you get too deep into it, know that Advanced SubStation Alpha (https://en.wikipedia.org/wiki/SubStation_Alpha#Advanced_SubStation_Alpha)already does drawing (http://docs.aegisub.org/3.2/ASS_Tags/#drawing-tags) (circles, rectangles, lines and beziers, but not polygons) plus much more.

Exactly this. It also does polygons because polygons are effectively a bunch of lines. In fact, you can even use Illustrator (https://github.com/TypesettingTools/AI2ASS) if you're into that kind of things. And even apply some motion-tracking (https://github.com/TypesettingTools/Aegisub-Motion) to it.

Truly, Aegisub is the best tool for creating overcomplicated masks and you should definitely try it before spending too much time writing your own tool.

Call now for 20% off.

[wonkey_monkey]

Okay, I see that vsfilter can take a .ass file and draw shapes, but it seems rather a convoluted way of going about it (especially if one were to start in Illustrator first) if all someone needs is a quick irregular mask over a region. It also can't, as far as I can tell, tween between polygons, which is another one of my key aims.

Should this have gone in Usage instead of Development?

[cretindesalpes]

Of course VSFilter can do this, but the plug-in is still useful to draw shapes directly from the script. Sometimes you just want to make a simple mask with known coordinates (or depending on script parameters) and having just to type a single line is really convenient (well, mt_lutspa often works too but I would not call it “convenient”).

If you want high quality polygon drawing, there is the Anti-Grain Geometry library. It did a good job last time I checked it (it was ~10 years ago), but there are probably other fine libraries around there too.

As improvement, make it work with YUV colorspaces out of the box too.

[poisondeathray]

Another good free tool that can interpolate and "tween" between keyframes with hand drawn masks, with bezier curves is cinegobs keyer . There is even a "portable" version so you don't have to install anything. It has avisynth import support, and you can export video directly (AVI , VFW formats) , or image sequences (PNG, BMP, JPG, TGA) . Even if you just needed a single static irregularly shaped mask it's pretty easy to use

Another old gem is claxa - which does basic motion tracking for rotoscoped masks. I think it's still available on sourceforge

And of course blender , but it's a bit complicated to use

But I don't want to discourage you; like cretindesalpes says - there are times where making the mask right in the avs script could be helpful, instead of having to export a mask from another program then having to import it back into the avs

[tin3tin]

Other projects which might interest you:
http://forum.doom9.org/showthread.php?t=161852
http://forum.doom9.org/showthread.php?t=167716

[TheFluff]

Windows's GDI+ API

2001 called and wanted their render engine back

No but seriously, just because Avisynth is 15 years old it doesn't mean you have to use 15 year old tech to write plugins for it. GDI+ makes absolutely no sense for a thing like this that doesn't even draw to the screen directly. There's a fucking ton of great 2D rendering libraries out there these days since 2D games are cool again. Please stop and think.

[wonkey_monkey]

Wow, harsh. I did stop. I did think. What I thought was "what methods do I have at my disposal that will do exactly what I want?" and the answer I came up was GDI+. I didn't have to download or compile anything extra.

GDI+ makes absolutely no sense for a thing like this that doesn't even draw to the screen directly.

Why not? GDI+ doesn't only draw to the screen. I create a Bitmap object and pass the AviSynth write pointer directly to it for it to use as its pixel data. The GDI+ call writes directly to the RGB32 clip. I get over 1000 frames per second for a 50-sided polygon at SD resolution (that's faster than an SD blankclip call), and 190fps at HD resolution (vs 210fps for an HD blankclip). That's for an anti-aliased and spline-smoothed polygon, by the way, and I haven't even started optimising yet.

I started out using a naive scanline polygon algorithm I'd (independently) came up with many years ago. When that got a bit tedious and the limitations became obvious, I stripped the code back to the bare essentials and from there the additional code to support and implement the GDI+ method amounts to about 15 lines, if that.

Please stop and think.

Think what? I don't understand what I've done that's so wrong. :confused:

[raffriff42]

You exist, that is enough to ruffle TheFluff. That's just his way.

Yes, GDI+ would work, and it is fast. I think DirectX or OpenGL (via FLuaG (http://forum.doom9.org/showthread.php?t=161852)) have better gradient support, if that matters to you. Built-in antialiasing too, I beleive. But starting from scratch is, well, be prepared for a major effort. It's the unknown unknowns (https://en.wikipedia.org/wiki/There_are_known_knowns) that will get you every time.

Anti Grain Geometry (http://www.antigrain.com/about/index.html) looks interesting, and FLuaG (http://forum.doom9.org/showthread.php?t=161852) even more so. I learned a lot in this thread, thanks guys. I will have to try them both.

But if I were doing this project I think I would write a thin wrapper for ASS. As TurboPascal7 mentions, use AegisSub (http://www.aegisub.org/) with the bundled ASSDraw3 to lay out your masks (yes it does polygons, I was wrong about that). Use Animate (http://avisynth.nl/index.php/Animate) to tween them.

[TheFluff]

I did stop. I did think. What I thought was "what methods do I have at my disposal that will do exactly what I want?" and the answer I came up was GDI+. I didn't have to download or compile anything extra.
No. Much like an electrical current (which doesn't stop and think either), you chose the path of least resistance.

Why not? GDI+ doesn't only draw to the screen. I create a Bitmap object and pass the AviSynth write pointer directly to it for it to use as its pixel data. The GDI+ call writes directly to the RGB32 clip. I get over 1000 frames per second for a 50-sided polygon at SD resolution (that's faster than an SD blankclip call), and 190fps at HD resolution (vs 210fps for an HD blankclip). That's for an anti-aliased and spline-smoothed polygon, by the way, and I haven't even started optimising yet.

I started out using a naive scanline polygon algorithm I'd (independently) came up with many years ago. When that got a bit tedious and the limitations became obvious, I stripped the code back to the bare essentials and from there the additional code to support and implement the GDI+ method amounts to about 15 lines, if that.

You haven't seen the limitations of GDI+ because you've barely even started coding yet. If I were to ask myself "when is GDI+ the appropriate tool to use", the only real answer would be some toy Win32 GUI app that is only ever going to run on Windows, will never need to draw anything complex and doesn't have to worry about performance.

GDI+ isn't hardware accelerated, it's limited to Windows only and it's actual drawing capabilities are pretty limited by today's standards. For your current needs - basically just a proof-of-concept prototype - that's all fine. The reason I'm telling you to stop and think is that you need to consider the future implications of choosing GDI+ now. If you, two years from now, decide that you want to port your by that point quite mature plugin to VapourSynth and introduce cross-platform compatibility, you'd have to redesign the entire thing from ground up. Same thing if you run into performance issues.

I don't know what exactly you want your plugin to be able to do nor how you like to code things but please consider alternative rendering libraries such as SFML (http://www.sfml-dev.org), Cairo (http://cairographics.org/), Skia (https://code.google.com/p/skia/) or even Direct2D (https://msdn.microsoft.com/en-us/library/windows/desktop/dd370990%28v=vs.85%29.aspx) if you insist on being platform-locked to Windows. Many of these use OpenGL, which you can also use directly yourself if you want, but it's probably a bit too low level for your needs.

[wonkey_monkey]

The reason I'm telling you to

Did you mean "asking"? I think you meant "asking".

stop and think is that you need to consider the future implications of choosing GDI+ now.

The future implications are not worth considering for such a trivial plugin. I know exactly what I require of it. I have no plans to use VapourSynth or move away from Windows. They are non-issues for me.

[jmac698]

Hi,
Thanks for making this plugin, good work! I've had need to draw shapes onto video myself, not for masks but for making calibration videos (colourbars, sinewaves, etc.). So I wrote a script to do it, and was also pointed to substation which I couldn't figure out. I think your approach is much easier.

DrawRect
https://www.doom9.org/showthread.php?p=1534248

I would like to be able to draw these shapes into proper YUV formats, myself. Substation and all these other libraries only do RGB, which is *still* a problem.

[wonkey_monkey]

Substation and all these other libraries only do RGB, which is *still* a problem.

Painting directly into YV12 or YUY2 would be a pain, I'm sure. YV24, perhaps, followed by a conversion... I prefer to think of YUV as an opening compression step. Everything starts and ends as RGB :)

Anyway, I've been sidetracked by another new plugin idea, but I will eventually return to this one. I was getting a little stuck anyway on how to provide animation.

[Reel.Deel]

Why not support Y8? As it is now all RGB channels are identical. Sure you can do something like pg_draw(...).ShowRed("Y8"), but wouldn't be faster/more efficient if you process/output one greyscale plane from the get-go rather than 3 identical ones? Just my 2 cents :).

[wonkey_monkey]

I don't think GDI+ has the ability to paint to an 8-bit single-channel image, so it's got to go to RGB in the first place. And what if someone wants YUY2, or YV12? And what if they want 16-235 instead of 0-255? Nah - leave the conversion to the user, I think.

[raffriff42]

Belated thanks - I suddenly found it very useful! Perfect for garbage mattes.

To make coordinate generation quick & easy, here's an AutoHotkey (https://autohotkey.com/) script to generate a list of alt+left mouse click coordinates, ready to paste into pg_define():/*
*************************************************
** MouseClickLogger: mouse click coordinates **
** for use with 'Polygon' filter for AviSynth **
*************************************************

NOTE:
alt-left clicks only
x,y origin at bottom-left
Polygon filter: https://forum.doom9.org/showthread.php?t=172388

USAGE:
- install AutoHotkey
- save this script anywhere as "MouseClickLogger.ahk"
- Run this script (double-click it)
- You will see a system tray tooltip showing it is alive
- Alt+left-click the mouse around the target window;
you will see a tooltip each time;
coordinates are appended to the Clipboard
- Right-click the tray icon to quit
- Paste coords into Polygon::pg_define
(don't include the trailing comma)
*/

#SingleInstance on
#Persistent

;;*************************************************
title:="MouseClick Logger"
about=
(
MouseClickLogger
by raffriff42, version 19-Feb-2017
)

OnExit, ExitSub ;

;;*************************************************
clipboard =

TrayTip , %title%,
(
Logging Alt+left mouse clicks to Clipboard

(right-click tray icon to quit)
)
return

;;*************************************************
ExitSub:
{
ExitApp ;
}

;;*************************************************
~!LButton::
MouseGetPos, x, y

clipboard = %clipboard% %x%`,%y%`,%a_space%
ToolTip, click_x`, y = %x%`,%y%

return

;; END ;;


And here's a minimal test script:LoadPlugin(pathBase + "Polygon\polygon.dll")
#@ function pg_define(float [, float]*) ## list of x, y pairs
#@ function pg_draw(clip T, clip P) ## T=template, P=poly

## any old source
ColorbarsHD
ConvertToRGB32(matrix="Rec709")
KillAudio

## coords from MouseClickLogger (I traced a star shape)
m=pg_draw(pg_define(
\ 656,203, 704,335, 846,336, 733,427, 776,564,
\ 656,484, 537,563, 578,428, 467,337, 608,335
\ ))

## Windows' coords are flipped vs. AviSynth
m=m.FlipVertical

## adjust for window title & border (TODO: refine)
## and general fine-tuning
m=m.Shift(-12, 7)

## overlay on black BG
BlankClip(Last).Overlay(Last, mask=m)
return Last

## shift a clip up-down and left-right (with sub-pixel precision);
## this results in repeated edge pixels
function Shift(clip C, float offh, float offv) {
C
BilinearResize(Width, Height, -offh, -offv, Width, Height)
return Last
}

[wonkey_monkey]

Neat!

[jmac698]

A few updates in the modern era, could you make the special format array compatible with the array implementation of stainlessS (if it isn't)? Also, I was directed to substation for my rectangle drawing script, but all the solutions I checked out weren't pure yuv, which I needed. Also drawing on top of an existing video is pretty useful. I think speed isn't such a big deal as drawing a few lines is nothing compared to other processing you'd likely do. Converting from rgb just won't work, the values don't map exactly so some colours or even luma of yuv can't be reached.

This is a good reminder actually for my next plugin.

[wonkey_monkey]

What is StainlessS's array implementation?

To be honest this script is waaay at the back of my mind, and the idea was never well-formed in the first place...

[StainlessS]

Rt_stats implements DBase and up to 3 dimensional arrays.
Mobile

[wonkey_monkey]

Zip file updated to include an x64 DLL:

http://horman.net/avisynth/download/polygon0.1.zip

[Reel.Deel]

Zip file updated to include an x64 DLL:

http://horman.net/avisynth/download/polygon0.1.zip

https://i.ibb.co/prNFx8h/yesawesome.gif

Thank you wonkey_monkey!