OpenGl in Avisynth!

[tin3tin]

Here (http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/) it's descriped how to use OpenGl in Avisynth. Youtube example here (http://www.youtube.com/watch?v=64KJ3jvIcpE&eurl=http://royshil.blogspot.com/2009/01/opengl-for-avisynth.html).

Unfortunately there is no downloadable plugin file, but the whole idear of adding OpenGl to Avisynth sounds great. Maybe someone with the right plugin writing skills could be inspired to take this even further? :)

(I know... one of these days I need to take the leap into plugin writing myself).

[Leak]

Unfortunately there is no downloadable plugin file, but the whole idear of adding OpenGl to Avisynth sounds great. Maybe someone with the right plugin writing skills could be inspired to take this even further? :)
Well, OpenGL is just a rendering API that by itself doesn't do anything in particular. The blog entry lists code for an AviSynth filter to set up OpenGL to work on the frames the filter gets from AviSynth and how to return back the processed frames, but it's not doing any filtering per se - so it's just a building block, but no full filter.

np: Gold Chains & Sue Cie - Show Us Your Heart (When The World Was Our Friend)

[tin3tin]

Do you think that this is a valid/okay way to add 3D manipulation of clips in Avisynth? (I guess the glut stuff can't be included in a plugin, and it might be a bit dated?) Or do you think there are better ways?

[tin3tin]

Another interresting thing in this line of thought here. (http://renegadepixels.in/carmine.html)

This is a Blender 3D .py script which can make Blender 3D export to a OpenGl friendly format(For every scene exported, a corresponding .C and .H file is generated).

I imagine that this combined with the stuff above it would be possible to use Blender 3D for making 3D transitions in Avisynth.

[tin3tin]

Roy has released the sourcecode:
http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/
:)

[tin3tin]

Roy has released the sourcecode:
http://www.morethantechnical.com/200...-for-avisynth/ (http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/)

I've tried several times to get this to work - but still unsuccesful. Could I get one of you C++ enlighted people to check if this code is working, and if it does give me a few pointers on how to make more 3D transitions this way in C++/Avisynth?

[Youka]

This thread is old, but maybe someone helps this (http://youka.de/archives/182).
Create this object in your plugin constructor, use InitGL at beginning of GetFrame and CloseGL at the end. Destroy object in plugin destructor.
No need for GLUT.

Works for me in my plugin avisynth + opengl + lua (similar to Overlua (http://forums.animesuki.com/showthread.php?t=52757)).

[tin3tin]

Looks very interesting, but I'm pretty ignorant when it comes to c++. :(

[Youka]

Works for me in my plugin avisynth + opengl + lua (similar to Overlua (http://forums.animesuki.com/showthread.php?t=52757)).
See here (http://youka.xobor.de/t98f13-Tutorials.html). It's just at the beginning but enough.

[Wilbert]

Can't wait to try it out. Could you make your plugin open source so we can learn from it?

[tin3tin]

Wow, looks brilliant! This must have taken a lot of time to code! Nice examples :)

Is it possible to process two videoclips in the lua script - for making transitions? Or devide a clip into two clips and make a transition between them?

[Youka]

It's an avisynth plugin because it should use avs functions, like video splitting and merging, too. You can use FLuaG one time for saving video frames to .tga picture files and next round to use this pictures as video sequence or textures in another FLuaG call with another video.

[tin3tin]

I'm trying to look into the Gl functions.

Just a quick question: Is it possible to use a video as a texture on ex. a 3d box?

What do you use for testing the lua scripts(as you wrote somewhere AvsP crashes with there are errors in the Lua script)?

[Youka]

Save a video as .tga pictures and use picture-for-picture as texture for all cube sides - yes, it's possible.

For testing, i'm using AvsP but are careful. MPC hasn't this problem, because it allows error throwing during rendering too (AvsP just at the initialization of the plugin, bad programming fail).

[tin3tin]

Save a video as .tga pictures and use picture-for-picture as texture for all cube sides - yes, it's possible.
That's proberly going to take me some time to figure that stuff out. I guess it's not possible to just add the video to the memory and use it from there(would be faster)? Does this (http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/) make any sense to you?(I have no clue)

Are there any tutorial/examples on that process you mention of working with video on 3d objects?

[Youka]

I guess it's not possible to just add the video to the memory and use it from there(would be faster)?If i would add functions for video importing, encoding, etc. i could write a complete video editing program, but FLuaG is just a video filter with a powerful graphic library.

Does this (http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/) make any sense to you?(I have no clue)I know it. One way to write an avisynth plugin with OpenGL drawing on every frame (all in C++).
An example with the video on the cube in FLuaG:local w, h = flGetVideoWidth(), flGetVideoHeight() --Video resolution
local d = 150 --Cube size

--Render function
function Video_Cube(frame_i)
--Initialize 3D room
flInit3D(FL_ORTHO, -1000, 1000)

--Just draw the front of polygons (=faster rendering)
glEnable(GL_CULL_FACE)
glFrontFace(GL_FRONT)

--Video to texture
glEnable(GL_TEXTURE_2D)
local tex = glGenTextures(1)
glBindTexture(GL_TEXTURE_2D, tex[1])
glTexParameter(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR)
glTexParameter(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR)
glCopyTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 0, 0, w, h, 0)

--Clear frame to black
glClearColor(0, 0, 0, 0)
glClear(GL_COLOR_BUFFER_BIT)

--Drawing texture with full colors
glColor(255, 255, 255, 255)

--List drawing of one rectangle with video on it
local base = glGenLists(1)
glNewList(base, GL_COMPILE)
glBegin(GL_QUADS)
glTexCoord(0, 0); glVertex(-d, d)
glTexCoord(1, 0); glVertex(d, d)
glTexCoord(1, 1); glVertex(d, -d)
glTexCoord(0, 1); glVertex(-d, -d)
glEnd()
glEndList()

--Set cube rotation and position to center
glTranslate(w/2, h/2, 0)
glRotate(frame_i/2, 1, 1, 0)

--Draw front
glPushMatrix()
glTranslate(0, 0, d)
glCallList(base)
glPopMatrix()

--Draw back
glPushMatrix()
glTranslate(0, 0, -d)
glRotate(180, 0, 1, 0)
glCallList(base)
glPopMatrix()

--Draw right side
glPushMatrix()
glTranslate(d, 0, 0)
glRotate(90, 0, 1, 0)
glCallList(base)
glPopMatrix()

--Draw left side
glPushMatrix()
glTranslate(-d, 0, 0)
glRotate(-90, 0, 1, 0)
glCallList(base)
glPopMatrix()

--Draw top
glPushMatrix()
glTranslate(0, -d, 0)
glRotate(90, 1, 0, 0)
glCallList(base)
glPopMatrix()

--Draw bottom
glPushMatrix()
glTranslate(0, d, 0)
glRotate(-90, 1, 0, 0)
glCallList(base)
glPopMatrix()

--Finish
glDisable(GL_TEXTURE_2D)
glDeleteLists(base, 1)
end

--Register render function for the whole video
Register("Video_Cube", 0, flGetVideoFrames())Try it.

Are there any tutorial/examples on that process you mention of working with video on 3d objects?Decode a video into memory and load frame-per-frame as texture for an OpenGL 3D object. It's really simple.

[tin3tin]

WOW WOW WOW. That's super powerful!

Maybe I could just make two lua scripts(for transitions). Ex.
- One with the first clip processed (on one side of the box)and the rest of the image transparent.
- Process the second clip (on the other side of the box) with the rest of the image transparent.
- Then overlay those two together on a background.

[tin3tin]

Did a quick test with your Lua script and plugin in DVD slideshow GUI:
(I wish that this forum would allow posting of videos directly in the thread)
http://www.youtube.com/watch?v=ofTm1A3GNN4

[Wilbert]

@Youka,

Could you make your plugin open source so we can learn from it?

[Youka]

Comes next time.

[Gavino]

Youka, thanks for this - it looks to have a lot of potential.
What would make it even more powerful and flexible would be the ability to pass in more than a single source clip.
Also the ability to pass parameters to the lua script, or some way of accessing avs variables inside it (eg something like get_avs_var("x")).
Are those a possibility for later versions?

BTW, I have discovered that the plugin will not load on earlier versions of Windows (one of my machines is still on Win 2000, and for various reasons I don't want to upgrade it). I expect this is because it has been compiled with Visual Studio 2010 - any chance you could release a version compiled with say VS2008 or the ICC compiler?
(Of course once the source becomes available, I can compile it myself.)

[Youka]

What would make it even more powerful and flexible would be the ability to pass in more than a single source clip.Thought about it, but didn't needed it before (picture-for-picture method for short clips was enough until now). Maybe later, maybe another one want to join und write it.
Also the ability to pass parameters to the lua script, or some way of accessing avs variables inside it (eg something like get_avs_var("x")).
Are those a possibility for later versions?Don't understand, why variables of the .avs script couldn't be written in .lua script directly. A bit too much luxury :)
BTW, I have discovered that the plugin will not load on earlier versions of Windows (one of my machines is still on Win 2000, and for various reasons I don't want to upgrade it). I expect this is because it has been compiled with Visual Studio 2010 - any chance you could release a version compiled with say VS2008 or the ICC compiler?
(Of course once the source becomes available, I can compile it myself.) Compiled with msvc++ 10 express. Forgot the connection to .net 4. Will change it next version.

[Gavino]

:thanks: for posting the source code.

Don't understand, why variables of the .avs script couldn't be written in .lua script directly. A bit too much luxury :)
Suppose I want to write a general Avisynth function for some effect, like say your cube example. Attributes like the cube size and rotation speed are hard-coded in the lua script file, whereas I would like to have these as parameters of my Avisynth function and somehow make them available to the lua code.

[tin3tin]

Just found some OpenGL code for making custom transitions for iOS.

A YouTube video showing the transitions: http://www.youtube.com/watch?v=69TbdU5uJvU&feature=player_embedded

I'm not that familiar with OpenGL code, but the gl code looks similar to FLuaG: https://github.com/Split82/HMGLTransitions/commit/a5294ec90ef132b33b38d3c264013d12c5fa5f1e#diff-3

Would it be insane to think that the actual transitions could be ported to FLuaG?

[Youka]

I'm not that familiar with OpenGL code, but the gl code looks similar to FLuaG: https://github.com/Split82/HMGLTrans...5fa5f1e#diff-3It's written in objective-c, so it calls OpenGL functions like the standard C api. FLuaG is in Lua, so C arrays are tables and pointers don't exist, but i tried to define all gl* and glu* functions similar to C to allow copy&paste of C code without to change a lot.

Would it be insane to think that the actual transitions could be ported to FLuaG? Yes, it would be. FLuaG has enough functions to overview (becomes more with newer OpenGL versions), so it wouldn't be good to add combinations of them for limited effects. A collection of code snippets as templates is a better idea.

[Mr VacBob]

You'll probably be better off in terms of future compatibility (and look more like WebGL) if you expose ES2 API instead of the old OpenGL 2-style stuff.

[tin3tin]

@ Youka
Please, keep us posted on you development of that great plugin you're working on and thanks for working so hard on it. :)

[Youka]

You'll probably be better off in terms of future compatibility (and look more like WebGL) if you expose ES2 API instead of the old OpenGL 2-style stuff.- glBegin/glEnd blocks are faster than arrays. Converting a lua table into a c array needs more time than the win of rendering speed.
- Display lists are important! Drawing convex polygons, like text, with tesselation again and again is slow.
- ...
- Summary: too many limits (see here (http://www.khronos.org/registry/gles/specs/1.1/es_cm_spec_1.1.12.pdf)).

@ Youka
Please, keep us posted on you development of that great plugin you're working on and thanks for working so hard on it. :)Firstly i'm using/testing it for karaoke effect creation :)
Currently i'll just fix what i find. I'll continue when i've more time.
Alternative:Would be fine if someone want to support development. Source i'd posted.

[Youka]

Plans for next FLuaG version (http://youka.xobor.de/t97f12-Download.html#plan).
Release and some example videos + scripts follow next week (i hope).

[tin3tin]

Good to hear that you're making progress. Looking forward to your next release. :)

(btw. you should proberly make a new thread on your plugin here at doom9 - your plugin can do so much more than OpenGl)

[Youka]

The next version + material will start in a new thread (wish of splitting this thread was ignored :rolleyes:).
Extend it to other libraries than OpenGL... i think graphic is one thing. For audio there could be another plugin.

[tin3tin]

For audio there could be another plugin.

Wow, madness. :devil:

[wonkey_monkey]

I've been trying to put some of the stuff in this thread into practice using freeglut (http://freeglut.sourceforge.net/) (a static build in this case), and I've almost, but not quite, got something that works:

#include "windows.h"
#include "avisynth.h"
#define FREEGLUT_STATIC
#include "glut.h"
#include <stdio.h>

class glsynth : public GenericVideoFilter {
private:
void display();
public:
glsynth(PClip _child, IScriptEnvironment* env);
~glsynth();
PVideoFrame __stdcall GetFrame(int n, IScriptEnvironment* env);
};

#define RND (float)(rand()*(1.0/RAND_MAX))

void glsynth::display() {
glViewport(0,0,vi.width,vi.height);
glClearColor(0.0f, 0.0f, 0.0f, 0.5f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glBegin(GL_QUADS);
glColor3f(RND,RND,RND);
glTexCoord2f(0.0f, 1.0f); glVertex3f(-0.5, -0.5, 0.0f);
glColor3f(RND,RND,RND);
glTexCoord2f(1.0f, 1.0f); glVertex3f( 0.5, -0.5, 0.0f);
glColor3f(RND,RND,RND);
glTexCoord2f(1.0f, 0.0f); glVertex3f( 0.5, 0.5, 0.0f);
glColor3f(RND,RND,RND);
glTexCoord2f(0.0f, 0.0f); glVertex3f(-0.5, 0.5, 0.0f);
glEnd();

glFlush();
glFinish();
}

glsynth::glsynth(PClip _child, IScriptEnvironment* env) : GenericVideoFilter(_child) {
if (!vi.IsRGB32()) env->ThrowError("glsynth: input must be RGB32");

char *argv[]={"foo","bar"};
int argc=2;

glutInit(&argc,argv); glutInitWindowSize(vi.width,vi.height);
glutInitDisplayMode(GLUT_RGBA);
if (!glutCreateWindow("")) env->ThrowError("glsynth: couldn't create OpenGL context");
}

glsynth::~glsynth() {
}

PVideoFrame __stdcall glsynth::GetFrame(int n, IScriptEnvironment* env) {
PVideoFrame dst = env->NewVideoFrame(vi);

unsigned char* dstp = dst->GetWritePtr();

display();

glReadPixels(0,0,dst->GetPitch()>>2,vi.height,GL_RGBA,GL_UNSIGNED_BYTE,dstp);

return dst;
}

AVSValue __cdecl Create_glsynth(AVSValue args, void* user_data, IScriptEnvironment* env) {
return new glsynth(args[0].AsClip(),env);
}

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
env->AddFunction("glsynth", "c", Create_glsynth, 0);

return "`glsynth' glsynth plugin";
}


If I pass it an RGB32 clip it will display a coloured rectangle as expected in VirtualDub, and the colours change as I step through the frames. But if I hit play in VirtualDub, the OpenGL window no longer updates and the VirtualDub video display flickers between old frames and black frames (a little like what you get if your plugin fails to write anything to a new frame). Anyone have any ideas what's going wrong?

It also crashed VirtualDub on exit but I'm not so bothered by that yet.

David

[Youka]

glFlush();
glFinish();
Try to remove them. Both together are senseless anyway and you don't have to flush data before glReadPixels.

glReadPixels(0,0,dst->GetPitch()>>2,vi.height,GL_RGBA,GL_UNSIGNED_BYTE,dstp);
From Avisynth, you get BGRA with RGB32, not RGBA.

char *argv[]={"foo","bar"};
int argc=2;

glutInit(&argc,argv);
...
if (!glutCreateWindow("")) env->ThrowError("glsynth: couldn't create OpenGL context");
It's dangerous to give a window an empty name and glutInit (http://www.opengl.org/resources/libraries/glut/spec3/node10.html#SECTION00031000000000000000) a static array pointer.

[wonkey_monkey]

From Avisynth, you get BGRA with RGB32, not RGBA.

Yeah, I tried GL_BGRA but the compiler didn't recognise it. Once I get it working properly I was just going to pass the textures backwards.

Giving the window a name doesn't make a difference and I'm not sure what you mean about the static array pointer - glutInit seems to work as it is since the plugin is correctly producing frames on a frame-by-frame basis. I guess there must be something different about the way frames are fetched when VirtualDub goes into "play" mode.

David