BYTE - avisynth.h

[WarpEnterprises]

Maybe two dumb question:

1) When and why was

unsigned char pixel_type;

changed to

BYTE pixel_type;

and

2) where gets BYTE defined because when I try to compile a plugin which uses a newer avisynth.h which contains BYTE this type is not recognised (VC++6.0)

[Guest]

Get both internal.h and avisynth.h from the distribution and put them in your build directory. Then include just internal.h instead of avisynth.h.

[WarpEnterprises]

Thanks, now I found that the thing about internal.h has already been discussed.
It compiles now, but where gets BYTE defined?

[Guest]

Have a look at the headers that internal.h includes. :)

SPOILER: windef.h

[scmccarthy]

I was wondering how BYTE slipped into the avisynth.h also. I don't include windows.h. It is not nesessary except for that one define. I never worked with Visual anything, but I assume that windows.h is included automatically. Not having the define would make avisynth progamming more accessible to those of us who have to create our progamming environments with batch files.

In other word, don't we want avisynth filters to be programmable with any windows c++ compiler? I use Microsoft's cl.exe and link.exe, literally the minimum requirements. Well, also ml.exe.

It works fine after the BYTEs are replaced and ASSERTE is defined to void. Any experienced programmer should know how to deal with it, but still, it is mildly worrying. What if something truly incompatible slipped through?

Stephen

[Richard Berg]

When I split up the code for 2.0, some parts used BYTE and others 'unsigned char'. I decided to standardize on BYTE since it was clearer and easier to type -- perhaps a typedef to 'Pixel8' or something would be even clearer, but I doubt anyone's confused at this point...

ASSERTE has been around much longer still. If you don't want to include <windef.h> (though I can't see what advantage there is to excluding it), you should still paste in the definition IMHO -- it's quite a useful macro for debug-builds.

[scmccarthy]

Richard -

I don't know the advantages of including windef.h. My point is if it needs to be there, it should already be included with an include statement like this:

#include windef.h

BYTE should not have slipped in without making a concious desision to include the definintion. It takes me aback to realize that when I tried to compile YV12_example.cpp, I had to figure this out on my own.

When I added a define BYTE statement in avisynth.h I got an error saying that BYTE is defined twice. If I did not add it I got an error saying BYTE is not defined. So either I needed to figure out which windows header I needed to include that was not included already (and I could not decide which one) or replace every instance of BYTE with unsigned char (don't forget to check match case).
unsigned char is c++ code, BYTE is not, therefore using unsigned char (albeit with more to type) keeps the avisynth include file compatible with as many development environments as possible.

So the issue for me is whether it is a priority to make compiling filters as accessible to newbies as possible or not.

BYTE is fine with me as long as the definition is included by default. I know it does not make any difference to the final dll, but I would prefer it not to be there since defs can create problems that show up in some programming environments and not in others.

Stephen

[Richard Berg]

Yes, I agree it would be cleaner still to do the necessary #defines ourselves. However, that will generate spurious warnings for people who do have <windef.h> included. As Donald noted earlier, this task is currently delegated to internal.h instead of avisynth.h. Now that we're making the latter the "official" Avisynth interface (releasing it with a different license, etc.), moving all necessary inclusions to there may be worth examining.

In my mind the issue this actually raises is the total lack of a plugin SDK beyond the original version 0.xxx doc on BenRG's site. I've only compiled external plugins once or twice, but if someone more familiar with it wants to write up a guide then I'd fully support that being a major goal for v2.5 (though we should wait until the interface is finalized in the beta).

[scmccarthy]

When I wrote a filter that displays 75% color bars in YUY2 and now YV12, I ran into difficulties trying to remove IClip as an argument. With only Ben's site to go on, there is no model for writing any source filter except the 281002 source.cpp, which by the way everyone should consult when they have questions. Dividee had my back this time, he seemed to understand the problems someone will have when faced with this problem for the first time. I needed to add this line in the class descritption at the top:

vi.audio_samples_per_second = 0;

You almost had me with the 'what if the compiler does not have windef.h' argument. However, no windows c++ compiler can fail to have it. AviSynth is only compiled under windows, right? I have used Borland, MinGW32, and WINDDK. They all have it. It slows the compiler down to slog through this stuff unnessesarily, but if you add the define... well someone should try it first of course.. If you add the define and it works, everyone has that file. Also, the other method of including internal.h, if that were the way to go Sh0dan would be distributing it with avisynth.h.

As a principle, I still say avoid things like BYTE, because it is not an official c++ keyword. unsigned char is unambiguously defined already.

Stephen

[WarpEnterprises]

1) What's 281002 source.cpp ?

2) Some SDK and/or some template filters would indeed be good.

The program is really complicated (because of the C++ interface,...)
You have to learn:
- using the compiler
- nearly the complete C++ functionality
- an nearly undocumented interface (to AviSynth)
- struggle with many defines, similar names,... which for me (I'm no programming newbie, although just - being forced by AviSynth :) - starting to explore C++)
- debug a DLL (much more difficult than a standalone prog)
- understand the used formats (video, audio)

I'm sure it could be good for the AviSynth future if the possibility of developing filters is easier (would be a feature that is not available to "alternative products").

And getting 2.5 to the masses will need MUCH propulsion power - the advantages so far (a little more speed, a color format more to learn, complicated audio handling) are not convincing (not meaning to me of course but to the everday user)

[edit] and those dual version DLL thingie from the other thread is an absolute MUST for 2.5

[scmccarthy]

1) What's 281002 source.cpp ?

Although AviSynth 2.5 alpha has been updated since then, the newest available complete source code is from October 28, 2002, hence the name 281002. It is easier than you might think to find the code for specific functions in the sources. Although I started out by using grep, I soon realized that the names of all the functions in each cpp file are near the top, not in the comments, but just declared before they are defined.

Go to the AviSynth 2.5 alpha page and download the sources link just below the dll and header link.

Right now, I am trying to write a series of very simple, short filter that can serve as an example of every aspect of filter writing, starting with playing with the YV12 color space, converting to YUY2 both with and without interpolation, RGB, greyscale, color bars in YV12 and YUY2 instead of RGB, and next I'll try a single purpose BOB, of course aimed at avoiding the shimmering effect. Today I linked an object file from c and from masm. I think that is better than using __asm. The point being that eventually I might put together an SDK based on many more options than Invert supplies or at least explain how to do more.

The third point is that you can make dlls that work for both 2.0 and 2.5: eventually 2.5 will be the only version of AviSynth that exists. I am concentrating on the long term in case my AviSynth progamming skills are not up to snuff until 2.0 is just a memory anyway. This switch to YV12 is coming at a time when there are plenty of programmers who understand color space. It is not any harder to program for 2.5 than 2.0 to someone new to AviSynth programming. Gradually everyone will find compelling reasons to switch to 2.5. Anyway, I am banking on 2.5 being the only version available sometime in the future. On the other hand, the transition period is probably long enough to justify the effort to learn how to pack both versions in one dll. It is certainly a neat trick that might have other applications anyway.

Stephen

[Wilbert]

Anyway, I am banking on 2.5 being the only version available sometime in the future.
I don't see that happen in the near future. There are some import plugins (i'm thinking about IPCSource), that won't work for 2.5 yet. I hope Edwin will change that, but who knows ...

[Richard Berg]

As a principle, I still say avoid things like BYTE, because it is not an official c++ keyword. unsigned char is unambiguously defined already.
Both of the following are perfectly standard:

#define BYTE unsigned char
typedef BYTE unsigned char

In fact, providing such definitions is a net boon to portability. There's a good reason just about every platform provides typedefs for sized integers -- the C++ primitives vary! Thus in Win32 you have Int16, Int32, and a bunch of __otherstuff; GNU/Linux uses the C99 definitions like int16_t; even our friend VirtualDub uses Pixel32 in place of int.

In truth, we don't have to worry much about platforms where sizeof(short) == sizeof(int), since they're all really old, and the code where >32-bit registers may come into use is already mostly in assembly. (Pointers are another matter, though -- you can't assume sizeof(void*) == sizeof(int)!) And in this particular case, char is a bit unique in that it's fixed at 8 bits in the C++ spec. Nevertheless, given how much video code depends on fixed-width chunks of memory, it's a really good idea to abstract away from primitive types.

[scmccarthy]

@Richard

In the AviSynth source code as a whole, Richard, using BYTE is not a problem. The difference is that all the needed headers and their definitions get pulled in when the AviSynth dll is compiled. But by itself, there is an undefined token floating around in avisynth.h that does not belong there. Newbies get told to get the avisynth header and an example program and compiler them together, but it will not compile, and that is not fair to newbies. None of the explanations of what to do about it explains why it is there. At the top of the page, there where two questions, neither of which have been answered by anybody:

1) When and why was unsigned char pixel_type; changed to BYTE pixel_type;

Indeed, there is no good justification for it, so it is an unanswerable question. Except for when, I am curious about when myself, can anybody tell me when?

2) where gets BYTE defined (where does BYTE get defined) because when I try to compile a plugin which uses a newer (version of) avisynth.h which contains BYTE this type is not recognised (VC++6.0)

Well, the answer to this is that it should be defined within the avisynth header itself.

Theses are actual problems that can and should be fixed by changing avisynth.h.

Stephen

PS, the question 'when was it added' got my attention and when I realized that no one was going to answer it, then I jumped in.

[Richard Berg]

Like I said above, I did it sometime during last summer's reorganization. Before, different parts of the code used different types, and standardizing things seemed like a relatively easy way to improve the readability of what was then a very messy codebase. We've still got a long way to go in that department, but consider that I wasn't much more than a newbie at the time, so believe me the various names scattered everywhere was more confusing.

If you disagree with the reasons I gave for picking 'BYTE' then so be it, but don't say there's "no justification" right after I list several.

It should be clear from what I've written that I support (1) making avisynth.h directly compilable (2) creating better documentation for new developers, so I can't imagine what the conflict is. With any luck I'll have some time this weekend to help out myself.

[WarpEnterprises]

Thanks, for me the two questions are answered sufficiantly.

Maybe it should read more clearly somewhere in a filter SDK "learn the C++ basics first" which I'm doing now and which is much better than dumbly trying to compile CPPs ;)

if it only were easier...

[Si]

As a "dumb" C++ programmer, I'd just like to add the comment that if the gods themselves - Avery and Ben had not

a) Made it easy to "dumb" programme with their simple SDK's and simple examples (which at the time seemed very, very complicated)

and

b) Not assisted idiots like myself (and others not so thick) in learning C/C++ basics

then the whole shebang of VirtualDub and Avisynth filter writing would not have taken off.

I haven't a clue what you guys are on about (Well maybe a little bit) but us not so clever ones just would like the new 2.5 to be compiled with VC++, so we can do our


include Avisynth.h

for y = 0 to h
for x= 0 to w
pixel = some wonderful brilliant idea f(Pixel)
next x
next y

filters and then convert it to SSEX III (or whatever) at a later stage when we've had the brain transplant :D ;) :confused:


regards

Simon

[Richard Berg]

I know the feeling -- I was there once, and not as long ago as you might think :) I got tomorrow off, should be able to get some work done in this area.

[scmccarthy]

Yes, your right, I am picking at this.

Ben's clever use of C++ object oriented abstraction made it very easy to write filters for AviSynth. I actually get this stuff:)

@Sh0dan -

It seems I must add:

#include <windef.h>

to avisynth.h. But if that is all that is nesessary, then why not add it to the copy you distribute?

Stephen

[trbarry]

Ben's clever use of C++ object oriented abstraction made it very easy to write filters for AviSynth. I actually get this stuff

I am awed by the power and beauty of it.

However I'm also one of those programmers you've probably heard of who just doesn't think object oriented that way, and probably never will as a native language, so I have sort of a read-only understanding of how all that works.

I have to puzzle through it again each time, but I still appreciate it. ;)

- Tom

[scmccarthy]

Tom -

No, no, I agree. I need to go back to the drawingboard and relearn the diff between : and ::. That is an absolute must. You can pretty much write anything you want to write in C. So even though I studied C++, I never started to use it because I did not need it.

C++ is supposed to be good at hiding implemantation details so other programmers can write simpler code on top of it. AviSynth has been successful and leveraging that strength of the C++ programming paradigm.

I have seen examples of oop for assembler, too. So you could use it too.

Stephen

[WarpEnterprises]

A little follow-up question (because I think a seperate thread is a waste):

What's the difference between
__cdecl
__stdcall

regarding the use in the AviSynth-environment?

What's written in MSDN

Keyword Stack cleanup Parameter passing
__cdecl Caller Pushes parameters on the stack, in reverse order (right to left)
__stdcall Callee Pushes parameters on the stack, in reverse order (right to left)

is not really useful in regard when is which to be used.

[Guest]

Originally posted by WarpEnterprises
A little follow-up question (because I think a seperate thread is a waste):

What's the difference between
__cdecl
__stdcall

regarding the use in the AviSynth-environment?

What's written in MSDN

Keyword Stack cleanup Parameter passing
__cdecl Caller Pushes parameters on the stack, in reverse order (right to left)
__stdcall Callee Pushes parameters on the stack, in reverse order (right to left)

is not really useful in regard when is which to be used. The default for C language is _cdecl, in which the caller pushes the parameters and removes them (cleanup) when the function returns. This allows for a variable number of parameters to a function. But it is inefficient because the cleanup code is duplicated at each call instance. More efficient is the _stdcall, where the callee cleans up the stack, but then you can't have a variable number of parameters.

The caller always has to push the parameters, so your quoted text above is wrong.

[sh0dan]

Regarding BYTE, I'll change anisynth.h to include the library - I have a changed version at home, that doesn't give any errors/warnings with older filters - I'll upload it ASAP.

[Richard Berg]

Aha, so that's why I was getting weird CVS messages when I tried to check in my fix. Now that it's re-updated, merged, and committed, I invite people to look at avisynth.h v1.3.2.22 (http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/avisynth.h?rev=1.3.2.22&only_with_tag=avisynth_2_1&sortby=date&content-type=text/vnd.viewcvs-markup).

The new license is obviously very important -- I think we had agreement here (http://forum.doom9.org/showthread.php?s=&threadid=32863&highlight=license), though comments are of course welcome. The included draft allows closed-source development based off anything defined in avisynth.h, but perhaps we want this exemption only for the IClip interface (filters) as in the above thread? I.e., require GUIs that hook into the DLL (not script-writers like GKnot and VDubMod), tools like Link2, etc. to be open-source?

Second, I noticed that sh0dan included <objbase.h> in his update, which I originally didn't. This file defines the COM interface, which is actually an idea I'd been toying with. The standard 'C' interface exported by the DLL is rather limiting -- we want to allow the creation of GUIs and such in VB, Delphi, etc., but you can't use C++ types like the ones I considered adding for filter info. Needless to say this problem cropped up more than a decade ago, and Microsoft's answer was COM. I wish I knew more details on how we'd actually implement such a thing...perhaps sh0dan knows.

The remaining issue are the sized types. Above I claimed they were important so as to abstract away from platform-dependent types. As I copied over the definitions from internal.h to the avisynth.h version linked above, however, I noticed that our types are no better! I'm all for defining names for 8-bit ints, 32-bit ints, etc., but neither the VirtualDub types nor the Win32 ones are portable at present.

Microsoft has 64-bit versions of Windows, so we'd probably be safe with using their types (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/win64/win64/the_new_data_types.asp) as long as we used up-to-date MS compilers, but they extend the primitive types differently from everyone else. In Win64 pointers are 64-bit, of course, but 'long' is still the same as 'int'. GCC, on the other hand, makes 'long' a 64-bit type. Ick.

So...

// Raster types used by VirtualDub & Avisynth

// this is ugly, possibly incorrect or nonportable, and only
// used by Tweak...votes to remove?
#define in64 (__int64)(unsigned short)

// This doesn't handle pointer types...hopefully they'll
// automagically resolve if the API interfaces are kept straight...
#ifdef MS_32
typedef DWORD Pixel;
typedef DWORD Pixel32;
typedef BYTE Pixel8;
typedef void Pixel64; // 64-bit code would be in #ifdefs; force
typedef INT PixCoord; // type error if accidentally used
typedef INT PixDim;
typedef INT PixOffset;
#elif MS_64
typedef DWORD32 Pixel;
typedef DWORD32 Pixel32;
typedef BYTE Pixel8;
typedef DWORD64 Pixel64; // only at Microsoft could the meaning
typedef INT32 PixCoord; // of 'doubleword' be ambiguous
typedef INT32 PixDim;
typedef INT32 PixOffset;
#elif GNU_32
typedef unsigned int Pixel
typedef unsigned int Pixel32;
typedef unsigned char Pixel8;
typedef void Pixel64;
typedef int PixCoord;
typedef int PixDim;
typedef int PixOffset;
#elif GNU_64
typedef unsigned int Pixel
typedef unsigned int Pixel32;
typedef unsigned char Pixel8;
typedef unsigned long Pixel64; // sizeof(long) > sizeof(int)
typedef int PixCoord;
typedef int PixDim;
typedef int PixOffset;
#endif

...and then change all those nasty BYTEs to Pixel8??

[scmccarthy]

...and then change all those nasty BYTEs to Pixel8??

There is nothing wrong with BYTE as long as it gets defined within the sources that use it.

It looks good now, looks like it is going to work, but that looks like a lot of types! I have to memorize every type in order to understand source code. It is easier when there are less types to memorize.

I see the value, importance, and nessesity in specifying the exact number of bits in the ints we are using, we all wish C had started out with rigerous definitions in the first place. Beyond that, however, it's six of one, half a dozen of the other whether all these typedefs really make the sources more readable or not.

Stephen

[Richard Berg]

Those types (except Pixel64) have been around (in internal.h) as long as I've seen the Avisynth source. Like you say, I wish C had just made fixed-width primitives.

It deserves mention that worrying needlessly about redefining them again in preparation for 64-bit compatibility is, to say the least, overengineering. If you haven't noticed, that's a common fault of mine -- when I was hashing out the details of (yet another) new P2P protocol with a fellow codemonkey last week, he went as far as to call our tendency "delusions of grandeur" -- but anyway I thought I'd post everything that's on my mind while I can, since I'll be offline for a couple weeks starting Saturday. On that note, more seriously, if there's anything sh0dan et al. want me to work on before the middle of January, let me know soon :devil:

[SansGrip]

Instead of including windef.h in avisynth.h, it might be preferable to simply


typedef unsigned char BYTE;
in there instead. According to Microsoft redeclaring types isn't an error provided they refer to the same "source" type. This way you avoid the compile-time cost of including windef.h.

As far as ASSERTE goes,


#include <assert.h>
needs to be in avisynth.h, and any calls to ASSERT() or ASSERTE() should be replaced with assert(), since the latter is portable to almost any platform.

[sh0dan]

Originally posted by Richard Berg
On that note, more seriously, if there's anything sh0dan et al. want me to work on before the middle of January, let me know soon :devil:

Your avisynth.h seems very nice. Unfortunately I don't know that much about the windows internals, and tend to stay away from it.

What would be great, would be if you have the time to rework the cache - it seems very inefficient in the current implementation.

I have implemented a fast "cache-hint" routine, but it's not very good, though more efficient compared to the original.

I can't find any good way of implementing it. The current way of reusing videframe buffers is good, but it should be extended so that frames are marked for reuse instead of just being put into a linked list.
On memory fragmentation AviSynth should be able to deallocate and reallocate buffers, so we can be sure the memory usage doesn't just explode.

I'll get back to you with a better explanation of the functional ideas, but you probably already have some. :)

[Richard Berg]

This way you avoid the compile-time cost of including windef.h.
Needn't worry about that, it's short and totally linear. It does need to go inside the #ifdef, now that you remind me, since it's not standard.

any calls to ASSERT() or ASSERTE() should be replaced with assert()
That's precisely what the current code does when not using an MS compiler. (When you are, I think it's best to stick with <crtdbg.h>, if only because all the existing code assumes its presence.)

@shodan -- been a long time since I've thought about the cache, but I agree it's about time we revamped it. I'll start fishing for ideas; if you have any good insights, start a thread.

[sh0dan]

Edit: [Moved to new thread]

[SansGrip]

Originally posted by Richard Berg
Needn't worry about that, [windef.h is] short and totally linear. It does need to go inside the #ifdef, now that you remind me, since it's not standard. I suppose there would be considerably bigger things to worry about if porting to a non-Wintel platform than windef.h :D.

That's precisely what the current code does when not using an MS compiler. Oh, ok. I need to read the source again :). I just assumed there was some big problem since this is an awfully long thread about a typedef.

(When you are, I think it's best to stick with <crtdbg.h>, if only because all the existing code assumes its presence.) I've always had to add #include <crtdbg.h> to any avisynth.h files I've downloaded. Maybe that one should be included by default too.

P.S. How's the power situation down in Durham now?? My web server went down for days and it's just outside Raleigh...

[sh0dan]

>I've always had to add #include <crtdbg.h> to any avisynth.h files I've downloaded. Maybe that one should be included by default to

It is (now) :p

[SansGrip]

Originally posted by sh0dan
It is (now) :p heheh I love open source :).

[WarpEnterprises]

I tried to compile my mpasource.dll with the 2.5 avisynth.h which includes windef.h
It didn't work - I had to include windows.h (which itself includes windef.h).
Do I have something special - does windef.h work at your compiles?