AVS Developers: AviSynth 2.5 alpha

[sh0dan]

As you might have noticed, there has been some talks of a new AviSynth, including YV12 support, float samples and multiple channels.

Now the time has come for a complete proposal for the revised framework. There are some radical changes. Sample handling has changed a lot, and YV12 support has also required some changes.
At the same time it has the intention to create a more extensible framework, so adding new features in the future shouldn't require too many changes.

Now, I call all the filter and core developers for feedback. This is not an enduser alpha test, because it is not ready for that yet!

I have created an alpha page, at http://cultact-server.novi.dk/kpo/avisynth/avisynth_alpha.html where you can read about the changes, and download the new avisynth.h, the complete source and a binary. I've created extensive documentation for developers.

Virtual Dub doesn't handle YV12 natively, so you need an YV12 decompressor. On my system it uses DivX 5.02. If your system doesn't support this, end your script with converttoyuy2(). This is also an issue that should be handled.

Until now, there isn't any speed improvement. The binary is a debug build, and will probably remain so some time. Speed is not an issue yet. Making it work has been the top priority.

So if you want to get working with planar YV12, test away!

[Wilbert]

Very nice Sh0dan, MarcFD and others!

Two dumb questions:

1) How do you make avi's in YV12? Did you make a function ConvertToYV12?

2) Is YV12 also supported by DivX4 or Xvid, or do I have to install DivX5?

Thx,

[sh0dan]

The easiest way to get YV12 is to use DirectShowSource, since it'll most often decode to YV12. Then you can use any DivX/Xvid file as source.
AviSource also decodes to yv12 on my computer, but that probably depends on your configuration.

I'll do a converttoyv12 soon - at least to get from yuy2 -> yv12 conversion.

Edit: If you compile your own version, AviSource will show which colorspace it's decoding to in debug mode.
(Late night edit): You can also force AviSource to decompress using YV12, by specifying it as parameter.

[HarryM]

Originally posted by Wilbert
Very nice Sh0dan, MarcFD and others!

Two dumb questions:

1) How do you make avi's in YV12? Did you make a function ConvertToYV12?

2) Is YV12 also supported by DivX4 or Xvid, or do I have to install DivX5?

Thx,


What I know...

All videocodecs work in YV12 natively. When you use YUY2's instruments (e.g. Avisynth v1.0x or 2.0x) actually, you must convert >>>

YV12 (e.g. MPEG-2 format) -> YUY2 (e.g. Avisynth) -> YV12 (e.g. XviD)

You must convert twice:
1. YV12 to YUY2
2. YUY2 to YV12

With Avisynth 2.5x you don't need conversion, additional YV12's videoprocessing routines are little quicker than YUY2's too.

[Wilbert]

@Sh0dan,

AviSource also decodes to yv12 on my computer, but that probably depends on your configuration.

Ok, thanks.

@HarryM,

All videocodecs work in YV12 natively

Ok, I didn't know this. Maybe MarcFD could add an additional argument to his MPEG2DEC3 which outputs to YUY2, YV12 or RGB32. (Maybe he's already working on this ...)

[Richard Berg]

I think we've talked about this before, but there should be an option we can pass to source filters to specify preferred format. In general, pointing to an encoded stream and telling the system (AviFile or DirectShow) "decode this" isn't deterministic the way users probably want.

Some other ideas we may want to support in the future and should thus at least make room for:
-Multiple streams per file. The most obvious case is having soundtracks in >1 language, but the AVI format actually supports up to INT_MAX audio and video streams in a single container file.
-More powerful scripting. Things like conditional primitives come to mind, as well as greater exposure of internal state to the scripting environment e.g. through reserved variables.
-Add your brainstorms here...if we're gonna be changing the interface in a non-compatible way, we should do it *once*.

Also, now that we got Ben's permission, we may as well use the occasion to revise the license statement at the top of avisynth.h to allow the creation of closed-source plugins, or perhaps just release it to the public domain.

[sh0dan]

@HarryM: Correct, but the problem is, that VDub cannot handle YV12 internal, and it therefore requests a codec to return the data as YUY2 or RGB.

So what happends is:
AviSource -> AviSynth -> Vdub -> Compressor
[YV12] [YV12] [YUY2] [YV12]
So what happends is a conversion, since Vdub cannot forward the data as YV12.
I'm pretty sure (although not 100%) that this is true for "fast recompress" (this is what i can see from the code). So until Vdub is modded for passing YV12 in "fast recompress", we cannot make the process 100% YV12.

My initial performance tests seems to confirm this. Although some filters most probably will work faster in YV12 mode, there isn't a big speed gain in YV12 (yet!). It does however work very nice with zoomplayer for instance, with very fast overlay, so using AviSynth for postprocessing is much better now!
Coding for planar is however a great thing, and we should also have a look at the ffdshow filters at one point.

[hakko504]

Has anyone tried to contact Avery Lee to see if he could implement YV12 support in VD?

[HarryM]

Originally posted by hakko504
Has anyone tried to contact Avery Lee to see if he could implement YV12 support in VD?

VirtualdubAVS could implement 'direct YV12 support' in future.

[sh0dan]

Originally posted by Richard Berg
I think we've talked about this before, but there should be an option we can pass to source filters to specify preferred format. In general, pointing to an encoded stream and telling the system (AviFile or DirectShow) "decode this" isn't deterministic the way users probably want.

I'm not sure I'm with you here - could you give an example?


-Multiple streams per file. The most obvious case is having soundtracks in >1 language, but the AVI format actually supports up to INT_MAX audio and video streams in a single container file.

Interesting. Could it be done by extending the return() statement, so it could do: return(video_lang_uk,video_lang_dk,video_lang_de). Is this supported by any software?

I have no idea how this would end up in main.cpp - so you'll have to help there.


-More powerful scripting. Things like conditional primitives come to mind, as well as greater exposure of internal state to the scripting environment e.g. through reserved variables.

I have no personal oponion on this - it would be great, but you'll have to kick in there as well - I'm not the best on this area.

I have actually complete forgot to mention the one feature, that isn't implemented yet, which is better support for GUI's, making an optional function to GenericVideoFilter, that returned information about the filter.


Also, now that we got Ben's permission, we may as well use the occasion to revise the license statement at the top of avisynth.h to allow the creation of closed-source plugins, or perhaps just release it to the public domain.
Agree! If you have the text, just commit it to the 2_1 branch.

[sh0dan]

Originally posted by HarryM
VirtualdubAVS could implement 'direct YV12 support' in future.
Yes - There are someone who has been playing with a modified Vdub (http://forum.doom9.org/showthread.php?s=&threadid=33586) that support MPEG2 input, decodes it to YV12 and passes it directly to the codec. It does however seem to be completely broken for Avi input.

I don't think it would be a very big modification, from what I can judge, but it'll probably still need some coding.

[Belgabor]

Originally posted by HarryM


VirtualdubAVS could implement 'direct YV12 support' in future.

This is certainly something for the planned combined project since this is surely (atm at least) beyond my coding ability.

Belgabor

[sh0dan]

I'm looking at yuvcodec.cpp - but I'm not sure how it interfaces with the rest of AviSynth. It does seem to be a passthrough filter, that only does minimal work of itself, besides forwarding compressor commands to the decompressor.

Maybe it no YV12 decompressor can be found, preview will be disabled, but I'll have to look much further into that. First of all I'll have to

[hakko504]

Originally posted by sh0dan
I'm looking at yuvcodec.cpp - but I'm not sure how it interfaces with the rest of AviSynth. It does seem to be a passthrough filter, that only does minimal work of itself, besides forwarding compressor commands to the decompressor.

Maybe it no YV12 decompressor can be found, preview will be disabled, but I'll have to look much further into that. First of all I'll have to press the right button? :D :cool: :D

[sh0dan]

... get some more time perhaps? ;)

I think I meant, I'll have to make it compile, it seems to be missing some ogg stuff (ogg.h). Strangely enough I can't really see where to put it.

Edit: Just tested on a plain Vdub, and it seems like it's more Dub.cpp that's the interesting one. It does seem to require some tweaking, for which I don't have the time now - making the inputsource on fast repack accept YV12 isn't enough.

[vlad59]

@Sh0dan

At first : great work, thanks a lot for this huge work !!!!

Next, I think I would be interesting to add to your "alpha buil page" some info about the YV12 format especially that horizontal size must be mod 4 (IIRC ???), vertical size mod 2 or 4, .....

Just my 2 cents

[sh0dan]

You're right - I probably should. But to sum it up:

In YV12:
- Width must be mod 2 (as YUY2)
- Height must be mod 2.

The image data is divided into three planes. One luma, and two chroma planes. You operate on these planes seperately.
The two luma planes are half the size of the luma planes (in width and height).
That means you have three different pointers you operate on. In principle your filter can operate independant on the three planes.

[Marc FD]

too dumb ^^
i just start to be _very_ used to work in YUY2 now.
it doesn't make me any problem anymore to do SIMD on YUY2.
even vertically ^^ (msoften use a SIMD 2d average blur)

i think i would keep YUY2 until we have YV12 support in VDub.

then i would add YV12 support in MPEG2Dec3 and release my SIMD verion of distortion_measure (thx marco ^^) who works in YV12

BTW, keep in mind handling interlacing in YV12 is really hard ^_^

Cheers,
MarcFD

[Richard Berg]

That was more or less my intuition -- filters like CNR2 can probably be sped up greatly, but something like Decomb will require a lot of work and possibly more processing time to boot.

Replies to sh0dan forthcoming, got a lot on my plate ATM...

[stickboy]

Originally posted by sh0dan
I'm not sure I'm with you here - could you give an example?I think Richard is referring to the pixel_type argument added to AVISource and its ilk in version 2.05... based on your "late night edit", though, I assume you've already taken care of that.

[sh0dan]

Originally posted by Marc FD
BTW, keep in mind handling interlacing in YV12 is really hard ^_^


Unless I'm mistaking, it's much easier in YV12. Every second line in the Y plane is interlaced, just as YUY2. The chroma planes cannot be interlaced, so no special code there.

Is it more complicated than that?
i just start to be _very_ used to work in YUY2 now.
If I recall correctly, you were the one pushing very hard for YV12 support?!!?

[trbarry]

There are design issues to be worked out but I think deinterlacing is probably easier in YV12. And probably at least 25% faster.

- Tom

[Marc FD]

>here are design issues to be worked out but I think deinterlacing is
>probably easier in YV12. And probably at least 25% faster.

of course, everything would be theorically 25% faster in YV12 ^^
but i though the vertical upsampling would make deinterlacing harder.
BTW, you and don are the deinterlacing gurus here. if you say it's easier, it's good news ^_^

>Unless I'm mistaking, it's much easier in YV12. Every second line in the
>Y plane is interlaced, just as YUY2. The chroma planes cannot be
>interlaced, so no special code there.
>Is it more complicated than that?

chroma _is_ interlaced, from what i understood.
and it's why i though it's going to be more difficult to handle :(

>If I recall correctly, you were the one pushing very hard for YV12
>support?!!?

of course :) . i never say i was against. i just slowly got used to YUY2 and now i'll need to learn how to use YV12 correctly :D

keep the good work :)

Cheers,
MarcFD

[sh0dan]

Originally posted by Marc FD

chroma _is_ interlaced, from what i understood.
and it's why i though it's going to be more difficult to handle :(


Do you have ANY information about this, since my implementation of YV12 separatefields, weave and doubleweave assume the opposite. Many Nic, Koepi or other XVID devels can help here?!?

Interlacing isn't a big issue in AviSynth, due to the methods mentioned above. No filters really deal with interlacing, except the ones actually designed to modify interlacing (decomb, mocomp). All other filters doesn't really care about interlacing, even though some probably should (resize, smoothers, etc), but doesn't have to because the user can use separatefields/weave.

So even if interlacing isn't simple, it won't affect that many filters, if they are seperated correctly.

[SansGrip]

sh0dan: VDub cannot handle YV12 internal

Do you (or anyone else) know if other "destinations" (TMPGEnc, CCE, Nandub etc.) can handle YV12?

[-h]

Do you have ANY information about this, since my implementation of YV12 separatefields, weave and doubleweave assume the opposite. Many Nic, Koepi or other XVID devels can help here?!?

I can only comment on interlacing behaviour in MPEG-4.

edit: which I misunderstood anyway :)

-h

[trbarry]

-h

I'm not sure that is completely correct. First, I don't think you have to divide by 2 because you already have the fractions, .25,.75.

But the idea is, like with resizing, you don't want to blend the values from even and odd fields which may be from different points in time, because you would get interlace artifacts.

So it is just a weighted average of lines 0 & 2 and and a weighted average of lines 1 & 3. The weights follow easily from knowing what sample points the YV12 chroma values are supposed to represent.

This is often done wrong, causing the various DVD player "chroma bugs" and I harp on it a bit. ;)

But see DVD Benchmark - Special Report - The Chroma Upsampling Error in DVD Players - April, 2001 (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html) by Stacy Spears, who is even more obsessed with the idea than me. (and understands it better) ;)

That page graphically displays the layout and explains the common problems that we shouldn't make. There is also code in MPEG2DEC2 or DVD2AVI that does this in what I hope is a correct fashion when converting to YUY2.

I think the book "Video Demystified" also explains it well. Don't know if that is online.

- Tom

edit: In the DVD2AVI source for instance ( www.trbarry.com/DVD2AVIT3.zip ) in member filters.cpp function conv420to422() converts 4:2:0 planar to 4:2:2 planar. It does it differently for interlaced vs frame pictures, and does the top and bottom fields separately for field pictures.

edit2: It looks like we are going to be relying more on the "frame based" and "field based" Avisynth values, so IVTC or deinterlace functions should set this to frame when done. Mine don't. ;)

edit3: It looks like when vertically cropping interlaced YV12 it should maybe be done in multiples of 4, not 2. I don't think anyone honors this anywhere. Somebody correct me if I'm wrong here but I was just staring at Stacy's pictures and worrying. I'm not sure I'm right about this.

edit4: These are some of the things I brushed off as "design considerations". ;)

[-h]

I'm not sure that is completely correct. First, I don't think you have to divide by 2 because you already have the fractions, .25,.75.

Ha! Good points, I probably wouldn't have noticed the division and field-blending errors until I tried implementing it.

edit3: It looks like when vertically cropping interlaced YV12 it should maybe be done in multiples of 4, not 2. I don't think anyone honors this anywhere. Somebody correct me if I'm wrong here but I was just staring at Stacy's pictures and worrying. I'm not sure I'm right about this.

Yes. Heights being a multiple of 4 is enforced for field-based video in MoMuSys, and now I see why. Otherwise the top and bottom fields would have differing amounts of chroma information.

-h

[Wilbert]

@Sh0dan,

I downloaded v2.5, but I can't install it (I mean I can install it but it doesn't work). I changed the dll's, run the reg file, rebooted a couple of times, removed every entry in my registry file about AviSynth manually and reinstalled. Every time I try to play the avs with "version" or with "AviSource("C:\clip.avi")" it says

Cannot open 'F:\version.avs'
Please verify that the path and filename are
correct and try again.

Win98SE + Athlon, let me know if you need more info.

edit: I also noticed that the file size is eight times bigger.

[sh0dan]

If AviSynth is already installed, you should just overwrite the .dll and do nothing more.

The size of the dll is because it is a debug build, so it contains debug information.

I have it running on both Win2k, WinXP and Win98SE. Is Virtual Dub giving you these errors?

[Wilbert]

If AviSynth is already installed, you should just overwrite the .dll and do nothing more.
I also tried that a couple of times.

Is Virtual Dub giving you these errors?
WMP6.4 did. I forgot to try it with Virtual Dub.

[sh0dan]

So what happends in VDub?

[sh0dan]

Another issue:
- I would really much like to make YV12 mod 4 on width/height.

It seems like there isn't much of an option on making it mod 4 in height, because interlaced processing requires this.

Making mod4 restriction on width is for speed reasons. I'd very much like this to be mod8 (so chroma can work on 32 bits at the time) - but I'm afraid you'd all cry out in anger. :(

Another way for this could be to make sure that pitch is always at least mod 8 or 16. That way filters could be allowed to process beyond the rowsize without causing distortions.

So how about YV12 has mod 4 restrictions, and AviSynth must always deliver a pitch that is at least mod16?

[Wilbert]

Sorry I'm at work now. But I also installed it here (Win98SE PIII). When I try WMP6.4 it says the same. Opening in Virtual Dub gives "Cannot determine file type of "C:\Temp\test.avs"".

The avs file contains "AviSource(some DV file)", also tried it with "version".

(Overwriting v2.05 works fine.)

[vlad59]

Originally posted by sh0dan
So how about YV12 has mod 4 restrictions, and AviSynth must always deliver a pitch that is at least mod16?

I agree

[sh0dan]

Originally posted by Wilbert
(Overwriting v2.05 works fine.)
So.... does the alpha work, when you install 2.05 first, or does it work, when overwriting the alpha with 2.05? :confused: :confused:

[-h]

So how about YV12 has mod 4 restrictions, and AviSynth must always deliver a pitch that is at least mod16?

I would be happy so long as the pitch is mod16 and the image itself is allocated with 16-byte memory alignment.

-h

[Richard Berg]

Ditto to -h's request.

[dividee]

It would be nice if 16-byte alignment was guaranteed, but then Crop would (usually) need to do a memcopy, because nobody would like to be constrained to crop on mod16 boundaries only.

[-h]

It would be nice if 16-byte alignment was guaranteed, but then Crop would (usually) need to do a memcopy, because nobody would like to be constrained to crop on mod16 boundaries only.

A good point!

Perhaps the script handler could be smart enough to determine where the image should be allocated, such that after cropping the top-left pixel will be 16-byte aligned. I assume there are other complications I'm not aware of.

-h

[trbarry]

I think mod 16 alignment is important enough that it is worthwhile to ensure it even if crop had to be complicated to do a copy in the non-mod-16 cases. This is true for RGB and YUY2 also.

It would simplify programming and probably still give a performance boost in the average case even if it slowed down crop sometimes.

- Tom

[trbarry]

If multi-channel of ac3 is working, would it make sense now to have DVD2AVI/MPEG2DEC (optionally) also return that along with the video to the MPEG2SOURCE function?

Maybe it would help with some sync problems, or at least make it possible to fix them more easily.

- Tom

[WarpEnterprises]

This would be really great since creating the WAV is not really elegant.
But then the DVD2AVI downmix/dynamic range control would be useful IN AviSynth.

[sh0dan]

@All:

This is how I've tried to implement it:

How it is in 2.0x: When you request the rowsize, you get the width of the actual image in bytes. This may, or may not be the same value as the image pitch.

How it works in the alpha: When you request the rowsize, you can add the parameter "PLANAR_Y_ALIGNED" or "PLANAR_U_ALIGNED"/"PLANAR_V_ALIGNED". This will give you the width of the image, rounded up to nearest mod16 (or if already mod16, just rowsize as usual).

When a planar image is allocated, it is FORCED to have mod16 pitch. For source filters that may not wish to comply with that I've created a filter "AlignPlanar" which must be called with a clip argument. This will align (and copy is necessary) the pitch to mod 16.(See here (http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/source.cpp.diff?r1=1.17.2.12&r2=1.17.2.13&only_with_tag=avisynth_2_1&sortby=date) how to use it.)

Also env->NewVideoFrame(vi,[align]) has changed, so pitch is rounded up to 16 for all images, and align is only forced, if align is negative (-4 = 4 align forced).

Please ask, if you have any questions - this explanation is a bit technical I know.

Other YV12 changed include:
- *Resize now works vertically.
- VerticallyReduceby2() works
- Added converttoyv12()
- Added experimental plugin info function.
- Greyscale() now works.
- (Edit) MMX optimized ConverttoRGB24() (don't ask why - it's faster though)

New alpha will be up later today. I still have some stuff to debug.

[Wilbert]

So.... does the alpha work, when you install 2.05 first, or does it work, when overwriting the alpha with 2.05?
Sorry for my bad english. I meant: alpha doesn't work (win98se athlon + pIII).

[sh0dan]

The AviSynth alpha page (http://cultact-server.novi.dk/kpo/avisynth/avisynth_alpha.html) has been updated with a new alpha.

[Wilbert]

You mean this one "Download : Binary [21-10-2002]"? The link to that file is broken.

[sh0dan]

Refresh the page - it's here :)

[Wilbert]

Replaced the dll, rebooted. I still got the same error messages.

[sh0dan]

Try this Release build (http://cultact-server.novi.dk/kpo/avisynth/avisynth_release.zip). It shouldn't be any different from 2.06.

Alternatively try creating the reistry entry:

HKEY_LOCAL_MACHINE\\Software\\Avisynth\PluginDir2_5 and set it to an empty directory. (Even though there is only a VERY small chance it'll work).

Btw, rebooting shouldn't be necessary at any time. This is a very frustrating thing to hunt down - we don't seem to get anywhere :(