AviSynth 2.5 updates

[sh0dan]

Definately YUV. Alpha can be implemented the same way as RGB.

However! I'm not sure that a in-picture alpha is necessarily a good thing. Implementation and maintenance-wise it would be much better if alpha are used from luma from another image.
Making filters as layer take the chroma from another image as alpha is a much easier and IMO just as good solution. That would also make YV12 alpha blending possible.
RGB32 alpha is at best unsafe - many filters trash it when they process the image.

[scmccarthy]

@Richard@Steady&sh0dan

As long as we're making up our own format, how does YUVA sound? This may not be quite true, depending on what you mean. In my investigations of color space, I found reference to 4:2:2:4, where the last 4 represents an alpha channel. It is not a fourcc code, but it is a format. It is a 4, because it is always as large as the Y samples regardless of how many chroma samples there are (ie 4:1:1:4, 4:2:0:4, 4:4:4:4). I envision the alpha channel for YUV as a separate plane for both planar and interleaved versions of YUV. The only way this solves anything for you is if you investigate exactly how the 4:2:2:4 format is implemented elsewhere.

And as long as we are talking about this now, I want to reiterate my desire to see avisynth support the NY12 fourcc code. In the future, avisynth will probably expand its support of Direct Show, which might cause problems as Microsoft is making noises threatening to drop support for YV12 in favor of NV12. Did you notice? On the Video Rendering with 8-Bit YUV Formats page: It is expected to be an intermediate-term requirement for DirectX VA accelerators supporting 4:2:0 video. The same statement is made regarding YUY2.It is expected to be an intermediate-term requirement for DirectX VA accelerators supporting 4:2:2 video. Obviously we already support the second one. Microsoft likes NV12 because both planes have the same pitch. It has a Y plane and a plane of interleaved uv samples. It would be very easy to convert between the two except, I need a way to put the NV12 code in the header. Actually, all you'ld need implemented is the following code:vi.pixel_type = VideoInfo::CS_NV12;
GetPitch(PLANAR_UV) // returns the same as GetPitch(PLANAR_Y)
GetHeight(PLANAR_UV); //returns the same as GetHeight(PLANAR_U) Only the first one is needed really. Just give me the NV12 code in the header.

Maybe it doesn't matter if the fourcc codes are not used with Direct Show anyway.

Here's some links that refer to 4:4:4:4 YCbCr with an alpha channel:
Quicktime for Linux (http://heroinewarrior.com/quicktime.php3)
LMA - Linus Media Arts (http://www.lmahd.com/SDIO.html)
I guess mainly it is standard for professionals, not for consumer products. Still, it would be nice to have some form of YCC alpha channel support internally. From there anyone can write a plugin that implements a specific format.

Stephen

[Bidoche]

Damn, I knew it !!!
(that I should have made a YV12VideoFrame subclass of Planar and not assumed Planar = YV12)
:p

[hakko504]

Originally posted by Bidoche
Damn, I knew it !!!
(that I should have made a YV12VideoFrame subclass of Planar and not assumed Planar = YV12)
:p Well there's a lot of time to change that before 3.0 hits the market :D

[sh0dan]

I don't think MS is going to drop YV12 anytime soon. It's widely used, almost as much as YUYV (YUY2).
Anyway I cannot see any significant gains in using NV12 internally - adding it as an output option should still be possible, but I see no need for filters to support it - it's clumsy, and only usable when U and V are interdependant. Even then it's just as easy processing YV12.

[scmccarthy]

I fail to see a difference between an output plugin and an internal filter. All I am asking for is a new pixel type for the video info:vi.pixel_type = VideoInfo::CS_NV12; with a matching fourcc type in the header.

Attached is a plugin that won't compile yet, but I hope can be added as an internal filter for avisynth.

and I agree that this filters main usefulness would be to add it at the bottom of a script for codecs that can read NY12. I think Microsoft likes all frames to have the same pitch. Perhaps it saves memory that way.

Stephen

[sh0dan]

Guess it could be added - however I fail to see the purpose - do you have conversion routines to/from all the currently supported colorspaces?

[scmccarthy]

@sh0dan

The above attachment is a conversion routine from YV12 to NV12. I did it to demonstrate what is needed to allow conversion routines to work in NV12 space. Attached is the converse conversion routine from NV12 to YV12.

Altogether, this is a proposal to add three things to the avisynth header:

1) CS_NV12, of course.
2) PLANARUV, which returns a plane that is the same width as the Y plane, but half its height.
3) I created IsNV12() in the source of the attachment below. This would also go in avisynth.h however.

The big problem with NV12 is what it should return for interleaved. It has a plane of interleaved chroma samples. In all other ways it is the same as YV12. The MSDN YUV site does explain both color spaces adequately. Again, the advantage is supposed to be that having all planes with the same pitch and width allows the data to be stored in one big rectangle. (I hope that makes sense to you, you'd see what I am talking about if you look at the pictures on the MSDN site.)

My interest in this is being curious if there are or soon will be applications that can read NV12 directly and if this would allow those applications to work that way. For instance, if Microsoft is so gung-ho on this color space, does that mean that it already works with graphedit or WMP? It would be easier to implement this to find out than try to look for the answer.

Honestly, I am not sure what DirectX VA accelerators are, except it is part of DirectShow and definitely part of DirectShow for Windows XP service pack 1. Well, and that VA must stand for video accelerator. Most importantly to me, if we throw it out there, it will probably be useful sometime in the future and someone might use it in a way we cannot anticipate. My best guess is it will work with graphedit.

Stephen

[sh0dan]

To be frank, I think you're better off testing it yourself - do you have the software to build AviSynth? Not to be arrogant, but there are still some issues we need to resolve that to me seems more important.

When I implement it, it'll mean that we'll have to support it, ie. adapting the internals for it every time there is a change. It also means "one more potential error", which will only bring confusion, if it is added as a hack.

Do you have any concrete docs on why it should be good - what software and hardware is supporting it?

[scmccarthy]

This is just a proposal. Bioche already mentioned that AviSynth assumes that PLANAR and YV12 as equivalent, as though there are no other planar color spaces worth supporting. I don't know what they mean by it, but Microsoft's statement: intermediate-term requirement for DirectX VA sounds ominous.

So it is good to consider right now that NV12 might be needed in the future.

For now I'm concentrating on how to write a good plugin, rather than building Avisynth. But everything I write could be useful as an internal plugin, so I've thought of it.

I'm not as anxious for this as it seems. My thinking is if Microsoft thinks NV12 is better than YV12 for some reason then we should support it.

Stephen

[Bidoche]

I guess maybe we may want to add NV12 support as import and export, but as an internal colorspace to work with... I dunno

[scmccarthy]

@bioche

I honestly don't understand the difference. Unless you mean you do not want to require every filter to handle it. That I whole heartedly agree with. The only support I offer for using it is the vague theory that some applications *might* support it. Plus Microsoft's approval of it, which may mean nothing. Nothing, that is, of practical importance to avisynth.

My thought is that if setting pixel_type to CS_NV12 causes avisynth to output the appropriate fourcc code in the header now, we might be prepared in the future if NY12 becomes a more important colorspace.

Meanwhile, the only sure method for finding out if applications can use it is to try it. It is not that important to me, but all I am asking for is the minimum that would allow the two filters I already wrote to compile. I wrote them mainly to figure out what that minimum is. They are not software so much as a umambiguous proposal.

I suppose that if I don't research it to the point that I find at least one application that can really use it, there is not much point in it for now.

Stephen

[sh0dan]

AviSynth 2.5 does NOT assume YV12 == planar. Everything has been designed so that other planar formats can be accepted without any problems.
YUV 4:4:4 could in principle be added easily but since there is little use for it, and it would require rewrites of about 25% of the filters it isn't. Adding other kinds of planar formats (with is just as easy) would be just as easy - only problem is that not all filters know they're capable of processing not only YV12, but all planar images.


Adding a 16 bit YUV is another matter though, but still doable within the current design.

But NV12 isn't planar - it's a strange planar/interleaved hybrid, hopefully doomed as a strange idea MS got and nobody ever implemented. ;)
Adding support for NV12 will probably only create a mess.

[Bidoche]

@sh0dan

I was the one assuming that planar == YV12 (in my 3.0 framework)
but that's not really a problem, just some subclass renaming.

@scmccarthy

I could probably add NV12 support in 3.0 since VideoFrame is polymorphic it won't mess the rest, but for 2.5 it's out of question.
It's no time to try structural changes in 2.5 when everything is not even ok and we are planning a bigger jump anyway

And my nick is Bidoche with a d.
I don't understand how all of you manage to miswrite it..

[Guest]

@Bidoche

Originally posted by Bidoche
And my nick is Bidoche with a d.
I don't understand how all of you manage to miswrite it.. It's intentional; you're French. ;)

Seriously though, as English speakers our brains latch onto the "Bi" prefix and that round part of the "d" and subconsciously we perceive it as "Bio". Careless Yanks, I know. :)

[trbarry]

Assuming this is still the update thread, see

www.trbarry.com/DctFilter.zip

Avisynth 2.5b, pluginit2, YV12, and SansGrip's YUY2 support.

- Tom

[scmccarthy]

Everything has been designed so that other planar formats can be accepted without any problems. Part of why I bring it up is to make sure avisynth IS being designed so that any color format can be added later, as things change.@sh0dan That's right, I was just repeating what biDoche said. (Neuron2 was almost freakishly right, very perceptive.)hopefully doomed as a strange idea MS got and nobody ever implemented. Although NV12 interests me, I see your point. Unless someone can demonstrate that people are implementing it, there's no point in implementing ourselves.I could probably add NV12 support in 3.0 since VideoFrame is polymorphic it won't mess the rest, but for 2.5 it's out of question. Well, in a way I am pushing it just to get it on the table. Due to Microsoft's backing, I am guessing that all sorts of video companies have started support for NV12, card makers like NVidia perhaps, and it's definitely part of DirectShow now. So in a couple of years, it might be an important standard, or not.

Stephen

[trbarry]

A format like that was a seductive idea and I even thought I had invented it once. ;)

An early (unreleased) version of my CheckeredInterlace filter used something like that. But as I got more used to YV12 it became obvious that in YV12 I could often call the same code for both the U and V planes and sometimes even for the Y plane. So YV12 programming is really easier than NV12 as far as I'm concerned. I'd just let codecs handle it.

- Tom

[scmccarthy]

@tom

I don't want to belabor the point, but I think the reason Microsoft likes it is because all planes have the same pitch and rowsize. Maybe that lets you stack them on top of one another and processes them both in the same loop? Whatever advantages other formats have, as you learned, it is *often not* worthwhile to change the format in a filter just to make processing easier. (Did I say never?) We can't appreciate how much easier NV12 is until we live with it for a while. It just always seems easier at first to work in more familiar color spaces. But it is a moot point unless it becomes more widely supported anyway. I am very happy with sh0dan's sense of priorities.

Stephen

[Guest]

The fact that the UV planes in YV12 are smaller than the Y plane makes low-level optimization of Decomb very difficult and I still don't know what I'm going to do about it. I can't just do the planes totally separately. There is a common combing map that must be an OR of all the planes. I just mention this as a concrete instance of Stephen's point.

[issa]

I am now trying avisynth 2.5 beta with mpeg2dec3 v1.0.0.
I found out that SeparateFields().Weave() will not product
right interlace result. Is there a problem in Weave()?

*edit*
I fixed the problem with adding AssumeBFF() before Weave().
However, TMPEG said the clip is TFF, is there a problem with
field detect in mpeg2dec3 or avusynth ?

[Guest]

Originally posted by issa
I am now trying avisynth 2.5 beta with mpeg2dec3 v1.0.0.
I found out that SeparateFields().Weave() will not product
interlace result. Is there a problem in Weave()? There certainly seems to be something strange because the following scripts give output clips with different heights by a factor of 2:

AssumeFieldBased() // produces double height clip!
SeparateFields()
Weave()

...versus:

AssumeFrameBased()
SeparateFields()
Weave()

[sh0dan]

Originally posted by Steady
Just missed the beta deadline - arg. Here is the final optimized BitBlt, with (ugly) testing routines.

I still get crashes in the unaligned loop on complex scripts.

Point of crash:

memoptU_prewrite8loop: //write out odd QW's so 64bit write is cache line aligned
cmp ecx,eax //start of cache line ?
jz memoptU_pre8done //if not, write single QW
-> movq mm7,[esi+ecx]

Script:

stackvertical(last,last)
stackhorizontal(last,last)
reduceby2()
stackvertical(last,last)
stackhorizontal(last,last)
reduceby2()
flipvertical()
fliphorizontal()
swapuv()
reduceby2()

I don't think all (or any) the filters are necessary. It crashes right before output, so it's probably because the output isn't very well aligned.

The size of the video at the crash is:
rowsize: 180
dst_pitch is 180, src_pitch is 184
height: 144

[Steady]

I am sorry I missed this till sh0dan PM'd me. It turns out that it is crashing because with YV12 on the U and V planes the pitch was not mod8 (width 184, UV width 92 gave a pitch of 92).

I had made the assumption that the pitch was ALWAYS QW aligned (modulo 8). Evidently that is not true for YV12. I can make it check the alignment for every row, but this will slow it down a little bit. I guess my question is; Is the pitch allowed to be not mod8 for YV12 or is this a bug?

[Richard Berg]

Yes, pitch is allowed to be non-mod-8. There was a big discussion awhile ago where we decided to allow unaligned sizes up to the minimum required by the format (mod 2 in general, mod 4 when interlaced).

Nothing says we have to give unaligned sizes preferred treatment, though -- if it'll increase speed for "normal" frame sizes, I wouldn't feel bad about doing a single test at the beginning and letting it revert to the C code for non-QW-aligned sizes, so long as we add a FAQ entry or similar as documentation.

[trbarry]

Yes, pitch is allowed to be non-mod-8. There was a big discussion awhile ago where we decided to allow unaligned sizes up to the minimum required by the format (mod 2 in general, mod 4 when interlaced).

I though it was agreed size could vary non-8 but pitch would be 8 even in chroma. (so total pitch at least 16)

Not sure, but I think some of my filters rely on that too.

- Tom

[scmccarthy]

@Richard

You are confusing pitch with height and width. As tom points out, the decision was to let the Y plane pitch be mod16, making the U and V planes mod8.

Interlaced versus frame-based only affects the modulus base 2 for the height.

Stephen

[sh0dan]

(Copy/Paste from my PM)

Pitch is only guarranteed to be aligned for filters! When bitblitting from AviSource for instance pitch can only be mod4. That's why there is an AlignPlanar class, that always gets applied efter AviSource, DirectShowSource and some conversions, that realigns to a mod 8 pitch, if it is not so already.

Output (as the case here) also requires a bitblit to a mod4 pitch - that's AVI specs.


So - you filter writers can still safely assume pitch = mod8, luma pitch = mod 16!

[Steady]

The fix turned out much simpler than I thought. Shouldn't slow it down significantly. I should have thought of it before.

[Richard Berg]

You're right, I read his post incorrectly. Don't listen to me ;)