Layer and Mask cfr

[poptones]

I'm finally ready to release my layer additions into the wild.
Here's the source for my misc.cpp (http://personal.ayrix.net/~poptones/layer/misc.cpp) I added it to misc.cpp for two reasons:

I'm a novice with vs and c in general, so I don't trust myself to add it in a new page.

I hate screwing with loading plugins, and much prefer to just invoke commands. It's not like this is a 500K DLL or anything.

As you may or may not know, I originally created this to use with either YUY2 or RGB. And truth be told there's a modest speed advantage in YUY2 (or there was before I made it all mmx). But layers have the most power when used with alpha masks, and YUV formats don't support an alpha channel.

Now, since I screwed up some months ago and lost my only working source, I've had to fudge with getting an "in between" version working again. And frankly, I just don't see YUV support as being worth the trouble right now. Much better to get something cogent out there that works so it can be thrashed and refined.

So, here's the specifics: Layer() works only with RGBA. Not YUY, not even RGB. If you don't apply a mask after you create RGBA, you'll have a default "white" (on) mask anyway, so it doesn't matter.

The positioning commands still have a behavior I kinda dislike, but I have an extreme headache today and can't concentrate well enough to mess with it. Since I've been granted posting priviledges, my focus now is to get this moving. So, it gets released with "logical, but inconsistent" variable usage.

The coordinates follow a polar plot, rather than something we all should know: css and absolute positioning. If you want a layer positioned ten pixels down from the top, then you use a negative number. If you want a layer positioned ten pixels BEHIND the left edge, then you use negative. Positive numbers go up and right, just like we count. Negative goes down and left. Hope that's clear. If it ain't, mess with it and you'll see.

Now, if you position an element completely out of site, rather than get a null operation, you'll get an exception. This is the bug I was working a couple hours to fix, but I'm just not thinking today. I'm sure it's a simple problem in the math, but I can't see straight enough right now to fix it. And frankly, I don't see it as any worse than the undefined exception one gets when trying to crop() a clip past its borders, so there ;)

Layers are most powerful when used with masks. So, there's a mask() command to go along with layer(). Mask will take an RGB clip and apply the greyscale version of that clip as a mask to the first clip.

sourceclip.mask(maskclip)

Very simple. Only requisite is sourceclip has to be RGBA, and maskclip has to be either RGBA or RGB.

Layer usage:

sourceclip.Layer(overlayclip,amount,xpos,ypos,use_chroma)

If you set use_chroma to false (or zero) then only the lum information in overlayclip will be applied to sourceclip. The default is one, or use_chroma=true.

There are four methods for applying the overlay: add, subtract, lighten and darken. These words must be in quotes as a text string, and these are the only four choices. They are also very, very useful; try making a blur base layer and applying various amounts of lighten and darken over the blur. Very cool effect when applied over a very blurry layer, and handy for cleaning up noise in a clip when applied over a slightly blurry layer. try this:

blur=clip.blur(1.58,1.58).blur(1.58,1.58).blur(1.58,1.58).blur(1.58,1.58)
clip=blur.Layer(clip,"lighten",192,0,0,1)
return clip

Change to darken, subtract, etc and you'll see how useful this can be.

Also, this can be used (if you like) to replace the subtract(clip,clip) function. Last I checked it was a tiny bit faster, but that could be negated if all you need is subtract() and you have to convert everything to RGB32 to use this method.

Now, this might seem at first rip like only meant for people wanting to do sophisticated NLE stuff. And yeah, I do a good bit of that with this (I've been using this code for more than six months now, with rarely a problem). But the fact is, if you look at most any nonlinear filter, they inevitably use some means of "masking" in the function. For example, the smartsmoother functions usually make use of a blur layer and an unfiltered layer, comparing the values between the two layers and creating a new layer.

This can be done using a layered system and a threshold filter that compares two clips (blur and unfiltered source).

But where it gets really handy is making use of photoshop type effects that allow pixel by pixel masking of sharpening and blurring filters. Apply an edge detection matrix to your source clip, apply it as a mask to itself, then layer that over a blurred clip and you have damn near perfect MPEG artifact removal without losing any of the detail that makes images look sharp to us. If this sounds confusing, as soon as I get a general purpose 5x5 convolution filter for avisynth I'll put up a webpage littered with examples of the power that lies in using layered images. If you're a photoshop/gimp wizard, you'll already know the world of cool stuff this makes possible.

You've probably seen the thread I posted to at ars with the "de-logo" usage of layers. If you've seen that, it should also be obvious to you this is a very easy way to add your own bugs (if you're the cruel sort) to your own rips. So you can replace that big ugly MTV2 box with your own bug that says "I ripped this, nya."

Anyway, that's it. My first post here; my thanks to management for helping me get this posted today.

[trbarry]

Hi poptones -

Glad to see ya drop in.

This sound complicated, but powerful. I'll check it out.

- Tom

[dividee]

Looks great, thanks poptones.
Some testing is in order to fully understand how it works ;)

For those of you who can't compile avisynth, I attach a compiled version (untested, gonna sleep first, but it compiles without warning with VS7). Please use this version only for testing purposes.

[EDIT:] Removed attachment, new one below

[dividee]

I've done some modifications on your filter, I don't remember them all, but here is a quick list:

mask & layer now works in-place (a bit faster and conserve memory)
coordinate system: x=0,y=0 is still the upper left corner, but positive goes to right and bottom (more classic I think)
greyscale was wrong: you used the formula to get luminance in YUV but that gave you a [16,240] range; for a RGB greyscale you need the full [0,255] range. rgb2lum is a 64bit integer. greyscale MMX code was wrong too.
shouldn't crash anymore when the overlay is positonned out of sight (is now a null op)
map arrays calculations moved into the constructor.
more consistent AVSFunction definition
some code cleanup
some other small bugfixes I forgot

There are probably some bugs left and I may have introduced new ones.
If you don't apply a mask after you create RGBA, you'll have a default "white" (on) mask anyway, so it doesn't matter.
Not in my experience. For instance, AVISource returns a RGB32 clip with garbage in alpha channel. Easiest way to apply a "white" (opaque) mask:
clip1.Mask(clip1.BlankClip(color=$FFFFFF))

The exact prototype of Layer is now:
clip.Layer(overlay_clip,amount=255,x=0,y=0,treshold=0,use_chroma=true)
Default values follows the equal sign.
I left "treshold" in, altough it's not used at the moment.

What I'll do next:
* convert it to a plugin
* check alpha calculation. At first sight, it seems it may cause some darkening, since you multiply by 255 and divide by 256. Not sure. But this is probably hard to fix.

This is a great filter! I guess some of you will find very creative uses for it.

[dividee]

Oops I forgot the attachement!

[Richard Berg]

mask & layer now works in-place (a bit faster and conserve memory)
Not sure if it will actually work; makeWritable() initializes a new clip if there's >1 reference, and it seems like there would be.

Note that if you are adapting an algorithm that works in continuous pixel values in [0,1], you should divide by 255, not 256, to do the conversion!
-Avery Lee, VDub Filter SDK documentation. I'll fix this and merge dividee's fixes into my new source tree.

But where it gets really handy is making use of photoshop type effects that allow pixel by pixel masking of sharpening and blurring filters. Apply an edge detection matrix to your source clip, apply it as a mask to itself, then layer that over a blurred clip and you have damn near perfect MPEG artifact removal without losing any of the detail that makes images look sharp to us. If this sounds confusing, as soon as I get a general purpose 5x5 convolution filter for avisynth I'll put up a webpage littered with examples of the power that lies in using layered images. If you're a photoshop/gimp wizard, you'll already know the world of cool stuff this makes possible.

Your move :)

[poptones]

For the record, the reason I went to using the bitblitter is because using makewritable leaves artifacts if you have more than one layer. In fact, it was in my early days (before I added the mmx stuff) and this really drove me nuts. I thought my idea wouldn't work at all until I just went for it and learned how to make a new buffer (all my previous efforts had been pretty much using "subtract" as a template for new filters).

Unless you know how to work around this, I suggest that change be left out. It's all well appreciated, 'tho, and would you believe it's been another very hot day here in mississippi? Yes, I have a terrible headache again. I hope this isn't establishing a pattern.

The math error has always bugged me. I even thought about making it all 16 bit, then I thought about making a whole new port using 16 bit math everywhere - 16 bit pixel values, 16 bit command variables - everything. Then I realized I can barely even drive the compiler, so I had a lot of toddling to do before I tried running.

[Richard Berg]

Ok, done. Binary posted here; source posted in the "core" thread.

For my sanity, can future modifications be done to the modularized version? Merging them was a bitch. If you have problems compiling the revamped source tree, good! Ok not really, but let's get them fixed so we can make all the refactoring work worthwhile :)

To those who know what they're doing, you should probably test this not only to make sure the artifacts poptones spoke of haven't reappeared (if the avisynth internals are doing their thing correctly, the worst-case scenario *should* be that it's merely equivalent to writing to a separate dst buffer), but to make sure no more bugs have been introduced in this process.

Thanks all for some great work the last couple days!

[poptones]

If you don't apply a mask after you create RGBA, you'll have a default "white" (on) mask anyway, so it doesn't matter.
----------------------------------------------------------------------

Not in my experience. For instance, AVISource returns a RGB32 clip with garbage in alpha channel. Easiest way to apply a "white" (opaque) mask:
clip1.Mask(clip1.BlankClip(color=$FFFFFF))

Thanks for bringing this to my attention. Actually, I should have been right because this was fixed (by me) months ago. I thought it had been made part of the cvs. It'll soon be fixed in 2.0, 'tho.

Did your changes work well for you? I'm having fits with this right now because a couple of the changes have broken my avs scripts, but I'm not sure yet exactly why.

[dividee]

Thanks for bringing this to my attention. Actually, I should have been right because this was fixed (by me) months ago. I thought it had been made part of the cvs. It'll soon be fixed in 2.0, 'tho.

Then we should also check that EVERY filter that works in RGB32 also either leaves the alpha channel untouched or does something sensitive with it.


Did your changes work well for you? I'm having fits with this right now because a couple of the changes have broken my avs scripts, but I'm not sure yet exactly why.


Works well here (else I wouldn't have posted them).
The paramter string was changed to:
cc[op]s[amount]i[x]i[y]i[threshold]i[use_chroma]b
from:
ccs[x]iii[threshold]i[use_chroma]b
which was inconsistent: the first integer is not x but amount, and when you give a name to an argument it also makes it optionnal, so you have to give a default value for it, which wasn't the case. Now every argument (except the clips) has a default value (op="Add",amount=255,x=y=threshold=0,use_chroma=true)

And BTW, in avisynth, you can't use 0 and 1 for booleans, you have to use true and false.So the example you gave:
clip=blur.Layer(clip,"lighten",192,0,0,1)
works because 1 is interpreted as the threshold, not use_chroma

[Richard Berg]

Good explanation, thanks. That info should allow me to make GeneralConvolution's bias optional, finally.

I'll also be committing your changes to the CVS, dividee...though I'll leave the release packages the way they are until, err, it's time to release again.

[poptones]

Thanks for the help. Turns out the anomalies I was getting was due to my expecting the alpha channels being returned in a known good state. What "broke" the avs files I was using wasn't your changes, but the fact I went from one build with my changes in the rgb conversion asm routines to this "newer" version which didn't have them anymore. Grr.

Question: is there anything (besides inelegance) wrong in reading past the end of the file? I was actually exploiting that in a couple of scripts, 'tho it can be replicated using freezeframe, etc. I'm just curious about the old version causing stack overflows or other odd behavior.

So far as checking alpha in other filters, I will be. I did this some months back when I first started actually using the alpha channels via this filter, and I suspect those changes are still here, but I'm gonna verify it asap. I have better things to do than chase my own tail around in circles...

[Richard Berg]

Forget if I mentioned it before, but I did add alpha support to GeneralConvolution. Gunnar's algorithm would just write zeros to the high doublebyte, but my filter now stores/restores the original values via an extra buffer. Real-world testing always appreciated, though.

[dividee]

Question: is there anything (besides inelegance) wrong in reading past the end of the file? I was actually exploiting that in a couple of scripts, 'tho it can be replicated using freezeframe, etc. I'm just curious about the old version causing stack overflows or other odd behavior.

It's safer; when calling GetFrame you ask the previous filter(*) in the graph, and you never know how that filter will behave if asked for a non existent frame. To be really safe, you should also check for frames <0.

(*): indirectly, since avisynth insert the Cache filter after every filter in the graph.

[poptones]

BTW if anybody wants to experiment, here's a sample (enhanced to really show off) of the things you can do combining some of these basic tools...

LoadPlugin("MPEG2DEC.dll")
clip1=mpeg2Source("combine.d2v").crop(16,0,688,576).ConvertToRGB32()
noise=AVISource("noise.avs").ConvertToRGB32()
clip1=clip1.Layer(noise,"add",8,0,0,use_chroma=true)

clip2=clip1.trim(125766,0)
clip=clip1.Layer(clip2,"add",128,0,0,use_chroma=true).Trim(9180,86780).AssumeFrameBased().ComplementParity().Bob()

#build de-logo mask
logosrc=AVISource("nologo.avi").weave().ConvertToRGB32().BilinearResize(59,13)
masklogo=AVISource("whitelogo.avi").ConvertToRGB32()
logo=logosrc.mask(masklogo).BilinearResize(60,27)

#now cover up that ugly white logo...
clip=clip.Layer(logo,"add",255,582,36,use_chroma=true)

blur=clip.generalconvolution(0,"10 10 10 10 10 10 10 10 10 10 10 10 10 16 10 10 10 10 10 10 10 10 10 10 10")

#sobel edge detection... the power of the matrix!
hor=clip.generalconvolution(0,"-1 0 1 -2 0 2 -1 0 1")
vert=clip.generalconvolution(0,"-1 -2 -1 0 0 0 1 2 1")
edgemask=hor.layer(vert,"lighten",255,0,0,1)

#now sharpen up just the edges of the blurred image...
clip=clip.mask(edgemask)
clip=blur.layer(clip,"add",128,0,0)

return clip


This runs about 1fps on my P2/450 "TV set." It's more tweakable (try messing with the levels of the mask, especially gamma), and faster than many of the "smart smoothers" - especially when you combine functions into the convolution matrix.

[scav]

Hey poptones,

I've fiddled in a strictly amateur fashion with Photoshop layering tricks, and ran across a very handy one for cleaning up photos taken with digital cameras/scanners. Since such devices evidently can't capture info in the blue channel as well as the red and green, a lot of the noise is in this channel and simply smoothing the blue channel only, produces a very nice overall perception of sharpening. So, two questions: 1) Does your Layer filter lend itself to doing a similar function with video and 2) Does digital video exhibit this same behavior, i.e., is one color channel usually more noisy than the others?

Also, a general question about averaging multiple video streams, I assume this is only beneficial if you use two (or more) streams that were actually captured in separate capture sessions. Or can you cheat and use a single captured stream, and source it twice and average? Or perhaps source it twice and apply dithering to one to 'offset' it slightly, then average. Is that what you are doing with your 'noised' version of the stream?

[poptones]

Averaging the same capture twice won't buy you anything. You can, however, apply a blur to one layer, sharpen the other, and apply the sharpen through an edge detection mask (as above) and you'll have sharp edges AND a substantial decrease in noise.

I'll put all that to gether as soon as I get these actually working properly. There's all sorts of things you can do, and in fact I often hit photoshop sites looking for inspiration. Being able to "glassify" moving video has gotta be useful to more people than just those who want to make copies of packaged DVDs, and this stuff runs a whole lot better (on my machines, anyway) than AfterEffects.

But right now there's still a few bugs, and I really want to get the YUY2 layers (as well as mask) mmx optimized and released. I put that together this morning (took me all of an hour and it actually worked the first time I ran it! woohoo!) but so far it's just the add and subtract, and subtract still ain't even right in yuy2. The hard part, I think, is gonna be learning to use the mmx to do lighten and darken.

So far as blurring separate color channels, I really hadn't thought of that but I suppose it could be done pretty readily, and it probably should be an option. Maybe richard can add that to his convolution filter, since you'd need it in RGB form to do what you're talking about anyway.

[Richard Berg]

If you want to apply something to just the luma or just the chroma, that's precisely what MergeChroma & MergeLuma do. Unfortunately it still requires you process the full YUV image in both parts of your chain, but I thought about it a lot and there was no good way to "cheat" on processing just one channel while retaining compatibility with existing filters.

I've thought about making similar filters to separate/combine the RGB channels but haven't found a case yet where it would be useful. It could also be a feature of GeneralConvolution, but there'd be no real advantage since you still have to extract & repack the entire pixel -- and if I ever optimize it for SIMD, it'll go in parallel anyway -- meanwhile you give up the ability to use existing filters that are better optimized for common tasks. So if there's a good use for this, I'd do it in the same style as sh0dan's Merge filters.

[poptones]

Quite frankly, I think the intel invented these "standard storage formats" to be a pain in the butt, and if I were designing from scratch there's no way I'd do it this way. These should all be linear planes, with a pointer to each plane. Then it's simply a matter of blasting away at the layers 8 pixels at a time. Only wanna filter red? Simple, just run your convolution on the red plane. Wanna filter just the luminance? Fine, blast across the lum plane.

Problem now is it would require a complete rewrite of the existing filters. Maybe if there was a single new pixel type called "plane?" Then the filters could be added if/when it mattered. Doesn't anyone else think it's ridiculous such a huge amount of time in every single filter is spent masking and moving around bit planes that SHOULD be linear?

[Richard Berg]

Actually, given the details of x86 assembler (I've been poring over docs lately if you haven't noticed...), I think the current method is best. The reason manipulating just the red channel is "slow" (compared with the theoretical throughput of a CPU) is because manipulating ARGB all at once is so damn fast using PADDUSB and its brethren.

Sure, if you had planar video you could do something similar (assuming the dimensions were multiples of 4), but unless you were just manipulating one channel you'd run into major crunches in the register stack & cache lines trying to keep up with 4 pointers at once. With YUV video there would be the additional problem of distinguishing 4:4:4 from 4:2:2 from 4:2:0 and so on -- anything that requires averaging local pixels or similar would have to be "smart" in ways you get for free with YUY2.

[trbarry]

compulsive rant ......

Ignoring the legacy filter issue, I would LOVE to see Avisynth be able to accept and handle a planar 4:2:0 format.

This is the internal format used by DVDs, HDTV, Xvid, and DVD2AVI. (can you pass it to an overlay?, or Vdub?) Folks here occasionally see me rant about it but I have not actually done anything otherwise useful in this area.

But when you convert YV12 to YUY2 you use some extra CPU time and also create some new chroma pixels by interpolation. These can be a source of error since they all represent new sample points (from the wrong places) and also increase the processing time on all subsequent operations, not only because of the various YUY2 splitting operations but because there are now more bytes to process.

Later on when Xvid or whatever converts back to YV12 then more errors probably occur. And every color space conversion fuzzes things up a bit.

YUY2 more closely matches what you might get out of a capture card but 4:2:0 planar is almost certainly a better match for MPEG2/4 processing.

But again, it's a legacy issue and I don't know what a practical solution would be.

- Tom

[Richard Berg]

It would take 2 seconds to modify the VideoInfo struct in avisynth.h to support YV12. I'll do it right now if someone tells me what the significance of these numbers are: enum { UNKNOWN=0, BGR24=0x13, BGR32=0x14, YUY2=0x22 };

Backwards-compatibility would still be a relatively simple matter: check all the filters to make sure they don't assume (!RGB24 && !RGB32) implies YUY2, or similar logic.

Forwards-compatibility is the toughie: getting a critical mass of filters with a YV12 codepath. Are there any that are particularly important to dealing with YV12 sources? I don't think it's unreasonable to get the above stuff + a couple of key filters in place for v3.0.

As far as output -- I don't see a YV12 codec on my system, so we'd have to add that to HuffYUV or write a DX codec from scratch, though that kinda defeats the purpose since said codec would have to just convert it to RGB32 anyway in order to feed it to apps like VDub. I don't know of any vidcard drivers that accept anything but YUY2 & RGB for overlay. The best bet would be collaboration with the XviD folks to find a way to input YV12 directly into the compression engine.

[poptones]

I honestly think a more meaningful bob() filter will address much of this, as it will provide a better picture where that "extra" info is justified. The speedup from dropping those few pixels would be exceedingly tiny (at best perhaps five percent when actually moving/filtering). I've not seen the "purple bands" problem in any of the MPEG2 decoders we use, so I don't see the problem with having a bit of "extra" info - just so long as it looks nice.

[dividee]

some time ago I experimented with this. I wrote a filter ConvertToYV12 (without telling avisynth) and hacked CAVIStreamSynth::Info to return a FCC of '21VY' instead of '2YUY'.
I was able to open it in vdub, and when I looked at the file information I saw it used the divx5 decoder. I was also able to play it in WMP. But I was unable (didn't look too far) to figure out alignment issues for putting the 3 planes in a single buffer and it only worked for width multiple of 8 or 16, can't remember.

@Richard:
I think these numbers were chosen arbitrarily, for easy implementation of the VideoInfo methods:
pixel_type&0x10 = RGB
pixel_type&0x20 = YUV
pixel_type&0x07 = Bytes per pixels (won't work for YV12 :( )

[Richard Berg]

Thinking about it more, planar formats would make some filters do a lot more work. Crop() would have to recopy the whole buffer...unless we changed the whole architecture to support addressing based on 3/4 plane-pointers instead of size & pitch, which WOULD break everything.

As I started explaining before, we more or less have the whole CPU industry optimizing away the (former) disadvantages of packed-pixel formats for us. The only real savings would come when you wanted to modify just one channel, which isn't that often.

[trbarry]

I think the time spent on most operation in Avisynth is probably proportional to (some power of) the number of bytes of video data that are being processed.

So if you have 6 bytes (YV12) instead of 8 bytes (YUY2) then things might be proportionally faster.

But I agree it might be a lot of work and not immediately obvious how to implement or how to predict where the problems would be. So it's maybe not worth it.

And to be honest I don't even know how YV12 would appear in a file format. I've only looked at the planar data in DVD2AVI and Xvid a bit.

- Tom