I made a new SimpleResize filter

[trbarry]

I also looked into the performance of AddBorders. I think it does seem to make another copy of the image, which has some performance effect. I'm still not sure whether crop does that also.

For performance (but not user friendliness) considerations I'm considering adding an additional couple of commands to SimpleResize:


InterlacedResize(Width, Height)

and

NotSimpleResize(OutWidth, OutHeight,
CropLeft, CropTop, CropWidth, CropHeight,
LeftBorder, TopBorder, RightBorder, BottomBorder,
HwarpFactor, VWarpFactor, InterlacedFlag)


This would add interlaced support and the Crop and AddBorders functions with the same parms as Avisynth currently uses. Crop may be unneeded if it turns out it already runs instntly without making another copy of the data. I'll check.

The 2nd parm list would be as confusing as GreedyHMA, but it would run fast. ;)

Comments?

- Tom

[dvd2svcd]

Maybe also a Boolean for Addingborders/cropping before or after resizing.

[trbarry]

Maybe also a Boolean for Addingborders/cropping before or after resizing

DVD2SVCD -

My first intention (out of programming laziness) was to crop first, which speeds things up by having less data to resize. Crop of course means do nothing, just not process some data, so it is just pushing pointers.

I was then going to simultaneously resize into the center of the output area and add any needed borders. But most of this was just for speed and the programming convenience of how easily it would fit into my existing code. And offhand I can't come up with any example that it couldn't handle if I did it this way.

Does this way create any problems? If so, could you give an example to solve?

I somewhat wanted to leave it undefined because depending upon the cropping I may be able to play games with the data alignment this way. Stuff runs faster on a 16 byte boundary so sometimes it may be possible to run a little faster if I resize some of the cropped area. I confess I have not really thought that part through properly yet. That was one reason I considered an integrated crop but there are probably many ways to do it that might not take much more work.

I don't suppose anyone knows how to easily make Avisynth filters accept keyword parameters with defaults. That would certainly make the interface easier to use.

- Tom

[Guest]

>>I don't suppose anyone knows how to easily make Avisynth filters accept keyword parameters with defaults. That would certainly make the interface easier to use.

What do you mean by this? In Decomb, you can say things like:

Telecide(blend=false,postprocess=true)

with the parameters in any order and combination (any can be omitted or included). Is that what you mean? If so that is easy using existing Avisynth kernel features. If not, what do you mean?

An alternative would be to do everything with a string:

Telecide("blend,postprocess,threshold=10")

Your code would have to parse the string.

[dvd2svcd]

I agree Tom, Crop, Resize, Addborders. That is the prime way (and what I always do), and when I think about it, I can't even see a reason to do it any other way.

Reagarding default parameter values, I know that the internal routines (like bicubicresize) has default values, and I thought that you might do it the same way in your filter.

[trbarry]

What do you mean by this? In Decomb, you can say things like:

Telecide(blend=false,postprocess=true)

with the parameters in any order and combination (any can be omitted or included). Is that what you mean? If so that is easy using existing Avisynth kernel features. If not, what do you mean?


neuron2 -

Yep, that is indeed what I mean. Did I miss some doc that tells how to do it without writing another command parser? I guess I'll go look again but I don't suppose you'd happen to have a spare link?

And can I mix positional & keyword types? SimpleResize(640, 480, LeftBorder=2) ?

- Tom

[Guest]

@trbarry

I figured it out from reading Ben's document describing the plugin example: avisynth-extensions.html.

It's easy, here's how:

1. Make your plugin init line like this:


env->AddFunction("Telecide", "c[swap]b[firstlast]b[postprocess]
b[threshold]i[blend]b[chroma]b[debug]b", Create_Telecide, 0);


...the things in brackets are the names that will be used.

2. Make your create call as follows:


AVSValue __cdecl Create_Telecide(AVSValue args, void* user_data,
IScriptEnvironment* env) {
return new Telecide(args[0].AsClip(),
args[1].AsBool(false), // swap
args[2].AsBool(false), // firstlast
args[3].AsBool(true), // postprocess
args[4].AsInt(15), // threshold
args[5].AsBool(true), // blend
args[6].AsBool(false), // chroma
args[7].AsBool(false), // debug
env);
}


3. Then do your class as follows:


class Telecide : public GenericVideoFilter
{
bool swap, firstlast, postprocess, blend, chroma, debug;
int threshold;

public:
Telecide::Telecide(PClip _child, bool _swap,
bool _firstlast, bool _postprocess, int _threshold, bool _blend,
bool _chroma, bool _debug, IScriptEnvironment* env) :
GenericVideoFilter(_child), swap(_swap), firstlast
(_firstlast), postprocess(_postprocess), threshold(_threshold),
blend(_blend), chroma(_chroma), debug(_debug)
{
...
}

PVideoFrame __stdcall GetFrame(int n, IScriptEnvironment* env);
};


It seems you can mix, but the positionals will all have to come before the named ones, otherwise the positioning gets confused.

[trbarry]

Neuron2 -

Quite nifty. Thanks!

You've probably just written the most compresensive guide to programming Avisynth filter parms that's ever existed.

- Tom

[trbarry]

Hi -

I have not done any more optimizations or coding for the more complex crop/borders functions yet but I did today release a new version supporting:

InterlacedResize() and
InterlacedWarpedResize()

These are experimental but should support resizing of interlaced video without the usual problems in trying to do that.

I'm going to even break a standard rule here by suggesting that you can sometimes go faster by downsizing before the deinterlace/IVTC step if you so choose. Now it is possible that the way I did it you can theoretically lose a small amount of vertical detail with InterlacedResize but so far I haven't seen it. And it is also possible that InterlacedResize can confuse a subsequent deinterlace/IVTC function but at least with testing GreedyHMA that also does not seem to be a problem. But all this is brand new and YMMV.

The new version 0.3, including DLL & source, is available at www.trbarry.com/SimpleResize.zip

A readme/changelog is in the zip and also at www.trbarry.com/Readme_SimpleResize.txt

As always, I appreciate any feedback.

- Tom

[b0b0b0b]

Hi Tom I like your idea for interlacedresize.

Does it do a separatefields; resize a; resize b; combinefields; ?

Here are my results:

I tried it on a resize from 720x480 => 320x240 on a straight ntsc interlaced stream and deinterlaced with decomb. No interlace artifacts came through, but the result was a lot more jagged than when I did a straight simpleresize after deinterlace. Doing a bilinearresize after deinterlace came out soft with no jaggies. I don't understand why interlacedresize was more jagged than simpleresize. I will keep thinking about it.

I just now realize I should have done a separatefields and discarded one of them because I ended up halving the vertical resolution.

[trbarry]

b0b0b0b -

Thanks. AFAIK, you are the only one who has tried it so far.

Does it do a separatefields; resize a; resize b; combinefields; ?

Not exactly. That wouldn't usually produce (quite) the correct results, but it does something similar. It calculates the odd pixel values as an interpolation of other odd values only, but using the coordinates of the whole screen. Same for even.

I tried it on a resize from 720x480 => 320x240 on a straight ntsc interlaced stream and deinterlaced with decomb. No interlace artifacts came through, but the result was a lot more jagged than when I did a straight simpleresize after deinterlace. Doing a bilinearresize after deinterlace came out soft with no jaggies. I don't understand why interlacedresize was more jagged than simpleresize. I will keep thinking about it.

This may be because even SimpleResize will tend to blend a little, though I usually claim it has no additional filtering. But if you blend adjacent odd and even lines after deinterlace it can tend to hide some deinterlace artifacts. So it could be either but keep in mind that, even more than SimpleResize, InterlacedResize won't help you filter or hide defects. Or it may be a bug. ??

And yeah, you can probably go a lot faster and get better results just throwing away a field and forgetting the deinterlace completely.
But in that case I still personally use SimpleResize (after) instead of HorizontalReduceBy2 (if needed) because HRby2 still softens more.

- Tom

[trbarry]

A bit more thinking about the excess jaggies observed above when doing InterlacedResize before an IVTC or deinterlace process.

It is possible to do an almost perfect IVTC given good material but when there is video interlaced source there will almost always be some jaggies or artifacts introduced. And many of us have tended to use the inherent smoothing of a resize method to somewhat hide these.

So while I still think we can get away with InterlacedResize before IVTC on good film based material I would recommend doing it before deinterlacing video material only under if you turn on the vertical filter option using GreedyHMA or add some other filtering using another deinterlace method. Because I don't make extremely low bit rate encodings I'm usually against a lot of filtering but in some cases I guess it becomes necessary.

Of course if you intend to keep your video interlaced then InterlacedResize may still be your best bet, with or without filtering. I'm not currently aware of others that handle this.

- Tom

[Blight]

trbarry:
I am in need of adding a high-performance image resizer into ZP (for resizing the DVD/Media folder images.

Is there a way I can use your resizer (considering I'm using delphi)... Is it fully ASM? something I may be able to add as an ASM block inside delphi?

[Blight]

Hmm, since my understanding of math is limited, I wrote my own fuzzy-math scaler using RGB space and factors. It's got nice quality on down-sizing, but rather bad on upsizing:


procedure DrawBGImage(SrcW{XOfs},SrcH{YOfs},BGWidth,BGHeight : Integer);
type
TMyRGB =
Record
mBlue : Byte;
mGreen : Byte;
mRed : Byte;
End;
TMyScanLine = Array[0..4095] of TMyRGB;

var
Speed1 : Integer;
Speed2 : Integer;
Speed4 : Integer;
PixR : Integer;
PixG : Integer;
PixB : Integer;
SX,SY : Integer;
X,Y : Integer;
PixXShl : Integer;
PixYShl : Integer;
PixXRnd : Integer;
PixYRnd : Integer;
PicWidth : Integer;
PicHeight : Integer;
PixCount : Integer;
PixXPos : Integer;
PixYPos : Integer;
PixX : Real;
PixY : Real;
P : ^TMyScanLine;
PL : Array[0..4095] of ^TMyScanLine;

begin
For Y := 0 to BGBitmap.Height-1 do PL[Y] := BGBitmap.Scanline[Y];
PicWidth := BGBitmap.Width shl 2{Factor};
PicHeight := BGBitmap.Height shl 2{Factor};
PixX := PicWidth / BGWidth;
PixY := PicHeight / BGHeight;
PixCount := Round(PixY)*Round(PixX);
PixXShl := (PicWidth shl 8) div BGWidth;
PixYShl := (PicHeight shl 8) div BGHeight;
PixXRnd := Round(PixX);
PixYRnd := Round(PixY);

For Y := 0 to BGHeight-1 do
Begin
PixYPos := (Y*PixYShl) shr 8;
P := BGImage.Picture.Bitmap.Scanline[Y+SrcH];
For X := 0 to BGWidth-1 do
Begin
PixR := 0;
PixG := 0;
PixB := 0;
PixXPos := (X*PixXShl) shr 8;
For SY := 0 to PixYRnd-1 do
Begin
Speed1 := ((PixYPos+SY) shr 2);
For SX := 0 to PixXRnd-1 do
Begin
Speed2 := (PixXPos+SX) shr 2;
Inc(PixR,PL[Speed1]^[Speed2].MRed);
Inc(PixG,PL[Speed1]^[Speed2].MGreen);
Inc(PixB,PL[Speed1]^[Speed2].MBlue);
End;
End;
Speed4 := X+SrcW;
P^[Speed4].MRed := PixR div PixCount;
P^[Speed4].MGreen := PixG div PixCount;
P^[Speed4].MBlue := PixB div PixCount;
End;
End;
end;


BGBitmap is a TBitmap source image
BGImage is a TImage in which the resized bitmap is written to, with SrcW, SrcY being offsets within that image (since I don't always cover the entire image with the resized image).

I doubt anyone came up with this retarded factor scaling method before, but hey, it works for me! feel free to use ;)

And I bet it can be ramped up in speed if optimized ...

[trbarry]

Hi Blight -

SimpleResize is mostly assembler with C/C++ code for setup and interface to Avisynth. The code is out there in the same zip, but it currently only handles YUY2. If you use it you could probably quickly port the small amount of C code to Delphi and imbed the assmembler run time loop if Delphi allows this, and accepts the same syntax. I've never used Delphi.

And BTW, SimpleResize doesn't really upsize that well either. That's where BiCubic excels. But except for folks without overlay hardware, who cares? ;)

You are indeed a coding madman if you whupped that together just while waiting for my reply. ;)

But no matter how fast Delphi is by some standards I don't think it can do real time video scaling. That's an assembler job. (or hardware)

- Tom

[Blight]

I don't really use this code for real-time video. It's for resizing a standard windows bitmap (hbitmap) for a background image while MP3 are playing or for the DVD image when in stop mode.

But I guess my solution is fast enough, it simulates grabbing sub-pixel data using an x4 image size (all simulated, the image isn't actually in there). I'm not sure what the technical name for this approach is. I just like to call it factor scaling. And it's all done in RGB, since windows bitmaps are RGB, converting to YUY2 first isn't really an option.

[tg0021]

I've just made a try to compare bilinear, bicubic and simpleresize. Your filter is amazing : it's quite as sharp as sharp bicubic and a lot faster than bilinear.

one problem : I got a few green lines on the right of the pictures with simple resize. Of course I could crop a few the right of the pictures but it's not a nice way. Do you know why those lines appeared.


Here is the avs script
LoadPlugin("D:\mpeg2dec.dll")
Mpeg2source("D:\Resize\vob.d2v")
LoadPlugin("D:\SimpleResize.dll")
crop(6,0,705,574)
SimpleResize(544,304)
#Neutral
#BicubicResize(544,304,0,0.5)
#Sharp
#BicubicResize(544,304,0,0.75)
#Bilinear
#BilinearResize(544,304)

[hakko504]

LoadPlugin("D:\mpeg2dec.dll")
Mpeg2source("D:\Resize\vob.d2v")
LoadPlugin("D:\SimpleResize.dll")
crop(6,0,705,574)
SimpleResize(544,304)
#Neutral
#BicubicResize(544,304,0,0.5)
#Sharp
#BicubicResize(544,304,0,0.75)
#Bilinear
#BilinearResize(544,304)
Never use odd values in crop. Change it to 704 and it should work.

[tg0021]

it works great w/o odd numbers in the crop. Thanks for the tricks

[kythorn]

@trbarry:

Out of curiosity, and at the risk of sounding like an idiot: Do you think this resize filter will be useful for resizing what started as a digital satellite capture? Obviously, this isn't true actually a digital capture, but captured via svideo @ 720x480 via huffy (I'm still working on extracting digital streams from my dtivo, but haven't gotten there yet)

There's *is* a fair amount of noise in my satellite feed anyway (DTV's using some hideously low bitrates on a lot of channels lately), and in general I like to use a temporal (not spatial though) smoother on it. This results in a fairly good picture in and of itself, but then any resizing method I've tried so far filters it even MORE and it winds up looking ridiculously oversmoothed/filtered.

I plan on trying this out when I get home later, but as thats a long ways off, I'm interested in your (or anyone else reading this) thoughts on this matter.

[trbarry]

kythorn -

It should do fine in this case. Good results with SimpleResize depends mostly on 2 things:

1) You already have done all the filtering you want (or will add it later). If you want more then you might be better off with BiLinear or a very soft BiCubic.

2) You are downsizing, or at least not upsizing by very much. To upscale a lot you might want BiCubic.

- Tom

[MaTTeR]

trbarry,

I got pretty amazing results with both quality and performance. It appears I'm late for the party though :-) Thankfully Acaila pointed me to the dll. My FPS has shot from 27FPS to an average of 36FPS, peaking at 41FPS.

Very nice work though, I'll be using the dll from here out.

Dual PIII 850's
768MB RAM (Highly Tweaked with BXTune)
WinXP Pro

Anyways nothing more than a thank you message and I should also point out great work on the optimization of the XviD DSF:D

You don't have a PayPal acct setup by chance?

[trbarry]

You don't have a PayPal acct setup by chance?

MaTTeR -

Thank you for you kind comments. I do have a PayPal account, but not for this. Instead please see the section in my ReadMe file ( www.trbarry.com/Readme_SimpleResize.txt ) about 'Philanthropy-Ware'.

That is, make any (totally non-obligatory) donation to the Electronic Frontier Foundation to help fight silliness like the DMCA or the new Disney/Hollings CBDTPA (Conniving Broadband and Digital TV Prevention Act) legislation. But if you do, be sure to tell everyone about it so they do too.

And thank you again for being willing to listen to me rant. ;)

- Tom

[trbarry]

Since this thread has opened up again I thought I'd mention something.

I've been reading up on it a bit and it looks like I really just re-invented BiLinear Resize, as MMX/SSE/SSE2 code. That is, SimpleResize is more of a standard BiLinear Resize like you might see in the text books.

The BiLinearResize() in Avisynth (and therefore in practically everything else) has a softening filter added to it, but SimpleResize just omits this.

So it could have been called UnFilteredBiLinear(), but I didn't know that at the time. ;)

- Tom

[Acaila]

Any chance you could make an UnfilteredBicubicResize() as well? :)

I really like this filter's speed (I use it mostly for testing codec and filter settings) and the added quality of bicubic over bilinear would make me switch permanently. VDub's and Avisynth's various BicubicResize filters either blur or sharp too much most of the time.

[trbarry]

Acaila -

I don't really yet understand the mathematics that evaluates the two parms for sharp/soft in BiCubicResize(). I suspect you can already get the effect you want by varying those. Then use BiCubicResize for upsizing, SimpleResize for downsizing moderate amounts or interlaced, and BiLinearResize for extreme downsizing or noisy material.

But BiCubicResize probably can be made to go faster for the most common case of resizing both horizontal and vertical in YUV color format. This could be done just by keeping the output of the vertical resize in a form that is efficiently used by the horizontal resize. Right now it resizes vertical, converting the output to YUY2 format. Then it horizontal resizes, converting one line at a time to a more convenient work format before resizing. This creates an extra pass through the data and probably slows it down some.

I was actually looking at optimizing the copy of BiCubicResize in DVD2AVI when I decided to write SimpleResize. I probably won't be doing anything on this soon but I do want to also create a set of these resizers that work on YV12 4:2:0 format data. This is the format that is used internally inside DVD2AVI, Divx5, Xvid, and the way data is stored in mpeg2 (DVD's) and mpeg4 (Xvid). It would be nice to avoid all the unneeded color conversions that slow things down and introduce small errors.

But that format wouldn't help Avisynth much except it would allow a resizer in mpeg2dec.dll to go faster.

- Tom

[Acaila]

Thank you Tom for that elaborate reply. I wasn't aware that avisynth's build-in resizers did their work with colorspace conversions, but it would certainly explain the speed difference between those and your SimpleResize (yours sometimes doubles my encoding speed :) ).

I don't really yet understand the mathematics that evaluates the two parms for sharp/soft in BiCubicResize(). I suspect you can already get the effect you want by varying those.

Probably, except I'm not sure anyone knows what unfiltered would be. I know (0,0.75) sharpens and (0.333,0.333) softens the image, but is unfiltered (0,0.5) or (0,0.6) or some other value or does it always filter it? If someone knows for sure the mathematical unfiltered value please let me know. There still remais the issue of slowness of the existing resizers though....

but I do want to also create a set of these resizers that work on YV12 4:2:0 format data. This is the format that is used internally inside DVD2AVI, Divx5, Xvid, and the way data is stored in mpeg2 (DVD's) and mpeg4 (Xvid).

Did I understand this correctly and will XviD get an internal resizer and will you be working on it?

[trbarry]

Did I understand this correctly and will XviD get an internal resizer and will you be working on it?

I don't know if they want a resizer built into Xvid. But in looking at the very similar code in DVD2AVI, Xvid, and mpeg2dec I noticed that it would be more efficient to resize in the 4:2:0 color space before converting to the more common YUY2 format that is often passed around. The conversion to YUY2 or RGB is just creating redundant (interpolated) data bytes.

Now Xvid may already be able to pass YV12 format to the display/render filters, I'm not sure. But DVD2AVI and MPEG2DEC both convert it to YUY2 format first. In MPEG2DEC's case that is necessary to pass it to Avisynth but I could still put a version of SimpleResize into it that operated before that happened. For DVD2AVI it may even be possible to resize and pass the YV12 data directly to Xvid for even more performance gains and probably slightly better quality for any material that did not need Avisynth/Vdub filters.

So I probably will look into a YV12 version of SimpleResize. And if I do then I'll probably also put it into DVD2AVI & MPEG2DEC. But keep in mind that I'm always thinking and talking about a lot of things and not all of them ever get done. I'm just tossing out ideas here.

Maybe others can pick up and do some of these.

For instance, a version of BiCubicResize that operated on YV12 format would probably be twice as fast as one for RGB simply because YV12 has half as much data. But that really only works if you resize before you convert to another color format.

- Tom

[Acaila]

Most of us are using other filters as well before we resize the image. So resizing first probably wouldn't make things much faster because colorspace conversions are still neccessary for further processing, and some other filters work better when used before resizing.
Having other filters operating in YV12 directly in DVD2AVI would certainly help a lot, except we still need Avisynth to feed the data in VDub. If we ever get a program that can encode the movie without using VDub and which handles YV12, all these problems will be solved.
I heard the XviD people are working on a new encoding program because it's needed for B-frame support. If this can be made to read DVD2AVI project files we're settled. Or maybe encoding straight away in DVD2AVI, but unless someone severly upgrades the current version it won't work to satisfaction (wasn't Nic working on that a while back?).

So as far as resize filters go, I believe the best way to make them faster right now is throwing in some optimization code or make it so they work in YUY2 exclusively (if that's possible).

[trbarry]

For people that want to use a good collection of filters then probably Avisynth/YUY2 is still the best bet. There is no reason why a new Xvid encoder program wouldn't be able to use Avisynth if it can read avi files.

Nic or somebody released a new version of DVD2AVI 1.86 today, see Doom9's news page. I'm not sure what all this does but I don't think it is based upon the one in Sourceforge I've been working on. I'll probably hold off on any more DVD2AVI changes until I see what the differences are and where it's going.

- Tom

[Nic]

T'was not me :) But gloval.

Although im not working directly from your source we are both based on the same :) (ill try to encompass your source into mine soon....especially liaor's transport stream code)

Ive got beta testers testing mine for all my horrendous bugs :) But it works well (rui is a brilliant beta tester BTW).

But this is very OT.

-Nic

[trbarry]

Nic -

Great. I was worrying HDTV support had disapeared. Btw, did you notice that awile back I entered a fix from Dmitryr in the P4 IDCT code? (idctmmx or some such member) That might not show up in other folks testing if they weren't using P4's. And it didn't crash, just made it uglier.

Anyway, glad to hear your's is still coming.

- Tom