Scale2X for Avisynth (Source available)

[morsa]

Well, here is my first filter.It isn´t an incredible thing and is slow as hell cause I've no idea about how to optimize it or port the mmx version of it.
It works nice for me with video images, didn't test it with anime (hope mf will do it).
By now it only works in RGB (24 or 32).
It gives better results when used with msharpen first.
Usage : NoName() (sorry no params).
That's all.
Ahh, please do not link directly to the file cause it's just a geocities page.


http://www.geocities.com/morsa20002000/morsafilter.html?1077427198530


P.S: In fact it is not meant to give a 2X image but to antialias the original resolution.For example you take a 640x480 image apply it 2 times and then resize it again to 640x480 with the resizer you like.

Feedback welcome...

[hanfrunz]

Hello morsa,

where can we find de source code? How does your filter work?
Can't you use PointResize(1280,960) instead?

hanfrunz

[Didée]

Now, that was not really the question to become "senior member", hanfrunz :D

[hanfrunz]

why not? Do you understand the difference between his filter and pointresize?

hanfrunz

[Didée]

Oooops ... take my apologies, hanfrunz.

I only say the filter's name and packed it on HD without trying it out immediately ... :o

:stupid:

When I read "HQ2X" and morsa's comment about resolution antialiasing, I understood it would do some calculations to interpolate at the upscaled resolution.

Now I tried it, and it seems it is indeed a resolution doubler "only". (Well, I myself could not even code that...)
Hmmm... subtracting a NoName'd from a pointresize'd clip shows big differences, but side-by-side they look just the same.

Welcome to the seniors, hanfrunz ;)

morsa?

- Didée

[morsa]

Well, it isn't supposed to be the same as Pointresize.
May be I did something wrong.
It has the effect of of affecting the strong edges of the image giving a kind of antialiasing effect.
Did anyone tried NoName twice and then downsample again to original resolution.Does it do the same as using Pointresize all the way?.
Today I'm going to try shifting the matrix to work only with the corners instead of left, wright, top and bottom pixels.
I tested it with a Monty Python episode which has subtitles and the subtitles really became better than the original ones.

[MfA]

Is this the same algorithm as the Scale2X used in emulators? As was said before, it will only work well on palletized pixel art ... which would explain why it works on subtitles :) Ill agree with mf here, you should really use hq2x instead if you want that kind of effect ... because of the thresholded difference comparison instead of the is-equal testing it will be more robust.

[morsa]

Exactly, It Is the Scale2X.I chose it because HQ2X tends to blur too much the rest of the image that is not an edge.
I saw the comparison between both and I liked Scale more.
But, because this is just testing to get feedback from you (all) to see if this is worth the effort I will continue with this thing till it gets exhausted.:D

[mf]

NoName, now that's really a cheesy name, considering SangNom... ;)

[morsa]

Suggestions welcome. (if someone has an idea how is the HQ2X implementation in C++ please explain it to me cause I can't understand exactly what it does.)

[MfA]

You could also try this (http://www.compuphase.com/graphic/scale2.htm) algorithm ... although I would try 75:25 averaging for the first 3 mixing options.

Little harder (only a very little though) in YV12 instead of RGB space, except for the difference which you can just compute using luminance only, because of the subsampled chrominance planes ... but that is left as an excercise to the reader.

Or dont ... I was planning on trying it myself eventually, so ... :)

[morsa]

Hurray, THX to MfA!!!!!!!!!!!
It's done.
After reading the link carefully I noticed that I was just halfway to it.The problem with the first version was that (I'm really newbie about programming and I don't know most of the common operations with pixels, you know?) ,although I was using a diagonal way to read the pixels ,I wasn't taking their distance from the source pixel to accurately select the correct ones.
New version, faaaaarrrrrrrr better, although it only supports RGB24 and it doesn't interpolate but copy pixels following the rules of Scale2.
If anybody wants to test it go for it, if not wait for a better version coming soon.....
Bye!

My script for a source of 480x368:

ConvertToRGB24
sharpen(0.3,0.3)
NoName()
Lanczosresize(720,576)

[morsa]

RGB32 and YV12 coming soon...

[morsa]

Finally new version.
Supports RGB24, RGB32, YV12.

YUY colorspace not supported yet cause I have problems implementing it.

[Didée]

Ah - now it looks like something!

Gave it a short try on some Window$ screenshots. Yummy! No chance to get something similar with (lanczos/bicubic)resize and/or Xsharpen and/or Warpsharp.

One thing I noticed: The placement of the image in the upsized frame is different between RGB and YV12 - and both are different from normal resize's placement.
And: when feeding an RGB with odd framewidth, then noname().noname() will crash.

Good work so far, thanks!

- Didée

[morsa]

Thank you Didée!
Good feedback.
I know about the displacement of the image.It is a problem I got because it is still very beta but I wanted to release it to see if it is of use or not.
Those corrections will be made when I finish YUY2 support.I'm planning to add a deinterlacer mode, an enhanced source matrix , an antialiasing mode (no resize) and may be a 3X mode.
By now I'm thinking about using LUTs (HQ2X style).Suggestions?


P.S. Why do you want to use an odd frame width?
Another question: How fast is it? (mean fps)

[mf]

*chemicals mix inside of mf's brain*
*smokey explosion from the chemical reaction*
OO, IDEA!

OptimalResize() - discriminating image types (natural, soft computer gfx, hard computer gfx) within single frames, and applying edge-directed, directional rounding, or hq2x interpolation, or an optional mix when unsure :D. That'd be fun :p.

[Wilbert]

Did you also plan to add the source?

[morsa]

of course!!!
why are you asking that?

[Wilbert]

I wanted to look at the source code, but didn't see it. No problem if you add it in a later stage :)

[morsa]

At this moment I have a version which supports all the colorspaces Avisynth can manage (2.5.XX).Thanks to SiWalters.
I also added three modes: sharp =0,1,2.
0= 50/50 average.
1=75/25 average.
2= Direct pixel replacement.
(Not available for download yet)


Corrected image position for YV12.
RGB32 still very buggy.I can't figure out why it throws an error when the filter is called twice.:confused:
It also suffers from something I will call "Acne" from now on.
It is mostly visible on Porno style skins.;)
Next step will be adding a 3 or 4X algo..( don´t know yet...)

[morsa]

Well, unless someone tells me the opposite I will stop development because I've been testing this filter and I believe Lanczos is really better at managing borders ( although it adds a little bit of ringing and mine decreases it).
Ideas welcome, again...

Source code available for download.

[Soulhunter]

Originally posted by morsa
Well, unless someone tells me the opposite I will stop development...

No, please dont stop until its finished... :o

Bye

[morsa]

Soulhunter , what's the speed (fps) in your machine?

[Soulhunter]

So, @ DVD resolution about 10-15 fps !!!

Bare I get about 24-28 fps... ;)


Bye

[morsa]

well, it isn´t so slow!!!
Could someone give me or lead me to a clear code for edge detection?
I also need high pass and low pass filter code.

[mf]

Originally posted by morsa
well, it isn´t so slow!!!
Could someone give me or lead me to a clear code for edge detection?
I also need high pass and low pass filter code.
Tried masktools yet? :)

[morsa]

But I'm looking for edge detection algorithms in C or C++.Like Prewitt or Sobel or something easier and better.
Are you saying I can use masktools source code?

[Manao]

Of course, you can fetch it here (http://www.geocities.com/manao47/Filters/masktools-v1.4.13.1.zip) (right click on the link, copy and paste in your adress bar)

But the C/C++ code is really simple for Sobel (it's just a convolution after all). The file to look in is EM_Func.cpp, and you can look at the code for Sobel, Laplace & co.

[MfA]

BTW I just read a paper the other day on Sobel and directionality ... if you want to find the direction of an edge from the responses to horizontal/vertical filtering you are better off with these masks :

3 0 -3 3 10 3
10 0 -10 0 0 0
3 0 -3 -3 -10 -3

[Xcandescent]

Originally posted by morsa
Well, unless someone tells me the opposite I will stop development
Please don't! I'm looking for a good line darkening/anti-aliasing filter for cartoons/anime, and the only one I've found that's come close to what I want is SangNom -- and that filter introduces a lot of black-dot/small black line artifacts (even at low settings like aa=15). I'd definitely be interested in seeing a final version of this filter with AA capabilities!

-XCN-

[pieter1976]

I calculated the direction of a edge the folowing way.



a = pixel[0 , 0]

vector.x := 0;
vector.y := 0;

for y := -1 to 1 do
begin
for x := -1 to 1 do
begin
dif := abs(a - pixel[x , y]);
vector.x := vector.x + x * dif;
vector.y := vector.y + y * dif;
end;
end;
result:=vector;

[morsa]

So?
Are you talking about border detection ?

[MfA]

That's an interesting approach. It gives the direction perpendicular to the edge I assume, does it work well? (Intuitively it would seem to have poor response if the edge goes straight through the pixel.)

[pieter1976]

It works oke but I haven't found any use for it yet.

If you draw a line from (x , y) to (x+vector.x , y+vector.y) on a gray image it gets clear what it does.
Use a position(x,y) close to a edge. And as MfA said The angle between the edge and the line is 90 degrees or perpendicular.

[pieter1976]

This I used for edge detection.
It's probably not the best but it works.


for y:=1 to imgheight-2 do
begin
for x:=1 to imgwidth-2 do
begin
value:=img[x,y]
edge:=0;
for y2:=-1 to 1 do
begin
for x2:=-1 to 1 do
begin
edge:=edge+abs(value-pixel[x+x2,y+y2]);
end;
end;
edge:=edge div somevalue;
if edge>255 then edge:=255;
ImgEdge[x,y]:=edge;
end;
end;

[pieter1976]

I have been thinking about using neuralnetworks for scaling.


you can first learn it how to scale by giving it image examples.
After the learning periode it should know how to do it.
I think that it is able to keep sharp edges sharp and blurry ones blurry

A friend of my is working on it at the moment.

He uses four inputs and one output and rotates it.
This way with one netwerk he can calculate four pixels.
It is a small network so it can be implemented easy.

I haven't see any results yet

[MfA]

There is a couple problems.

Such a small network is unlikely to be able to estimate the edge direction well ... and smoothness of edges is the most important part of the interpolation (more important than PSNR/MSE). That is the beauty of NEDI, as long as it doesnt fuck up, it is extremely smooth.

Also of course performance is dependent on the training, and the network is not guarantueed to perform well. HP recently published a paper (http://www.hpl.hp.com/techreports/2003/HPL-2003-26R1.html) where they weighted distortion such that the network was specifically trained to do well in hard and perceptually important areas.

BTW you might want try training the NN using the low-passed original rather than the original itself (I would suggest low passing with a 1 2 1 convolution mask, or rather it's 2D equivalent, and then subsampling to get the low rez version). Trying to synthesize high frequency data as well as doing smooth interpolation is probably not a good idea ... first things first.

Personally Im rather intrigued by the paper "Two-dimensional direction-based interpolation with local centered moments". It uses the same idea as "Image Magnification Using Level-Set Reconstruction" only instead of iteratively moving the interpolated pixels toward isovalue curves it determines the isovalue curves from isovalue directions determined from local centered moments (AFAICS a much more robust way of determining orientation than either sobel operators or your own edge direction measure). d'Oursse was implementing that second one, but I dont know if he intended to share any results.

This (http://www.securecms.com/icip2002/Papers/viewpapers.asp?papernum=2441) also sounds interesting, but I havent gotten a chance to read it yet.

[pieter1976]

I have some trouble reading IEEE documents.
This because I have no access.
Mfa are you a member?

[MfA]

Well no, I can just read the abstract (the uni has access to a lot of journals, but not the conferences). Which is why I havent read it yet, trying to get my hands on it though :)