My new UnFilter.dll (soften/sharpen)

[trbarry]

I capture some things like Buffy that are sent on digital TV but upconverted to some huge 1920x1080 resolution in the process. Even after I resize them to something more reasonable they are very soft.

Now previously I tried a number of times to use the built-in Avisynth Sharpen filter but it always crashed. But yesterday I read here (somewhere) that it was fixed in the 1b7 Avisynth release I'm now running, so I thought I'd try it. It did indeed work and looked pretty good, but it seemed very slow.

So I've ported some of my DScaler code that does a similar thing and released it today as an Avisynth filter, UnFilter.dll.

UnFilter is a simple and reasonably fast Avisynth Soften/Sharpen filter.

It implements 5-tap user adjustable horizontal and vertical filters designed to (slightly) reverse previous efforts at softening or edge enhancement that are common but ugly in DVD mastering. Since DVD's were intended originally for interlaced displays this has caused content providers to vertically filter them even a bit more to hide interlacing artifacts. I don't know why they sometimes over do the edge enhancement.

When softening, the UnFilter will attempt to approximate the inverse of a simple 3-tap edge enhancement filter. When sharpening it will attempt to approximate the inverse of a simple 3-tap softening filter.

More details are in the www.trbarry.com/Readme_UnFilter.txt file or see the source and DLL at www.trbarry.com/UnFilter.zip.

Any feedback appreciated.

- Tom

[ARDA]

@trbarry

As always I follow your work with great interest,so here my first report.
I have just made a short test,I daren't to give a depth opinion,but it
seems at first sight that at least we have a good and fast sharpen filter
for avisynth.Unluckly I've found some bugs or at least I think they are
with positive values in many cases it shows instead of the frame some lines,
most of them fine lines in black and some yellow areas.
Some examples where it shows those lines:
UnFilter(1,1) until (20,20) bug
but
UnFilter(1,31) is ok.
UnFilter(1,30) is ok.
UnFilter(1,29) bug
UnFilter(2,30) is ok.
UnFilter(20,30) is ok.
UnFilter(20,20) bug
UnFilter(31,30) is ok.
UnFilter(30,31) bug
UnFilter(31,31) bug
UnFilter(32,31) bug
UnFilter(32,30) is ok.

from UnFilter(40,40) until (100,100) it works ok,but I've also watched it accepts
values over 100 and under -100.

I hope this can be usefull.And thanks again for your great job.

ARDA

[trbarry]

ARDA -

Thanks for testing. Those are very strange results. I previously saw some of the lines you mentioned but only when I stacked up multiple instances of the filter to see what the effects would be. But I'll look into it.

But let me confirm; (1,29) will be a bug no matter what the source material is? I obviously did something silly here but hopefully it should be obvious. The interaction between the two values make me think there is a case where I get them mixed somehow. I'll check.

For now I'd suggest avoiding VSharp numbers right around 30. I think that is about the boundary condition where it actually will use a 5 tap filter since one term goes to zero below there abouts, and I optimized for that.

It is true that there is no range check currently. The results for values not -100 <= x <= 100 will be unpredictable, but probably ugly due to truncation errors. The range check in DScaler was in another part of the code I didn't port.

I only decided to do this yesterday afternoon, out for a walk with the dog. It just annoyed me that I liked the look of the now working Avisynth Sharpen command but it was too slow. So this whole thing was really just an impulse, designed to procrastinate working on finishing up something else.

- Tom

[trbarry]

It looks like I over optimized one case. The UnFilter will currently give incorrect results when the absolute value of VSharp is less than about 31 (32?) and that of HSharp is greater than that number (both non-zero).

I think there could also be a problem when the source and dest pitch don't match, which might happen right after a Crop() maybe. I should be able to get a fix out tommorrow sometime for both of these.

I still have no idea why UnFilter(20,20) would be a problem but I'll deal with that one later if it doesn't get fixed by serendipity tomorrow.

I think I also know why stacking UnFilters multiple times can cause a problem. This was interesting because it still seems to run faster with 5-6 UnFilter's instead of one Sharpen command, but gives incredibly ugly results.

- Tom

[trbarry]

I just put out a new version with fixes for the strange behavior with UnFilter(30,30), etc. Also added right click version info (0.0.1.1) and fixed a bug that could maybe have crashed P-II users if anyone had tried it.

Also fixed a potential problem with Avisynth buffer source and dest pitches not matched.

Same file locs:

www.trbarry.com/Readme_UnFilter.txt
www.trbarry.com/UnFilter.zip

- Tom

[dividee]

Is it just me or the archive is corrupt ?

[Doom9]

no.. I get the same..

[trbarry]

Grrrrrrrrr.....

Maybe SmartFTP hiccuped or something. The same zip is okay here but if I open it from the web it's busted.

But I just up-loaded it again and now I can open it and read all the members. So give it another try.

- Tom

[ARDA]

@tbarry
I couldn't make many tests with the new version.I've just
arrived home and I'm too tired; I've gotta work tomorrow.
I've still found some bugs round the 30 values.It is not
the same ,now it seems to me there is a problem in some width
loop,jumping some columns,every 4 maybe ,I can't be sure of that.
When the values of HSharp and VSharp are both negative I've watched
a problems of darkening ,or maybe some problems with
"potential problem with Avisynth buffer source and dest pitches not matched"
as you say.What I see are the four edges with a different bright.
The bigger the negative values ,the darker the image.
If HSharp > 0 and VSharp< 0 ,is ok. But If HSharp<0 and VSharp>0 the problem
is similar the one watched around the "30".Other information I'm working
on Athlon 1.4 , W2k and this the script:

LoadPlugin("C:\mpeg2dec_dll\mpeg2dec2\Release\mpeg2dec2.dll")
LoadPlugin("C:\Sharing\Release\Sharing.dll")
LoadPlugin("C:\copydvd\avisynth\UNFILTERBARRY\UnFilter300602\UnFilter.dll")
#####
MPEG2source"D:\MALDICIONESCORPIONDEJADE\MALDICION1771.d2v")
Crop(0,4,712,568)
SimpleResize(640,352)
UnFilter(51,-35)
Tweak(bright=5,cont=1.08,sat=0.75)

Thanks again ARDA

[trbarry]

ARDA -

Is UnFilter(51,-35) where the problem is? Or is it when ( HSharp<0 and VSharp>0 )? I'll test both. I admit more of my testing was with sharpening then softening, just because that's what I needed for myself at the moment.

But I don't notice any problems around the (30,30) area now. Could you post a pic?

It is true that UnFilter does 4 pixels at a time and the left 4 are not filtered. So it's something else to look at. But I thought I corrected the problem with the different pitches (I actually had had them backwards in the first version).

And I don't usually use Tweak. Are you trying to correct the problem with that?

Thank,

- Tom

edit: yep, I can dupl that every 4 pixels thing. Don't know why it happens yet though.

[ARDA]

@tbarry
I'm sorry but I had a lot of work today and I've just sat in front
of my computer.The UnFilter(51,-35),was accidentally the last value
I've tested ,of a serie of random ones.I've included Tweak to increase
contrast to be able to see if I could find any problemdirectly in VDub
using 2Xsize in a 21" monitor.My apologizes the problem when
HSharp<0 and VSharp>0 is always around "30".I'll try to attach two
examples.One for both negative and the other the case above mentioned.

ARDA

[trbarry]

I just put out another version with more bug fixes.

Same file locs:

www.trbarry.com/Readme_UnFilter.txt
www.trbarry.com/UnFilter.zip

ARDA -

This should correct your errors with small and mixed parm values, including the 'every 4 pixels' error and the dark screens.

- Tom

[ARDA]

@Tom

Just two quick tests ,I did't find anything wrong.
I'll do more when I'm back from job.
Once more great job.Thanks

ARDA

[trbarry]

ARDA -

Thanks for all the work you've done on this. Hopefully I didn't overlook anything else incredibly dumb.

- Tom

[trbarry]

Someone emailed me today about this so let me add a few words about the softer side of UnFilter ...

The negative (softening) parm values were a first attempt at an algorithm to remove overzelous Edge Enhancement. This is a problem that bothers videophiles with expensive hi rez projectors on some movies, though it might not show up anyway unless you were making very high rez high bit rate encodings. For more info, see Bjoern Roy's excellent page on the issue, at the Ultimate Guide to Edge Enhancement

My experiment was only partly successful as UnFilter is sort of a 5-tap filter that can unwind a 3 tap EE filter. So UnFilter with softness settings can help a bit only if the EE isn't too wide. Also, a little bit of softening can give some movies back a film look. But in order to have a really great effect on EE you might have to over soften the clip. OTOH, that might make it compress better.

Still experimental. YMMV.

Here's a sample matching one sample on Bjoern Roy's page from Terminator 2, a known EE offender, plus a couple sample efforts at removing the EE.

www.trbarry.com/T2_orig_EE.jpg
www.trbarry.com/T2_softer.jpg
www.trbarry.com/T2_Softest.jpg

The pics above were actually done using the real time DScaler version of UnFilter built into the Greedy/HM deinterlace code, but the UnFilter code there is substantially the same as the Avisynth version.

- Tom

[Bluedan]

couldn't reach your site, please take care, I'm curious about your filter...

[trbarry]

Bluedan -

Dunno. Maybe try again. It seems to be up now.

- Tom

[Boulder]

@trbarry: This might be a rather simple (and stupid!) question, but does UnFilter(40,40) do almost the same sharpening effect as Sharpen(0.4)? As markrb suggested using the sharpen function for optimal SVCD quality, I'd much rather use this excellent and fast filter you've created than the slower one built in Avisynth.

Keep up the good work - the scene needs people like you

[trbarry]

@trbarry: This might be a rather simple (and stupid!) question, but does UnFilter(40,40) do almost the same sharpening effect as Sharpen(0.4)?

Boulder -

Nope. Almost certainly not. The algorithm is designed to be the inverse of something else I previous wrote and the '40' is scaled to be about 40% of the way to where I start getting artifacts due to overflow errors.

I just wanted it normalized to a range of -100 through 100 because these values first appeared slider controls in the DScaler project.

- Tom

[Dreassica]

Great filter!! The encoding time in CCE 2.5 creating SVCD's using unfilter.dll in the avs script has dropped from 50 min. to 30 in each pass and the results are the same as the slow sharpen command!!