My new UnFilter.dll (soften/sharpen)

[trbarry]

I capture some things like Buffy that are sent on digital TV but upconverted to some huge 1920x1080 resolution in the process. Even after I resize them to something more reasonable they are very soft.

Now previously I tried a number of times to use the built-in Avisynth Sharpen filter but it always crashed. But yesterday I read here (somewhere) that it was fixed in the 1b7 Avisynth release I'm now running, so I thought I'd try it. It did indeed work and looked pretty good, but it seemed very slow.

So I've ported some of my DScaler code that does a similar thing and released it today as an Avisynth filter, UnFilter.dll.

UnFilter is a simple and reasonably fast Avisynth Soften/Sharpen filter.

It implements 5-tap user adjustable horizontal and vertical filters designed to (slightly) reverse previous efforts at softening or edge enhancement that are common but ugly in DVD mastering. Since DVD's were intended originally for interlaced displays this has caused content providers to vertically filter them even a bit more to hide interlacing artifacts. I don't know why they sometimes over do the edge enhancement.

When softening, the UnFilter will attempt to approximate the inverse of a simple 3-tap edge enhancement filter. When sharpening it will attempt to approximate the inverse of a simple 3-tap softening filter.

More details are in the www.trbarry.com/Readme_UnFilter.txt file or see the source and DLL at www.trbarry.com/UnFilter.zip.

Any feedback appreciated. ;)

- Tom

[ARDA]

@trbarry

As always I follow your work with great interest,so here my first report.
I have just made a short test,I daren't to give a depth opinion,but it
seems at first sight that at least we have a good and fast sharpen filter
for avisynth.Unluckly I've found some bugs or at least I think they are
with positive values in many cases it shows instead of the frame some lines,
most of them fine lines in black and some yellow areas.
Some examples where it shows those lines:
UnFilter(1,1) until (20,20) bug
but
UnFilter(1,31) is ok.
UnFilter(1,30) is ok.
UnFilter(1,29) bug
UnFilter(2,30) is ok.
UnFilter(20,30) is ok.
UnFilter(20,20) bug
UnFilter(31,30) is ok.
UnFilter(30,31) bug
UnFilter(31,31) bug
UnFilter(32,31) bug
UnFilter(32,30) is ok.

from UnFilter(40,40) until (100,100) it works ok,but I've also watched it accepts
values over 100 and under -100.

I hope this can be usefull.And thanks again for your great job.

ARDA

[trbarry]

ARDA -

Thanks for testing. Those are very strange results. I previously saw some of the lines you mentioned but only when I stacked up multiple instances of the filter to see what the effects would be. But I'll look into it.

But let me confirm; (1,29) will be a bug no matter what the source material is? I obviously did something silly here but hopefully it should be obvious. The interaction between the two values make me think there is a case where I get them mixed somehow. I'll check.

For now I'd suggest avoiding VSharp numbers right around 30. I think that is about the boundary condition where it actually will use a 5 tap filter since one term goes to zero below there abouts, and I optimized for that.

It is true that there is no range check currently. The results for values not -100 <= x <= 100 will be unpredictable, but probably ugly due to truncation errors. The range check in DScaler was in another part of the code I didn't port.

I only decided to do this yesterday afternoon, out for a walk with the dog. It just annoyed me that I liked the look of the now working Avisynth Sharpen command but it was too slow. So this whole thing was really just an impulse, designed to procrastinate working on finishing up something else. ;)

- Tom

[trbarry]

It looks like I over optimized one case. The UnFilter will currently give incorrect results when the absolute value of VSharp is less than about 31 (32?) and that of HSharp is greater than that number (both non-zero).

I think there could also be a problem when the source and dest pitch don't match, which might happen right after a Crop() maybe. I should be able to get a fix out tommorrow sometime for both of these.

I still have no idea why UnFilter(20,20) would be a problem but I'll deal with that one later if it doesn't get fixed by serendipity tomorrow. ;)

I think I also know why stacking UnFilters multiple times can cause a problem. This was interesting because it still seems to run faster with 5-6 UnFilter's instead of one Sharpen command, but gives incredibly ugly results.

- Tom

[trbarry]

I just put out a new version with fixes for the strange behavior with UnFilter(30,30), etc. Also added right click version info (0.0.1.1) and fixed a bug that could maybe have crashed P-II users if anyone had tried it.

Also fixed a potential problem with Avisynth buffer source and dest pitches not matched.

Same file locs:

www.trbarry.com/Readme_UnFilter.txt
www.trbarry.com/UnFilter.zip

- Tom

[dividee]

Is it just me or the archive is corrupt ?

[Doom9]

no.. I get the same..

[trbarry]

Grrrrrrrrr.....

Maybe SmartFTP hiccuped or something. The same zip is okay here but if I open it from the web it's busted.

But I just up-loaded it again and now I can open it and read all the members. So give it another try.

- Tom

[ARDA]

@tbarry
I couldn't make many tests with the new version.I've just
arrived home and I'm too tired; I've gotta work tomorrow.
I've still found some bugs round the 30 values.It is not
the same ,now it seems to me there is a problem in some width
loop,jumping some columns,every 4 maybe ,I can't be sure of that.
When the values of HSharp and VSharp are both negative I've watched
a problems of darkening ,or maybe some problems with
"potential problem with Avisynth buffer source and dest pitches not matched"
as you say.What I see are the four edges with a different bright.
The bigger the negative values ,the darker the image.
If HSharp > 0 and VSharp< 0 ,is ok. But If HSharp<0 and VSharp>0 the problem
is similar the one watched around the "30".Other information I'm working
on Athlon 1.4 , W2k and this the script:

LoadPlugin("C:\mpeg2dec_dll\mpeg2dec2\Release\mpeg2dec2.dll")
LoadPlugin("C:\Sharing\Release\Sharing.dll")
LoadPlugin("C:\copydvd\avisynth\UNFILTERBARRY\UnFilter300602\UnFilter.dll")
#####
MPEG2source"D:\MALDICIONESCORPIONDEJADE\MALDICION1771.d2v")
Crop(0,4,712,568)
SimpleResize(640,352)
UnFilter(51,-35)
Tweak(bright=5,cont=1.08,sat=0.75)

Thanks again ARDA

[trbarry]

ARDA -

Is UnFilter(51,-35) where the problem is? Or is it when ( HSharp<0 and VSharp>0 )? I'll test both. I admit more of my testing was with sharpening then softening, just because that's what I needed for myself at the moment.

But I don't notice any problems around the (30,30) area now. Could you post a pic?

It is true that UnFilter does 4 pixels at a time and the left 4 are not filtered. So it's something else to look at. But I thought I corrected the problem with the different pitches (I actually had had them backwards in the first version).

And I don't usually use Tweak. Are you trying to correct the problem with that?

Thank,

- Tom

edit: yep, I can dupl that every 4 pixels thing. Don't know why it happens yet though.

[ARDA]

@tbarry
I'm sorry but I had a lot of work today and I've just sat in front
of my computer.The UnFilter(51,-35),was accidentally the last value
I've tested ,of a serie of random ones.I've included Tweak to increase
contrast to be able to see if I could find any problemdirectly in VDub
using 2Xsize in a 21" monitor.My apologizes the problem when
HSharp<0 and VSharp>0 is always around "30".I'll try to attach two
examples.One for both negative and the other the case above mentioned.

ARDA

[trbarry]

I just put out another version with more bug fixes.

Same file locs:

www.trbarry.com/Readme_UnFilter.txt
www.trbarry.com/UnFilter.zip

ARDA -

This should correct your errors with small and mixed parm values, including the 'every 4 pixels' error and the dark screens.

- Tom

[ARDA]

@Tom

Just two quick tests ,I did't find anything wrong.
I'll do more when I'm back from job.
Once more great job.Thanks

ARDA

[trbarry]

ARDA -

Thanks for all the work you've done on this. Hopefully I didn't overlook anything else incredibly dumb.

- Tom

[trbarry]

Someone emailed me today about this so let me add a few words about the softer side of UnFilter ...

The negative (softening) parm values were a first attempt at an algorithm to remove overzelous Edge Enhancement. This is a problem that bothers videophiles with expensive hi rez projectors on some movies, though it might not show up anyway unless you were making very high rez high bit rate encodings. For more info, see Bjoern Roy's excellent page on the issue, at the Ultimate Guide to Edge Enhancement (http://home.t-online.de/home/bjoern.roy/Guide_EE/Html/Page_01.htm)

My experiment was only partly successful as UnFilter is sort of a 5-tap filter that can unwind a 3 tap EE filter. So UnFilter with softness settings can help a bit only if the EE isn't too wide. Also, a little bit of softening can give some movies back a film look. But in order to have a really great effect on EE you might have to over soften the clip. OTOH, that might make it compress better.

Still experimental. YMMV.

Here's a sample matching one sample on Bjoern Roy's page from Terminator 2, a known EE offender, plus a couple sample efforts at removing the EE.

www.trbarry.com/T2_orig_EE.jpg
www.trbarry.com/T2_softer.jpg
www.trbarry.com/T2_Softest.jpg

The pics above were actually done using the real time DScaler version of UnFilter built into the Greedy/HM deinterlace code, but the UnFilter code there is substantially the same as the Avisynth version.

- Tom

[Bluedan]

couldn't reach your site, please take care, I'm curious about your filter...:)

[trbarry]

Bluedan -

Dunno. Maybe try again. It seems to be up now.

- Tom

[Boulder]

@trbarry: This might be a rather simple (and stupid!) question, but does UnFilter(40,40) do almost the same sharpening effect as Sharpen(0.4)? As markrb suggested using the sharpen function for optimal SVCD quality, I'd much rather use this excellent and fast filter you've created than the slower one built in Avisynth.

Keep up the good work - the scene needs people like you:)

[trbarry]

@trbarry: This might be a rather simple (and stupid!) question, but does UnFilter(40,40) do almost the same sharpening effect as Sharpen(0.4)?

Boulder -

Nope. Almost certainly not. The algorithm is designed to be the inverse of something else I previous wrote and the '40' is scaled to be about 40% of the way to where I start getting artifacts due to overflow errors. ;)

I just wanted it normalized to a range of -100 through 100 because these values first appeared slider controls in the DScaler project.

- Tom

[Dreassica]

Great filter!! The encoding time in CCE 2.5 creating SVCD's using unfilter.dll in the avs script has dropped from 50 min. to 30 in each pass and the results are the same as the slow sharpen command!!

[trbarry]

Dreassica -

Thanks for your encouraging words. AFAIK you are only the 2nd person to try this filter, besides myself.

Glad I wasn't trying to sell it. ;)

- Tom

[manono]

Hi-

You sell yourself short Tom. I jumped on it as soon as it came out, as I've been trying to find something to lessen the effects of Edge Enhancement (I've run across a lot of it recently). Temp Smoothers usually damage the film before cleaning up EE and the associated pixelation. Unfortunately, my tests show that you have to blur-soften the film too much before it has much of an effect on a movie with bad EE. But combined with a light TemporalSoften2, it's not too bad.

However, as Dreassica said, as a sharpener for soft sources, it's great, and way faster than the Sharpen filter. They're just now getting wind of it down at the SVCD Forums where a lot of people like to sharpen up the source (for some reason) and have only had the Sharpen Filter before now. So you may hear some more reactions before too long.

[Boulder]

Since you wanted feedback, here's some: it's simply magnificent! I've encoded three TV captures and using a rather small value for UnFilter gives the same results as Sharpen ( +bicubic 0.00/0.75 ) but is a helluva lot faster!

Although, it seems to crash Avisynth (the infamous read access violation error) in Win XP sometimes. Win98SE has worked flawlessly.

[Holomatrix]

Hello, I just read about this new filter. What values would you use to get a sharpen of .5? I tried UnFilter(15,15) and did not notice any difference. My max ave bitrate was 2230, would the filter work better with low bitrates? and do you recommend using the higher values for Bicubic? (Increase from .6 to .8?)
Thanks

[trbarry]

Sorry, the UnFilter sharpen values don't correspond nicely to other filter values in Sharpen, BiCubicResize, etc. I'd just try raising them until you either get too much sharpening or reach 100.

- Tom

[trbarry]

Although, it seems to crash Avisynth (the infamous read access violation error) in Win XP sometimes. Win98SE has worked flawlessly.

Is UnFilter somehow implicated in the crash? Could you provide some more info?

And you might try updating to Avisynth beta 6 or 7 if it's not just an UnFilter problem.

- Tom

[trbarry]

@all

Glad to hear that some of you are using this after all. Thanks for the feedback.

- Tom

[Gant]

i come a little late... but still.. i'm using this filter and i'm finding it very usefull to me... thnaks for this great filter m8 :)

Again Thnaks for u'r hard work :)

Matta.

Gant

[Dreassica]

I haven't had any crashes with the unfilter on yet with my XP system!!

[Boulder]

Originally posted by trbarry
[B]

Is UnFilter somehow implicated in the crash? Could you provide some more info?

And you might try updating to Avisynth beta 6 or 7 if it's not just an UnFilter problem.


I'm currently using beta 6 I think. I've had crashes when using UnFilter but not without it. However, I've found out that XP sometimes speeds up the encoding so much that it crashes (sic!), especially when Sharpen is present. Win98 doesn't go as fast but is more stable.

Unfortunately I can't give any specific details yet as I've only done a couple of encodes, but as I said, Win98 works like a charm!

[soulfx]

I just ran across a strange side effect of using the UnFilter. I use an Animated MergeChroma and MergeLuma to blend in my name and episode info into the first couple seconds of video. This will usually drop the FPS during those seconds of blending down to around 8 FPS. However I noticed that if I have some form of UnFilter in the AVS script (and it doesn't even have to be set to do anything (Like UnFilter(0,0))) it will increase my FPS up to it's normal real-time 23-24FPS! That's sweet, but uh, I don't think it's supposed to do that.

I'm not sure if this speed improvement can be applied to other Filter's as I haven't tried yet, but I sure did a double take when I noticed it.

Any info or thoughts?

[trbarry]

soulfx -

I'm not sure how Avisynth does it but any time you combine multiple clips of different frame rate into one (using ANY filters) you maybe will end up with a single constant frame rate.

- Tom

[Holomatrix]

Any chance on getting some samples to show or if you can let me know what settings would you use (unfilter(20,20))?
Thanks

[trbarry]

I don't have any sharpening samples, or even any additional recommendations, but I posted some softening samples in the discussion on removing EE earlier in this thread.

- Tom

[Dreassica]

Maybe a stupid question, but isn't DivX Yuy2 compliant, because i used the unfilter in my avs to convert divx to SVCD, but it won't open the AVS in Vdub, stating that unfilter only supports Yuy2 color format. When i tried searching for explanation i found out it was the first DivX 3.11 file i tried encoding (the earlier ones were all DivX5).. I distictly remember reading that divX was Yuy2 native.

[trbarry]

I don't know what format Divx 3.11 uses. But you could probably just put in a ConverttoYUY2 statement.

- Tom

[Dreassica]

I did that, it now will load into Vdub, but i still get the crashes in CCE!!

[h00z]

I'm having some issues with Unfilter in my anime encoding. It seems that I end up with vertical "bars" running through my encodes with the settings I'm using (tried more than a few different settings of course).

Are you guys finding that you need different values for horizontal and vertical when sharpening?

[trbarry]

I have seen the vertical bars when I sharpen it too much, or especially when I tried sharpening it twice. But it seems to depend upon the material. Some video already has some edge enhancement and sharpening will make it worse.

Also, Vlad just sent me a fix, pointing out that this can crash on a K6-II, using an unsupported instruction. I'll try to incorporate that and make it available soon but anyone experiencing this please PM me.

- Tom

[lovelegend]

I'm a newbie amd I'm going to find a suitable filter chain for myself. Therefore, I would like to ask if I want to convert DVDrips to SVCD's, do I still use unfilter?

[WarpEnterprises]

Unfilter is exactly made for that purpose.
But use it only if the picture looks unsharp, because by sharpening unnecessarily you need more bitrate afterwards.

[iago]

Up! UnFilter back into the scene! ;)

@Tom,

Recently I've been experimenting using UnFilter with both (+,+) and (-,-) settings in various DVD test rips. I have a few questions I'm still looking the answers for and I'd be very glad if I get some reply directly from the author of this great filter ;).

If you remember the "lumi-masking" thread in the XviD forum, it was discussed that using UnFilter (somehow ;)) "helped" solving the black-blocking issue. I guess this may be due to some darkening effect introduced by UnFilter with both + and - settings. Can you confirm/reproduce such a darkening effect introduced by UnFilter?

Also, the issue of vertical bars with even some light settings such as UnFilter(5,5) seem to be more or less but consistently appearing in almost all my encodes with many different sources. Is it a bug or actually related to the sharpness/EE of the source?

Finally, I get some thin light lines at all four sides of the image when using UnFilter, seeming to be regardless of the resizing method and resizing parameters I use. Any ideas on that?

best regards,
and thanks again for your great filter,
iago

[iago]

@Tom

To be more specific:

* darkening effect -> both when softening and sharpening

* vertical bars -> when sharpening

* thin light lines on all four sides of the frames -> when softening (especially)

regards,
iago

[trbarry]

iago -

I have absolutely no idea why UnFilter helps with the lumi masking problem, though enough people have confirmed it that I can't argue.

I don't think unfilter darkens anything. If so, that would be a bug. But, if it turned out that UnFilter was slightly darkening everything, should I fix it? Or leave it alone to help with the lumi problem? ;)

I suspect the vertical bars are UnFilter sharpening some cyclical compression (block?) artifact that is already in the source material but have no way to prove that. The only thing I can say here is be careful not to over sharpen. I've previously searched long and hard to see if UnFilter was doing something wrong in this area but could not find anything.

Could you post a pic with the light lines at the edge? And what release are you using? I think there is version info if you right click on the dll.

- Tom

[iago]

@Tom,

First, thanks for the quick reply and your interest.

* The UnFilter.dll version is 0.0.1.1, dated 29.06.2002.

* The (slight) darkening effect of UnFilter is certain imho (and I can also post screenshots for comparison from many different test rips to prove it), but as you said it would be much better to leave this effect "as is" imho ;), since it is really very helpful against one of the ugliest problems of the encoding scene -> black-blocking! :p

* I have attached a sample image with light lines at edges (UnFilter(-10,-10) / XviD / BicubicResize(640,352,0,0.5) / MPEG2DEC2.dll / avisynth 2.06)

regards,
iago

[iago]

Sorry,

The sample image with light thin lines at edges is attached to this post.

regards,
iago

[iago]

Tom,

Any ideas on these light lines at edges (or the UnFilter.dll version I use)?

iago

[trbarry]

iago -

Sorry, I can't even see the light edges, but maybe that is my image preview on this machine with white borders around it.

But there is a newer version 0.0.1.2 (from July 2) that fixes a couple things when using low parm values. You might try that first.

www.trbarry.com/UnFilter.zip

- Tom

[iago]

-> Sorry, I can't even see the light edges, but maybe that is my image preview on this machine with white borders around it. (trbarry)

Tom,

It's OK, with the 0.0.1.2 version I don't see them light edges either! ;)

regards,
iago

[trbarry]

Ha! I weasled out of another one! ;)

- Tom

[Dark-Cracker]

@trbarry

hi, i am actually working on a guide for my tools, i have make some screenshot for the unfilter, i post it here because perhaps this will help u if u want some graphical exemple in your doc.

i have make a subtract beetween the 2 pictures (the original source and the filtered source) the substcart function show the different pixel beetween the 2 pictures, an a modification on the color show the new pixels appeared .

unfilter light : 15,15
unfilter medium : 25,25
unfilter heavy : 40,40

only UNFILTER :
http://www.eclipsedvd.firstream.net/autodub/Filter_screenshot/unfilter.jpg

the sharp filter increase noise and noise increase final file size it for that we generaly use a denoise filter before use the unfilter, i have make some screen shot :

UNFILTER & temporalsoften2(2,5,5) :
http://www.eclipsedvd.firstream.net/autodub/Filter_screenshot/unfilter_soften.jpg

UNFILTER & convolution3d(1,4,5,3,4,2.8,0) :
http://www.eclipsedvd.firstream.net/autodub/Filter_screenshot/unfilter_convolution.jpg


UNFILTER & Temporalsmoother(1) :
http://www.eclipsedvd.firstream.net/autodub/Filter_screenshot/unfilter_smooth.jpg

i hope this will perhaps help some people.
i knox unfilter was not specified for mangas, and when i have make the sceenshot with the movie sample i will paste it.

Ps: sorry for my bad english and for the huge size of the picture (jpeg quality 100%).

Bye

[trbarry]

Dark-Cracker -

Interesting samples. But the total extent of my doc is usually just a Readme.txt file so there's not much place for nice graphics.

I suppose I should work on this but there always seems to be something else to do. I am not very good at packaging.

- Tom

[cult]

@trbarry
will you please make yv12 version for unfilter?

[iago]

@cult

Was asked before by an addict of the filter! ;)
Not in Tom's top-priority list, though I wish it were ;).

regards,
iago

[cult]

But I am addicted too.And his great tomsmocomp.Although I live in a country with pal,sometimes I have to use it.Its the best deinterlacer filter imho.
@Iago
thank you Iago for all your suggestions with xvid.

[trbarry]

FWIW, I just put out an experimental YV12 UnFilter, for Avisynth 2.5 Alpha testing only. See:

www.trbarry.com/Readme_UnFilter.txt and
www.trbarry.com/UnFilter.zip (source and dll)

Somebody please let me know if this immediately crashes. My recent version of YV12 TomsMoComp seems to crash for everybody except me so maybe I'm using the wrong libraries or something. But if it crashes please post your script and say which CPU and OS you are using.

There are 2 versions in the zip, one for Avisynth 2.5a with YV12 support and another which is a copy of the old one. Of course use the right one one.

For YV12 support only, I decided to also soften/sharpen chroma values, unlike for YUY2 and the old one. It seems to work okay but if it turns out to be a dumb idea let me know and I'll make that optional or just turn it off. But it was easier to do with YV12.

- Tom

[scmccarthy]

@Tom

Both versions 0.0.1.1 and 0.0.1.2 are kept on my computer because the latter caused vertical lines unfilter(-15,0) that look a little like television. I like the noise it produces even if it is inadvertant. Too bad mmx code really baffles me, I am curious how unfilter works.

[Gaia]

Sorry Tom just tried your YV12 Unfilter. There's huge green and black line in the right border and also "pixel width to be a multiple of 4" causes huge cropping problems.

Anyway thanks for lots of very nice filters. I like Undot a lot.

[trbarry]

I forgot to update the version info on the one I posted last night. But could both of you guys with the problems look at the copies of UnFilter.dll you are specifying for YV12 and verify that the date on the module was yesterday?

Also what processor and parameters are you using when you have YV12 problems.

- Tom

[iago]

Tom,

First, thanks a lot for working on the YV12 version of UnFilter and making a release. I can confirm the problem of "thick green line at the right side of picture" when using the YV12 version.

Also, the UnFilter.dll in the "UnFilter_Avisynth_2.5_Alpha" directory of the package is dated 14.11.2002.

I'm using Windows XP SP1 and Celeron 900.

thanks and best regards,
iago

edit: I tried different -/+ settings but the green line at the right border is always there.

[Gaia]

I am using latest version. Tried lots of different settings. Example 5,5 and 10,10 etc. I have Duron 950 and 98se. No crashes but green and also black line at the right border.

[cult]

Thank tom for your great filter.
The good news no green line or any line anywhere...
Just a small clip encoded though.
My settings were these:
LoadPlugin("C:\Program Files\AviSynth2\plugins\Convolution3DYV12.dll")
LoadPlugin("C:\Program Files\AviSynth2\plugins\MPEG2Dec3.dll")
LoadPlugin("C:\Program Files\AviSynth2\plugins\Unfilter.dll")
mpeg2source("D:\ÍÝïò öÜêåëïò (2)\cop.d2v")
crop(0,2,716,569)
BicubicResize(704,384,0,0.75)
Unfilter(-100,-100)
Used koepis 9/11 build,bframes(3,150)chroma motion
celeron @1ghz w2k

[scmccarthy]

@trbarry

I have XP SP1 and Pentium 1.8 GHz processor (doesn't get any better than that does it?). I cannot see any problems with Unfilter(-30,0).
Well, I do think UnFilter adds a pleasing amount of noise that I think helps to avoid blocking. I know that is not why you wrote the filter in the first place, but I think inadvertantly you stumbled on something pretty good.

Previously, I asked you for an example of mmx coding, what I really want is a way to compare the algorythm in cpp with the same in assembly. Similiarily, I'd love to see UnFilter in pseudocode.

However, I have been asking for that in the full knowlegde that I already have a great example of what I am looking for in the AviSynth 2.5 sources. Convert.cpp RGB helper class method RGB32To24() was converted to mmx code with the original cpp code commented out. This is a great chance to compare the two. So you see, I am not helpless; I can figure this stuff out for myself given enough time. I don't want to come off like I am really demanding a lot of your time, but this kind of thing (i.e. releasing some of your programs in pseudocode or c and the example I sited) might interest a lot of people.

Back to green in UnFilter: I haven't encoded anything with it yet, but it looks good in the preview.

[drebel]

A big thanks to the whole community for the great filter and for all the testing so far..
About Unfilter.dll,the problem with the green bar on the right occurs only when i use it AFTER RESISING in the following script:

LoadPlugin("C:\Program Files\Avisynth2.5\MPEG2Dec3.dll")
#LoadPlugin("C:\Program Files\Avisynth2.5\TomsMoComp.dll")
LoadPlugin("C:\Program Files\Avisynth2.5\UnDot.dll")
LoadPlugin("C:\Program Files\Avisynth2.5\UnFilter.dll")
LoadPlugin("C:\Program Files\Avisynth2.5\Convolution3DYV12.dll")
#LoadPlugin("C:\Program Files\Avisynth2.5\Focus2.dll")
clip=mpeg2source("D:\Movies\war\Hart's war.d2v",cpu2="ooooxx")
Part1=trim(clip,0,422)
Part2=trim(clip,423,172720)
Part3=trim(clip,172721,0)
Start=Part1.crop(2,76,716,424).LumaFilter().BilinearResize(672,272).Convolution3D (1, 32, 128, 32, 128, -1, 0)
Movie=Part2.crop(2,76,716,424).LumaFilter().Undot().LanczosResize(672,272).UnFilter(5,5)
Credits=Part3.crop(2,76,716,424).LumaFilter().BilinearResize(672,272).Convolution3D (1, 32, 128, 32, 128, -1, 0)
Return Start+Movie+Credits

Of course you can detect the use of YV12,but the use of the filter is without further problems if used BEFORE CROPPING(although i'm trying to keep cropping and resizing within mod4 limits...).I can't notice any mosquito noise with the combination of deringing (cpu2="ooooxx") and UnFilter(5,5) in such a clean movie(HART'S WAR-R2).FPS loss is also quite tolerable,as usual for unfilter(about 15 fps on a Duron 1GHZ) WITH Undot.dll.Hope i helped...

regards,
George

Edit: Trying to reproduce the "bug",i noticed that i accidently posted the WRONG script.It only appears when Unfilter is placed AFTER cropping ,also AFTER LumaFilter(),but NOT after Undot() or LanczosResize(strange?)...Sorry for the inconvenience

[MaTTeR]

Vertical green line showed up on right hand side of the screen for me using-

LoadPlugin("C:\Program Files\AviSynth2\plugins\MPEG2DEC3_YV12.dll")
LoadPlugin("C:\Program Files\AviSynth2\plugins\undot.dll")
LoadPlugin("C:\Program Files\AviSynth2\plugins\unfilter_yv12.dll")
mpeg2source("C:\test.d2v",iDCT=2,cpu2="xoooxo",moderate_h=35,moderate_v=55)
crop(4,54,708,366)
Unfilter(-3,-3)
BicubicResize(608,260,0.5,0.5)
Undot()
Limiter()

Encoded using-
Koepi's Dev Build- 11-09-2002
AVIsynth build 11-11-2002
Dual AMD 1600s w/512MB DDR
Win2k w/SP3

Thanks for the effort on this port Tom!

BTW- To the right of the vertical green line I also noticed a white vertical line flashing sometimes. Anyone else notice this yet?

[drebel]

Sorry,a black line here on the right, 3 times thinner than the green one ...

[cult]

My cousin drebel was right.This avs:
Movie=part2.crop(4,4,712,566).lumafilter().Unfilter(5,5).Convolution3d (preset="movieHQ").BicubicResize(704,384,0,0.75)
produses green line
This one:
Movie=part2.crop(4,4,712,566).lumafilter().Convolution3d (preset="movieHQ").BicubicResize(704,384,0,0.75).Unfilter(5,5)
doesnt.
Its the same avs,only the place of the unfilter changed.

[Gaia]

Sorry quys but i doesn't matter in which order filters are. There's always black and green vertical line. I tried to put Unfilter in every possible place. I used very basic scripts. Just cropping and resizing.

[MaTTeR]

Tom,

FWIW, I've been testing Vlad's SMP/multithreaded Convolution3D this past week and have noticed the "green line" problem with his dll as well. However, the problem goes away if I use Convolution3D(preset"movieHQ") after the BicubicResize line. I suspect the same will apply to Unfilter. Perhaps this issue is with the latest 2.5 AVS(11-11-2002)?

[cult]

I just finished the encoding of a whole movie with simple resize.No green line,no black line,no nothing:)
But,I noticed a strange thing.I copied the whole avs in a new file.No green line.Replaced Bicubicresize with lanczos.Green line appeared.Replaced lanczos with bicubic as before.GREEN LINE!
I had to copy again the whole avs to make green line disapear.Strange...

[trbarry]

Everybody -

Sorry, I spent yesterday figuring out why TomsMoComp crashs so I didn't even look at UnFilter. But I should be able to get it today.

The green lines at the right are usually a multiple of x problem, 4,8,16 ... where filters leave garbage at the right. Binary zero's (YUV=<0,0,0>) show green in YUY2 or YV12.

BTW, I forgot to mention I owe Vlad59 credit for sending me a new version of DoSoftSharp.inc a couple months ago that does not crash on K6 400 and similar CPU's. Sorry Vlad, I just finally added that this week.


cult -

If you are using SimpleResize then you are not using YV12. I don't think there is any support for it yet unless somebody else wrote it. (but that would be nice)


scmccarthy -

Not full vertical pseudo code, but basically the vertical softening just does:


For all pixels P(x,y) on the screen:
outputP(x,y) = ( QVA * P(x,y)
+ QVB * Average( P(x,y-1), P(x,y+1)
+ QVC * Average( P(x,y-2), P(x,y+2) ) >> 6)


... taking care to not overflow 0 < x < 64k and not fall off the edge of the screen. There is a huge block of comments in UnFilterAll.inc showing how the QVA, QVB, QVC are calculated to sum to 64 in a way that somewhat approximates the inverse of a simple 3 tap soften or sharpen filter. That code is done in C.

It looks like for YV12 I'm not quite handling the "falling off the edge of the screen" part quite right yet. ;)

The actual softening/sharpening code is in member DoSoftSharp.inc.

- Tom

[vlad59]

Originally posted by trbarry
BTW, I forgot to mention I owe Vlad59 credit for sending me a new version of DoSoftSharp.inc a couple months ago that does not crash on K6 400 and similar CPU's. Sorry Vlad, I just finally added that this week.


Don't worry, it's not a problem ;) . In the meanwhile I dropped my K6II400 and I bought a Duron 1.2 so I almost forget that unfilter had problem on 3dnow CPU :cool: . There is less and less non-iSSE CPU so it should have been only my problem.

I just tested Unfilter without any resize and I get not green line. I'll make more test tonight.

[cult]

@ trbarry and everybody
Just a typo mistake.As you can see from the rest of my msg it was meant to be bicubic resize.
Sorry.So yes I use YV12

[MaTTeR]

Originally posted by trbarry
The green lines at the right are usually a multiple of x problem, 4,8,16 Yep, that did the trick for me. I had to crop in multiples of 8 instead of 4. Thx!

[cult]

croping multiple of 4 always works for me.Have the feeling that more problems appear with amd cpus...Am I wrong?

[MaTTeR]

I can't see how the CPU type would have anything to do with it at all. Cropping in multiples of 8 instead of 4 fixes the vertical line problem on my dual PIII system as well.

[trbarry]

Thanks for verifying it was indeed a multiple of x problem. I'll fix that. I don't think there is any particular reason UnFilter needs any special multipls. It is probably just that with YV12 I can now process 8 pixels at a time instead of 4. But I'll fix the boundary problems.

- Tom

[Gaia]

Yeah i confirm it too. It's mod4 problem. One sad thing is that you can't use iago's trick with YV12 :( I just tried Lumfilter() and it looked really really bad. Atleast short Matrix clip looked totally crap. So Unfilter might not be that usefull in YV12 encoding :(

[Koepi]

Thanks a million Tom,
really great work! Now I can do proper YV12 encoding again, your unfilter is needed e.g. for the second edition of 5th Element to get a compressability that makes some sense to put it on 1CD :)

I really appreciate it!

Best regards
Koepi

[scmccarthy]

It's a small think Tom, but you never updated the version number to 0.0.1.3 in the dll.

I mentioned vertical lines before, but I just realized that I have an LCD screen with soft vertical lines for anti-glare. No wonder I liked the look of them, it's part of the screen!

Thanks for the tips on how to read your code, that will get me to where I want to go.

Stephen

[MaTTeR]

Originally posted by Gaia
One sad thing is that you can't use iago's trick with YV12 :( I just tried Lumfilter() and it looked really really bad. Atleast short Matrix clip looked totally crap. Do u have a screenshot? I've been using Lumfilter(-1) with Unfilter and the results don't look bad at all. Your seeing more blocks? Maybe slightly off topic and needs a new thread:)

[trbarry]

Do u have a screenshot? I've been using Lumfilter(-1) with Unfilter and the results don't look bad at all. Your seeing more blocks? Maybe slightly off topic and needs a new thread

The ability of UnFilter to remove certain dark macro blocks on TV's is one of the great unexplained mysteries of life, at least to me. ;)

Nobody has yes explained why it works. And more importantly, does it work in YV12, or can it be made to? I was concerned about filtering chroma also (YUY2 doesn't) just because of the possibility it could maybe destroy this happy unintended consequence.

So I'm fairly interested in reports of this phenomena with YV12.

- Tom

[MaTTeR]

Well I'm quite sure iago has been testing it and I've yet to hear complaints from him, that might be a good sign:)

For me it still seems to work the same against the DCT blocks as in YUY2 mode. However, I have noticed some "flickering" lately on walls but that might be due to me playing around with UnDot in the same script. I'll run some Quant2 tests later and see if I can pinpoint it.

BTW- Have I already mentioned that Unfilter and UnDot are wicked fast?:D At 640 resolution I'm seeing peaks of around 64FPS.

[iago]

Hello all,

First, I agree with MaTTeR that both UnDot and UnFilter are amazingly fast! I want to thank Tom one more time for these two great and most successful filters working in YV12.

Well, actually I can say that the effectiveness of UnFilter/LumaFilter(-2) combo (speaking in YV12 / Avs 2.5 Alpha terms ;)) depends on the black richness, sharpness, noise, etc. of the source. That is, while for some sources only LumaFilter(-2) would suffice, for others LumaFilter(-2) might have to be accompanied by UnFilter too.

Actually, it's easier to explain how UnFilter(+,+) helps eliminating dark and undark macroblocks or DCT blocks, since it increases the detail level and thus the codec cannot quantize these sharpened areas highly. In other words, codec "cannot" remove details (or maybe noise) to such a high degree that it begins to generate macroblocks/DCT blocks.

However, how UnFilter(-,-) assists reducing blockiness in "dark areas" still remains as a mystery! ;) The only explanation I can bring to this is that UnFilter with either (-,-) or (+,+) settings introduces some sort of subtle darkening effect too, which acts as an additional Luma Offset -x parameter, and hence decreasing the appearance of dark blocks just like the LumaFilter(-2) parameter in mpeg2dec3.dll does.

In short and imho, "UnFilter(+,+)" helps eliminating "macroblocks in all areas" (in both dark and undark parts of the movie) with its "sharpening effect", increasing the detail levels in critical/undetailed areas of the encode. And "UnFilter(-,-)" helps eliminating "dark macroblocks" with its "darkening effect", decreasing the luma offset slightly.

As for Tom's question, yes I think the YV12 implementation works as effectively too, just like the previous YUY2 version.

best regards to all,
iago

[iago]

Well, I want to add a few more points to the above argument:

Actually, LumaFilter(-2) parameter as a separate line to be added to the script when using Marc's mpeg2dec3.dll for Avisynth 2.5, or lumoff=-2 parameter to be added at the end of the "mpeg2source(.....)" line when using Marc's mpeg2dec3.dll for Avisynth 2.0.6 are just simple tricks, playing with the brightness level of the encode, and the same effect can also be achieved via ffdshow or divx decoders during the decoding process by taking down the brightness level or luma offset a bit.

The important point (and the most significant result to be derived from all these discussions) to be noted is that sharpening (or keeping enough necessary noise of the source) "as a method" is really effective against blocking and it improves the overall quality of the whole encode.

I can mention the movie U-Turn as an example to this argument, which is really hard-to-compress but with its high detail level and fine source noise you can still get decent rips even with relatively high average quantizers! ;)

Maybe we should revise or consider all our encoding/denoising habits again, attaching more importance to "noise" and "sharpness" especially in low-detail parts of the encodes; and refrain from ruining our encodes by unnecessary/extra filtering. Maybe only motion-adaptive denoising will be the better way to go soon for normal/real life encodes. Otherwise it's really possible to get crappy encodes with even an average quantizer of ~2 due to unnecessary, over-filtering.

best regards again,
iago

[Marc FD]

hi iago, matter and tom ^^

>I'm seeing peaks of around 64FPS.
64 FPS ! whow ! i'm curious to know how fast would go simple 3d cleaner with YV12 SIMD code ^^
(for me 65-70 FPS is the speed of MPEG2Dec3 to memory decoding. but i've only 1.4 Ghz)

>motion-adaptive denoising

i personnaly prefer motion-compensated denoising ^_^

i've some cool stuff here, like YV12 high quality 3d denosing, and YV12 motion compensated denoising, but it's currently too experimental. i'll need beta-testers a day ^^

BTW, i'm currently investigating a bit the sharpening world.
my favourite sharpen filter is unsharp mask. i tried it on a very clean anime DVD source with a very high threshold (127). it was impressive. maybe too strong, because it was blocky ^^
on a average good source (1000 kbps CBR XviD) it looks really better, but all macroblocks are very very noticable...
i see that the real video postprocessing is using sharpening too, but never shows the macroblocs. It gived me an idea to do post-processing sharpening (who'll not enhance blocks edges. maybe a good candidate for ffdshow after). my second idea is a filter doing adaptive sharpening/denoising. it would remove noise and enhance edges in the same time. i could even merge the 2 ideas :)

but i need to take a look to unsharp mask sources before.

[trbarry]

Okay, glad to hear all that still works for some reason. ;)


I just made another mini-release of UnFilter, now properly labled v 0.0.3.0, at the same URL:

www.trbarry.com/UnFilter.zip

This one does not need any horizontal multiple greater than 2, though Avisynth and some codecs may.

I also added proper version info and removed the incorrect readme references to Avisynth 2.05 and MPEG2DEC_YV12.

- Tom

[MaTTeR]

Originally posted by Marc FD
64 FPS ! whow ! i'm curious to know how fast would go simple 3d cleaner with YV12 SIMD code ^^
(for me 65-70 FPS is the speed of MPEG2Dec3 to memory decoding. but i've only 1.4 Ghz) Hi Marc,
Though I wouldn't consider Convolution3D a "simple" 3D cleaner, I can tell you that I'm averaging about 36FPS with it along with UnFilter, UnDot and of course slight PP. Those numbers are with Vlad's unreleased SMP/multi-threaded build of C3D which gave me another 2-4FPS:D

Dual AMD 1600s @ 139FSB along with a 512MB stick of PC2700/333Mhz CAS2 RAM. I'm not sure what my maximum decoding speed would be. How did you determine 65-70FPS? Make a simple script with an extremely low resolution I guess;)

Edit- Tom,
Thx for fixing the width restrictions!

[iago]

@Tom,

Yeah, whatever the logic behind it, UnFilter really works! ;) Thanks again, this time for the new release! ;)

regards,
iago

[jorel]

@trbarry

thanks for the "news",
i use Unfilter in all my encodes for a "long time".
for me is the first(best)filter from all!

:cool:

[Gaia]

My point is that somehow luomoff -2 made it too dark in my short matrix test. I couldn't see any "black blocks! but the picture was too dark for me. I didn't like it. Colours are really nice in YV12 but lumoff -2 seemed to screw that. Anyway have to do more testing.

Some update. I tried luomoff-1, it seems to look better but still...
I quess i have to do full 2-pass encode to fully confirm this but it still seems to be too dark. I use Matrix for testing so it might look better with not so dark movies.

[Marc FD]

>Though I wouldn't consider Convolution3D a "simple" 3D cleaner

s3dc is one of my filters. it's about as fast than Convolution 3d (but it's very different from c3d) with C code. i think a SIMD version would be damn fast ^^

>How did you determine 65-70FPS?

with VDub. peaks to 80fps ^^

> Make a simple script with an extremely low resolution I guess

no, only MPEG2Dec3 alone. YV12. stream mode. (source : PAL, 6 mbps MPEG-2)

[MaTTeR]

Hi Tom,

Just wanted to report a small problem I ran across with the YV12 build. I stumbled over what appears to be small bug when using UnFilter(-1,-1)[/i}. Using [i]-1 seems to cause a small pink vertical bar down the left edge about a 1/4" wide and a pink bar about 1/8" wide down the right side. The odd thing is that I can't reproduce the problem using any other integer values + or -. Here's my basic script- crop(4,64,712,352)
Lumafilter(-1)
UnFilter(-1,-1)
BicubicResize(624,264,0,0.5)
Undot()
Limiter()

Of course I tried cropping and resizing with different values and using different AVS 2.5 builds but with no luck. I'm attaching a small JPEG(uf_pink.jpg) that displays the problem. Let me know if you need any other information. TIA:)

[trbarry]

MaTTeR -

I see the problem and will look for it. Would it be possiible for you to post a short clip somewhere. Or email it? I'd probably only need 2-3 frames since UnFilter only looks at one frame at a time.

It is probably related to the fact that YV12 UnFilter also sharpens chroma, which is not done for YUY2. So likely I can find it anyway, just a clip would be easier.

Or if everybody prefers I could just turn off chroma softening & sharpening, keeping it like YUY2. It would run faster.

- Tom

[MaTTeR]

Tom,

I'll PM you a URL where you can grab a short clip. If it's easier to get rid of the chroma code then I'd say it's safe to do so. After all, we were doing without it before;) Thx

[iago]

Or if everybody prefers I could just turn off chroma softening & sharpening, keeping it like YUY2. It would run faster. (trbarry)

@Tom

Not for the speed concern (since it is already lightning fast!) but I would actually prefer having UnFilter as it was before, just like MaTTeR! ;)

regards,
iago

[trbarry]

Okay, I'll make YV12 like YUY2 for now, filtering only luma. Maybe later I'll make chroma filtering optional, if anyone sees the need for it.

- Tom

[Koepi]

Optional sounds fine for me :)

Thaks for your efforts, Tom!

Best regards
Koepi

[Holomatrix]

Ok, so what version of UnFilter works with AVISynth 2.06 I don't have 2.5 and if I want to sharpen DVD conversions to CVD what is the best option to use ?? Resize then UnFilter (20,20) ?
Thanks

[Marc FD]

there is no "best" filter. look at :

http://www.avisynth.org/index.php?page=Section+3%3A+Filters+and+colorspaces#q28

setion sharpenning, the most common filters are listed.
asharp may be interressing (hq result), but it's only working in YV12.

[trbarry]

Ok, so what version of UnFilter works with AVISynth 2.06

I left the old one in the same new zip, under the UnFilter_Old directory. I don't think anything has to be recompiled for 2.06 (and wish they didn't for 2.5 ;) ).

www.trbarry.com/UnFilter.zip

- Tom

edit: In case of confusion, this is NOT the fixed version I just promised above. soon

[trbarry]

Just wanted to report a small problem I ran across with the YV12 build. I stumbled over what appears to be small bug when using UnFilter(-1,-1)[/i}. Using [i]-1 seems to cause a small pink vertical bar down the left edge about a 1/4" wide and a pink bar about 1/8" wide down the right side. The odd thing is that I can't reproduce the problem using any other integer values + or -. Here's my basic script-

MaTTer -

Did you send me part of the original clip? Or the UnFiltered one?

This clip you sent me already has those pink bars even without UnFilter, though they are being amplified somewhat. I ran the following script
LoadPlugin("D:\MPEG2DEC3\MPEG2Dec3")
LoadPlugin("D:\Avisynth_205\UnFilter\Release\UnFilter.dll")
avisource("D:\vcr\test\UnFilter\pink_clip3.avi")
a=AddBorders(8,8,8,8)
b=a.UnFilter(-1,-1)
return stackvertical(a.subtitle("Original"),B.subtitle("UnFilter"))


and got the attached image: (Mod? :o )

You can easily notice 3 things:

1) The pink bars on left and right are there in the source.

2) UnFilter YV12 is turning clips ugly green, making it more obvious. :( I'll remove the chroma filtering for now.

3) StackVertical for YV12 now seems to put the first clip on the bottom. That confused me before I put subtitles on. I thought it used to be on the top.

- Tom

[trbarry]

All YV12 testers should probably get this one. The previous green push was kinda ugly. I'm not sure why I didn't notice it before.

This version does not filter chroma at all, much safer.

Get it as usual at www.trbarry.com/UnFilter.zip .

The old (YUY2, chroma avoiding) version is still also in the zip for non-Avisynth 2.5 users)

It should be a safe fix but pls let me know if I totally trashed something else.

- Tom

[MaTTeR]

Originally posted by trbarry
Did you send me part of the original clip? Or the UnFiltered one? Sorry Tom, I totally missed your follow-up on this somehow; maybe too much wine yesterday ;)

The clip I sent you was the Unfiltered processed. Your images above produce exactly what I was seeing though I failed to mention the color shift. Going to grab the new build now and give it a go. Thx for the speedy update.

[bond]

Originally posted by trbarry
I don't think unfilter darkens anything. If so, that would be a bug. But, if it turned out that UnFilter was slightly darkening everything, should I fix it? Or leave it alone to help with the lumi problem? ;)lo trbarry
sorry for bringing this up again

we are currently playing with a script (http://forum.doom9.org/showthread.php?s=&threadid=70734) which is able to smooth motion parts in a frame and sharpen edges in a frame using your unfilter
now as only parts in each frame are treated by unfilter, this means that this darkening effect can now kick in badly (cause the non unfiltered parts are not darkened)

well my wish would be now that if possible i would be glad if you could have a look and fix this darkening behaviour of unfilter, as its really a pretty good and fast filter (and lumi masking/adaptive quant. is now working as it should be in xvid, meaning this darkening effect of unfilter shouldnt be usefull anyhow anymore)

thanks a lot :)

[ben13]

can't download your filter!!:(
says, 'site's down'

[manono]

can't download your filter!!:(
says, 'site's down'
Look here:

http://avisynth.org/warpenterprises/

And welcome to the forum.

[kedautinh12]

anyone have x64 ver???

[Reel.Deel]

anyone have x64 ver???

There's no need, RemoveGrain(mode=1) is identical.

[kedautinh12]

Oh, thanks

[StainlessS]

There's no need, RemoveGrain(mode=1) is identical.

You sure bout that ? [Unfilter]

From Wiki describing RemoveGrain modes.
Mode 1 does pure minmax clipping with all eight of a pixel's spatial neighbors. Thus it is identical to trbarry's Undot filter. It is the safest mode with regards to artifacts, and it preserves thin lines including their endpoints.

[kedautinh12]

You sure bout that ? [Unfilter]

From Wiki describing RemoveGrain modes.

Yeah and i checked function aaa(), it's need xshrp and yshrp from unfilter. RemoveGrain don't have that parameter
https://github.com/Dogway/Avisynth-Scripts/blob/ea48309b83f59904d075e7371460b35466acc846/EX%20mods/LSharpAAF1.60.avsi#L556

[Reel.Deel]

You sure bout that ? [Unfilter]

From Wiki describing RemoveGrain modes.

Ooops, my bad ... Got Undot and Unfiler confused :o ... Undot is equivalent to RemoveGrain(mode=1), not UnFilter.