New Smart Smoother HiQ?

[Zarxrax]

I was just wondering if there was a version of smart smoother hiq available for avisynth 2.5? I tried searching but all I could come up with was a post from back in september where Shodan said he was working on an optimized version :p
If nothing is currently being done, maybe someone out there would like to try porting it?

[scmccarthy]

I think there's a concensus that although it is a great filter, it is not an optimal smoothing solution in YV12 or YUY2 for avisynth.

Stephen

[Zarxrax]

I'm not sure I understand. What do you mean by "optimal"? There is no good way to convert the code to process YV12 data?

[scmccarthy]

Someone else should weigh in on this, I might be wrong, but I mean that newer smoothers have replaced it.

Stephen

[Zarxrax]

Hmmm, I've tested on pretty much every smoother that comes out, and I can't find anything else that even begins to approach it :\

[digitize]

hmmm, I would think filters like convolution3d would give better results than sshiq. c3d is a temporal & spatial cleaner, where as sshiq is just a spatial. And even though c3d only has a 3x3 spatial matrix, you could use a filter like dnr2 to stabilize the source before the use of c3d.

[Zarxrax]

IMO convolution 3d does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything. Even if I zoom in on the video, I cant see hardly any difference between the source video and the filtered video. It does look ever so slightly different, but it looks like it kinda just moved the noise around a little instead of actually cleaning it :confused:

[angelyote]

Give deen a try. It's a MarcFD filter.

Most recently I did an encode of some cowboy bebop episodes and also castle of cagliostro and it did an amazing job of cleaning the noise and smoothing the flat color sections and the compressibility increase was pretty impressive. (enough to allow me to add both english and japanese ac3's onto my 2cd rip)

Dave

[Lothar]

SSHiQ is much better than any other filter i tried in removing macroblocks and artifacts form prevoius encodes, like it's shown on the page of the virtualdub version.

It also does a good job in removing noise... it woul be really great to have a YV12 version (at least a version that can be used on avs 2.5)

[OUTPinged_]

temporal axis helps in removing noise very much.

C3d is better at "remove noise/keep details" than sshiq. You guys just didnt take some time at tweaking the thresholds.

does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything.
See?

@angelyote:

spatial filtering for bebop and CoC DVDs? O_o These rippers are evil..

[digitize]

@zarxrax
Really you can't just use the presets, you have to play around with the actual settings. And animeBQ might cause ghosting / motion trails b/c of the strong temporal settings.

Edit (actually just an addition): @zarxrax, im guessing you just use a spatial cleaner like sshiq and something like temporal cleaner afterwords, right?

[Zarxrax]

@angelyote: Thanks, this filter seems pretty nice from first glance, though it appears to blur a little too much. I'll keep playing around with it though and see what I can get out of it.

@digitize: Ok I just tried Convolution3d again with settings Convolution3d(1, 128, 128, 0, 0, 0)
And even with these insanely strong spatial settings, I was unable to tell any difference!
So then I tried Convolution3d(1, 128, 128, 8, 8, 0) (any higher temporal caused bad ghosting)
I could tell the temporal cleaner was working, but thats not what I'm after. I need a spatial cleaner, and I just dont see this doing anything!
And you're right, I use temporal cleaner in conjunction with SSHiQ.

[digitize]

@zarxrax
Yeah the big drawback about c3d (imo, and probably the same with everyone else) is that the spatial matrix is only 3x3, I think if it could have a spatial matrix of 5x5 (sshiq & 2d cleaner can) it would clean much better, but lose a little more detail, but like you said it doesn't do enough in noisey anime tv caps.

[Zarxrax]

Hmmm, the "a2d" mode in Deen seems to work really well for me and its very fast too! I think this could make a fine replacement for SSHiQ in my encodes :D

[angelyote]

@OUTPinged_
deen has a 2dcleaner setting that worked pretty nicely on CoC. The source I had was very grainy. deen("a2d",2,10,12) did a nice job of removing the grain without losing too much detail. I like the look of 2dcleaned anime but didn't want to converytoyuy2() to use _2dcleanyuy2()

@Zarxrax
I was going to suggest you use "a2d" instead of "c3d" but I see you already found it. I hope it works out for you. I've liked it so far. Working on Doomed Megalopolis at the moment.

Dave

[IanB]

I have to agree SShiq hack all over most other smoothers. It has exceptional
edge detection algorithms and can be set to average over quite a high radius.
Married with a good temporal smoother, I find it unbeatable as long as I can
afford the processing time.

Wussy 3x3 averaging just doesn't cut the mustard.

Maybe sh0dan can find time in his busy life as a developer to at least recompile
a YUY2 version for 2.5.

IanB

[Richard Berg]

Here you go: http://www.avisynth.org/FileHosting/users/richardb/files/SmoothHiQ.zip

Tested only briefly; it rejects YV12 appropriately, but you may find other incompatibilities.

[IanB]

:D

[digitize]

hmmmm, i've been gone all weekend, but i'll have to check out the a2d (along with a temp cleaner) option of deen. Like IanB said, 3x3 spatial matrix isn't enough, but the big advantage of a hybrid (spatial + temporal) cleaner is that the cleaner decides when to use which cleaner, rather than cleaning already processed frames which is done when using 2 seperate filters.

[Lothar]

Thank you very much for the filter :D !

Isn't it possible to convert it to work in YV12? :confused:

Thans :)

[scmccarthy]

If I am not mistaken, in order to avoid a memory leak, you need to add the following method:

SmoothHiQ::~SmoothHiQ() {
delete[] weight;
delete[] dweight;
delete[] difftable;
delete[] data;
delete[] mmxdata;
delete[] mmxdata2;
}

Stephen

[digitize]

someone forget to add the destructor? :p

[scmccarthy]

Not in the source code for the 2.0x version.

Stephen

[sh0dan]

I hope I get the time for reworking the code for YV12 - I have an idea for a speedup, but I'm not sure how much it'll gain.

[scmccarthy]

Not in the source code for the 2.0x version. What did I say, it did not make a lot of sense!

I meant there was no destructor in the original code, but Robert did not give his code, so I can't be sure if he forgot too.

Stephen

[sh0dan]

He forgot it :)

Being a former Java programmer, the automatic Garbage Collection spoils you. ;)

[scmccarthy]

I am basically following up on our discussion about the ~cnr() destructor. Not being the most experienced programmer, I am running it past you guys in order to confirm that I am applying this new knowledge correctly.

Thanx for the comfirmation, Sh0dan and please put in the destructor, Richard.

Stephen

P.S. for those of you following this, every instance of the new operator in an AviSynth plugin should *not* be pared with a destructor. PClip and PVideoFrame are smart pointers, meaning they take care of their own 'garbage collection'.

[digitize]

Hmmm, I tested out deen's a2d option and recieve pretty good results with it. It cleaned up noise better than c3d's spatial cleaner, so I'll probably stick with a2d.