New Smart Smoother HiQ?

Zarxrax · 7th February 2003, 05:09

I was just wondering if there was a version of smart smoother hiq available for avisynth 2.5? I tried searching but all I could come up with was a post from back in september where Shodan said he was working on an optimized version

If nothing is currently being done, maybe someone out there would like to try porting it?

scmccarthy · 7th February 2003, 06:32

I think there's a concensus that although it is a great filter, it is not an optimal smoothing solution[edit] in YV12 or YUY2 for avisynth[/edit].

Stephen

Zarxrax · 7th February 2003, 22:56

I'm not sure I understand. What do you mean by "optimal"? There is no good way to convert the code to process YV12 data?

scmccarthy · 7th February 2003, 23:22

Someone else should weigh in on this, I might be wrong, but I mean that newer smoothers have replaced it.

Stephen

Zarxrax · 7th February 2003, 23:48

Hmmm, I've tested on pretty much every smoother that comes out, and I can't find anything else that even begins to approach it :\

digitize · 8th February 2003, 05:41

hmmm, I would think filters like convolution3d would give better results than sshiq. c3d is a temporal & spatial cleaner, where as sshiq is just a spatial. And even though c3d only has a 3x3 spatial matrix, you could use a filter like dnr2 to stabilize the source before the use of c3d.

Zarxrax · 8th February 2003, 05:58

IMO convolution 3d does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything. Even if I zoom in on the video, I cant see hardly any difference between the source video and the filtered video. It does look ever so slightly different, but it looks like it kinda just moved the noise around a little instead of actually cleaning it

angelyote · 8th February 2003, 06:30

Give deen a try. It's a MarcFD filter.

Most recently I did an encode of some cowboy bebop episodes and also castle of cagliostro and it did an amazing job of cleaning the noise and smoothing the flat color sections and the compressibility increase was pretty impressive. (enough to allow me to add both english and japanese ac3's onto my 2cd rip)

Dave

Lothar · 8th February 2003, 13:36

SSHiQ is much better than any other filter i tried in removing macroblocks and artifacts form prevoius encodes, like it's shown on the page of the virtualdub version.

It also does a good job in removing noise... it woul be really great to have a YV12 version (at least a version that can be used on avs 2.5)

OUTPinged_ · 8th February 2003, 13:44

temporal axis helps in removing noise very much.

C3d is better at "remove noise/keep details" than sshiq. You guys just didnt take some time at tweaking the thresholds.

does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything.
See?

@angelyote:

spatial filtering for bebop and CoC DVDs? O_o These rippers are evil..

digitize · 8th February 2003, 15:29

@zarxrax
Really you can't just use the presets, you have to play around with the actual settings. And animeBQ might cause ghosting / motion trails b/c of the strong temporal settings.

Edit (actually just an addition): @zarxrax, im guessing you just use a spatial cleaner like sshiq and something like temporal cleaner afterwords, right?

Zarxrax · 8th February 2003, 18:15

@angelyote: Thanks, this filter seems pretty nice from first glance, though it appears to blur a little too much. I'll keep playing around with it though and see what I can get out of it.

@digitize: Ok I just tried Convolution3d again with settings Convolution3d(1, 128, 128, 0, 0, 0)
And even with these insanely strong spatial settings, I was unable to tell any difference!
So then I tried Convolution3d(1, 128, 128, 8, 8, 0) (any higher temporal caused bad ghosting)
I could tell the temporal cleaner was working, but thats not what I'm after. I need a spatial cleaner, and I just dont see this doing anything!
And you're right, I use temporal cleaner in conjunction with SSHiQ.

digitize · 8th February 2003, 18:21

@zarxrax
Yeah the big drawback about c3d (imo, and probably the same with everyone else) is that the spatial matrix is only 3x3, I think if it could have a spatial matrix of 5x5 (sshiq & 2d cleaner can) it would clean much better, but lose a little more detail, but like you said it doesn't do enough in noisey anime tv caps.

Zarxrax · 8th February 2003, 19:24

Hmmm, the "a2d" mode in Deen seems to work really well for me and its very fast too! I think this could make a fine replacement for SSHiQ in my encodes

angelyote · 8th February 2003, 19:31

@OUTPinged_
deen has a 2dcleaner setting that worked pretty nicely on CoC. The source I had was very grainy. deen("a2d",2,10,12) did a nice job of removing the grain without losing too much detail. I like the look of 2dcleaned anime but didn't want to converytoyuy2() to use _2dcleanyuy2()

@Zarxrax
I was going to suggest you use "a2d" instead of "c3d" but I see you already found it. I hope it works out for you. I've liked it so far. Working on Doomed Megalopolis at the moment.

Dave

IanB · 9th February 2003, 04:25

I have to agree SShiq hack all over most other smoothers. It has exceptional
edge detection algorithms and can be set to average over quite a high radius.
Married with a good temporal smoother, I find it unbeatable as long as I can
afford the processing time.

Wussy 3x3 averaging just doesn't cut the mustard.

Maybe sh0dan can find time in his busy life as a developer to at least recompile
a YUY2 version for 2.5.

IanB

Richard Berg · 9th February 2003, 05:46

Here you go: http://www.avisynth.org/FileHosting/.../SmoothHiQ.zip

Tested only briefly; it rejects YV12 appropriately, but you may find other incompatibilities.

IanB · 9th February 2003, 11:21

digitize · 10th February 2003, 03:54

hmmmm, i've been gone all weekend, but i'll have to check out the a2d (along with a temp cleaner) option of deen. Like IanB said, 3x3 spatial matrix isn't enough, but the big advantage of a hybrid (spatial + temporal) cleaner is that the cleaner decides when to use which cleaner, rather than cleaning already processed frames which is done when using 2 seperate filters.

Lothar · 10th February 2003, 08:42

Thank you very much for the filter

!

Isn't it possible to convert it to work in YV12?

Thans

7th February 2003, 05:09	#1 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	New Smart Smoother HiQ? I was just wondering if there was a version of smart smoother hiq available for avisynth 2.5? I tried searching but all I could come up with was a post from back in september where Shodan said he was working on an optimized version If nothing is currently being done, maybe someone out there would like to try porting it?

8th February 2003, 13:36	#9 \| Link
Lothar Registered User Join Date: Apr 2002 Location: Italy Posts: 24	SSHiQ is much better than any other filter i tried in removing macroblocks and artifacts form prevoius encodes, like it's shown on the page of the virtualdub version. It also does a good job in removing noise... it woul be really great to have a YV12 version (at least a version that can be used on avs 2.5) __________________ Never loose that feeling

8th February 2003, 13:44	#10 \| Link
OUTPinged_ MooPolice 1st division Join Date: Dec 2001 Location: VIlnius,LT Posts: 448	temporal axis helps in removing noise very much. C3d is better at "remove noise/keep details" than sshiq. You guys just didnt take some time at tweaking the thresholds. does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything. See? @angelyote: spatial filtering for bebop and CoC DVDs? O_o These rippers are evil.. __________________ ___________________MooPolice is watching you!____.o/________

8th February 2003, 15:29	#11 \| Link
digitize Registered User Join Date: Aug 2002 Posts: 142	@zarxrax Really you can't just use the presets, you have to play around with the actual settings. And animeBQ might cause ghosting / motion trails b/c of the strong temporal settings. Edit (actually just an addition): @zarxrax, im guessing you just use a spatial cleaner like sshiq and something like temporal cleaner afterwords, right? Last edited by digitize; 8th February 2003 at 15:33.

9th February 2003, 11:21	#18 \| Link
IanB Avisynth Developer Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,167	Ta! very muchly

7th February 2003, 06:32	#2 \| Link
scmccarthy Registered User Join Date: Oct 2002 Posts: 462	I think there's a concensus that although it is a great filter, it is not an optimal smoothing solution[edit] in YV12 or YUY2 for avisynth[/edit]. Stephen

7th February 2003, 22:56	#3 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	I'm not sure I understand. What do you mean by "optimal"? There is no good way to convert the code to process YV12 data?

7th February 2003, 23:22	#4 \| Link
scmccarthy Registered User Join Date: Oct 2002 Posts: 462	Someone else should weigh in on this, I might be wrong, but I mean that newer smoothers have replaced it. Stephen

7th February 2003, 23:48	#5 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	Hmmm, I've tested on pretty much every smoother that comes out, and I can't find anything else that even begins to approach it :\

8th February 2003, 05:41	#6 \| Link
digitize Registered User Join Date: Aug 2002 Posts: 142	hmmm, I would think filters like convolution3d would give better results than sshiq. c3d is a temporal & spatial cleaner, where as sshiq is just a spatial. And even though c3d only has a 3x3 spatial matrix, you could use a filter like dnr2 to stabilize the source before the use of c3d.

8th February 2003, 05:58	#7 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	IMO convolution 3d does not do a good job at all for anime. Even with the "animeBQ" preset, I can hardly tell that it is doing anything. Even if I zoom in on the video, I cant see hardly any difference between the source video and the filtered video. It does look ever so slightly different, but it looks like it kinda just moved the noise around a little instead of actually cleaning it

8th February 2003, 06:30	#8 \| Link
angelyote The frumious Bandersnatch Join Date: Sep 2002 Posts: 78	Give deen a try. It's a MarcFD filter. Most recently I did an encode of some cowboy bebop episodes and also castle of cagliostro and it did an amazing job of cleaning the noise and smoothing the flat color sections and the compressibility increase was pretty impressive. (enough to allow me to add both english and japanese ac3's onto my 2cd rip) Dave

8th February 2003, 18:15	#12 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	@angelyote: Thanks, this filter seems pretty nice from first glance, though it appears to blur a little too much. I'll keep playing around with it though and see what I can get out of it. @digitize: Ok I just tried Convolution3d again with settings Convolution3d(1, 128, 128, 0, 0, 0) And even with these insanely strong spatial settings, I was unable to tell any difference! So then I tried Convolution3d(1, 128, 128, 8, 8, 0) (any higher temporal caused bad ghosting) I could tell the temporal cleaner was working, but thats not what I'm after. I need a spatial cleaner, and I just dont see this doing anything! And you're right, I use temporal cleaner in conjunction with SSHiQ.

8th February 2003, 18:21	#13 \| Link
digitize Registered User Join Date: Aug 2002 Posts: 142	@zarxrax Yeah the big drawback about c3d (imo, and probably the same with everyone else) is that the spatial matrix is only 3x3, I think if it could have a spatial matrix of 5x5 (sshiq & 2d cleaner can) it would clean much better, but lose a little more detail, but like you said it doesn't do enough in noisey anime tv caps.

8th February 2003, 19:24	#14 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	Hmmm, the "a2d" mode in Deen seems to work really well for me and its very fast too! I think this could make a fine replacement for SSHiQ in my encodes

8th February 2003, 19:31	#15 \| Link
angelyote The frumious Bandersnatch Join Date: Sep 2002 Posts: 78	@OUTPinged_ deen has a 2dcleaner setting that worked pretty nicely on CoC. The source I had was very grainy. deen("a2d",2,10,12) did a nice job of removing the grain without losing too much detail. I like the look of 2dcleaned anime but didn't want to converytoyuy2() to use _2dcleanyuy2() @Zarxrax I was going to suggest you use "a2d" instead of "c3d" but I see you already found it. I hope it works out for you. I've liked it so far. Working on Doomed Megalopolis at the moment. Dave

9th February 2003, 04:25	#16 \| Link
IanB Avisynth Developer Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,167	I have to agree SShiq hack all over most other smoothers. It has exceptional edge detection algorithms and can be set to average over quite a high radius. Married with a good temporal smoother, I find it unbeatable as long as I can afford the processing time. Wussy 3x3 averaging just doesn't cut the mustard. Maybe sh0dan can find time in his busy life as a developer to at least recompile a YUY2 version for 2.5. IanB

9th February 2003, 05:46	#17 \| Link
Richard Berg developer wannabe Join Date: Nov 2001 Location: Brooklyn, NY Posts: 1,211	Here you go: http://www.avisynth.org/FileHosting/.../SmoothHiQ.zip Tested only briefly; it rejects YV12 appropriately, but you may find other incompatibilities.

10th February 2003, 03:54	#19 \| Link
digitize Registered User Join Date: Aug 2002 Posts: 142	hmmmm, i've been gone all weekend, but i'll have to check out the a2d (along with a temp cleaner) option of deen. Like IanB said, 3x3 spatial matrix isn't enough, but the big advantage of a hybrid (spatial + temporal) cleaner is that the cleaner decides when to use which cleaner, rather than cleaning already processed frames which is done when using 2 seperate filters.

10th February 2003, 08:42	#20 \| Link
Lothar Registered User Join Date: Apr 2002 Location: Italy Posts: 24	Thank you very much for the filter ! Isn't it possible to convert it to work in YV12? Thans __________________ Never loose that feeling