Constructive criticism needed, want to improve on script

[Chainmax]

Just compare the before-after screenshots and tell me if the filtering has improved or worsened the source and in either case, please tell me how would you improve my results. Since I want as unbiased an opinion as possible, I won't post the script. Without further ado, here are the pics:

Before:
http://members.arstechnica.com/x/master_burrfoot/Before1.png

After:
http://members.arstechnica.com/x/master_burrfoot/After1.png

Before:
http://members.arstechnica.com/x/master_burrfoot/Before2.png

After:
http://members.arstechnica.com/x/master_burrfoot/After2.png

[Guest]

The edge ringing is reduced, which is valuable.

How would seeing your script make us biased? :p

[Leak]

Originally posted by neuron2
The edge ringing is reduced, which is valuable.

How would seeing your script make us biased? :p

Maybe it's because you can't attach barf bags to postings? ;) :D

*ducks*

np: MicroNost - Got Mad Love (Clicks & Cuts 4)

[Chainmax]

neuron2: seeing a particular filter can make you direct your thoughts to just one issue instead of trying to see all of them.

Leak: huh? What is that supposed to mean?

[Leak]

Originally posted by Chainmax
neuron2: seeing a particular filter can make you direct your thoughts to just one issue instead of trying to see all of them.

Leak: huh? What is that supposed to mean?

It was a joke. But a joke really isn't one if you have to explain it... :(

(I meant you might not want to post your script because it was so bad - which of course it can't be if it produces such results...)

np: Khonnor - Cantelopps (Intelligent Toys 1)

[2Bdecided]

I like the fact that the nasty little horizontal lines around the edge of the hat (in the first image) have been removed by the filter.

I can't see anything to dislike - I suppose the process is just for cartoons?

Cheers,
David.

[Didée]

Looks rather fine to me (though animated stuff isn't my world).

One thing: there is still some color bleeding left over. Did you try to merge chroma from some aWarpSharp with cm=1? Or is that already in ..?

[Chainmax]

Leak: sorry for spoiling the joke, I was assuming you were referring to the quality of the filtering :).

2Bdecided: I never saw those lines any of the times I tried this before, they must be just YV12 chroma sampling issues since they only happen in red areas and from what people have told me that's a telltale sign of it. And yeah, the script is probably only for cartoons since it has a whole lot of smoothing/denoising.

Didée: I would like this to look outstanding, since it's meant to completely replace my crappy R4 DVDs. As for the color bleeding: I haven't merged the chroma, but I was thinking of using ChromaShift to eliminate it.


Everyone: I am going to make sample clips (source and filtered) and the script soon, but in the meantime I'd be mostly interested if you think the halos have been completely eliminated or not and if you think I have oversharpened or not, plus whatever else you think might be wrong with the results so far.

[Soulhunter]

Just a quick "AAA+Sharpen/darken+SSHQ" try... :\

http://img91.exs.cx/img91/6830/after1b2om.png


Bye

[Chainmax]

The floor and the curtains seems to have less artifacting and the halos seem to have gone too (what did you use for halo removal?). The darkening helps a lot on the girls's contour except on the feet, they stand out too much now. Maybe I should use FastLineDarken without the thinning?

Anyway, here's the script I am using for now:

TFM(mode=2,PP=7)
TDecimate(mode=1)
BlindDeHalo2(2,2,100)
HQDering(255)
Undot()
ConvertToYUY2()
Pixiedust(limit=5).Trim(0,FrameCount-2)+Trim(FrameCount-1,FrameCount-1)
ConvertToYV12()
Deen("a2d",1,3,5)
Crop(12,0,700,476,align=true)
LimitedSharpen(ss_x=2,ss_y=2,dest_x=640,dest_y=480,strength=128)

You know this script, Didée :).

In order to use HQDering I replaced the YV12Layer line with Didée's suggestion (it's in the function's thread), and I changed all 3 instances of lanczos in to lanczos4 in LimitedSharpen.

I will post sample clips for source and encode sometime this week. So, can anyone tell me how to improve this script? Maybe higher settings for Deen and some FastLineDarken after LimitedSharpen?

[Soulhunter]

Originally posted by Chainmax

...what did you use for halo removal?

Think its a side-effect of SmartSmootherHQ... ;)


Bye

[Chainmax]

Ok then, I am going to try replacing BlindDeHalo2 with SmartSmootherHQ. Since I am going to do that and linking to a question I made in another thread a short while ago, I am trying to use DePanInterleave() to enhance the smoothing on pans but I get an "unrecognized expression" error wherever I put it on the script. Does anyone have an idea why this happens?


Mmm, warpenterprises's page has SmartSmoother and Smoother HiQ, which one is SmartSmootherHQ (Avisynth version)?

[Didée]

And don't forget about SmoothUV by Kurosu, which also includes an SSHiQ ...

[Soulhunter]

Originally posted by Chainmax

Ok then, I am going to try replacing BlindDeHalo2 with SmartSmootherHQ...

Erm, I dont think this is a good idea... :o

Better use SmartSmootherHQ and BlindDeHalo2 !!!


Originally posted by Chainmax

Mmm, warpenterprises's page has SmartSmoother and Smoother HiQ, which one is SmartSmootherHQ (Avisynth version)?

Well, I use the VDub version...

Always got problems with AviSynth's SmartSmootherHQ !!!


Bye

[Chainmax]

The problem is that I already have two colorspace conversions due to Dust. I will try the avisynth version (any ideas where to download it from) first and if that doesn't work then I'll try the VDub version. I have two questions though:

1)What settings did you use?
2)I assume the latest VDub version is the v2.1 that can be downloaded here (http://cultact-server.novi.dk/kpo/vdub/index.html) and you have to use this (http://forum.doom9.org/showthread.php?s=&threadid=23804) in order to use it in Avisynth, right?

[Soulhunter]

Originally posted by Chainmax

The problem is that I already have two colorspace conversions due to Dust...

You shouldnt care about this...

Under this conditions you wont see much difference !!!


Originally posted by Chainmax

What settings did you use?

Cant remember anymore... :o

Its was just a quick try @ a single frame anyway !!!

Just place it like this and tweak SSHQ till all areas look flat...

TFM(mode=2,PP=7)
TDecimate(mode=1)
BlindDeHalo2(2,2,100)

# Not really sure if you need HQDering... :\
# HQDering(255)

Undot()
ConvertToYUY2()
PixieDust(Limit=5)
ConvertToRGB32()

# Removed Deen...

SmartSmootherHiQ

ConvertToYV12()
Crop(12,0,700,476,Align=True)
LimitedSharpen(ss_x=2,ss_y=2,Dest_x=640,Dest_y=480,Strength=128)


Originally posted by Chainmax

I assume the latest VDub version is the v2.1 that can be downloaded here (http://cultact-server.novi.dk/kpo/vdub/index.html) and you have to use this (http://forum.doom9.org/showthread.php?s=&threadid=23804) in order to use it in Avisynth, right?

I got it here... (http://www.neuron2.net/hiq/smoothhiq.html)

Actually I use it only in VDub directly !!!

Mainly coz I need the realtime preview for proper tweaking... :D


Bye

[Chainmax]

The test clip can now be downloaded from here (http://www.31012.com/~azulftp/TestClip.vob). I expect to be able to upload an encode sometime this week.