Looking for filters to clean up a cartoon

[chilledinsanity]

I'm converting some cartoons over from MPEG to an Xvid encoding because the originals have problems, like not enough saturation or too blurry. I already know how to fix those problems, but I figure since I'm already re-encoding it, are there any filters you'd recommend to get rid of compression artifacts?

Normally I wouldn't even consider something like this because I normally work from a ripped DVD source and it would only lead to quality loss. However since this is a cartoon and the compression artifacts are a bit prevalent, I'm thinking there is more potential for quality gain than loss.

Anybody have suggestions for Vdub filters?

[chilledinsanity]

I ended up finding my own, but to anyone who reads this later, I found one that I think work pretty well.

The one I decided to go with was Smart Smoother High Quality. It has a lot of good options and got rid of a lot of noise and compression artifacts from the original. After that I run Vdub's sharpening filter with a value of 10 to bring back some of the detail (the original clip probably needed some sharpening anyway).

[Edit]

I've done some more stuff now. I'm still using Smart Smoother, but now I resize it a little bit larger (from 352x240 to 480x320, it had a low resolution) with the lanczos filter, then I run warpsharp on it. Currently I'm using a value of 30, but like Smart Smoother, it's dependent on the source material as to how high the values are. Finally I sharpen the image, using a value of 20 at the moment. If I can find someplace to upload them, I can show you some screenshots of before/after.

[ScottZ]

I can host them for a bit - send me a pm

[tigerman8u]

@ chilledinsanity: what version of smart smoother are you using. I'm using version 1.1 by Donald Graft and it has a setting for diameter and thresh hold. I'm trying to clean up some video similiar to what you're doing and not having to much luck. Could you post or pm me as to what filters you're using and the order you're applying them. Thanks and look forward to seeing the results you have achieved.

[chilledinsanity]

Tigerman:

I'm using Smart Smoother HiQ 2.11, it says it's by Klaus Post

Take in mind you have to tweak the values depending on your content, but here are mine right now for my current encodes (I messaged ScottZ, hopefully I'll have some shots to show soon):

Diameter = 7
Threshold = 31
Amount = 254
Maintain Difference = 7

Weighed Average selected
Weighed With Difference checked


I run that filter first, the Lanczos resize (I decided my last resolution was a little overkill, I'm going to 384x288 since the aspect ratio was off on the original encoding). After that I run Warpsharp, usually with a depth of 30 and a blur of 1x. Finally, I run Vdub's sharpening filter. I have to play around with that, decided how much to use before colors get too distorted and curved lines turn jagged. I'm typically using a value between 10-20.

At the original resolution of the clip being 352x240, it takes about 3 hours for me to encode in Xvid with 2 passes for a 22 minute episode on an AMD 2400+ 1GB RAM. Nothing that great in terms of speed, but not nightmarish like Mftoon's avisynth filter (I tried that in my research, it took it over 3 minutes to do a 1 pass encoding on a 3 second clip. The results were too brute force for my taste too. I hate tweaking in avisynth and it made my clip look like an oil painting by default).

Let me know if you have any more questions, I'm very pleased with my results right now considering the source.

[tigerman8u]

thanks. I had a different version that didn't have the same settings. got it and will give it a try. again thanks

[chilledinsanity]

Well I have some bad news in the ScottZ has been taken into custody by INTERPOL for international kidnapping and human trafficking charges. :(

The good news, however, is that I found another place to host my pics! So here they are:

(this isn't modem friendly as I'm using .pngs)

Before:
http://www.radford.edu/~rscott31/1_original.png

After:
http://www.radford.edu/~rscott31/1_processed.png

Before:
http://www.radford.edu/~rscott31/2_original.png

After:
http://www.radford.edu/~rscott31/2_processed.png

Before:
http://www.radford.edu/~rscott31/3_original.png

After:
http://www.radford.edu/~rscott31/3_processed.png


For each shot there were different things I was trying to highlight:

Shot 1 shows a typical scene from my video clip and also illustrates the line thinning caused by sharpening warpsharp.

Shot 2 shows a scene where there are definite compression artifacts present from the source and shows that this isn't a miracle solution, there's still going to be some ringing in the clips if you want to preserve the detail.

Shot 3 shows off the detail gain to be made from this process. Again, it's not a miracle solution, but I'm pretty damn happy with it considering the source.

All of the original shots were 352x240 and the end result shots are 384x272 because I wanted to give it some higher resolution to work with and the aspect ratio was off in the original. Since it's much easier to compare changes if they are the same size, I took these shots straight out of virtualdub with only the Lanczos resizing filter applied to the originals since I use that with the processed ones anyway. All of the processed shots were made with the settings I listed above, although I do change those settings for other clips. Anyway, hope this helps!

[ScottZ]

Well I have some bad news in the ScottZ has been taken into custody by INTERPOL for international kidnapping and human trafficking charges.

Yea.. Yea... as I mentioned.. I had a system meltdown and the originals you sent before that had problems. But again, glad you got them online anyway.

Hmmm... the idea of kidnapping those who know how to fix this shtuff and putting them to work is appealing though.. :D (just kidding of course).

Anyway... are you using vdub for all this, or are you importing the filter into an avs script?

[tigerman8u]

looks good

[chilledinsanity]

ScottZ: I'm using the exact settings I mentioned earlier for Tigerman8u in this thread. Warpsharp I've been keeping at 30 pretty consistently, but the smoothing and sharpening (along with contrast and HSV adjust for clips that need it) have all been tweaked a little for each clip. I usually take a scene with some detail in it (like shot #3 there) and turn the slider up on the threshold for for smoothing until I see detailed things in the background start to blur, then back down a little.

I'm not using any avisynth script, the source is an MPEG file which I just opened straight in virtualdub. I tried messing around with a line thickening script for avisynth, but found that the benefits weren't really worth the trouble of me going through and experimenting with the number values each time until I got the ratio right (thank god for active preview in vdub). Speaking of which, does anyone know if there are brightness/contrast filters and a sharpening one that have an option for active preview, vdub's don't by default.

[mg262]

if there are brightness/contrast filters...that have an option for active preview, vdub's don't by default.

Search for 'Color Mill' by Eugene Khoroshavin