Log in

View Full Version : Anime filtering: weird color error


Akatsuker
16th June 2008, 00:48
Hey you guys.
I was working on avsp, doing some filtering on a raw.

Here's the code:

vmToon(sharpen=false)
Tweak(sat=1.6,cont=1.1,hue=1,bright=-10)
Undot().Deen("a2d",7)
msmooth()
msharpen(threshold=1,strength=20)

Here is the BEFORE (raw):

http://img205.imageshack.us/img205/4811/originallg0.png

Here is the AFTER:

http://img259.imageshack.us/my.php?image=filteredjg5.png

But some parts are now with a big kind of chroma noise, I guess.
Look and the girl (the edge of her hair, her face):

http://img440.imageshack.us/img440/9892/chromaerroreo3.png

It's a kind of shadow, I don't know.

1 - Does it have an exact name?
2 - How could I clean the video of this crap?
3 - Do you guys have any suggestions for improving my script?

thetoof
17th June 2008, 04:28
Edge issue... so I guess it could be caused by vmtoon. Try disabling it.
A few other issues :
- Your filtered image looks more aliased than the original. Try calling maa(), nnediaa() or daa() after putting animeivtc.avsi in your plugins folder. All the plugins you need are included in the requirements.7z
- Banding. Try gradfun2db(x) ajust x to your source
- You were a bit intense on the saturation boost... I don't know the anime, but over-saturated colors might go against the mood of it.

TheRyuu
17th June 2008, 04:48
Edge issue... so I guess it could be caused by vmtoon. Try disabling it.
A few other issues :
- Your filtered image looks more aliased than the original. Try calling maa(), nnediaa() or daa() after putting animeivtc.avsi in your plugins folder. sangnom All the plugins you need are included in the requirements.7z
- Banding. Try gradfun2db(x) ajust x to your source
- You were a bit intense on the saturation boost... I don't know the anime, but over-saturated colors might go against the mood of it.

1.) if you don't overfilter, you shouldn't need these in the first place.

2.) I don't really see enough banding to justify it...
Edit: Oh, a third pic... my bad, look below

3.) No kidding over-saturated... :p
Just a little much with that. I don't like screwing with the original color much if not at all. Keep it to 1.1 max boost if you are going to change it, 1.05 is probably a better limit.

Most of your problems are from overfiltering it and can be avoided in the first place. The original raw looks fine and pretty damn clean to me so most of the filtering you used I can't say it's justified but it is personal taste...

Edit:
Oh, I didn't see the third pick... (that might need gradfun2db)
Is that third picture the raw?

Edit2:
Tweak(sat=1.6,cont=1.1,hue=1,bright=-10)
Little strong on the sat boost. 1.1/1.05 max. Not sure about the cont, never really change any of these unless it's like a 1980's anime which looks real bad.

Undot().Deen("a2d",7)
Deen with a radius of 7... I bet that worked well... :confused:
I wouldn't really use deen, and if I did, radius of 1 max.

msmooth()
Do we want one big blur? (we already made it a blur with deen, we need more?)

msharpen(threshold=1,strength=20)
Why not just use unsharp at this point.

thetoof
17th June 2008, 04:58
All true. You have an excellent source and it needs minimal processing.

TheRyuu
17th June 2008, 05:48
To expand on the minimal filtering:
dehalo_Alpha(darkstr=0.2,brightstr=0.8)

fft3dgpu(sigma=2,sigma2=1,sigma3=1.25,sigma4=1.75,bt=4,plane=4)
FluxSmoothT(1)

Worked well for me. As much as I hate to dehalo anime. Furthermore, the sigma values might be a tad too high. Was just something quick I made up. Hard to judge with just a single frame.