Color stains after denoising in anime

[dodone]

I'm using this script on some anime

MPEG2Source("15.d2v",cpu=4)
colormatrix(interlaced=true)
blockbuster(method="sharpen",block_size=4)
#
dedot(22,18,15,7)
ConvertToYUY2(interlaced=true)
GreedyHMA(1,0,5,0,1,10,80,170)
ConvertToYV12
TomsMoComp(1,-1,1)
sangnom(1,10)
undot()
#
crop(8,0,702,960)
LanczosResize(640,480)
DeSpot(p1=27, p2=15, pwidth=14, pheight=14, mwidth=8, mheight=7, mthres=8, motpn=false, dilate=1, seg=1, show=0)
deen("c3d")
msharpen(7,75)
tweak(bright=-1,sat=1.09,hue=1,cont=1.04)
converttoRGB
RGBAdjust(1.10, 1.12, 0.99, 2.0)
assumefps(25)

and i get color stains expecially on green areas

here some samples

http://gnagnogno.altervista.org/ColorStains.rar

Does anybody know how to get rid of them?

[Tripx]

I believe it is called posterization, clumps of color caused by the denoising filter (or made more visible by the filters) It is very annoying with anime as it can be easily seen in areas that should be solid flat color. If you try the search function you should find other people with the same problem.

[foxyshadis]

Why do I suspect you're the one doing Princess Princess. -_- (Not seriously.)

All attempts to get perfectly smooth, clean animation in YV12 are destined to fail. It won't happen, the colorspace simply can't support it and you need some grain to hide the banding. I'm surprised you have anything left to encode, after washing every detail away like that.

[dodone]

It won't happen, the colorspace simply can't support it and you need some grain to hide the banding. I'm surprised you have anything left to encode, after washing every detail away like that.

I'm not washing, i'm using only one denoise filter.
If the reason is the denoising itself i cant find a solution as the original source is really bad.
I cant find a solution using search function too :(

[foxyshadis]

3 denoise filters (dedot, undot, deen). 2 deinterlace filters. 2 sharpeners (one way too strong). 3 gamma/levels correction filters. 2 extra fixer functions. A conversion to RGB and (implicitly) back to YV12. This is seriously going to rock your source's world, especially the color adjustments - yv12 has extremely low chroma resolution and colorspace, messing with it should be done gingerly, as it's already very susceptible to rounding error accumulation.

And if it really is blurry enough to need that much sharpening, consider using a strong vmtoon or limitedsharpen.

Anyway, I get some error accessing your site, maybe it has an external referrer blocker.

[dodone]

Vmtoon and limited sharpen are two scripts, the encoding time would increase too much and i'm backing up an entire tv series:( .

There is some loss of details but not critical cause it's an old anime. Only one denoise filter (deen) weight upon that as dedot and undot have really light settings. Same for sangnom. Toms is not used for a pure deinterlace and it's necessary to adjust indexed margins (AAA doesnt help in this case) and it's the reason why i do a strong sharpening at the end. You're right on color conversion but source colors and levels are completely incorrect.

The link i had posted was working, probably too many accesses in the last hours, it's a free space.

Ty very much for your help:thanks:

[Mug Funky]

if at all possible, could you post a sample up here of the original source? i'd like to have a crack at it. for example, you could use EEDI2 to double the height and make sangnom and tomsmocomp unnecessary. also, you could run undot before the height doubling, as you're halving it's effectiveness that way.

i suspect the source could be handled adequately with fft3dfilter and limitedsharpen... but i'd have to see the source to confirm that.

btw, what's your destination medium and bitrate? (like xvid, 700kbps, or x264, 300kbps, or DVD @ 3500 kbps... this stuff is important to know to determine how much pre-filtering is desirable).