Help - Anime Cleaning

[mit_mis]

hi, i've got those really horrible raws, and i wanted to ask if someone could tell what filters can i use to clean it, thanks in advance

http://img457.imageshack.us/img457/1605/villgust1jx.jpg

[Warpman]

try
fft3dfilter or fft3dgpu (slow but very good)
removegrain
fluxsmooth could also help

But "denoising" has been discuissed quite often so :search: thx

[Chainmax]

A sample would be of much more use, but for that I'd recommend starting with DeGrainMedian(), Lanczos4Resize(w,h) and LimitedSharpenFaster(). Maybe some DeCross() before IVTC/Deinterlacing too as it seems to have some chroma noise.

[mit_mis]

sorry, i'm a complete newb at avs, and i made a simple script, but it gives me errors:

LoadPlugin("degrainmedian.dll")
Directshowsource=("D:\Villgust - 1 [AT Raws].mp4")

DeGrainMedian(5,10,0)
Lanczos4Resize(640,480)
LimitedSharpenFaster(Smode=4,strength=20,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)

error:
Avisynth open failure:
Scipt error: Invalid arguments to function "DeGrainMedian"
(D:\villgust.avs, line 4)

[Chainmax]

A sample filterchain would be something like:

LoadPlugin("X:\wherever\DeCross.dll")

load the filters needed for IVTCing or deinterlacing

LoadPlugin("X:\wherever\DeGrainMedian.dll")

LoadPlugin("X:\wherever\MT_MaskTools.dll")
LoadPlugin("X:\wherever\RemoveGrain.dll")
Import("X:\wherever\LimitedSharpenFaster.avs")


DirectShowSource(whatever)
DeCross()
IVTC or Deinterlace
DeGrainMedian()
Lanczos4Resize(640,480)
LimitedSharpenFaster()

In order to know what is interlacing, what is telecine and how to remove them, I suggest you to read Decomb's (http://www.neuron2.net/decomb/decombnew.html) readmes and tutorials, the IVTC tutorial (http://www.doom9.org/ivtc-tut.htm) and the capture guide (http://forum.doom9.org/showthread.php?t=78797).Those are a great source of starter knowledge and will explain telecining and interlacing with detail.

Also, take your time to read the readme of each filter you use, as they contain descriptions of the filters and lots of extremely useful knowledge. And, like Warpman said, do use the search, maost novice questions have already been answered many times.

[mit_mis]

kk, i have it like this:
LoadPlugin("D:\degrainmedian.dll")
LoadPlugin("D:\DeCross.dll")
Import("D:\LimitedSharpenFaster.avs")

Directshowsource=("D:\Villgust - 1 [AT Raws].mp4")

DeCross(28, 80, 2, false)
DeGrainMedian(5,10,0)
Lanczos4Resize(640,480)
LimitedSharpenFaster(Smode=4,strength=20,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)

but it still gives me the same error, but instead of DeGrainMedian, now there are bad arguments in function DeCross

[hartford]

[QUOTE=mit_mis]hi, i've got those really horrible raws, and i wanted to ask if someone could tell what filters can i use to clean it, thanks in advance

There is a considerable amount of "ringing."

Try HQDering v0.1 by mf:

===

######
##
## HQDering v0.1 by mf
##
## Applies derining by using a smart smoother near edges (where ringing occurs) only.
##
## Usage: Import("HQDering-v0.1.avs")
## HQDering()
##
####

# Requires Masktools 1.4.1.6, Deen 1.0b
#

function HQDering(clip input, int "strength", int "overall", string "smoother", string "params") {

strength = Default(strength, 255) # strength, 0-255
overall = Default(overall, 0) # overall smoothing, 0-255
smoother = Default(smoother, "Deen") # filter that smooths
#params = default defined below # filter parameters - use Chr(34) for quotes

defaultnull = Default(params, "")
defaultdeen = Default(params, Chr(34)+"a3d"+Chr(34)+", 4, 15, 15, 20")

params = (smoother=="Deen") ? defaultdeen : defaultnull

try {
smoothedwparams = Eval(smoother + "(input, " + params + ")")
}
catch(err_msg) {
smoothedwoparams = Eval(smoother + "(input)")
}

smoothed = (params=="") ? smoothedwoparams : smoothedwparams

input.EdgeMask(3, 255, 255, 255, "sobel", Y=3, V=1, U=1)
normalmask = last

normalmask.Levels(0, 3.3, 90, 0, 255).Blur(1.0)
amplifiedmask = last

normalmask.Inflate().Inflate().Inflate().Levels(0, 3.3, 90, 0, 255).Blur(1.0).Inflate().Inflate().Inflate().Levels(0, 3.3, 255, 0, 255).Inflate().Inflate()
thickmask = last

YV12Layer(amplifiedmask.Invert(), thickmask, "mul", 255, chroma=false, Y=3, V=1, U=1)
#Overlay(amplifiedmask.Invert(), thickmask, mode="multiply").greyscale

Levels(60, 3.0, 140, overall, strength)

ringingmask = last

MaskedMerge(input, smoothed, ringingmask)

}
===

Copy between the ===

and save to "HQDering.avs"

In your script, do HQDering() at some point and observe.
This function requires YV12 colorspace, so do the conversion,
if necessary.

It is very difficult to prescribe a solution based only on one
frame.

[mit_mis]

i really want to try these solutions, but it's not working >_< each try gives me the same error, that there are bad arguments, but they're all fine, maybe i have to install something beside AviSynth?? btw i'm using AviSynth 2.5.6

[Chainmax]

mit_mis: being a novice user, it will take you some time before you can experiment with filter settings and reach satisfactory results. Please try using default settings for each filter just like in the filterchain I suggested.

[mit_mis]

you don't understand me -.- i'm using all the default settings, but it says, that there are bad arguments -.- pls, maybe i don't have something that is needed

[foxyshadis]

DeGrainMedian(5,10,0)

Don't do this kind of thing. AVISynth is very reliant on named arguments, and many filters use that to reorder arguments. They may not match the order in the readme at all.

Now, your actual error is here:

Directshowsource=("D:\Villgust - 1 [AT Raws].mp4")

Drop the equals and it'll work.

Personally I find frfun7 and tbilateral are excellent for deringing, as well as mipsmooth and degrainmedian for light noise (but I wouldn't call that light at all).

[DryFire]

I was also pleasantly surprised when frfun7 removed some ringing and other edge noise. It's also considerably faster then hqdering.

It seems that anime doesn't have too much detail to kill anyway.

I'd also like to suggest hqdn3d.

[Chainmax]

Does frfun7 do halo removal like HQDering and DeHalo_Alpha? If so, how does it compare?

[mit_mis]

i'm experimenting with it, and now i have something like this:
LoadPlugin("D:\degrainmedian.dll")
LoadPlugin("D:\DeCross.dll")
LoadPlugin("D:\MT_MaskTools.dll")
LoadPlugin("D:\RemoveGrain.dll")
LoadPlugin("D:\hqdn3d.dll")
LoadPlugin("SmoothDeinterlacer.dll")
Import("D:\LimitedSharpenFaster.avs")

Directshowsource("D:\Villgust - 1 [AT Raws].mp4")

SmoothDeinterlace()
DeCross(28, 80, 2, false)
DeGrainMedian(5,10,0)
Lanczos4Resize(640,480)
LimitedSharpenFaster(Smode=4,strength=20,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)

source:
http://img455.imageshack.us/img455/9573/source9wf.jpg

result:
http://img317.imageshack.us/img317/2299/result8rp.jpg

and it's the same frame, pls if you could tell me what would you add to my script

[Chainmax]

First, DeCross() goes before deinterlacing/IVTCing. Second, why are you using SmoothDeinterlace?

[mit_mis]

cause i didn't knew what to use, can u recommend something else?

[Chainmax]

First read all of the IVTCing/Deinterlacing documents I linked you to. Once you find wether your source is telecined or interlaced, I'll recommend you something.

[mit_mis]

well it seems to be telecined after all, so pls tell me how my script should look

[Chainmax]

First you have to know wether your source is Top Field First (TFF) or Bottom Field First, Decomb's readmes and tutorials tell you how to find out. Once you do, grab EEDI2, TIVTC, TDeint (http://forum.doom9.org/showthread.php?t=82264&page=32) and use the following lines:

For TFF clips:
AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)


For BFF clips:
Interp = SeparateFields().SelectEven().EEDI2(field=0)
Deinted = TDeint(order=0,field=0,edeint=Interp)
TFM(mode=6,order=0,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

[mit_mis]

First you have to know wether your source is Top Field First (TFF) or Bottom Field First, Decomb's readmes and tutorials tell you how to find out. Once you do, grab EEDI2, TIVTC, TDeint (http://forum.doom9.org/showthread.php?t=82264&page=32) and use the following lines:

For TFF clips:
AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)


For BFF clips:
Interp = SeparateFields().SelectEven().EEDI2(field=0)
Deinted = TDeint(order=0,field=0,edeint=Interp)
TFM(mode=6,order=0,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

thanks:D it really helped, but now i need a good smoother, can you recommend something?

[Chainmax]

For this kind of noise I'd suggest you to use DeGrainMedian like you've been doing. Its readme has many useful tips on how to tweak its parameters.

[mit_mis]

and maybe you have an ide what can i do with frames like that:
http://img460.imageshack.us/img460/9632/moon2da.jpg
?

[Chainmax]

My monitor isn't doing so well these days, but it looks like blocking. Grab MaskTools v2.x, DCTFilter, DGDecode, DeBlock_QED_MT2 and use the following somewhere in your script:

Deblock_QED(quant1=30,quant2=51)

[foxyshadis]

Indeed, today is your lucky day.

function BlockTerminator(clip c, int "ss", float "noise", int "dest_x", int "dest_y", int "cx", int "cy", int "cw", int "ch", bool "gpu", bool "deblock", bool "edi", bool "sharp", bool "npp", bool "fast") {
dest_x=default(dest_x,c.width) # final size
dest_y=default(dest_y,c.height)
ss=sqrt(default(ss,1.5)) # supersampling
noise=default(noise,2) # extra noise
cx=default(cx,0) # cropping params
cy=default(cy,0)
cw=default(cw,0)
ch=default(ch,0)
fast=default(fast,false)
gpu=default(gpu,false)
deblock=default(deblock,true)
edi=default(edi,fast ? false : true)
npp=default(npp,false)
sharp=default(sharp,true)
c
deblock ? DeBlock(quant=45) : last
crop(cx,cy,cw,ch,align=true)
Spline16Resize(m(16,width*ss),m(16,height*ss))
edi ? EEDI2().TurnRight().EEDI2().TurnLeft() : last
fast ? removegrain(mode=1) : frfun7(lambda=2,t=8,tuv=10) # gpu ? FFT3DGPU(sigma=2,bt=3,sharpen=0.7,precision=2) : FFT3DFilter(sigma=2,bt=3,sharpen=0.7)
sharp ? (fast ? sharpen(.7) : LimitedSharpenFaster(SMode=3,LMode=3,Strength=400,undershoot=2,wide=true,soft=5,ss_y=1,ss_x=1)) : last
LanczosResize(dest_x,dest_y)
noise > 0.0 ? AddGrain(noise,0,0) : last
npp ? removegrain(mode=5) : last
}

function m(int r, float x) {return(x<16?16:int(round(x/float(r))*r))}

Just add it to your script and use default settings (or use fast=true for some speedup at the expense of some quality). Slow, but you'll have no more blocks at all. Other deblockers are only posers.

This requires, by default:
LimitedSharpenFaster
Deblock (from fizick)
EEDI2
FrFun7 (Can be swapped out with FFT3DFilter/GPU)
Addgrain
Removegrain

Fun! Sorry I can't package them up, I don't have the sources to some of the gpl'd ones.

[mit_mis]

Indeed, today is your lucky day.

function BlockTerminator(clip c, int "ss", float "noise", int "dest_x", int "dest_y", int "cx", int "cy", int "cw", int "ch", bool "gpu", bool "deblock", bool "edi", bool "sharp", bool "npp", bool "fast") {
dest_x=default(dest_x,c.width) # final size
dest_y=default(dest_y,c.height)
ss=sqrt(default(ss,1.5)) # supersampling
noise=default(noise,2) # extra noise
cx=default(cx,0) # cropping params
cy=default(cy,0)
cw=default(cw,0)
ch=default(ch,0)
fast=default(gpu,false)
gpu=default(gpu,false)
deblock=default(deblock,true)
edi=default(edi,fast ? false : true)
npp=default(npp,false)
sharp=default(sharp,fast ? false : true)
c
deblock ? DeBlock(quant=45) : last
crop(cx,cy,cw,ch,align=true)
Spline16Resize(m(16,width*ss),m(16,height*ss))
sharp ? LimitedSharpenFaster(SMode=3,LMode=3,Strength=1000,wide=true,soft=75,ss_y=1,ss_x=1) : last
edi ? EEDI2().TurnRight().EEDI2().TurnLeft() : last
fast ? removegrain(mode=2) : gpu ? FFT3DGPU(sigma=1,bt=3,sharpen=0.7,precision=2) : FFT3DFilter(sigma=1,bt=3,sharpen=0.7)
LanczosResize(dest_x,dest_y)
noise > 0.0 ? AddGrain(noise,0,0) : last
npp ? removegrain(mode=5) : last
}

function m(int r, float x) {return(x<16?16:int(round(x/float(r))*r))}

Just add it to your script and use default settings (or use fast=true for some speedup at the expense of some quality). Slow, but you'll have no more blocks at all. Other deblockers are only posers.

This requires, by default:
LimitedSharpenFaster
Deblock (from fizick)
EEDI2
FFT3DFilter/GPU
Addgrain
Removegrain

Fun! Sorry I can't package them up, I don't have the sources to some of the gpl'd ones.

Avisynth open failure:
Evaluate: left of '?' must be boolean (true/false)
(...\BlockTerminator.avs, line 18)

and this is line 18:
sharp ? LimitedSharpenFaster(SMode=3,LMode=3,Strength=1000,wide=true,soft=75,ss_y=1,ss_x=1) : last

what can i do with that?

[foxyshadis]

Sorry, I edited a bunch of times as I changed it around and a couple versions were bad, oops. Try copying it over again. A readme will be forthcoming soon, along with a final version, which will be a little faster.

Samples:
http://foxyshadis.slightlydark.com/random/moon2da.jpg
Original
http://foxyshadis.slightlydark.com/random/badmoon-deblock.jpg
Deblock(quant=45)
http://foxyshadis.slightlydark.com/random/badmoon-slow.jpg
BlockTerminator
http://foxyshadis.slightlydark.com/random/badmoon-fast.jpg
BlockTerminator (fast)

[mit_mis]

yeah, it works great, but it goes 0-1fps and it wants 23h encoding in first pass >_< maybe you have some sugestions how to improve my script to go faster?
LoadPlugin("D:\avisynth\degrainmedian.dll")
LoadPlugin("D:\avisynth\DeCross.dll")
LoadPlugin("D:\avisynth\MT_MaskTools.dll")
LoadPlugin("D:\avisynth\RemoveGrain.dll")
LoadPlugin("D:\avisynth\TBilateral.dll")
LoadPlugin("D:\avisynth\FluxSmooth.dll")
LoadPlugin("D:\avisynth\TIVTC.dll")
LoadPlugin("D:\avisynth\EEDI2.dll")
LoadPlugin("D:\avisynth\TDeint.dll")
LoadPlugin("MaskTools.dll")
LoadPlugin("DeBlock.dll")
LoadPlugin("AddGrain.dll")
LoadPlugin("frfun7.dll")
Import("D:\avisynth\LimitedSharpenFaster.avs")
Import("D:\avisynth\BlockTerminator.avs")

Directshowsource("D:\avisynth\Villgust - 1 [AT Raws].mp4")

DeCross(28, 80, 2, false)

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

ConvertToYV12()
BlockTerminator()
ConvertToYUY2()

DeGrainMedian(5,10,0)
Lanczos4Resize(640,480)

LimitedSharpenFaster(Smode=4,strength=20,overshoot=1,wide=false,ss_x=1.3,ss_y=1.3)

[Chainmax]

foxyshadis: OMFG!

/me faints :)

mit_mis: if you want good results, you have to suffer the pain ;). IIRC, one time an intermediary huffyuv file took like 16 hours to complete :D.

[foxyshadis]

Well, you could always try the fast one, or simply deblock(quant=45) with a sharpener attached. I'll try to refine it more so it's not so slow, but it's unfinished.

[lamer_de]

Is it just my eyes, or is there no significant difference between deblock 45 and Blockterminator other than Blockterminator added some slight noise to the deblocked areas (which, for anime, I think is inappropriate).

CU,
lamer_de

[foxyshadis]

You won't notice there so much because the blocks are all in featureless areas. It's much different where strong blocking occurs in detailed areas, deblock leaves lots of edge disconnects. In this sense I'm replicating (and trying to better) MSU smart deblock. EEDI2 is where the reconnection comes in, it's quite the versatile filter (although the singular nature of the output hamstrings it a bit).

However, the filter still has a ways to go before I'm happy with it. I'm putting together a fast weighted (adaptive?) deblock to go with it, as this blurs too much, and still have different orderings and smoothing filters to test. I'd like it to be much faster in the end, without losing too much quality.

Development definitely goes in spurts though, can't when I'll be done. ;_;

[zambelli]

@Foxyshadis:

I've been playing around with BlockTerminator this last week. Nice work! It looks very promising. Are you ready to release an "official" version of the script yet?

[zlab]

Sorry, may I ask for wher I can D/L Decross.dll & DeGrainMedian ?

[Chainmax]

zambelli: it's not quite there yet. We're discussing some of its features, including the fast mode.

I uploaded DeCross along with an english readme (provided by niiyan) here (http://rapidshare.de/files/22711944/DeCross_v0.0.02___English_readme.rar.html). DeGrainMedian can be downloaded here (http://avisynth.org.ru/fizick.html), like all filters made by Fizick.

[foxyshadis]

zambelli, I was pinning some of my hopes on smoothdeblock, but I don't think that's going to be fast enough. So I'll go back to working on it once I wrap up a couple other projects.