Some advices for avs anime scripts :)

[kyousuke]

hello, i see that some people here use avs for encoding animes. so i'm going to give you some setting i use in general.

nota: the method can be generic, the setting aren't. you must find the best setting, and your video'll be more better.

so let's go.

i use a certain order in my avs script. i don't know if this order is the best, marcFD, and the other coders'll be able to answer to this.

# 1) import video stream
# 2) IVTC or correction frame rate
# 3) Denoising and/or smoothing, etc...
# 4) virtual dubs plugins import
# 5) tweak (hue, saturation, brightness)
# 6) subtitler for the subbers ;)

so here an extract of my script for patlabor3 trailer and other exemples very useful for some animes.

/--------------------------------------------------------------------/
# 1--- SOURCE VIDEO ---
VideoSource("c:\TEST ENCODE XDIV PATLABOR_3_Trailer\anime.d2v")


/--------------------------------------------------------------------/
# 2--- DESINTERLACE (DECOMB.DLL) ntsc guided with post processing + cropping + IVTC 23.9fps
Telecide(Guide=1,Gthresh=40,Threshold=20,Chroma=True,dthreshold=10,Post=true).Crop(16,68,690,344)
Decimate(cycle=5)

#but you can resize at the same time like this

Telecide(Guide=1,Gthresh=40,Threshold=20,Chroma=True,dthreshold=10,Post=true).Crop(16,68,690,344)
clip=Decimate(cycle=5).BicubicResize(640,352,0,0.75)
return clip


/--------------------------------------------------------------------/
# 3---- DENOISING and/or chroma and/or motion and/or etc... ---
# there're A LOT OF METHOD ! and a method can be exellent for a video and not for an other.
# so check your virtual dub ouput :) (marc FD, please, make an output gui for avs plugins ;)))

# --- Chroma Noise Reducer 2 ---
cnr2()
# or

# --- Chroma Noise Reducer 2 ---
Subtract(clip.cnr2(),clip).Levels(129,1,179,0,255)
# i don't remember the values ^^;
# or

# --- Motion Adaptive Denoiser --- (2,4,4,4) soft, (2,10,2,6) normal, (6,14,1,10) hard
MAD(2,10,2,6)
# MAD became MAM for now

# ---- DENOISING with TemporalSmoother/soften ---
# 1) little noise (fast), 2) medium noise (slow), 3) heavy noise (very slow)
#Clip=TemporalSmoother(Clip,2,1)
#
#Clip=SpatialSoftenMMX(Clip,1,4,6,false,false,4,4,6,8)
#Clip=TemporalSmoother(Clip,2)
#
#Clip=SpatialSoftenMMX(Clip,2,4,6,false,false,4,4,6,8)
#Clip=TemporalSmoother(Clip,3)

so you can denoise with a smoothing and add the chroma and the luma with this :
# --- Smoother HiQ ---
# SmoothHiQ(Diameter, Luma Threshold, Chroma threshold, Amount, Maintain Difference)
# Temporal=Source.TemporalSoften2(1,5,25).MergeChroma(Source)
# Spatial=SmoothHiQ(Temporal,3,25,10,192,3).MergeLuma(Temporal)

you can combine all this filters with the last plugin of marc FD like this :
# MAM + SmoothHIQ
MAM(Cnr2().TemporalSoften2(1,10,0),SmoothHIQ(3,15,25,192,10))
# or MAM + SpatialSoften
MAM(Cnr2().TemporalSoften2(1,10,0),SpatialSoften(2,15,25))

i have to say that finding the good values is very hard and very long. i found last days a method called Layer wich use TomsMoComp, seems to be interesting :

clip=AVISource("c:\TEST ENCODE XDIV PATLABOR_3_Trailer\anime.d2v")
clip=clip.ConvertToYUY2.TomsMoComp(0,15,1)
Levels(clip3,3,1.03,255,8,240)

i haven't tested it, so i don't know certains values.

and after making a lot of tests, i don't think that a denoising is always necessary. marcFD isn't it ?
sometime, when you find anime raws, 2d cleaner is the best ^^;;;


/--------------------------------------------------------------------/
# 4--- virtual dubs plugins import
Now that your video was denoised we can use the filters, wich can really help us too smooth the video without lose all the details. i had separated the only usefull filterd from the w.j.dijkhof's script.
i called it Anime_VD_plug.AVS. there're this filters : 2D Cleaner, Smart Smoother, WarpSharp, Xsharpen, Cartoon, VHS Filter, Subtitler.

In general i use 2D cleaner + warsharp but don't forget, the characters into an anime are made on celluloid (today on computer ;) but they have only few colors, but background are painted and have a lot of details, so don't lose the details of background for smoothing the characters.
just find the best settings, and warpsharp can help to don't abuse of 2d cleaner.

Xsharpen, Cartoon, VHS Filter can help you to sharp your video but i don't use it. don't forget they're very slow and all the benefits you would be able to have will be loose after the mpg4encoding. xvid/divx smooth the video ^^;

so the script is attached at the end of this thread, and here 2 two exemples :

# --- 2Dcleaner
ConvertToRGB()
VD_2DCleaneroptk7(threshold,x,y)
ConvertToYUY2()

# --- SHARPENING
ConvertToRGB()
VD_WarpSharp(depth)
ConvertToYUY2()


/--------------------------------------------------------------------/
# 5--- tweak (hue, saturation, brightness)
here are the default values. this filters can help you to correct the saturation and the brightness in general, don't forget this's is very slow and the xvid post processing can make this in real time.

# Tweak(hue[0.0], sat[1.0], bright[0.0], cont[1.0])


/--------------------------------------------------------------------/
# 6--- subtitler for the subbers ;)
there're two ways. you can use SSA or ASS. ssa can be used with subtitler into VirtualDub or be imported into AVS. ASS is a new format derived from SSA, and has advanced features. it can be used with texsub into VirtualDubn but equally with AVS.


that's all folk ^^;

sorry for my bad english.

any suggestions, corrections are welcome :)

++

[kyousuke]

rename it into .avs ;)

erk, the attachement don't work.
moderators !!! can i post a long script here ?

[soulfx]

Nice work kyousuke, as this will help out many new to the AVS scene and give them an idea of what it's all about and where to start. At the same time it's an awesome arrangement to compliment even the experienced AVS'er.

I noticed you mentioned the Tweak filter slowing down the encode, but actually I never really noticed it until you said something. I only get about 1fps drop in speed between when it's used vs. not used.

Peace,
SoulFX

[DJ Bobo]

The better IVTC alternative: IVTC 2.2 downloable by doom9

ivtc(44,11,95)

works alot better than Decomb.
If you're a perfectionist, combine IVTC & Decomb for excellent results that way:

ivtc(44,11,95)
FieldDeinterlace()

And an alternative for the subs: use TextSub (coming with VobSub) to load your sub (MDVD) or srt files that way for example:

Textsub("filename.srt")

[sh0dan]

If using avisynth 2.04, you should try to use ConvertBackToYUY2() in the VDub scripts - It'll give you better chroma preservation.

[kyousuke]

soulfx > it's my fault ^^;
i forgot to say that : tweak under avs is rapid, but hut/sat/intensity under vdub is slow, it's the old version of tweak under vdub, i think... and too obsolete...

DJ Bobo > interesting, for my part in general, i use decomb (ivtc) + telecide to remove all interlaced lines and ghosting effects + area based v1.3 if there're few interlaced artefacts. but i'll try your method :)

sh0dan > sugoi! i forgot to tell this! ^^

to everybody, if your anime raw has a rgb video stream, use ConvertBackToYUY2() as sh0dan said or ConvertToYUY2() to have a YUY2 video stream into VirtualDub. it's really the best.


anybody use ASS ?
i tried it yesterday, it works very well for scrolling text, zoom, fade, rotation (this's great) etc...
anybody wanted to make a gui ? the editing is very long without, like avs...

[OUTPinged_]

kyo,


You have to use spatial filtering _only_ as a last resort.


I really missed much and i dont know/remember good threshs for sshiq and MAM, but i do so for temporal/spatialsoften. And your setup for them will produce unacceptable results.

TemporalSoften2(1,10,0),SpatialSoften(2,15,25)) <--- 1,Spatialsoften will act pretty badly here. Moreover, Tempsoften will not work well with 0 chroma thresh even after heavy cnr. You will have much more effective use of it by setting it to 1-10-1.

More on ss. It is _very_ slow. How much fps is your setup producing?


However I am a filter hater so I may be a wrong person here. The bad thing is, you cant completely remove even pretty small noise without making output show noticable differences on some scenes, even after setups like 1-5-0 of TS (which doesnt remove much noise, too). Masking out motion areas helps though.

Good mask setup for small noise was a motion masked 1-10-1 TS for stills and low thresh sshiq for motion areas with most aggressive motion detection. I wasnt able to find scenes with excessive artifacts yet, but I am still hoping :-)



I am sure that temporalsoften clone coupled with real motion processing will be _the_ filter to use.

[kyousuke]

i forgot to use 1 for the chroma.

my setup produce 5-6fps on my XP1700 with ivtc+denoising+some vdub filters... very slow but i noticed that cnr2() and SHiQ are better rapid than i hope... MAM effect? don't know ;)

i'm going to search the best generic setting for standard anime video with noising effect, but i've to say that 2d cleaner is an (too) easy solution and it works well... :sly:

[jarthel]

what anime series have you tried this scripts? Also, what the target output filesize in your encodes.

Seems like a great script but I only have an Athlon 1.1ghz (overclocked) and by using SSHiQ, my encoding time almost triples.

Thanks

Jayel

[bill_baroud]

if notepad is a pain , for you, you could just add ASS tag in a SSA file (and edit it with SSA) ... Textsub will render is properly.

Ok, it's just usefull to set up the colors of fonts, but it's better than nothing ;)

@OUTPinged_ : well everybody can't have R2 DVD , or get some wonderfull hdtv rip, when there isn't dvd ... so filter and spatial filter are usefull because the raws you get are (too) often messed up.

[kyousuke]

jarthel > for 4:3 i use 640x480 @ 23.996 - ratio 1.33
> for 16:9 i use 640x352 @ 23.996 - ratio 1.85
for the filesize, it's very easy, you can find it yourself :)
for SSHiQ, use it with MAM. it's the best solution, for exemple on my little amdk7, vdub run at 8-10fps with mam+SSHiQ, and run at 30fps on my amdXP. hiii ^--^

bill_baroud > i don't use notepad, i'm not crasy. for editing my "ass" (humhum..) , i use subresync and ass-quickref.txt (lol)

for all > i tried these night white blance, what do you think about this filter ? it seems interesting, but i don't know if it's better then tweak. anybody has an idea about this ?

[jarthel]

Well when I meant target output filesize, I meant if you're target is 350mb/episode then the quality would really be good.

Also, you didn't answer my other question on what anime series who have used this settings on.

7FPS using SSHIQ? that's alot. with my previous encodes, with just decomb,SSHiQ,crop and resize (all avisynth functions), it processes at 2 to 4FPS.

Thanks for the reply. :)

Jayel

[jarthel]

. .

[manono]

Hi Jayel-

He's using Diameter 3, not 5. Big difference in speed.

[jarthel]

. .

[lovelegend]

Hi, I'm new in using Avisynth too. I would like to ask if I am going to convert sharp clips such as DVD rips to SVCD, do I still need to use a filter chain (since DVD rips' image is sharp already)? Also, I need ivtc only if I see interlaced lines or ghosting effects, right? Can somebody post a pic of capture which has interlaced lines? Since I've heard this term many times but I haven't seen an exmaple... ^^"

My first anime project is converting "spirited away". As you may know, the Red color in DVD version is too high. Therefore, I will use a RGB filter to adjust the color. Will this filter add noise to the anime? If so, then which denoising filter should I use?

P.S. Hope I didn't have too many questions... :D

[lovelegend]

Originally posted by jarthel
Thanks Manono. :)

Well aside from speed difference, can you tell me what the difference between diameter 3 and 5?

Jayel

/me waits for kyousuke to send him the sample avs script

The explanation of SSHiQ filter is here:
http://sauron.mordor.net/dgraft/hiq/smoothhiq.html

I think it's better to ask kyousuke to post his avs here... since not only you wanna try it... :D

[OUTPinged_]

kyo,
the slowest filter in a mentioned chain was a regular spatialsoften. sshq is faster, use it (i am serious).


lovelegend,

for spirited away, setup swiss army knife filter like that:

1. get correct color clip from dvd trailers.

2. find a scene which exist in trailer and movie and where in trailer it has pure white and in movie doesnt.

3. get color values with a photoshop, check how much red channel you need to subtract.

4. Absolutely no "i know there is too much red, so i will make red channel go, say, 10 notches down" approach. You will end with crappy movie in that case.

bill_baroud: Sadly, it's true. Still, that is no reason to produce over-filtered encodes. Sometimes you are better to leave source a bit noisy than filter it to death.


If you still have to filter it to death because of temporal noise, try this:

vid=mpeg2source("C:\temp\bzzz.d2v").Telecide(reverse=false,swap=false,firstlast=false,guide=1,gthresh=25,post=true,threshold=25,dthreshold=5,blend=true,chroma=true,debug=true).Cnr2(mode=0,ln=35,lm=192,un=47,um=256,vn=47,vm=256)
clip2=vid.smoothhiq(3,12,12,255,5)
clip1=vid.TemporalSoften2(2,12,0)
mask=MotionMask(clip2,1,1,true)
MaskedMix(clip1,clip2,mask)


That is for _completely_ removing temporal noise that doesnt exceed 12 luma value dispersion. Also note that cnr settings are probably crappy and can be reduced greatly.

That thing does:

1. filter motion parts with sshiq and stills with huge tempsoften.
2. tempsoften artifacts are somewhat reduced by 0 chroma thresh, but there are still some left because motion masking isnt perfect :-(
3. produces pretty nice uniform color areas, that sshiq freaks are so proud of.

The above setup is extremely slow, but who cares...


PS. btw what's wrong with writing scripts with notepad? :-)

[Belgabor]

Does anyone know where to find information about the ASS format? Serching for it... well, let's say it's a pain in the ass ;)

Cheers
Belgabor

[OUTPinged_]

lol

[manono]

Hi lovelegend-

Spirited Away is your first ever anime project? As OUTPinged said, get the trailers off Disk 2 and compare with them. If you don't have Photoshop, the trailers are good enough to match against. But there's a whole lot more to fixing Spirited Away than just lowering the reds. You have to desaturate, add brightness and contrast, and boost the blues and greens in differing amounts as well. It wasn't easy.

And if you want to see a picture of an interlaced frame, then go Here (http://nickyguides.digital-digest.com/pictures/original-dvd-interlace.jpg).

[OUTPinged_]

Oh well, got the piece of spirited away vob, watched it, compared it.


You better leave that red stuff alone. Removing it will require some pretty good skill and it is definitely not a task for a first encode.

[kyousuke]

jarthel > find the setting by yourself and you'll understand how this filter works :p

nota :
SmoothHiQ(Diameter, Luma Threshold, Chroma threshold, Amount, Maintain Difference
Diameter
Defines on how big a diameter the filter works. Diameter can be 3,5,7,9,11 or 13. The greater the diameter, the more smooth will the video appear. Has especially influence on big evenly colored surfaces. The greater the diameter, the slower will the filter operate. Recommended settings are 5 or 7.

OUTPinged_ > wow, big script, i never used MotionMask and MotionMix, in fact i haven't red anime :p like chihiro but if a i've chihiro i day i'll try

recap :
MAM = very rapid
SShiQ = rapid with 16:9 mpeg2 source, slow with 4:3 avi source, don't know why...
spatialsoften = slow
TemporalSoften2 = i'm going to test it this afternoon
2d cleaner = slow
warsharp = medium
xsharpen = rapid
white balance = rapid
VHS = very slow

Belgabor > arfff, read ass-quickref.txt or ass-specs.doc.
you don't believe me? try this :
{\t([6000,9000,][2,]{\fscx200\fscy200} and here you text :devil:

[Belgabor]

Originally posted by kyousuke
Belgabor > arfff, read ass-quickref.txt or ass-specs.doc.


Thats kinda hard if you can't find those documents, ne? :p

Cheers
Belgabor

[kyousuke]

yeah :D

there're given with vobsub. hmm don't remember the url ^^;
check doom9 links or in the news, the last version is vobsub 2.6 :cool:

[Schultz]

Originally posted by kyousuke
yeah :D

there're given with vobsub. hmm don't remember the url ^^;
check doom9 links or in the news, the last version is vobsub 2.6 :cool:
http://vobsub.edensrising.com

[Belgabor]

YES! I love to search the web like a madman for stuff that already was somewhere on my hd.

To sum it up in one word: DOH!

Thanks, ppl

Belgabor

[jarthel]

Originally posted by kyousuke
jarthel > find the setting by yourself and you'll understand how this filter works :p

I'm not really after your settings.I just need to see the general layout of the script.

[rubic42]

I've been working on the same thing... try these color settings on for size to see if it helps that R2 version of Spirited Away:

VD_RGBAdjustment(0.83,1.08,1.35)
Levels(0,0.946,241,0,255)
VD_Hue(0,false,1.22,1.12,true,true,true,true,true,true)

It helps a lot to play with the settings in VirtualDub to get instant visual feedback, and then translate them into AVISynth settings. It also helped to load up the trailers on the extras DVD as someone else suggested earlier.

[Marc FD]

whow. big script !
i don't use so much filters.
because i try some XviD pre-filtering.
but raws are far more noisy than R2 DVDs :D

[Suzahara]

Originally posted by kyousuke
yeah :D

there're given with vobsub. hmm don't remember the url ^^;
check doom9 links or in the news, the last version is vobsub 2.6 :cool:

Vobsub 2.6? As far as I know, it's only up to 2.18...