Anime Encoding

[Kenshin5]

Hi guys im new in a beginner in this section and i want to know more about how to do perfectly encode an Anime footage. I followed few guides from here and from other sections, but i guess they weren't enough because it still doesn't makes me satisfied. So i can say a week section of my encoding is the filtering. Can somebody point me give me a few hints about Anime filtering, or point me to a guide where i can learn more about that stuff. Thank you for your help guys :).

[Sharktooth]

Welcome to doom9.
Filtering anime is not an easy task, i would start from the AviSynth forum where filters and stuff is discussed.
Plus, there are some anime addicted members that know a lot about anime filtering and encoding.
A good idea is to visit also the Real Video forum, since most anime are encoded with that codec.
Sarching the forums will make you find a lot of useful threads too.
Hope it helps.

[L'il Jerry]

Originally posted by Sharktooth
A good idea is to visit also the Real Video forum, since most anime are encoded with that codec.

Eh, not really. Most anime is encoded with XviD or DivX.

But for a guide, here's (http://forums.animesuki.com/showpost.php?p=427252&postcount=15) a pretty decent one. But the amount and type of filtering is completely dependent on the source material. So without knowing anything about the source(s) you are encoding its really hard to give you any filter recommendations.

[Neo Neko]

Encoding anime just got really easy. Using AVISynth, DOOM9's mencoder gui, and mencoders x264 encoding abilities. If you encode to fit a single CD then it would be way overkill for your average anime etc. And I am talking about full resolution anamorphic encoding.

I realize that this is the Xvid section and no disrespect to Xvid. But h264 is just damned impressive. Even though X264 is only about where Xvid was a year or two ago. The only downside is it is a bit more resource intensive.

[Teegedeck]

I'm very much inclined to agree here. Id prefer watching an x264 encode with inloop filtering over an XviD encode with the amount of preprocessing people typically like to use...

Concerning XviD, please try the 'cartoon mode' (yes, it does work well with anime...), but be sure to use the latest XviD CVS-head build (http://www.aziendeassociate.it/cd.asp?dir=/). Don't use adaptive quantization. Have fun.

[Spyn]

if your source is clean, you don't have to use any filter except
if you want to increase compressibility.

so if your source is noisy or if you want to increase compressibility (you may also loose some details) you can use MipSmooth AviSynth Filter

http://www.avisynth.org/warpenterprises/files/mipsmooth_25_dll_20031217.zip

use MipSmooth(preset="AnimeHQ") or MipSmooth(preset="AnimeLQ")

(read the manual include in the zip)

i don't think Xvid Cartoon mode works good with detailed anime, i think this option is for cartoon with large areas like the simpsons, tex avery...

[Teegedeck]

Have you actually made a comparison of a non-cartoon mode encoding against a cartoon mode encoding?

[Spyn]

Originally posted by Teegedeck
Have you actually made a comparison of a non-cartoon mode encoding against a cartoon mode encoding?

no but it don't seems to work good with anime :

http://forum.doom9.org/showthread.php?s=&threadid=65938

[Kenshin5]

Tnx for the reply guys, i appreciate it. But im interested in finding more about using noise filters, and how to clean up interlaced sources, and also hybrid sources where the footage is mixed (NTSC or Film).

[Sharktooth]

Then head to the avisynth forum. It's where you'll find all those infos.

[Teegedeck]

Originally posted by Spyn
no but it don't seems to work good with anime :

http://forum.doom9.org/showthread.php?s=&threadid=65938

Because of an errand pixel I wouldn't discard a feature that saves... let me see (/me performing comparison) - more than 4% of bitrate for me.

You do know that better efficiency like this leads to lower quantization and thus higher level of detail in two-pass?

Let me try to set these misconceptions about cartoon mode straight by performing the comparison that you neglected in favour of listening to hearsay:

This is how XviD typically looks in cartoon mode, compared against normal mode (at constant quantizer):

http://img214.exs.cx/img214/5331/6218vv.th.png (http://img214.exs.cx/my.php?loc=img214&image=6218vv.png)
cartoon mode

http://img237.exs.cx/img237/4751/621nocartoon7if.th.png (http://img237.exs.cx/my.php?loc=img237&image=621nocartoon7if.png)
normal mode

Please download the fullsize images and switch back and forth between them with an image-viewer. What are your results?

Yes. They are exactly the same. Only that one is from a file that's 4.11% smaller than the other one; at the same quantizer i.e. level of compression.

Now, if we do a two-pass to identical target filesize with and without cartoon mode, the result is something like this:

Average quantizers for cartoon mode encoding:
I: 3,04
P: 3,18
B: 5,18
Average quantizers for normal mode encoding:
I: 3,12
P: 3,33
B: 5,54

Translated into something visible this means quite some frames in the cartoon mode encoding are less compressed. Like this frame:

http://img32.exs.cx/img32/3417/tp18040pn.th.png (http://img32.exs.cx/my.php?loc=img32&image=tp18040pn.png)
cartoon mode

http://img239.exs.cx/img239/6003/tp1804nocartoon3mb.th.png (http://img239.exs.cx/my.php?loc=img239&image=tp1804nocartoon3mb.png)
normal mode

There really are some popular myths about anime and XviD that let people encode to a lesser quality than possible. One of the reasons why I like to recommend x264 is that is still is relatively unharmed by people making false claims as to settings without offering any proof. Though I recently read something horrible about b-frames not being good for anime in an x264-thread, too. Probably in good tradition... I think it is by root a psychological problem; someanime fans just reject to see their beloved medium as nothing but asian cartoons. But for the codec it doesn't make a difference whether you encode the Flintstones or Pokemon; both are flat surfaces and well fit for cartoon mode. On the other hand it doesn't matter whether the source is an anime with high-fps-fantastically-detailed-CG or an old Disney cartoon produced with a higher framerate than anime (more actually differing frames handdrawn per second) and drawn meticiously detailed; both are probably not good to encode with cartoon mode.

Edit: a typo; the comparison of average quantizer read 'cartoon mode' in both instances, where it should of course have been 'cartoon mode' and 'normal mode'.

[ChronoCross]

After trying out Nero and x264 I don't think AVC is ready for use yet. Not only is it slow as a fat man at a track meet(1fps encoding compared to 10fps encoding with xvid) but I just don't feel the compression/quality is as good as it needs to be in order for it to appear as a superior codec.

As for filtering Mipsmooth is a waste of time as well as convolution. for anime your gonna have to try more powerful things like deen or hqdn3d as well as sharpening (awarpsharp and Xsharpen work well). but like they said everything is relative. The most important part with anime however is IVTC, cause you can really screw this up if you don't know what your doing.

[Spyn]

Originally posted by Teegedeck

http://img32.exs.cx/img32/3417/tp18040pn.th.png (http://img32.exs.cx/my.php?loc=img32&image=tp18040pn.png)
cartoon mode

http://img239.exs.cx/img239/6003/tp1804nocartoon3mb.th.png (http://img239.exs.cx/my.php?loc=img239&image=tp1804nocartoon3mb.png)
normal mode



the second (normal mode) has a quality slightly lower cause of the higher quantizer but the quality is still very acceptable and looks a bit more sharp than the first screenshot (cartoon mode)

i will made my own test, i was just afraid about this "bug" :

Originally posted by haibane
Here is some screen shots of the artifacts I'm talking about.......
the good ones are w/o cartoon mode........

http://www-personal.umich.edu/~liusu/bug/good.jpg
http://www-personal.umich.edu/~liusu/bug/cartoon.jpg
on the cartoon.jpg you can clearly see a black dot on top of the line in the middle, which is not suppose to be there. It might be a block carried up from the line below.



http://www-personal.umich.edu/~liusu/bug/good.png
http://www-personal.umich.edu/~liusu/bug/cartoon.png
here is more pictures, on the cartoon.png, there are some black dots above the black lines, which is no present in the encode w/o cartoon mode checked.

I've notice this for a few months, I don't know is this a bug or it is a trade off for samller filesize.

[Sharktooth]

Originally posted by ChronoCross
After trying out Nero and x264 I don't think AVC is ready for use yet. Not only is it slow as a fat man at a track meet(1fps encoding compared to 10fps encoding with xvid) but I just don't feel the compression/quality is as good as it needs to be in order for it to appear as a superior codec.
If you have doubts about the h264 efficiency look at what can it do: http://www.aziendeassociate.com/screenshots.rar
It's a full DVD res anamorphic encode (720x424) @ 722kbps and not even at max quality!!!

[Spyn]

Originally posted by ChronoCross


As for filtering Mipsmooth is a waste of time as well as convolution. for anime your gonna have to try more powerful things like deen or hqdn3d as well as sharpening (awarpsharp and Xsharpen work well). [/B]

by using the filter mipsmooth with the preset "animeLQ" I increase the compressibility of approximately 10 %, i don't think this is a waste of time...

[Teegedeck]

Originally posted by ChronoCross
After trying out Nero and x264 I don't think AVC is ready for use yet. Not only is it slow as a fat man at a track meet(1fps encoding compared to 10fps encoding with xvid) but I just don't feel the compression/quality is as good as it needs to be in order for it to appear as a superior codec.
Not true:

http://img126.exs.cx/img126/82/x2646219ax.th.png (http://img126.exs.cx/my.php?loc=img126&image=x2646219ax.png)
From a constant quantizer encode, the same size as the constant-quantizer XviD encodings.

http://img197.exs.cx/img197/5158/1804x2647me.th.png (http://img197.exs.cx/my.php?loc=img197&image=1804x2647me.png)
From the same encode, which is 20% smaller than the two-pass XviD encodings from which the screenshots of this frame in my previos post stem.

(Edit: I'm gonna update the second screenshot with one from a two-pass encoding with x264 lateron, for an exact comparison. And insert the correct frame for the first one, too... ;))

Edit: corrected swapped image descriptions.

[dragongodz]

I think it is by root a psychological problem; anime fans just reject to see their beloved medium as nothing but asian cartoons
sorry but i cant let that pass. please do not try and lump all anime fans in to some negative preconcieved opinion you have. yes anime is cartoons made for Japan(to be specific), with higher quality than your average U.S.A. cartoon in many cases however. there are of course plenty of cheap tv shows made though, pokemon being a good example. thank you doctor freud for your opinion on our psychological problems though.

for the codec it doesn't make a difference whether you encode the Flintstones or Pokemon; both are flat surfaces and well fit for cartoon mode.
i agree completely. this could have been said without the insults could it not ?

[dragongodz]

oh and if you are going to reccomend H264 for people then Nero would be the thing since its relativly stable while x264 is constantly being updated and ffdshow(free viewing solution) is constantly behind. with Nero you know what the encoder produces the decoder will play. so x264, at the moment, is more for those wanting to try out or test H264 rather than people wanting to do permanant backups IMHO.

[Teegedeck]

Originally posted by dragongodz
sorry but i cant let that pass. please do not try and lump all anime fans in to some negative preconcieved opinion you have. yes anime is cartoons made for Japan(to be specific), with higher quality than your average U.S.A. cartoon in many cases however. there are of course plenty of cheap tv shows made though, pokemon being a good example. thank you doctor freud for your opinion on our psychological problems though.

:lol: Excuse me please, present company exceptet.

No offense meant.

Probably my reaction is from country-studies; so often I hear people claiming that some phenomenon is typical of only the country they study, while it can actually be found in many places.

Edit: Please accept the correction of my initial generalizing statement as a way of apologizing.

[Teegedeck]

Originally posted by Spyn
the second (normal mode) has a quality slightly lower cause of the higher quantizer but the quality is still very acceptable and looks a bit more sharp than the first screenshot (cartoon mode)

Not for my eyes. It simply has more ringing.

That 'bug' is the stray pixel that I wrote about (magnifying lense and all...). The danger of a stray pixel seems small (to me and IMHO) in comparison to the very real fact of having more highly quantized frames. Better deal.

[ChronoCross]

Originally posted by Spyn
by using the filter mipsmooth with the preset "animeLQ" I increase the compressibility of approximately 10 %, i don't think this is a waste of time...

well compression is great but I don't really think it improves on the look of the anime all that much especially when dealing with very noisy sources. using a custom set of values for mipsmooth even when done correctly still does not produce the desired result. I'm only speaking from the things I;ve worked with perhaps newer animes do better with mipsmooth than with the filters I suggested but as for now I still recommend the others.

[Spyn]

Originally posted by Teegedeck
Not for my eyes. It simply has more ringing.

That 'bug' is the stray pixel that I wrote about (magnifying lense and all...). The danger of a stray pixel seems small (to me and IMHO) in comparison to the very real fact of having more highly quantized frames. Better deal.

I will test :D

I just want to know how do you show frame type and quantizer on the video ?

[Sharktooth]

ffdshow OSD function

[Spyn]

i don't use ffdshow, any other method ? avisynth plug-in ?

[Sharktooth]

No, ffdshow.

[Spyn]

i don't like ffdshow :/

[Teegedeck]

You can use ffdshow and still let it use XviD to decode XviD - it's all in the codec configuration tab. And BTW, such snapshots are easily made by creating an .avs like

directshowsource("your_encoding.mkv")

and then opening it with VDub.

It's also an easy workaround if you don't want to use that dreaded packed bitstream feature but still want to fetch the correct frame in spite of b-frame delay.

BTW, ffdshow's OSD + Visualizations functions are really cool... You can even display the quantizer of each macroblock, if you want to check how well adaptive quantization works.

[Sharktooth]

Originally posted by Spyn
i don't like ffdshow :/
Infidel!!! :D

[Blue_MiSfit]

Honestly, what's not to like about ffdshow? :)

[Spyn]

ok someone get a link to download the latest stable release of ffdshow ?

i don't know if the word "stable" can be used when i talk about ffdshow :D

[Kenshin5]

As i know from my experience most of the anime groups use XviD as a good compression and high quality codec. I've tried the cartoon mode and there were slight differences in the quality, but the final result is still noisy. Any solutions?

[Teegedeck]

Noise-filtering? ;)

[Spyn]

Originally posted by Kenshin5
As i know from my experience most of the anime groups use XviD as a good compression and high quality codec. I've tried the cartoon mode and there were slight differences in the quality, but the final result is still noisy. Any solutions?

use an avisynth denoise filter !

convolution3d
deen
mipsmooth
fluxsmooth
temporalsoften
...

you have the choice.

[dragongodz]

Please accept the correction of my initial generalizing statement as a way of apologizing.
accepted. unfortunatly whenever you get a large group of people for any cause or subject there will always be atleast a small group that approach zealotry with their opinions.

use an avisynth denoise filter !
i personally like msmooth since it was designed for animated content. i usually use lower than default settings however because the defaults can be a little harsh.

[[MA]Debito]

I find that for modern (clean analog/digital sources) all ya really need is a simple script like this:

(settings and values are secret :))

Mpeg2Source
Telecide
Decimate
XSharpen
deen
TemporalSoften
Convolution3D
WarpSharp

My best settings for XviD so far are:

Profile @ Level: AS @ L5
->Quantization type H.263
->Everything Unchecked but B-VOPs and Closed GOV
->Max consecutive BVOPs:2
->Quantizer ratio: 1.50
->Quantizer offset: 1.00

The rest I must keep secret :)

[Spyn]

Originally posted by [MA]Debito
I find that for modern (clean analog/digital sources) all ya really need is a simple script like this:

(settings and values are secret :))

Mpeg2Source
Telecide
Decimate
XSharpen
deen
TemporalSoften
Convolution3D
WarpSharp

My best settings for XviD so far are:

Profile @ Level: AS @ L5
->Quantization type H.263
->Everything Unchecked but B-VOPs and Closed GOV
->Max consecutive BVOPs:2
->Quantizer ratio: 1.50
->Quantizer offset: 1.00

The rest I must keep secret :)

the values and filters to used depend on the source, Xvid settings and their effects are not kept secret :rolleyes:

[dragongodz]

I find that for modern (clean analog/digital sources) all ya really need is a simple script like this:

(settings and values are secret )

Mpeg2Source
Telecide
Decimate
XSharpen
deen
TemporalSoften
Convolution3D
WarpSharp
you think sharpen+smooth+smooth+smooth+sharpen is simple and good ?:eek:
ooookkkkk each to their own. :cool:

[Teegedeck]

I would call it overfiltering. And at least for "clean" sources unnecessary and very probably quality-degrading at that.

[ChronoCross]

Originally posted by [MA]Debito
I find that for modern (clean analog/digital sources) all ya really need is a simple script like this:

(settings and values are secret :))

Mpeg2Source
Telecide
Decimate
XSharpen
deen
TemporalSoften
Convolution3D
WarpSharp

My best settings for XviD so far are:

Profile @ Level: AS @ L5
->Quantization type H.263
->Everything Unchecked but B-VOPs and Closed GOV
->Max consecutive BVOPs:2
->Quantizer ratio: 1.50
->Quantizer offset: 1.00

The rest I must keep secret :)

oro.....jesus your anime script for clean sources insane. perhaps our definitions of clean digital/analog are different. but all thothe filters even on the defualts would tear your clean sources details to pieces. that would be a more appropriate filter combination for dirty analog funimation DB/DBZ/DBGT sources that have already horrible noise and artifacts and not too much detail to preserve.

Here's something a bit more simple for clean anime

mpeg2source()
Telecide()
Decimate()
awarpsharp().Xsharpen() //Modify Xsharpen to lower values because the defaults can be a bit strong and cause aliasing something fierce.
Convolution3d(preset="animeHQ")

it's just that simple and 10x faster. you can replace convolution with deen("c3d") if you like deen better or even use mipsmooth.

[Leak]

Well, I'm using the following on the R1 Ghost In The Shell: Stand Alone Complex episodes since I want to get rid of the telecine while speeding it up to PAL's 25 FPS:

IVTC with TIVTC.

BlindDeHalo2 (gotta test 3) to get rid of the absurd amount of edge enhancement in the video.

My own AWarpSharpDering (posted about it in a thread here somewhere if you want to take a look) to get rid of some extra lines around the edges caused by the edge enhancement that BlindDeHalo2 didn't remove.

DeGrainMedian (spatial) and FluxSmooth (mainly temporal) with very low settings to get rid of some noise that's still left to increase compressibility.

LimitedSharpen with low strength to sharpen things a bit after all of the above.

Resize with LanczosResize to the final frame size.

That's it - it's slow enough as it is, but it looks fine in the end. :)

np: C.J. Bolland - The Prophet (The Analogue Theatre)

[Spyn]

Originally posted by Leak
Well, I'm using the following on the R1 Ghost In The Shell: Stand Alone Complex episodes since I want to get rid of the telecine while speeding it up to PAL's 25 FPS:

IVTC with TIVTC.

BlindDeHalo2 (gotta test 3) to get rid of the absurd amount of edge enhancement in the video.

My own AWarpSharpDering (posted about it in a thread here somewhere if you want to take a look) to get rid of some extra lines around the edges caused by the edge enhancement that BlindDeHalo2 didn't remove.

DeGrainMedian (spatial) and FluxSmooth (mainly temporal) with very low settings to get rid of some noise that's still left to increase compressibility.

LimitedSharpen with low strength to sharpen things a bit after all of the above.

Resize with LanczosResize to the final frame size.

That's it - it's slow enough as it is, but it looks fine in the end. :)

np: C.J. Bolland - The Prophet (The Analogue Theatre)

can you post screenshot before / after please ? :)

[Leak]

Originally posted by Spyn
can you post screenshot before / after please ? :)

Sure, why not? :)

Also, here's the script - nothing secret about it:

source=MPEG2Source("Video.d2v",cpu=0,moderate_h=30,moderate_v=60,idct=5)

a=source

a=a.TFM(mode=1,pp=6,chroma=true,mi=50,display=false,d2v="Video.d2v")
a=a.TDecimate(mode=1,hybrid=1,vidThresh=0.5,vidDetect=3,conCycle=2,chroma=true,display=false)

#return a

newWidth=(Int(a.width*1.05+1)/4)*4
newHeight=(Int(a.height*1.05*(400.0/480.0)+1)/4)*4

a=a.BlindDeHalo2(2.25,2.25,100)
a=a.AWarpSharpDering (http://leak.no-ip.org/AviSynth/AWarpSharpDering/)(depth=20,diffThresh=16)

a=a.DeGrainMedian(mode=2,limitUV=12,limitY=6)

#a=a.LanczosResize(newWidth,newHeight)
#a=a.Crop(18,10,720,400,align=true)

a=a.LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=2,edgemode=1,strength=15,Lmode=1,soft=false)

a.AssumeFPS(25.0)


Before:
http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_7500_before.png

After:
http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_7500_after.png

I've commented out the resize so the images can easily be compared, and I've thrown out FluxSmooth because on very close inspection I found it to be overkill...

np: Animals On Wheels - Build a Church with Your Fear (Xen Cuts (Disc 2))

[aeternitas]

Thats a nice after. You cut down on the halo effect in many of the high contrast areas. Some of the black lines have thinned and look sharper. I'm not really sure how this would look better highly compressed or not, but as a backup solution this look like a good filter sequence for anime.

[Leak]

Originally posted by aeternitas
Thats a nice after. You cut down on the halo effect in many of the high contrast areas. Some of the black lines have thinned and look sharper. I'm not really sure how this would look better highly compressed or not, but as a backup solution this look like a good filter sequence for anime.

Well, I'm going for 13 episodes per DVD+R, so I'm mostly doing fixed Q2 encodes... :D

np: fscking annoying German webradio from across the office... *sigh*

[aeternitas]

I've been compressing animation on and off for almost a decade (woa were the days of motion jpeg and early versions of sorenson).

First of all before you start compressing, ask yourself what medium youre going to store these on. A year ago typically anime and movies were compressed for CD storage, but as DVD burners and media become cheap, DVD storage should be your encoding rate point.

20 episodes will fit on a DVD with very little visual quality loss. 223MB
30 episodes will fit on a DVD with slightly visual quality loss. 149MB

Usually one episode will last 24min. Typically a series will comprice of 1 - several beginning sequences and ending sequences that can be cut and encoded by themselves once and later reattached to cut down on encodeing times.

I have seen some 175MB encodes that looked almost better than the original with virtually no artifacts. There is a fine line to walk with blur and sharpness due to the high contrast native to animation. You basically want it to full-screen elgantly and blur or h.263 matrix does this (to start).

Right now I'm trying to get the best possible image at 149MB. One has to also take into consideration that anime is mostly spread by fansub, and downloading 149MB is better than 175MB is the quality is going to be hard to tell apart.

[helix]

Just out of curiosity, what do you guys usually use as your matrix? I've been doing anime for a about a year now, and have really just stuck with H.263.

[Spyn]

Originally posted by helix
Just out of curiosity, what do you guys usually use as your matrix? I've been doing anime for a about a year now, and have really just stuck with H.263.

i always use H.263 but i will test EQM v3 ULR cause it seems to be better than H.263.

http://forum.doom9.org/showthread.php?s=&postid=570152#post570152

[unmei]

I played with

undot
(dup)
tbilateral
tisophote
fastlinedarken
awarpsharp

a lot recently.

But i change filter chain all the time so..

And as i only used this for low bitrate AVC encodes, it might not be so suitable for xvid and/or hibitrate. Also the large diameters in tbilateral and the high iterations/low step in tisophote i usually used make this dead slow.

[edit]
@Helix: i too use for anime almost always the h.263 matrix. Maybe it is just because i am too lazy to seriously evaluate all matrices and learn to know they characteristics, but h.263 is quite the safe bet for anime i think.

[Kagura]

Warning: This is not a blatant act of self-promotion.

http://forums.animesuki.com/showpost.php?p=427252&postcount=15

Those are my anime encoding settings. There. Don't let anyone tell you that anime encoders are bums who hoard their secrets. -_- Give me a break. Half of the people doing anime encodes just throw a few global filters and bam, two pass. Keep in mind that this is for low quality sources, usually.

Staying on topic, I have to say that H.263 beats the crap out of all the other matrices at the sweet spot of 70% compression from first pass, with all the quality enhancing modes enabled. The quantization used by different matrices differs in its culling method. H.263 creates slight edge noise at low compressions and blocking at even lower ones. Most other matrices form NOTICEABLE edge noise and compression artifacts at around 90% compression. They are simply not suited for anime.

[helix]

Originally posted by Kagura
Warning: This is not a blatant act of self-promotion.

http://forums.animesuki.com/showpost.php?p=427252&postcount=15

Those are my anime encoding settings. There. Don't let anyone tell you that anime encoders are bums who hoard their secrets. -_- Give me a break. Half of the people doing anime encodes just throw a few global filters and bam, two pass. Keep in mind that this is for low quality sources, usually.

Staying on topic, I have to say that H.263 beats the crap out of all the other matrices at the sweet spot of 70% compression from first pass, with all the quality enhancing modes enabled. The quantization used by different matrices differs in its culling method. H.263 creates slight edge noise at low compressions and blocking at even lower ones. Most other matrices form NOTICEABLE edge noise and compression artifacts at around 90% compression. They are simply not suited for anime.


I'm inclined to agree with you. It seems that the quality of fansubs inparticular these days has greatly decreased from what it used to be. Such so, that it seems they do indeed just toss some filters in and not even bother to take the time to do a good job on thier encodes. I only wish more people would ask about anime encoding, I'd love share what I've learned so far to people that are looking for information on how to best do it.

[Shinigami-Sama]

Iuno
my fansub encoder friend has pretty standard settings
no deen though
keeps saying it was a mem leak like a sive
but I've been to buzy with learning everything I can find latly, to keep me from being bored to tears, to be able to spare cpu time to encode latly; which is quite sad now that I think about it
but once I get my new psu which will give me clean power I should start endoding again
last encode wasn't to bad but with all the stuff I've learn I should be able to make them 2x better
*edit* uber typos

[Kagura]

Well, the thing is, most fansub encoders have standard setting that they don't even bother to change between episodes and shows, which is rather sickening. Ah well. Can't blame people. They want maximum credit for least amount of work. Maybe that's why they don't share their techniques, since it'd just be avisource, undot, deen, textsub, 2 pass. hahaha

[Shinigami-Sama]

lol he has differant setting for het diffraent one
they;re not terribly diffreant but still slightly modifided
three pass QC doesn't let much through ;)
but yeah
most don't eraly like to share maybe next time I see pastic on I'll ask him about theier settings

[Soulhunter]

Another vote for h.263... :D

Imo no CQM is able to cope better with anime content !!!


Bye

[Shinigami-Sama]

Originally posted by Soulhunter
Another vote for h.263... :D

Imo no CQM is able to cope better with anime content !!!


Bye
hara for h263
:D
makes me look less newbish
>_>
yeah plastis uses prety standard settings also
but I was told notto tell vaulues
like I rembmer anyways >_>

[Kagura]

*shrug*

I'll try to restrain myself from lashing out, but you know that I am thinking, "The guy probably calls a bunch of global filter. He doesn't share his technique because he doesn't want everyone laughing at him for how simplistic it is."

[Shinigami-Sama]

both my fan subbing encoder friends told me their exact filter and values; they didn';t want me to tell anyone though, and yes they aer simple nothing as filtered as soulhunter there. mostly ivtc, de-noisers, sharpen/smooth resize some other one I can't remember what it is,
and other standard filters that everyone usles without thinking about it. so realy ther scripts aren't that complicated..

[Kenshin5]

Ok guys im encoding Love Hina Again, i don't know if anyone seen the source, it's mostly clean but there is some interlacing and a little noise. I've tried to use Convolution3D to smoothen it up but let's say it didn't go so well, and FluxSmooth is giving me some error in the scripting ("The Function FluxSmooth can't be found") and i already loaded the correct plugin in my avs script. Also my deinterlacing method isn't going so well can somebody give me a good sample for deinterlacing.

My Method:

Telecide(order=1,guide=1)
Decimate(cycle=5, mode=3, threshold=1.0)
FluxSmooth()
Crop(8,0,-8,0)
Lanczos4Resize(640,480)
AssumeFPS(23.976)

[Kagura]

Try:

IVTC
Crop
Resize
Denoise
Decimate

After, you can also maybe use sangnom to clean up ivtc artifacts.

The latest fluxsmooth is called as fluxsmoothst(spat_value, temp_value) or fluxsmootht(temp_value).

[lordadmira]

I've had great success with the Smart Smoother filter with strength 7,75 to 7,100. It keeps good detail while smoothing within color areas. I'll pretreat it with DNR if there's a lot of static.


LA

[Kenshin5]

Ok i got that fix now my method for making the footage progressive isn't working well, there is some mouth interlacing and stuff and also some blending aswell.

[Kagura]

Originally posted by lordadmira
I've had great success with the Smart Smoother filter with strength 7,75 to 7,100. It keeps good detail while smoothing within color areas. I'll pretreat it with DNR if there's a lot of static.


LA

YAY! Great to see another fan of older filters. Honestly, most of the newer avs "smoothers," "cleaners," and "denoisers" can't hold a candle to the versatility of the old ones.

The way I see it, if I want to denoise, I'll use a brute force denoiser and not one that uses new fancy techniques, since most of these experimental ones can't match the quality of the old ones. If I need to fine tune it to be smarter, I can use a combination of masking and supersampling. =)

[guada 2]

Hello my friends Kagura :)

Say me, did you tried "the guadafilter": the essence of the source
no ever, it's strange you should try it but,but, but....



Note: NO, it's a joke JUST an april fish :D :D :D :D :D

To soon ;)

[Kagura]

Man, it took me a while to puzzle out that one. XD

say me = dites moi?
to soon = a bientot?
april fish = poisson d'avril?

dunno what "essence of the source" is unfortunately =(

=P Are you using something like babelfish to translate?

Ah well, I got the gauda part though yea yea. Now, how is that on topic...?

[_E_]

Originally posted by Kagura
YAY! Great to see another fan of older filters. Honestly, most of the newer avs "smoothers," "cleaners," and "denoisers" can't hold a candle to the versatility of the old ones.

The way I see it, if I want to denoise, I'll use a brute force denoiser and not one that uses new fancy techniques, since most of these experimental ones can't match the quality of the old ones. If I need to fine tune it to be smarter, I can use a combination of masking and supersampling. =)

I beg to disagree, brute force filtering should only be kept for bad sources IMO. Sure, you can apply 2DClean for example and brute filter the source & it's details and be done with , but the new filters try to use algorithms which gets rid of the noise while minimising the amount of detail loss. Take FFT3D for example, which is frequency spectrum-dependant if I remember correctly, I'd rather sacrifice encoding speed and use that instead. Ultimatly, filtering is always depending on the source, there isn't a standardised filtering chain ( though admitdly, I do SS alot,heh :P ). Ofcourse, if you think you can mask a layer and brute filter it, then more power to you ;)

[guada 2]

Hello Kagura (english)

My translation lacks punch that is true.
The report with this topic rests on the filters.
As you gave your opinion on suitable filters to anime encoding, I proposed you the filter guada2.

It is an invention of my part. A way to express you my recognition and to tell you by the same opportunity that it is April first, 2005

I apologize for the translation, I am very bad.
"Nevertheless i didn't think that April first was disappointing". sorry


Salut Kagura (french)

Ma traduction manque de punch c'est vrai.
Le rapport avec ce topic repose sur les filtres.
Comme tu donnais ton avis sur des filtres appropriés aux encodage de dessin animés, je t'ai proposé le filtre guada2.

C'est une invention de ma part. Une manière de t'exprimer ma reconnaissance et te dire par la même occasion que c'est le premier avril 2005.

Je m'excuse pour la traduction, je suis très mauvais.
"Néanmoins je ne pensait pas que le premier avril était aussi décevant".
Désolé

[Kagura]

I have compared all the denoise generation of filters. Nothing atm beats 2dclean in visual quality. By brute force filtering I do not mean 2dclean at high settings. Rather, 2dclean is a "dumb" filter, which means it does not try to distinguish between noise and detail. Other filters do, which is what makes them fail. Humans are smarter than machines, and no adaptive detection algorithm can preserve as much detail as per scene techniques.

The variety of tricks involving 2dclean include:

upsizing
masking
iteration
yv12lutxy per pixel compare with before and after filtering (basically a smarter version of masking)

Hehe. I guess I'm kinda known as a "detail whore," so any filter that destroys unacceptable amounts of detail is a no-no for me. That is why 2dclean is imo, the best cleaning filter =D.

EDIT: My "filter chain" (which changes between shows and episodes of course) takes around 8 hours first pass and 7 hours second pass, and those two passes are before compression. 2 cleaning passes to a huffy intermediate file. That is pretty much how slow I make my 2dclean.

[Kenshin5]

Hi guys posting after awhile, i haven't finished this Love Hina project still and im haveing some problems with interlacing at the end of the video at the credits, when they scroll up there is some interlace but there is no interlace at the rest of the episode tough. Any suggestions on how to fix this problem? Filters or something...

[helix]

I have compared all the denoise generation of filters. Nothing atm beats 2dclean in visual quality. By brute force filtering I do not mean 2dclean at high settings. Rather, 2dclean is a "dumb" filter, which means it does not try to distinguish between noise and detail. Other filters do, which is what makes them fail. Humans are smarter than machines, and no adaptive detection algorithm can preserve as much detail as per scene techniques.

The variety of tricks involving 2dclean include:

upsizing
masking
iteration
yv12lutxy per pixel compare with before and after filtering (basically a smarter version of masking)

Hehe. I guess I'm kinda known as a "detail whore," so any filter that destroys unacceptable amounts of detail is a no-no for me. That is why 2dclean is imo, the best cleaning filter =D.

EDIT: My "filter chain" (which changes between shows and episodes of course) takes around 8 hours first pass and 7 hours second pass, and those two passes are before compression. 2 cleaning passes to a huffy intermediate file. That is pretty much how slow I make my 2dclean.


Could you explain in a bit more detain that whole encoding process of yours? I usually just load the script right up into Vdub and compress it with two passes. Are things better when you first run the filters through, then encode? It seems like it may be better, but would take longer and use more HD space.

[Leak]

Could you explain in a bit more detain that whole encoding process of yours? I usually just load the script right up into Vdub and compress it with two passes. Are things better when you first run the filters through, then encode? I seems like it may be better, but would take longer and use more HD space.

Well, if your filter chain is really slow then doing the filtering to HuffYUV or similar lossless format first and feeding that file for both XviD passes will actually speed up encoding. Also, if you need to re-compress the file with different XviD settings it saves even more time... :)

But you're of course right, it takes up more space on your harddisc, but these are just temporary files, after all...

[yaz]

@leak

sorry for being quite ot but ... a can't reach your site on no-ip anymore. where do u keep your avisynth stuffs ?
(in an 'accident' i lost all of my avs thingies and now i'm about to (re)collect them)


sorry again and thx
y

[Leak]

@leak

sorry for being quite ot but ... a can't reach your site on no-ip anymore. where do u keep your avisynth stuffs ?
(in an 'accident' i lost all of my avs thingies and now i'm about to (re)collect them)


sorry again and thx
y

Works fine here; the no-ip.org address is just a redirect to http://140.78.145.103:8888/~kp/AviSynth/ ...

[helix]

Well, if your filter chain is really slow then doing the filtering to HuffYUV or similar lossless format first and feeding that file for both XviD passes will actually speed up encoding. Also, if you need to re-compress the file with different XviD settings it saves even more time... :)

But you're of course right, it takes up more space on your harddisc, but these are just temporary files, after all...


How exactly does one process the the loaded script to HuffYUV using Vdub, a search didn't yield much. I feel rather foolish asking though since I suspect this a common process. Well besides that, when the HuffYUV file is made, is that loaded directly into Vdub and then twopassed just like I would a regular script? When my 160Gb harddrive comes in a few days I'll try this out and see how it goes.

[Leak]

How exactly does one process the the loaded script to HuffYUV using Vdub, a search didn't yield much. I feel rather foolish asking though since I suspect this a common process. Well besides that, when the HuffYUV file is made, is that loaded directly into Vdub and then twopassed just like I would a regular script? When my 160Gb harddrive comes in a few days I'll try this out and see how it goes.

Well, I'm using ffdshow to encode to HuffYUV, since it also supports YV12 which produces smaller files.

Just load your AviSynth script, choose ffdshow as the codec for compression, hit "Configure", select "HuffYUV" as your encoder, choose colorspace (probably YV12, as that's what XviD uses), predictor (speed vs. compression ratio trade-off) and adaptive huffman tables (better compression, less speed) to your liking, OK out of all dialogs and then save the AVI file in VD.

Then load the AVI file you just produced (setting up ffdshow's VfW interface to decode HuffYUV) and use it as the source for the first and second XviD pass.

That's all...

np: Luke Vibert - Orch Garage (Lover's Acid)

[helix]

Ah, I see now. Thank you.