Anime Encoding ... ummm HELP!

[Xndo]

I've looked everywhere, tried to read alot of stuff, but still there is no profound way to get good quality anime encodes. I've even searched all the fourms, hardly anything is mentioned about anime.

I've been trying to use gknot .28 to .28.2beta and yet I get errors with avs crashing with the "access violation" error, or with the lovely "-2" error. I've done a few anime flawlessly with the first release of .28 now it seems nothing wants to work.


I don't know how to setup the avs scripts that's for sure, and would like some pointers ( if anyone has any ) on possible scripts or method that might be used for encodeing anime ( that's worth looking at).

i'm useing the Divx 5.05 codec and the .28.2beta rippack but still haveing not very good results.

[jeremymacmull]

you did not search very well :P

I would suggest u look in the divx 3/4 forum there is a very very long thread (250+ replies) (search for anime or filters)

about good settings for anime scripts or manually or filters and all

while these are for divx 3 they work fine with other codecs not with gknot so much most need u to manually encode them

JEREMY

[TelemachusMH]

Well ... sorry to tell you this, but there is no 'profound' way to encoding anime. You use the same process as you do with regular movies. The only real difference is what filters, and how strong you set them.
How noice is your anime picture? You can find tons of threads on filter settings for anime videos, and you'll need to find out which fit you're needs.

TelemachusMH

[MasterYoshidino]

do you research, some DVD are really clean while some (darnit hentai dvd's) are super crappy.
But if you did a search for these words you will learn, o n00bie one

Spatial Smoothing, Temporal Smoothing, and SmartSmoothHiQ
:devil:

some like "Edge Enhancement" as well (if you use _2DCleanYUY2 / _2DCleanYV12, you may want to look into this, as SSHiQ already can emphasize edges if used correctly)

[Xndo]

so over in the divx3/4 section there is a post over which filters might be good?? hummm okay well their is noise in like 90% of all anime, i was just hopeing someone had some filter choices that i could try to read up one, but ya I'll go check out that one form ( even though the search didn't yeild any results for that thread).


thanks everyone

[Xndo]

okay i read that thread in the Divx3/4 forum, and all i have to say is "MY GOD" lot of bitching went on.

Settings i know will change from anime to anime, but what I'm really looking for are filters that would yeild the better results. I'm not very good with avs, and all it seems to want to do is error out consitantly so it would have to be that the settings I'm useing are either conflicting ( i'm useing gknots .28.2beta ) but i found even half the filters that avs has isn't even compataible with ther current version. or 2.08 will crap that it got a access violation.


so far I've seen 2d cleaner which was common among everyone. I'm sure there is another one better than that right now.

MasterYoshidino, Posted that
Spatial Smoothing, Temporal Smoothing, and SmartSmoothHiQ

Looked to be a good combination.

What I'm also finding is when to add the logo filter, if you ad it too earlier the logo looks like shit and you want to slap you computer a million times.

I'll try useing them and see if i can get any better results

[manono]

Hi-

sh0dan has very recently developed the MipSmooth filter mainly for anime. It's sort of a combination of C3D and SSHiQ. I've just started using it with good results. Real slow, though. You can find it in this thread (http://forum.doom9.org/showthread.php?s=&threadid=53548).

[Xndo]

is there away to put in a logo from the avs script ?

also here is my avs script maybe someone can tell me what I'm doing wrong
# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\Convolution3d.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\FluxSmooth.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\TomsMoComp.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\VSFilter.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
#
# SOURCE
mpeg2source("C:\DVD\Projects\blah.d2v")
# IVTC
Telecide(guide=1).Decimate(5)
# CROPPING
crop(4,2,706,478)
# RESIZING
BicubicResize(640,480,0,0.5)
# 3) heavy noise
#Temporalsoften(4,8,8,mode=2,scenechange=10)
#Convolution3d("animeHQ")
#FluxSmooth(10,15)
mipsmooth(downsizer="lanczos",upsizer="bilinear", scalefactor=2.0, method = "strong")
"I really didn't know how to insert that mipsmooth filter"

[TelemachusMH]

Yes there is. You can do it with the layer command. I am not sure exactly how to do it right now ... but search for it. I know it's there (I've written it before).

TelemachusMH

PS Can't help you with mips, I haven't tried testing it yet.

[manono]

Hi-

"I really didn't know how to insert that mipsmooth filter"

It might help if you load the Plugin first. :)

[Xndo]

I really haven't had much time to play around with some of these settings, but I seems I'm havieng trouble keeping a "crisp" image without haveing alot of noise in the video. so i have a few questions maybe someone can help answer them.


1. Do you have to reconstruct the fields manualy or is there another method.

2. Would useing TMPEGnc be better than just useing vdub ?

3. Filter placement, should resizeing go first before all of the other filters or last ?

[Xndo]

here is a thought, would doing a 1-pass useing the huffyuv codec work better for anime ??? then use like divx

[Dali Lama]

Xndo,

Using HuffYUV as an intermediate is only for speed gain in two-pass encoding. It does not yield higher quality in conversion to Divx. Actually, if you convert to YUV from YV12 and otherwise wouldn't then it *could* lower quality.

Oh and always listen to manono's advice ;)

Good Luck,

Dali

[Xndo]

well after I made that mpismooth post I found out I didn't load the stupid plugin so all i can say to that is DOH!!!!!! /action/ Xndo Slaps Xndo /action/ hehe.. anyways, i was also wondering if i should resize than use a filter like deinterlace, or is it visa versa ?

[Dali Lama]

@Xndo,

Always deinterlace or IVTC (or both) a source before resize. Try to read some doom9/avisynth.org guides if you have time, it will help you a lot.

bye,

Dali

[Xndo]

also another stupid question, is .... is wharpshape really a common pratice among anime enocdeing ?

[Dali Lama]

Yes, Warpsharp is used by a lot of poeple in anime encoding. I don't use it now, but just try it out yourself and see what it does. It can thin lines in anime. I also talked about it in "an attempt at cleaning anime"

Bye,

Dali

[Xndo]

i read the post you made and it said you used the filter 3 times to try to clean the source. so I'm going to try to use that and see if i can get the same result. the fact of getting the same quality in a .avi file as i have seen in a .ogm file, has me wondering why have i been doing it wrong.. hehe.

anyways I've tried to edit the .avs script that Gordian Knot made, maybe it would help, I'm going to try to setup tests to see if it works out well.. here's the script.

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\Convolution3d.dll")

mpeg2source("C:\DVD\Projects\WORLD_TOURNAMENT_2_VTS_01_PGC7\WORLD_TOURNAMENT_2.d2v")
Telecide(guide=1).Decimate(5)
crop(12,2,694,476)
Convolution3d("animeHQ")
BicubicResize(640,480,0.333,0.333)

maybe this would help out a little.

let me know if I need to change something

[crusty]

I'm not very good with avs,
It's not that hard really unless you do a lot of difficult stuff like conditional filtering etc.

If you're starting with avisynth, be sure to use the latest, it's avisynth 2.5, and it's out of beta right now so it's considered stable.

Gordian Knot will do most of the difficult avs stuff for you and you can then save the avs and edit it further if you want.
It will do the trimming, loading of plugins, cropping, and resizing automatically for you. And you can then add any stuff you want.
Try www.avisynth.org ....it has the description of most if not all of the built-in avisynth commands. Bookmark it, as you're gonna be there from time to time.
is there away to put in a logo from the avs script ?
Yes but please don't do it in the entire movie...I hate that. It's annoying. Only put it in the opening or end credits, if you want to take some credit for your rips, that's the place the put it.

1. Do you have to reconstruct the fields manualy or is there another method.
Decomb is a very intelligent filter and will do a lot for you. But some source simply is f*cked up and you will have to do some tweaking.
Overall, Anime is not the best place to start with encoding, it's a bit like throwing yourself off the deep end. :)
Decomb has an extensive readme, give it a try.

Filter placement, should resizeing go first before all of the other filters or last ?

Cut&paste from other post of mine:
In general, it just hurts speed if you put noise filters before resizing, because they have more pixels to work with then. But it could help quality if you do.

would doing a 1-pass useing the huffyuv codec work better for anime ???
Answer from Dali Lama:
Using HuffYUV as an intermediate is only for speed gain in two-pass encoding.

That's one thing, but it will also speed up your testing of different filter settings, because Huffyuv is generally faster than Mpeg4(divx, Xvid,wmv) encoding. Find a difficult-to-filter part in your source and try different filters/filtersettings and see.
There is no spoon...uhh, one answer to your problem. :D


And to end all this: This kind of conversation should be in the avisynth usage forum, since you really haven't asked anything about DivX 5. Right?

[manono]

Hi-

Good post, crusty. Thanks for helping him out.

This kind of conversation should be in the avisynth usage forum

Yes, it probably should, except sometimes the guys in those advanced forums can get upset with beginner type discussions. I, for one, don't mind having it down here. The other rookies might learn something, too.

Anime is not the best place to start with encoding, it's a bit like throwing yourself off the deep end.

I agree with that one also. Working with anime when you're not real sure of what you're doing can get discouraging. I think it's a good idea to do a few easy Force Film Hollywood type movies before diving into encoding anime (to stick with the swimming pool analogy). But who are we to tell Xndo what to encode?

[Xndo]

it started with the entent of being a codec releated topic but then I started to notice that alot had to do with the script. In any case I did a test last night useing this with gknot

Resizing : Selected Output Resolution
Resize filter : Soft Bicubic
Noise Filter : Heavy ( had to edit the avs script to change it to "animeHQ" setting )
File Operations: i did a Field Deinterlace


Now for the Codec i used this
1st pass settings:
the HDTV Profile, Encodeing bitrate I let it choose what it should be.
in the general Parameters, I left everything the same but Soucr interlace, instead of leaveing it at "Ecnode as progressive" like most others tell you to do, i change it to Deinterlace all frames

on the profile i checked "Use Bidirectional Encodeing" it seems that its helped with this encodeing

I pretty much left all of it the same ont the nth pass, but I think it changed it from "Deinterlace all frames" to "Encode Progressive".

it still looked a little noisey but no were near before, any suggestions ?

ALso: The guides on the avs's website are not even done yet. so I can't really get alot of information on what I need. unless I'm looking wrong

[bkam]

If you're doing a fielddeinterlace I don't think there's any point to having the internal codec deinterlace for you as well (the script is the intermediary between the source files and the codec... since you deinterlaced in the script, you're having the codec deinterlace what you already did). I'm not sure how smart the internal codec deinterlacer is, it might not do anything, but it definitely slows down the encode.

C3d(animeHQ) is really not that strong of a filter for a very noisy source, if I'm not mistaken. You might try higher settings for C3d, or maybe try a different one. I'm not too familiar with anime but for cartoon stuff, using Msmooth usually can take out a ton of noise without losing almost any detail. Set the threshold pretty low and you can smooth a lot, although be warned that too low of a threshold can make stuff crawly. FluxSmooth, Undot, Cnr2 are some other filters I've used on cartoon. Oh, and you might fancy Dup. Hope that's (at least a little?) helpful...

[Xndo]

I'll try to give it a shot and see what happends.. thanks

[MasterYoshidino]

Faster (better?) smoother is _2DCleanYUY2, uses YV12 or YUY2. msmooth somehow does not fit my cup of tea in avisynth 2.5.
Undot + sharpen filters = no no
Tends to increase the artifacts when an edge has high contrast attached too close to a black line.

oops: forgot to mention, stay away from hybrid series dvds until you get the hang of anime movies like Escaflowne or Spirited Away.
It gets quite tedious figuring what IVTC process will best match frames w/o blended orphan fields/frames.

[Xndo]

with the new gknot the .28.3 I haven't downloaded the other one yet, thought by the time some people read this post I will have downloaded it. I've decided to use these settings

Temporalsoften(4,8,8,mode=2,scenechange=10)
Convolution3d("animeBQ")

which the over all look is getting better, but when i try to lets say add a logo after i encode it at like a 8mb bitrateing, that way i can bring it back down, I'm getting "image source is invalid"-2, which is kinda pissing me off. I'm also finding some problems with the codec putting in alot of artifacts that shouldn't be there. I guess its either that or something wrong with my computer... know knows =/

maybe someone can give me a insight to where I'm messing up again.

[Xndo]

okay from the help I've gotten from everyone before, and i mean it was a really big help. I've ran across another problem when trying to smooth out this one anime ( Dragonball GT ) me and a friend have been working on smoothing out the ( REALLY BAD SOURCE ) and have come up with 1 option at this time,

convolution3D "animebq" and
MSmooth(strength=9,highq=true) this has givin us a really good smoothing without the noise it was giving us on other settings, now the problem I'm running into is that this drops the fps down to 1, which is resulting in a 14 hour 2pass, which I'm really not happy about. I'm wondering if useing a 2dcleaner instead of the msmooth and useing convolution3D with preset "animeBQ" would give me the same output, but without haveing to wait 14 hours inorder to encode.

Any help would be great...... thanks