Tough Cartoon encode

[FUAll]

Hi all

Im trying to encode a cartoon movie - The Beauty and the Beast - and im having lots of trouble in reaching a good encode.

I'm trying to get it on a 1 CD encode with 2 ogg soundtracks what gives me about 600.0052 kB of video to deal with.
I've done some calculus and i've reached the limit of 937 kBits/s video bitrate.

I've also read somewhere that this one movie was very little compressible and i've reached to the following values:

Compressibility teste values: 44.3% of 0.556
with a resolution of 528x288 -> 0.246

I've also read almost every thread regarding cartoon and anime settings, filtering and problems with animation encoding.

My source is a PAL DVD although ist say interlacer in DVD2Avi it seems progressive to me as i did nothing else when i encodede it unless cropping and resizing it.

My Avs script:

SetWorkingDir("C:\SoftWare\DivX\GKnot0.28.5\AviSynth 2.5\plugins\")
LoadPlugin("mpeg2dec3.dll")
mpeg2source("E:\DVD\Projects\TIPO.d2v")
crop(10,12,702,552)
BicubicResize(528,288,0,0.5)

my Xvid settings are all default except:

Montion sp: 6 - Ultra High
Quantization type: H.263
VHQ mode: 4 - Wide Search

Use Chroma Motion: On
DX50 B-VOP: on

Debug:
Chrome Optimizer: On
Trellis: On

I've this Snaps of original Media and encoded:

Original 1 (http://homepage.esoterica.pt/~nx4qcu/Orig2.png)
Original 2 (http://homepage.esoterica.pt/~nx4qcu/Orig.png)
Encoded 1 (http://homepage.esoterica.pt/~nx4qcu/Comp2.png)
Encoded 2 (http://homepage.esoterica.pt/~nx4qcu/Comp.png)

If someone could give me some pointer of how i could improve the encode i'd apreciate it a lot.

Thanks
Fuall

[OBcecado]

Hi FUAll, be welcome to the forum.
Have you read anything about filtering ?
Have a look at thread (http://forum.doom9.org/showthread.php?s=&threadid=58961) I think you'll find it helpful.
Once it talks about filters, you can also go to avisynth section of this forum and search for it, you'll learn that filtering can help a lot in certain situations.


Hope it helps.




OT: És pt ? tou so a perguntar por causa do teu alojamento.

[Sirber]

You could try RV9 and keep the same rez as the DVD. 512x is pretty ugly... Also, use DeComb instead of DVD2AVI internal filters.

[FUAll]

Hi OBcecado.

As i said before i read (i think) all the threads that there are about Cartoons and Anime encoding, and this includes filtering.

I've also tryied filtering and i got some good results, but the thing i'm trying to reach is some "good" cartoon XviD settigs before i try to apply any filtering.

To much filtering damages the picture and i think that if i can get one good Encoder settings before i applçy any filter i'll certainly obtain better results than filtering to much before the encode itself.

As this is not a digital Cartoon it is so much hard of getting rid of the noise and artifacts that this encode makes.

Thaks for the sugestion anyway.
FUAll

OT: Eu sou Pt ;)

[OBcecado]

Hi, could you cut a vob sample so we could give it a look ?
10 mbs would do it, about your XviD settings i think it's ok, the only thing you could do is encode credits @ other lower quant, appart from that you could start thinking about filtering.



Best regards.

[FUAll]

Originally posted by Sirber
You could try RV9 and keep the same rez as the DVD. 512x is pretty ugly... Also, use DeComb instead of DVD2AVI internal filters.

Thanks for the suggestion Sirber.
I'm not interested in changing Codecs. I used to make Divx encodes but now i'm trying to change to XviD. I thinhk that with all the possibilities that it gives u it will become any time soon the most reliable codec to use in any kind and type of source(this is my oppinion). I've seen lots of animation encodes made with XviD and they looked awsome to me with the kind of quality obtained with so much compression rates.

Like i said, i didnt use any filtering besides the cropping and resizing so DVD2avi will give no prob to me.

FUAll

[FUAll]

@OBcecado

I've cut a 4 Mb piece of it if u want to check it out.

Vobcut (http://homepage.esoterica.pt/~nx4qcu/Cut.vob)

Thx
FUAll

[OBcecado]

Already downloaded the sample vob, will to look @ it asap =)




Greetz.






P.s. Why don't you show up @ ptnet ? it's easier to talk there.

[Joe Fenton]

You didn't say in your settings, but are you using b-frames? B-frames increase the compression quite a bit. That means for the same number of bits, you can have a better looking video. I find that the difference in anime encodes is tremendous. Most folks use b-frames settings something like 2/150/75.

[FUAll]

@Joe Fenton

I already used B-frames 2/150/75/0 it betters the compression a bit but the ringin is still there.

[OBcecado]

Hi, as i've talked a bit of filtering would be no bad.

encoded sample (http://ptbox.net/~ob/bela_e_o_monstro.avi)
[code]
mpeg2source("C:\Downloads\belamonstro.d2v")
LumaFilter()
undot()
unfilter(-5,-5)
Convolution3d (preset="animeHQ")
crop(12,18,694,544)
LanczosResize(536,288)
That's a simple avisynth script, I'm no anime encoding expert so i doubt a anime encoder will think this (http://ptbox.net/~ob/bela_monstro.mkv) acceptable.

Hope it helps.

[BoNz1]

Originally posted by FUAll

I've done some calculus and i've reached the limit of 937 kBits/s video bitrate.


I'm sorry but this is one of the funniest things I have seen in quite some time. Since when do you need calculus to figure out the bitrate? In fact I don't think you need any more than grade school math to do that. Lol. Ok, I will help you and not just make fun of you. What the other person said before was exactly right, it looks a little oversharpened use unfilter but I wouldn't use c3d or any denoise since it is really clean, just soften the edges a bit. As for XviD settings use 2,150,100,0 for b-frames, h.263 quants, chroma motion, vhq1-4 (depending on how long you want to wait), and trellis quantization. Basically the same as yours except for b-frames and turn off chroma optimizer personnally I think it does more harm than good.

[Sirber]

Originally posted by OBcecado
Hi, as i've talked a bit of filtering would be no bad.

encoded sample (http://ptbox.net/~ob/bela_e_o_monstro.avi)
[code]
mpeg2source("C:\Downloads\belamonstro.d2v")
LumaFilter()
undot()
unfilter(-5,-5)
Convolution3d (preset="animeHQ")
crop(12,18,694,544)
LanczosResize(536,288)
That's a simple avisynth script, I'm no anime encoding expert so i doubt a anime encoder will think this (http://ptbox.net/~ob/bela_monstro.mkv) acceptable.

Hope it helps. Everything is fine, except the Lanczos resize. The rez is far too small. Who will enjoy watching this after?I could produce it in something like 960x544, same quality (or better) with RV9. :mad:

[sh0dan]

@Sirber: How good an idea do you really think it is to encode source material in a resolution larger than your source?

Why don't you stick to the topic instead of shouting RV9 all the time?

@FUAll:

Try using:

MPEG2Source("D:\video\avisynth_tests\cartoon.d2v",cpu2="ooxxxx", ipp=false)

This will enable postprocessing on chroma (deblocking) and dering both luma and chroma. I got a 10% smaller file (fixed quant 2) and a more steady picture, and most of the ringing of the original is gone.

[Sirber]

I'm not encoding larger, but I'm necoding the real DVD size, then resize on decoder side. Never heard of Anamorphic? Also, I suggest RV9 to people that have problems like this guy in this very thread, where his current codec fail to encode it proprely. :)

[FUAll]

@Sh0dan

I'll surelly try ur suggestions, but why using a fixed quant 2, i thought that using two pass with variable quant would get a better result. If not just tell me why.

Which resize will be better to this bitrate?
Lanczos or Bicubic, i'm using bicubic cause for what i read it is better for bitrate < 900.

Thx for ur advice.
FUAll

[OBcecado]

@Sirber
So what resizer is the best one to use at these situations ? bilinear resize ?

[Sirber]

This situation with XviD or RV9? :confused:

[OBcecado]

With XviD, I don't encode rv9 still no curiosity :D

[Sirber]

hum... I don't use XviD anymore, but I would use Biculin :)

[OBcecado]

@Sirber I can't seem to find that resize filter can you point it for me ? I did search the forum with Biculin but didn't find matches except for this thread, or is it built in avisynth ?




Thanks in advance.

[yingx2]

I would apply very heavy 2d filtering and warpsharpening in your case.

take a look at this(852kbps@640x352)
http://myweb.hinet.net/home4/yingx2/test.avi

it's for my taste anyway:)

[Assault]

@ OBcecado

I think it was a typo. Sirber perhaps meant Bicublin Resize. It is an avisynth plugin by MarcFD. Search the forum for more information.

Assault

[OBcecado]

Thanks for your help Assault.
Will try that resize filter asap.



Greetz.

[FUAll]

@yingx2

Thanks for the info. I saw the sample and i would like to try something like that, perhaps a little bit softer.

Could u please state what filter u used for 2d filtering and what were the senting u used on both.

Thanks in advance for all ur help.
FUAll

[yingx2]

@FUAll

2D cleaner
threshold: 10
radii: x=y=5

smartsmoother HQ
diameter:7
everything else: default

Hope this helps:)

[FUAll]

@yingx2

Thanks for ur help
Sure gonna try with ur help to get something i like.

FUAll

[Chainmax]

yingx2, that clip looked awesome. Could you please post clips with:

a)pixiedust(limit=3) + MipSmooth (animeHQ) instead of 2D cleaner and smoothhiq?

b)mfToon()?

c) both a) and b) together?

[Chainmax]

Well?

[mf]

Originally posted by Chainmax
b)mfToon()?
Disney movies have colored lines, so darkening them would look bad.
Anyway, what's wrong with this: http://mf.creations.nl/beautybeastxvid2.avi ? Size is spot-on and quality is ok imho.

EDIT: I forgot to do /8 for bit->byte conversion, so it wasn't undersized. (I calced 5MB for 6 seconds :D) Uploaded a second pass that is 0% undersized.

[Sirber]

Q is nice...

[edit]

What about this? http://sirber.no-ip.com/~sirber/beast.rmvb :D

Diagnostic: File Size: 0KB
Diagnostic: Total Duration: 06.200
Diagnostic: Video Track
Diagnostic: Dimensions: 704 x 552
Diagnostic: Frame Rate: 25.000 FPS
Diagnostic: Format: YV12
Diagnostic: Duration: 06.200
Diagnostic: --------- End Input File Properties
Diagnostic: Using movie_video Audience
Diagnostic: Using video codec: rv9 (930000 bps)# load plugins
loadplugin("C:\Documents and Settings\sirber\Bureau\RMFactory\bin\mpeg2dec3.dll")
loadplugin("C:\Documents and Settings\sirber\Bureau\RMFactory\bin\Convolution3DYV12.dll")
#
#load source video
video=Mpeg2Source("C:\Documents and Settings\sirber\Bureau\Cut.d2v", idct=2)
#
#crop source file
video=video.Crop(10,12,704,552)
#
#Convolution3D Filter
video=video.Convolution3d(preset="animeHQ")
#
#Anamorphic Resize done by Producer !!
#
return(video)

[FUAll]

@mf

I really liked ur sample although i think that with that resolution it's gonna be hard to mantain the quality at 1 CD count for ~600KB of video, but i'm sure gonna try it.

If u can please tell me what u used for avs file, what were the filters u used and respective values.

Thx for ur support
FUAll

[FUAll]

@Sirber

Goog quality in ur sample and u mantained the original size but i'm not prepared to use other codec than XviD for the moment.

Was Convolution3D the only filter u used??

Thx for ur reply, i really appreciate all advices u guys are sending.

FUAll

[mf]

Originally posted by FUAll
@mf

I really liked ur sample although i think that with that resolution it's gonna be hard to mantain the quality at 1 CD count for ~600KB of video, but i'm sure gonna try it.

If u can please tell me what u used for avs file, what were the filters u used and respective values.

Thx for ur support
FUAll
Didn't you say your video bitrate was 937kbit/s ? I encoded that sample at that bitrate. My script was:

MPEG2Source("beautybeastcut.d2v", cpu2="ooooxx")
Deen("a3d", 4, 2, 10, 30)
Crop(12, 14, -10, -14)
Unfilter(-10, -20)
pre = last
LanczosResize(640*4, 352*4)
XSharpen(255, 255)
LanczosResize(704, 400)
aWarpSharp(depth=8.0, blurlevel=1)
LanczosResize(640, 352)

[FUAll]

Originally posted by mf
Didn't you say your video bitrate was 937kbit/s ? I encoded that sample at that bitrate. My script was:

MPEG2Source("beautybeastcut.d2v", cpu2="ooooxx")
Deen("a3d", 4, 2, 10, 30)
Crop(12, 14, -10, -14)
Unfilter(-10, -20)
pre = last
LanczosResize(640*4, 352*4)
XSharpen(255, 255)
LanczosResize(704, 400)
aWarpSharp(depth=8.0, blurlevel=1)
LanczosResize(640, 352)

Yes that is the bitrate. I meant to say 600Mb video total size, sorry for that misshap.

i just don´t undestand the double resize and what ist the function of:
pre = last

If u can bear the explanation i would like it very much...
I'm just trying to understand it as much as i can.

Thx for ur time
FUAll

[mf]

Originally posted by FUAll
Yes that is the bitrate. I meant to say 600Mb video total size, sorry for that misshap.

i just don´t undestand the double resize and what ist the function of:
pre = last

If u can bear the explanation i would like it very much...
I'm just trying to understand it as much as i can.

Thx for ur time
FUAll
The use of the double resize: http://mf.onthanet.com/supersampled_xsharpen
pre = last means that the last clip result (in this case what comes out of Unfilter) is now called "pre" so I can call for it at a later time. In this case I don't use it at all, so you can remove it. While I was working on the script, I resized back to the width and height of the cropped image, by using LanczosResize(pre.width, pre.height), but that was only temporary.

[FUAll]

@mf

Thx for ur support
Very much appreciated.

FUAll

[yingx2]

Originally posted by Chainmax
yingx2, that clip looked awesome. Could you please post clips with:

a)pixiedust(limit=3) + MipSmooth (animeHQ) instead of 2D cleaner and smoothhiq?

b)mfToon()?

c) both a) and b) together?
I am not so familiar with those filters.
sorry for being unhelpful:(

@mf & Sirber
I too have some tough clips...:)

[Sirber]

@all

I used only convolution3D with the setting "animeHQ". Nothing more. It was encoded in less than 15 sec, at 930kbps. IMO that clip wasn't hard :D

[mf]

Originally posted by Sirber
@all

I used only convolution3D with the setting "animeHQ". Nothing more. It was encoded in less than 15 sec, at 930kbps. IMO that clip wasn't hard :D
Imho it wasn't either.

[Sirber]

... and didn't need heavy filtering :p

[yingx2]

Originally posted by Sirber
... and didn't need heavy filtering :p
when using rv9:D

[mf]

Originally posted by Sirber
... and didn't need heavy filtering :p
Nope, only some anti-artifact measures, like getting rid of oversharpening (only very slightly though, thanks Disney ;)), then sharpening properly :D, and getting rid of the crappy MPEG compression stuff and especially the weird color bands (crap at you, Disney :p).

[Sirber]

IIRC, we can set a flag in matroska to use anamorphic with other codecs. I've never seen a full-rez XviD clip...

[Teegedeck]

...'cause people like me don't spread their encodings over the internet.

BTW, mf, could you post screen caps 'before' and 'after' that treatment where you exhibit how well you got rid of the mentioned problems?

[Sirber]

so that means you have some :p :D. Could you cut a part and send me a URL?

[Teegedeck]

1) Yes, I do

2) No, I won't. You tryin' to pull me down into dat pit o' dangerous filesharin' filth?!? No way, maaan! :D ;)

AFAIK ChristianHJW once made a full-res XviD-MKV for demonstration purposes. Don't know if it's still up; you can search for it.

Seriously, I only have one DVD that isn't encrypted and thus would still be officially legal to even extract in my country - and that one isn't even anamorphous. But I seem to recall that there is something like legal MPEG-demo-trailers; I believe it was mentioned on Xvid.org's mailing list.

But tell me right away whether you still insist that good encodes should be done at streaming-bitrates - because in that case I won't make any effort searching for the info and looking for some free webspace (don't have any, yet).

[Sirber]

IMHO encodes must fit on a CD (except 2h30+ movies), whatever codec I use. I just hate to switch CDs... :devil:

[Teegedeck]

Well, whatever suits you; but don't expect anyone else to do that. And don't fall for the illusion that a full resolution equals a reasonable level of details at that bitrate; even if it has been encoded with the wonderously amazing codec known as Real.

Edit: These bitrates you use certainly represent the 'sweet spot' of Real and thus you're right to use them - for as long as you insist on that codec.

[Sirber]

what I ment earlier is I wanted to see a full-rez XviD clip at any bitrate someone want to do it.