Bug in cartoon mode?

[Leak]

I'm just reencoding the first episode of Ghost In The Shell: Stand-Alone Complex for the n-th time (while zeroing in on an AVISynth filter combination that helps clean up the edge-enhancing mess that it is) and I got a garbled image sequence at about 2:10 into the file.

At first I thought it was caused by the custom MPEG matrix I was using, so I encoded the 20 seconds surrounding that part again using H.263, but the artifacts stayed:

http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_Cartoon_275.png

Next I tried disabling cartoon mode, and that fixed the problem for good:

http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_NoCartoon_275.png

I'm using Celtic Druid's current CVS head build from the 10th of January. I was doing a full quality first pass, i.e. a quant 2 encode, and the following options:

Unrestricted profile, H.263, adaptive quant, quarter pel (I think), default BVOPS, no packed bitstream, default PAR, chroma optimizer, MSP 6, VHQ 4, VHQ for B-frames, chroma motion, no turbo, max I-frame interval of 250, quants 2-31 each, trellis.

The two resulting 20 seconds of video can be found here:

http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_Cartoon.avi
http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_NoCartoon.avi

Here's the installer for Celtic Druid's build in case you need it - he's overwriting it with each compile:

http://140.78.145.103:8888/~kp/Stuff/XviD.cvs.head.exe

If you need any other info, just ask.

In the meantime, I guess I'll just go back to Koepi's older CVS head compile for the moment and don't use cartoon mode... :)

np: Zorn - Rock The Boat (The City's Collapsing (But Not Tonight))

[unmei]

I think it comes from the combination VHQ with B-frames plus cartoon mode. IIRC i stopped using cartoon mode for this very problem when VHQ for b-rames became available. I use some koepi's bild, but the symptom is the same.

[Leak]

Originally posted by unmei
I think it comes from the combination VHQ with B-frames plus cartoon mode. IIRC i stopped using cartoon mode for this very problem when VHQ for b-rames became available. I use some koepi's bild, but the symptom is the same.

I'm afraid that's not it:

http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_NoBFrameVHQ_275.png

http://140.78.145.103:8888/~kp/Stuff/GITS_SAC_01_NoBFrameVHQ.avi

Artifacts (though differently placed ones) are still there.

I wonder what sysKin will say when he finds the mess I made... :D

np: U2 - Original Of The Species (How To Dismantle An Atomic Bomb)

[CruNcher]

A known threshold problem i reported that a while back but looks like no solution was found yet, besides not everyone uses Cartoon Mode thats also the reason i deactivated it in my EDP Build.
You have only 2 choices @ this time to workaround it you can leave b-frame off cartoon on or the other way b-frame on cartoon off both on will couse this problem.
In Real footage it's much more visible and more anoying then in this sample here also look on frame 340 on her leg not really visible for non trained eyes :P

[Leak]

Originally posted by CruNcher
A known threshold problem i reported that a while back but looks like no solution was found yet, besides not everyone uses Cartoon Mode thats also the reason i deactivated it in my EDP Build.
You have only 2 choices @ this time to workaround it you can leave

That's strange; it only happened to me with Celtic Druid's newest build, but not the ones before them. Then again, I've just changed my filter chain to hopefully get rid of all the ringing along the lines, so maybe cartoon mode didn't kick in that much before... :confused:

Well, if you already reported this problem I guess I should have searched harder for it, but if it was just the one line in your "Extra Detail Profile" post maybe it went unnoticed by the devs? (At least I know I missed it... :D)

b-frame off cartoon on or the other way b-frame on cartoon off both on will couse this problem.

Yeah, I'm just disabling cartoon mode for now as my files are all coming out quite a bit smaller than needed to fit 13 episodes on a DVD, so slightly bigger files aren't a problem.

I really don't want to disable B-frames... :)

np: Zorn - Jacuzzi Suite (The City's Collapsing (But Not Tonight))