Xvid questions (blurry encode/high bitrates)

[Kangiten]

Hello everyone

I am using Gordian Knot and Xvid to back up movies from my DVD collection on my hard drive. Usually, I don't have any problems with encodes; they look just fine with the default Xvid settings and matrix. Still, I've come across two problematic DVDs.

1) I have recently encoded "Ghost In The Shell 2 : Innocence" without any problem. But my encode of "Ghost In The Shell" (the first movie) is blurry with Xvid. I'm using 2-pass, after running a compressibility check, and set the bitrate accordingly to get proper bitrate/resolution ratio (the movie is in 4/3 aspect; I used 640 as width). Normally, I don't have any problems when I encode movies like this but this one is blurry. I tried sharpening filters and none of them seemed to give the crisp image I usually get.
The bitrate I think was around 1000 Kbps and was more than enough according to the compressibility check. Could a matrix help fix the issue and if so, which one? I know next to nothing about matrices, let alone how to choose the appropriate one so any guidance regarding this (or any other info as to how I could fix this 'blurry' effect) would be more than welcome.

2) My other question is about a different DVD. It seems to be encoded at a fairly high bitrate (between 7,000 and 8,000 Kbps). I edited a clip from this DVD, which I then tried to encode to Xvid with the usual GK method. The clip lasts 2 minutes and 13 seconds and the VOB file is about 120 Mb. Now, the compressibility check returned a really high value, which means that I have to use an insanely high bitrate with Xvid to obtain a correct result (around 4500 at the minimum). The resulting file is very big, which sort of defeats the purpose of compression. It's really the first time I've come across source material like this; does it mean it's, in a way, impossible to compress more than it is on the DVD? Am I missing something crucial?

Again, any help is appreciated. :)

[Axed]

What region are you? In PAL land im pretty sure all anime are interlaced in one way or another. Did you have to deinterlace it? I think that can cause some issues..

[Kangiten]

Yes, the GITS dvd is PAL and I had to deinterlace it. I think GITS 2 was progressive and didn't need deinterlacing. Could this cause the issue? Is there a particular deinterlacer to use to avoid this?

[setarip_old]

My other question is about a different DVD.What is the Title and Region of this DVD?

[xyloy]

The bitrate I think was around 1000 Kbps and was more than enough according to the compressibility check. Could a matrix help fix the issue and if so, which one? I know next to nothing about matrices, let alone how to choose the appropriate one so any guidance regarding this (or any other info as to how I could fix this 'blurry' effect) would be more than welcome.
Try with the CG-animation matrix(in the file "some quantization matrices.zip") given with XviD, or Sharktooth's CG Matrix. ;)

BTW, what options are you using(VHQ mode, GMC, Trellis, etc. ...) ?
One of them might be the one harming your encode.

[Kangiten]

setarip_old,

The other DVD (the one with the 'high-bitrate' issue) is called "Pour toi, public" by Frank Dubosc. It's a series of skecthes by a French funnyman. The DVD is region 2.

xyloy

Thanks for the matrix recommendations.
I use the options as provided in the "Comprehensive Xvid 1.1 guide" available on doom9.

Profile => Unrestricted
Quarter Pixel
B-Vops (with default values)
Packed bitstream
Closed GOV
Motion Search Precision => Ultra-high
VHQ Mode => 1 (Decision)
Use VHQ for B-Frames too
Use Chroma motion
Turbo
Cartoon Mode
Treillis

That's all. Thanks a lot for helping. :)

Also, is there a particular deinterlacer that's better to use? I use the "Field" mode in GKnot, could it be the issue?

[xyloy]

Profile => Unrestricted
Quarter Pixel
B-Vops (with default values)
Packed bitstream
Closed GOV
Motion Search Precision => Ultra-high
VHQ Mode => 1 (Decision)
Use VHQ for B-Frames too
Use Chroma motion
Turbo
Cartoon Mode
Treillis
I would suggest you to try:
No Quarter Pixel(or "QPEL")
Global Motion Compensation (or "GMC") enabled(not if you intend to play it on a standalone DivX player, but I guess not since you use the "unrestricted" profile@level instead of the Home Theatre PAL Profile and it's VBV, wich is quite useful for standalone DivX players)

No Cartoon Mode(that's for old noisy cartoons/animes according to the unnoficial XviD FAQ (http://ronald.vslcatena.nl/docs/xvidfaq.html))
No Packed Bitstream
2 B-VOPs(1 if you intend to play it on a standalone DivX player)
Chroma Optimizer enabled
Motion Search on "6 - Ultra High"
VHQ Mode on "4 - Wide Search"
Use VHQ for B-Frames too
Use Chroma motion
No Turbo
Trellis Quantization
and with the CG-animation Quantization Matrix(MPEG-Custom => load => select "CG-animation matrix.txt". It's given with XviD along with several other quant. matrices in the zip file named "some quantization matrices.zip").

All of that on both passes, of course.

Also, is there a particular deinterlacer that's better to use? I use the "Field" mode in GKnot, could it be the issue?
I'm not a expert of deinterlacers, but "field deinterlace" has always seemed to me to do it's job fine. ;)

PS: You have XviD 1.1.0 beta 2 I think. Well, the finale version is available (http://www.koepi.org/xvid.shtml)(Closed GOV's checkbox is no longer present in it because it's always used). ;)

[Kangiten]

Thanks Xyloy. I'll try reencoding the movie with these settings this week end. I'll keep you posted :)

[Teegedeck]

I would suggest you to try:
No Quarter Pixel(or "QPEL")
Global Motion Compensation (or "GMC") enabled
[...]
No Cartoon Mode(that's for old noisy cartoons/animes according to the unnoficial XviD FAQ (http://ronald.vslcatena.nl/docs/xvidfaq.html))
I whole-heartedly disagree on these points. Sorry. ;) Switching off QPel will make the picture seem a little less crisp while activating GMC will have hardly a visual effect at all (though it will improve quality a tiny little bit) while slowing down the encoding time radically. Well, OK; I don't really disagree on using GMC but I don't see a coherent logic in first deactivating QPel and then activating GMC.

Cartoon mode is a good idea with any drawn content and generally anything where only things that were meant to move actually move (i.e. artificial content). It's dangerous when detail is so fine that it might get confused for film-grain or if, the opposite way round, you want to keep film-grain that gives an illusion of actual detail. I'm not aware of any anime that had actually details that were so fine that cartoon mode prevented them from being recognized. As cartoon mode triggers huge filesize-savings I'd at least give it a try.

The CG matrix is only good if you want a really large file; if space is limited H.263 is much more effective, GC matrix will give ringing there.

I'm not a expert of deinterlacers, but "field deinterlace" has always seemed to me to do it's job fine. ;)Err; that was a joke, right...?

I'd say TIVTC for NTSC movies and TDeint for PAL movies and TV content.

[xyloy]

Well thanks for your advice. ^_^
What's said in the FAQ is not true(or outdated) about cartoon mode or did I misundertand it ?

I don't see a coherent logic in first deactivating QPel and then activating GMC.
See the Unofficial XviD FAQ about QPEL: Bits saved by QPEL must overcome the number of bits added by QPEL. So you never know if it helps unless you compare with and without it for each movie. It's a waste of time. And in most cases, the no-qpel result is quite better.
Whereas GMC can't be harmful.
At worst, it doesn't help much, but at least it does help, since no bits are added:
C3f. What's GMC and what is it good for?

GMC stands for Global Motion Compensation and that pretty much tells the story of what it does. If used, it will look at the whole frame and see if there is an amount of motion that all the parts of the frame have in common. It will then take this amount of motion and put it in a single value. The parts of the frame are the macroblocks, and the amount of motion is called a 'motion vector' which has both a direction and a value (as a sort of two-dimensional X,Y value) .

All the macroblocks normally have their own motion vectors, but with GMC the one motion vector that they all have in common (that's why it's called 'Global') will be compensated and put into a single motion vector. Some macroblocks' movement will be completely compensated for by the GMC vector, getting completely nullified by the compensation process. These macroblocks' motion vector will then be removed, as it is the same and is only extra information. The possible benefit is that you can remove many or all the motion vectors of the macroblocks (or even the blocks themselves if there is no altered texture information) in a frame by a single value, thereby making it much smaller.
Note however that this is for one-warppoint GMC. With Multiple warppoints the process is much more complex, but the principle is the same.

I know both options are for motion compensation. But the aim of video coding in two passes (on a limited space or bandwith) is to use available space as "wisely" as possible, isn't it ?

Err; that was a joke, right...?

I'd say TIVTC for NTSC movies and TDeint for PAL movies and TV content.
this option is it that bad ? I used it for a lot of PAL interlaced(according to DGIndex during preview) DVDs, and one (truly) NTSC interlaced DVD.
But only one of the PAL sources was truly interlaced I think, the others looked progressive. That's weird.
Maybe that's why the results were good, even if I always do a visual quality check(no PSNR stuff..) for my encodes so I think I would have noticed some problem for the two others DVDs that were truly interlaced.

The CG matrix is only good if you want a really large file; if space is limited H.263 is much more effective, GC matrix will give ringing there.
I thought that ASP's default quant. matrices were not for CG/anime/cartoon content, whereas the JVT matrix of H.264/AVC is very good for almost every content ?
So what about Sharktooth CG matrix ? Could it be used for a classic dvd-rip like this one(let's say 700 Mb for 90 minutes; 25 fps; 16:9 for example) of both cg and anime/cartoon content ?

[Teegedeck]

Well thanks for your advice. ^_^
What's said in the FAQ is not true(or outdated) about cartoon mode or did I misundertand it ?The FAQ doesn't say 'cartoon mode is for grubby old American cartoons'; the FAQ is just cautious enough to say, cartoon mode might not be optimal for high quality anime and advises users to see and test for themselves whether it helps. IMHO this cautiousness of the FAQ stemmed largely from the fact that anime encoding people at that time had a tendency to try and lynch people who would recommend cartoon mode because their beloved anime is oh so different from American cartoons. Where it isn't; really. It's still only just cartoon, only drawn in a differenct style. (Don't get me wrong, I adored Japanese animation at a time when you had to order LDs from Japan in order to get some.)

On the other hand I would not recommend to encode cartoons of the 30s and 40s (American as well as Japanese) with cartoon mode, because those contain lots of fine gradients and a lot of film-grain and 'dirt' that I wouldn't want to remove before encoding because I consider that to be part of those movies' charm.
See the Unofficial XviD FAQ about QPEL: Bits saved by QPEL must overcome the number of bits added by QPEL. So you never know if it helps unless you compare with and without it for each movie. It's a waste of time.Yes, when I recently compared QPel vs. no QPel, I got a staggering difference of below 1 MB on a 1CD encode. The question is not so much the additional space QPel takes up but diminishing returns. At strong compression, there's not much QPel can help. But for any 2CD encode I'd strongly recommend it. It's a visual difference simliar to low vs. high DC precision in MPEG-2.
And in most cases, the no-qpel result is quite better.For natural video I would protest. But for anime that might be true. On the other hand, the thread starter wanted more sharpness and, well, that's what QPel is good at.
Whereas GMC can't be harmful.Except that it takes insanely much time. And you're against time-wasting, I take it, from what you wrote above. ;)
At worst, it doesn't help much, but at least it does help, since no bits are added:Says who? :) Any additional information costs bits to store it; the information about global transformations of the whole picture certainly doesn't come free; it's added to the usual motion-vector info and costs (minimal) space. ;) ...but it saves those bits again by adding a higher motion-prediction precision.

But that really isn't the point. As long as the visual gain is worth the additional information/size the use of more bits doesn't matter. QPel certainly improves motion-vector accuracy so much that even slightly higher quantization is worth it. For as long as quantization isn't so high that motion vectors grow very short and details aren't there to start with.

this option is it that bad ?Not really that bad but certainly not state of the art. It's just very inaccurate.
I used it for a lot of PAL interlaced(according to DGIndex during preview) DVDs, and one (truly) NTSC interlaced DVD.
But only one of the PAL sources was truly interlaced I think, the others looked progressive. That's weird.Yep, I hope you trust your eyes there and not the output of the statistics window because that ain't always telling the truth. I don't know why exactly many PAL DVDs are flagged as 'interlaced' although they really are progressive... A question of DVD player compatibility?
Maybe that's why the results were good, even if I always do a visual quality check(no PSNR stuff..) for my encodes so I think I would have noticed some problem for the two others DVDs that were truly interlaced.That's the way to do it! :)
I thought that ASP's default quant. matrices were not for CG/anime/cartoon contentQuite the opposite, most encoders swear by H.263 for anime as far as MPEG-4 ASP goes.
So what about Sharktooth CG matrix ? Could it be used for a classic dvd-rip like this one(let's say 700 Mb for 90 minutes; 25 fps; 16:9 for example) of both cg and anime/cartoon content ?Just try it; as long as there's no ringing that annoys you it's OK.

Edit: BTW, I use GMC all the time because I don't care how long it takes.

[xyloy]

Well, thank you very much. This will do quite a change in this (http://emule-france.com/forum/index.php?showtopic=61884&view=findpost&p=373414) DVD =>.AVI or .MKV tutorial(careful, it's in french :D ).

A few last questions if you don't mind, but what arguments must be used with TDeint or TIVTC ? Can I add simply "TDeint()" in an avysinth script ?

Edit: BTW, I use GMC all the time because I don't care how long it takes.
Me too. :) But since my personnal "x264's "Exhaustive" M.E. option experience"(I use MultiHex now) I'm more careful about it(time is not important as long as it results in better quality, but not if less time-hungry settings give same quality).

edit: For GMC, I meant by "no bits are added" that the number of bits saved by GMC(and it must be even more true with 3 warpoints) is more than the small add of bits(for all frames).

[Teegedeck]

It's good enough for me.:o But look up the help-files for the other options. Tritical has been very thorough, thankfully enough.

[Mug Funky]

is this the GITS special edition that was released last year, or the oldschool one that came out shortly after DVD was invented?

the old PAL GITS is field-shifted progressive. telecide will handle it.

the new PAL GITS _should be_ pure progressive (i know the UK and australian ones are, dunno about French, but it should be too), whether it's flagged interlaced or not.

btw, if you have the old version, i recommend getting the special edition - it's a much much better transfer and a better encode. plus it has DTS, not that that really means much (it's less efficient than mp2, but it's a bit of an audiophile format).