Which kind of scenes should I use for settings comparison (anime)?

[Warwizard]

Hello-

I'm trying to find the best codec settings for anime through testing.

However, I don't know exactly when to take a screenshot. The 27 screenshots I took from every file could probably be reduced to 10 or less.

By the way, from the tests I've already made, the results are:

- Single-pass is always (much) better than 2-pass (lol);
- Curve compress 10/3 is better than the defaults;
- B-frames 2,1.62,0 is better than 2,1,1;
- AQ enabled is worse;
- B-frames 1,1,1 is worse;
- GMC enabled is slighly better in some situations, and slighly worse in others;
- Overflow=0: same as GMC;
- VHQ=4 is slighly worse than VHQ1;
- Quant. Restrictions 3,4,3,5,3,5: same as GMC, compared to 2,6,2,6,2,6;
- QPel enabled is slighly better, but I can't take most of the screenshots (in MPC) when QPel is enabled.

So what kind of scenes (darker/bright, static/motion-rich, etc.) should I aim for when taking screenshots, in order not to waste time looking at meaningless frames?

Also, which anime matrixes would you recommend me for testing?

Thanks.

[Teegedeck]

Screenshots are only good to illustrate something you actually see while watching the video (and be it in slow-motion); typically that would mean encoding problems like ringing, blocks in low-contrast scenes, or scenes whit tiny details where you can tell how well they get preserved. I don't think differences that can only be seen in a still frame matter much.

BTW, interesting observation about b-frame ratio. What average quantizers did you test at?

[Warwizard]

I tested at a quantizer of 4.33.

Anyway, why did I get insane quality with single pass?

I configured it with the same settings as if I was configuring a 2nd pass (in other words, I used 2nd pass settings for the single pass). And the final size is 8% smaller (single=11.8 MB; 2-pass=12.8 MB), at the same quantizer.

[Teegedeck]

In two-pass you will have quite some quantizer fluctuation, especially if you encoded with standard first-pass settings, i.e. the bitrate curve for second pass is scaled based on a encode at quant=2. The large discrepancy between first-pass filesize and second-pass filesize makes the prediction prone to error. Oftentimes XviD will choose a quantizer for a frame and the result will either be much too large (most often) or much too small and then the codec have to compensate for that; typically by using a much higher quantizer for the next frame(s).

So while at an average quantizer of 4.33 you mainly got quantizer=4 for p-frames and quantizer=6 for b-frames, and sometimes p-frame quant=5 and b-frame quant=8 you got many frames with larger quantizers in two-pass, like p-frame quant=7 and b-frame quant=11. that makes for quite a striking difference.

The best solution is to perform your first pass at the average quantizer you expect for 2nd pass. In this case that would have been a first pass at quantizer=4. This gives you very little quantizer fluctuation and thus a more consistent quality.

[Warwizard]

This gives you very little quantizer fluctuation and thus a more consistent quality.
But that is almost single-pass's job, except single-pass has no quant fluctuation at all. Does this mean I don't need a two-pass encode? That would save me 1/3 of the time spent.

BTW, now I remember that I performed the first pass at a quantizer=2 (as a zone), and, in the 2nd pass, I chose a quantizer of 4.33 and set weight=1 as a zone.

[Teegedeck]

But that is almost single-pass's job, except single-pass has no quant fluctuation at all....and except single-pass doesn't give you real control over the final filesize.
Does this mean I don't need a two-pass encode? That would save me 1/3 of the time spent.As for filesize: Yes, if you don't need to reach a final filesize precisely you can instead make a filesize prediction with Enc at constant quantizer and spare yourself the first pass if you can live with the predicted filesize.

As for quality: That's not so easy. Constant quantizer encodings have a theoretical quality advantage if quantizer doesn't fluctuate at all; i.e. if only even quanitzers are used (like 4.00). But if the predicted filesize for an even quantizer encoding is well below the filesize you wanted to reach, then a two-pass encoding (keeping in mind choosing a good first-pass quantizer) that reaches that filesize will always have an advantage.

Also, at high compression it can make sense that big frames are compressed a bit more strongly than small frames; that's what curve-compression is there for. But I strongly believe that this goes only for real-life material, not for anime!
BTW, now I remember that I performed the first pass at a quantizer=2 (as a zone)That was unnecessary because the default quantizer used for 1st pass already is '2'. , and, in the 2nd pass, I chose a quantizer of 4.33 and set weight=1 as a zone.You cannot set a constant quantizer in 2nd pass. I'm not sure what you have actually done there.

[Warwizard]

lol, I meant I set an average bitrate of 1030kbps.

Apart from the settings I tested, which settings might also worth a try?

[Teegedeck]

Cartoon mode. Apart from QPel and b-frame ratio settings your results are pretty much alongside what I thought.

[Warwizard]

I have always enabled cartoon mode. I don't think it will rise the quality when disabled.

And yes, some results were unexpected to me, especially VHQ.

Should I leave ME precision always at 6, chroma ME/optimizer always enabled and matrix always h.263? I heard that there are some matrixes out there designed for anime.

[Teegedeck]

You could experiment with ME=4, VHQ=3; I'd be curious myself about the outcome.

For the average quantizer that you aim at there is nothing superior to H.263.

[Blue_MiSfit]

If you are trying to compare different versions of the same encode ie with different filters or encoder settings, AviSynth comes in handy.

Make a new script with code like this:

a=directshowsource("file1.mp4")
b=directshowsource("file2.mp4")
return stackvertical(a,b)


then open that script with a directshow player like media player classic. This will allow side by side playback of two different files so you can compare them.

You can also substitte another avisynth script for one of the MP4s in this example. This will allow you to compare the source (processed) to the encoded result. Good for checking for details and blur etc....

StackHorizontal(...) can also be used, both are useful depending on the files' resolution, and your monitor display resolution.

I would encourage you to learn a bit of avisynth. Especially with anime, much quality can be gained with noise reduction filters to make XviD's job easier.

Good luck
~MiSfit

ps just so you know
matrix = singular
matricies = plural

:)

[Prettz]

You could experiment with ME=4, VHQ=3; I'd be curious myself about the outcome.

For the average quantizer that you aim at there is nothing superior to H.263.
I hate to contradict Teegedeck, but this is extremely untrue. H.263 quantization is quite bad compared to a lot of custom quant matrices. The HVS trio are in my opinion the best tried-and-true custom matrices you can find. They always, always produce better quality and compression than MPEG and H.263, and with anime it's no exception. The standard matrices are too PSNR-tuned to be able to compete with the HVS matrices.

H.263 produces far more ringing and artifacting in general than the hvs-good matrix for most anime.

[Teegedeck]

H.263 quantization is quite bad compared to a lot of custom quant matrices.Hmm, that statement surprised me a little. I'd say: For movies, definitely yes; for anime, no. Not at an average quantizer of 4.3. At such high quants, MPEG and MPEG-custom matrices have a tendency stronger than H.263 towards ringing and blocking (anime) from what I saw (and I think this must also hold true for 'HVS-good'). This is highly annoying with anime.

The HVS trio are in my opinion the best tried-and-true custom matrices you can find. They always, always produce better quality and compression than MPEG and H.263, and with anime it's no exception. IMHO the 'HVS' matrices always were the worst of all XviD CQMs, whenever I compared them to others they exhibited a low level of detail and a high tendency towards blocking. The latter is something you don't want when encoding anime, I think. So I would at least strongly contradict the 'always' here. This seems to be very much up to personal taste.

I liked the HVS matrices when they came out (they were the very first custom matrices specifically made for XviD as far as I recall) but they were quickly outshone by other CQMs as far as my eyes tell me. True, they were not modelled for reaching good PSNR values, that is a good thing; but they were modelled after an HVS model that doesn't work for me at all. And even if it did work it wasn't meant for artificial video content where other rules apply (the same goes for XviD's AQ; an HVS-based approach which also works fine with natural video but horribly fails for artificial video like anime). I prefer matrices that are modelled after actual human perception. :)

H.263 produces far more ringing and artifacting in general than the hvs-good matrix for most anime.At quantizer 4 and above???

[CRCC]

I have to agree with teegedeck encoding anime low bitrate h.263 is good (i don't like low bitrate) otherwise 6of9,eqm_uhr or heini_mr are great (understatement) for anime/cartoon and movies.
for low bitrate encoding of movies i'm using eqm_v3lr, eqm_v3ulr they really give good results better than hvs matrices less blocking clearer picture.

[Prettz]

At such high quants, MPEG and MPEG-custom matrices have a tendency stronger than H.263 towards ringing and blocking (anime) from what I saw (and I think this must also hold true for 'HVS-good'). This is highly annoying with anime.

That's odd to me too. I would stand firmly behind just the opposite statement. As you said later on, this must definitely be up to personal taste.

What I like about hvs-good vs. H.263 is that it's quite good at retaining some graininess (assuming the source is a DVD) throughout the picture, while keeping that graininess from getting too ugly. This works wonders towards hiding ringing artifacts around cel borders, which are for me the absolute worst compression artifacts to see in anime. In addition, I would say that hvs-good just plain produces less ringing to begin with, as well.


IMHO the 'HVS' matrices always were the worst of all XviD CQMs...

Oh I never said the HVS series was the best. In fact, it's the ONLY set of custom matrices I've tested. Back when all the new matrices were coming out I stuck with the standard matrices just because I like to stick with what's tried-and-true and not risk screwing up parts of an encode because the matrix isn't totally perfect. When I tested the HVS ones they were much better than the standard ones in all situations.

So I guess basically, if you're like me and are unsure of all these amateur-made matrices, you should still definitely always use at least the HVS matrices over the standard ones. Beyond those, there's so many lesser-known custom ones it's really hard to keep track of them all and know which are good and which are very very niche.


whenever I compared them to others they exhibited a low level of detail and a high tendency towards blocking.

You'd have to explain a bit further what you meant by "high tendency towards blocking", it's a little vague. If you're talking about blocking of film graininess, then this might go back to personal taste. Ringing is my prime concern, blockiness not so much.


At quantizer 4 and above???
You know, I can't answer that actually. I'll have to go back and look at what quantizers my anime encodes came out to, but I'm pretty sure that nothing I've done has come out to an average quant of 4.


Perhaps I'll have to do some further testing of anime at such low bitrates. I certainly wouldn't have assumed that the situation changes drastically between hvs-good and H.263 once you go below a bitrate threshold.

[Teegedeck]

What I like about hvs-good vs. H.263 is that it's quite good at retaining some graininess (assuming the source is a DVD) throughout the picture, while keeping that graininess from getting too ugly. Hmm. Actual film grain is extremely hard to preserve; I'd say you have to use a 'very-high-bitrate' matrix and lower VHQ and ME precision values considerably. As far as I recall the HVS matrices, as do all MPEG-matrices, introduce some level of artifacting of their own at some point which can be perceived as noise (perhaps grain). I'm gonna give the HVS'es a whirl to test it out. :)
So I guess basically, if you're like me and are unsure of all these amateur-made matrices, you should still definitely always use at least the HVS matrices over the standard ones.The HVS matrices were also written by an 'amateur'... ;) As was XviD and most other worthwhile inventions in the field BTW. I tend to believe my eyes and just choose the matrices that produce the most pleasant results. Whether those matrices are based on an HVS model doesn't mean much to me.
Beyond those, there's so many lesser-known custom ones it's really hard to keep track of them all and know which are good and which are very very niche.Wellll... From what I see on the forum the HVS-matrices themselves have pretty much become niche matrices by now. The most talked-about (and also used, I think) matrices certainly are Sharkooth's EQM series and Didee's SixOfNine. Not sure about it but I think Sharktooth's matrices are also widely used in AutoGK.
You'd have to explain a bit further what you meant by "high tendency towards blocking", it's a little vague. If you're talking about blocking of film graininess, then this might go back to personal taste. Ringing is my prime concern, blockiness not so much.I meant actually visible macroblock borders. But, as said before, I'm gonna test that again, memory betrays...
I certainly wouldn't have assumed that the situation changes drastically between hvs-good and H.263 once you go below a bitrate threshold.Matters indeed change radically once a CQM is used beyond its 'sweet-spot'. For example, up to quantizer=4 I prefer SixOfNine over every other matrix. But beyond that, SixOfNine is easily worse than other matrices with higher coefficients, like EQMv3 LR, or even MPEG.
If bitrate wouldn't matter I'd always recommend using SixOfNine at quant=3 or something which should definitely look better. But well, nobody wants 'high-bitrate' anime.

[Prettz]

Wellll... From what I see on the forum the HVS-matrices themselves have pretty much become niche matrices by now. The most talked-about (and also used, I think) matrices certainly are Sharkooth's EQM series and Didee's SixOfNine. Not sure about it but I think Sharktooth's matrices are also widely used in AutoGK.

Well I would certainly have gone and given those matrices a good testing for the next DVD I rip, but I'm now learning x264, and plan to stick with H.264 for good :-( There's custom matrices for H.264 already, though.


But well, nobody wants 'high-bitrate' anime.
Well I do. By high-bitrate of course I mean 2CD rips of movies or ~200MB TV show episodes. If you mean more than that, then yeah, you're right.